2024-06-06T08:17:17.335 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-06T08:17:17.335 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --name teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} --verbose -- /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/orig.config.yaml 2024-06-06T08:17:17.393 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T08:17:17.482 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7743249' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5050 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 416/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI3XCK60BwihmK2lAgoOdo04qKaPtBzVQ24qvJG4iWu9ZGqfNWCzh+C6kk1hQYN6tvRP6HwFqEPsvEnJhSTTiXU= smithi176.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBITV/p8ua9np0IYn6rtPlqK99k5rNGal8F1Apo5ASxgUsMrZhSEN3nXQy9O0e9fmoxC1S9qR4U2RhFvf/5wPJDc= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJTv+yejbnSo1wJE4x57hX/NHf2GjSdl6JcMJdBuwxkt9UZjTs7IkiMjRoF+NeGsfN8L6FD8K0zNmIR2/y2xxDw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-05_21:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-06T08:17:17.482 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa; will attempt to use it 2024-06-06T08:17:17.483 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks 2024-06-06T08:17:17.483 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-06T08:17:17.484 INFO:teuthology.task.internal:Checking packages... 2024-06-06T08:17:17.504 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-06T08:17:17.505 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-06T08:17:17.505 INFO:teuthology.packaging:ref: None 2024-06-06T08:17:17.505 INFO:teuthology.packaging:tag: None 2024-06-06T08:17:17.505 INFO:teuthology.packaging:branch: squid 2024-06-06T08:17:17.505 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:17:17.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-06T08:17:17.700 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2960.g3d746ca5 2024-06-06T08:17:17.701 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-06T08:17:17.708 INFO:teuthology.task.internal:no buildpackages task found 2024-06-06T08:17:17.708 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-06T08:17:17.752 INFO:teuthology.task.internal:Saving configuration 2024-06-06T08:17:17.772 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-06T08:17:17.786 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-06T08:17:17.811 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 08:11:53.509297', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI3XCK60BwihmK2lAgoOdo04qKaPtBzVQ24qvJG4iWu9ZGqfNWCzh+C6kk1hQYN6tvRP6HwFqEPsvEnJhSTTiXU='} 2024-06-06T08:17:17.831 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 08:11:53.511207', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBITV/p8ua9np0IYn6rtPlqK99k5rNGal8F1Apo5ASxgUsMrZhSEN3nXQy9O0e9fmoxC1S9qR4U2RhFvf/5wPJDc='} 2024-06-06T08:17:17.852 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 08:11:53.512445', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJTv+yejbnSo1wJE4x57hX/NHf2GjSdl6JcMJdBuwxkt9UZjTs7IkiMjRoF+NeGsfN8L6FD8K0zNmIR2/y2xxDw='} 2024-06-06T08:17:17.852 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-06T08:17:17.858 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-06T08:17:17.858 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-06T08:17:17.858 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-06T08:17:17.858 INFO:teuthology.run_tasks:Running task console_log... 2024-06-06T08:17:17.971 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9a58d33be0>, signals=[15]) 2024-06-06T08:17:17.972 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-06T08:17:17.978 INFO:teuthology.task.internal:Opening connections... 2024-06-06T08:17:17.978 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:17:17.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:17:18.056 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi176.front.sepia.ceph.com 2024-06-06T08:17:18.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:17:18.131 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-06-06T08:17:18.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:17:18.209 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-06T08:17:18.216 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-06-06T08:17:18.234 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-06-06T08:17:18.234 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-06-06T08:17:18.293 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-06-06T08:17:18.293 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-06-06T08:17:18.293 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-06-06T08:17:18.293 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T08:17:18.294 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T08:17:18.295 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-06-06T08:17:18.316 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2024-06-06T08:17:18.334 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2024-06-06T08:17:18.335 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2024-06-06T08:17:18.391 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream" 2024-06-06T08:17:18.391 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="9" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="9" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el9" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:LOGO="fedora-logo-icon" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T08:17:18.392 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T08:17:18.393 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2024-06-06T08:17:18.416 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-06-06T08:17:18.434 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-06-06T08:17:18.435 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-06-06T08:17:18.491 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-06-06T08:17:18.491 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T08:17:18.492 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T08:17:18.493 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-06-06T08:17:18.516 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-06T08:17:18.527 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-06T08:17:18.561 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-06T08:17:18.561 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-06-06T08:17:18.564 DEBUG:teuthology.orchestra.run.smithi176:> test '!' -e /home/ubuntu/cephtest 2024-06-06T08:17:18.566 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-06-06T08:17:18.582 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-06T08:17:18.593 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-06T08:17:18.593 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-06-06T08:17:18.622 DEBUG:teuthology.orchestra.run.smithi176:> test -z $(ls -A /var/lib/ceph) 2024-06-06T08:17:18.625 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-06-06T08:17:18.714 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-06T08:17:18.796 INFO:teuthology.run_tasks:Running task kernel... 2024-06-06T08:17:18.810 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-06T08:17:18.811 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-06-06T08:17:18.811 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-06-06T08:17:18.811 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T08:17:18.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:17:18.828 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-06-06T08:17:18.887 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-437.el9.x86_64 2024-06-06T08:17:18.887 INFO:teuthology.task.kernel:Running kernel on smithi027: 5.14.0-437.el9.x86_64 2024-06-06T08:17:18.887 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2024-06-06T08:17:23.859 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.1 MB 00:03 2024-06-06T08:17:25.777 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-06-06T08:17:30.194 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-06-06T08:17:31.660 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 17 kB 00:00 2024-06-06T08:17:32.747 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-06-06T08:17:38.244 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-06-06T08:17:40.375 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-06T08:17:40.376 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-06T08:17:40.481 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-06T08:17:40.489 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:17:40.490 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-06-06T08:17:40.490 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-06-06T08:17:40.490 INFO:teuthology.orchestra.run.smithi027.stdout:Install 4 Packages 2024-06-06T08:17:40.490 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:17:40.490 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 80 M 2024-06-06T08:17:40.491 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 124 M 2024-06-06T08:17:40.491 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-06-06T08:17:40.720 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:17:40.722 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:17:40.731 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:17:40.862 INFO:teuthology.orchestra.run.smithi027.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 3.9 MB/s | 893 kB 00:00 2024-06-06T08:17:41.196 INFO:teuthology.orchestra.run.smithi027.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-06-06T08:17:41.554 INFO:teuthology.orchestra.run.smithi027.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 42 MB/s | 29 MB 00:00 2024-06-06T08:17:42.030 INFO:teuthology.orchestra.run.smithi027.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 25 MB/s | 35 MB 00:01 2024-06-06T08:17:42.032 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-06-06T08:17:42.033 INFO:teuthology.orchestra.run.smithi027.stdout:Total 52 MB/s | 80 MB 00:01 2024-06-06T08:17:42.761 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-06-06T08:17:43.169 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-06-06T08:17:43.169 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-06-06T08:17:44.142 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-06-06T08:17:44.142 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-06-06T08:17:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-06-06T08:17:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-06T08:17:47.472 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T08:17:50.235 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T08:17:50.376 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T08:17:55.462 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T08:17:55.515 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T08:18:00.625 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T08:18:31.817 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T08:18:31.860 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-06T08:18:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T08:18:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-06T08:18:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T08:18:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T08:18:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T08:18:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:18:33.030 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-06-06T08:18:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-06T08:18:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-06T08:18:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-06T08:18:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-06T08:18:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:18:33.030 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-06-06T08:18:33.378 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T08:18:33.456 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T08:18:33.456 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T08:18:33.456 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-06T08:18:33.456 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-06T08:18:33.456 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-06T08:18:33.456 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-06-06T08:18:33.457 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2024-06-06T08:18:33.457 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T08:18:33.510 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T08:18:33.511 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T08:18:33.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-06-06T08:18:33.543 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-06-06T08:18:33.543 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-06-06T08:18:33.543 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-06-06T08:18:33.543 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 68.el9 2024-06-06T08:18:33.543 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-06-06T08:18:33.543 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-06T08:18:33.543 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-06-06T08:18:33.543 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8264254 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-06-06T08:18:33.544 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-06-06T08:18:33.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:18:33.545 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-06T08:18:33.545 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-06T08:18:33.545 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-06T08:18:33.545 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-06-06T08:18:33.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:18:33.545 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-06-06T08:18:33.546 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-06T08:18:33.546 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 5.14.0-452.el9.x86_64 2024-06-06T08:18:33.547 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-06T08:18:34.114 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-06-06T08:18:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-06T08:18:35.598 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-06-06T08:18:35.600 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-06-06T08:18:35.620 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.ajQHTSV4xF 2024-06-06T08:18:35.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ajQHTSV4xF 2024-06-06T08:18:35.685 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.ajQHTSV4xF 2024-06-06T08:18:35.824 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.ajQHTSV4xF is 6KB 2024-06-06T08:18:35.875 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.ajQHTSV4xF 2024-06-06T08:18:35.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-06-06T08:18:35.954 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-06T08:18:35.955 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-06T08:18:35.955 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-06T08:18:35.955 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-06T08:18:35.956 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-06T08:18:36.101 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-06-06T08:19:06.104 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-06T08:19:06.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-06-06T08:19:06.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:19:40.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-06-06T08:19:49.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-06-06T08:19:49.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:19:52.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-06-06T08:20:04.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-06-06T08:20:04.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:20:23.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-06-06T08:20:38.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-06-06T08:20:38.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:20:38.466 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-06-06T08:20:38.912 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-06-06T08:20:38.912 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-452.el9.x86_64"... 2024-06-06T08:20:38.913 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-06-06T08:20:38.928 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-452.el9.x86_64 2024-06-06T08:20:38.928 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-06T08:20:38.928 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-06T08:20:38.928 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-06T08:20:39.930 DEBUG:teuthology.parallel:result is None 2024-06-06T08:20:39.930 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-06T08:20:39.937 INFO:teuthology.task.internal:Creating test directory... 2024-06-06T08:20:39.938 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T08:20:39.940 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T08:20:39.943 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T08:20:39.961 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-06T08:20:39.980 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-06T08:20:40.030 INFO:teuthology.task.internal:Creating archive directory... 2024-06-06T08:20:40.030 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T08:20:40.033 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T08:20:40.035 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T08:20:40.165 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-06T08:20:40.174 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-06T08:20:40.174 DEBUG:teuthology.orchestra.run.smithi176:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T08:20:40.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:20:40.191 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T08:20:40.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:20:40.208 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T08:20:40.210 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T08:20:40.234 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T08:20:40.265 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T08:20:40.279 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T08:20:40.296 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T08:20:40.316 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T08:20:40.343 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T08:20:40.690 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T08:20:40.692 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-06T08:20:40.700 INFO:teuthology.task.internal:Configuring sudo... 2024-06-06T08:20:40.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T08:20:40.702 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T08:20:40.705 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T08:20:40.750 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-06T08:20:40.760 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-06T08:20:40.760 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T08:20:40.762 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T08:20:40.774 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T08:20:40.792 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T08:20:40.904 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T08:20:40.972 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T08:20:41.042 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T08:20:41.115 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:20:41.115 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T08:20:41.176 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T08:20:41.222 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T08:20:41.273 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T08:20:41.347 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T08:20:41.399 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:20:41.400 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T08:20:41.465 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T08:20:41.506 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T08:20:41.572 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T08:20:41.611 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T08:20:41.656 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:20:41.656 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T08:20:41.723 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-06-06T08:20:41.725 DEBUG:teuthology.orchestra.run.smithi176:> sudo service rsyslog restart 2024-06-06T08:20:41.727 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-06-06T08:20:41.764 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:20:41.795 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:20:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:20:42.429 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-06T08:20:42.799 INFO:teuthology.task.internal:Starting timer... 2024-06-06T08:20:42.799 INFO:teuthology.run_tasks:Running task pcp... 2024-06-06T08:20:42.918 INFO:teuthology.run_tasks:Running task selinux... 2024-06-06T08:20:43.013 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-06T08:20:43.013 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-06-06T08:20:43.085 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-06-06T08:20:43.087 DEBUG:teuthology.orchestra.run.smithi176:> sudo service auditd rotate 2024-06-06T08:20:43.188 INFO:teuthology.orchestra.run.smithi176.stdout:Rotating logs: 2024-06-06T08:20:43.190 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-06-06T08:20:43.239 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-06-06T08:20:43.240 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-06T08:20:43.240 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-06-06T08:20:43.264 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-06-06T08:20:43.264 DEBUG:teuthology.orchestra.run.smithi176:> /usr/sbin/getenforce 2024-06-06T08:20:43.281 INFO:teuthology.orchestra.run.smithi176.stdout:Permissive 2024-06-06T08:20:43.281 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-06-06T08:20:43.303 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-06-06T08:20:43.303 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi176.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-06-06T08:20:43.304 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:20:43.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:20:43.330 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:20:43.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:20:43.355 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:20:43.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:20:43.379 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-06T08:20:43.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-06-06T08:20:43.405 DEBUG:teuthology.orchestra.run.smithi176:> sudo /usr/sbin/setenforce permissive 2024-06-06T08:20:43.429 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-06-06T08:20:43.454 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-06T08:20:43.464 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-06T08:20:43.477 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-06T08:20:43.478 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi176.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-06T08:25:17.261 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi176.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-06-06T08:25:17.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-06-06T08:25:17.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:25:17.345 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-06-06T08:25:17.422 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-06-06T08:25:17.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-06-06T08:25:17.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:25:17.499 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-06-06T08:25:17.578 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-06-06T08:25:17.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-06-06T08:25:17.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T08:25:17.652 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-06-06T08:25:17.734 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-06-06T08:25:17.734 INFO:teuthology.run_tasks:Running task clock... 2024-06-06T08:25:17.747 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-06T08:25:17.748 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T08:25:17.748 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:25:17.750 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T08:25:17.751 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:25:17.753 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T08:25:17.753 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:25:17.780 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T08:25:17.780 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T08:25:17.794 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T08:25:17.795 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T08:25:17.804 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T08:25:17.817 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: ntpd: command not found 2024-06-06T08:25:17.818 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T08:25:17.820 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-06-06T08:25:17.827 INFO:teuthology.orchestra.run.smithi176.stdout:506 Cannot talk to daemon 2024-06-06T08:25:17.832 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-06-06T08:25:17.840 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T08:25:17.842 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-06-06T08:25:17.846 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T08:25:17.852 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-06-06T08:25:17.853 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T08:25:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T08:25:17.866 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T08:25:17.879 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T08:25:17.911 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:25:17.915 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:25:17.915 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-06-06T08:25:17.915 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.915 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.915 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.915 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.923 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:25:17.927 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:25:17.927 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-06-06T08:25:17.927 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.928 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.928 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.928 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.938 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:25:17.942 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:25:17.942 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-06-06T08:25:17.942 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.942 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.942 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.942 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:25:17.942 INFO:teuthology.run_tasks:Running task install... 2024-06-06T08:25:17.950 DEBUG:teuthology.task.install:project ceph 2024-06-06T08:25:17.950 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-06T08:25:17.951 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-06T08:25:17.951 INFO:teuthology.task.install:Using flavor: default 2024-06-06T08:25:17.956 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-06T08:25:17.956 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-06T08:25:17.957 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T08:25:17.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:25:17.959 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T08:25:17.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:25:17.960 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T08:25:17.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:25:18.112 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T08:25:18.112 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T08:25:18.190 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T08:25:18.190 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T08:25:18.193 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T08:25:18.193 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T08:25:18.284 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T08:25:18.284 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:25:18.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T08:25:18.298 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T08:25:18.298 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:25:18.298 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T08:25:18.305 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T08:25:18.306 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:25:18.306 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T08:25:18.311 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T08:25:18.311 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T08:25:18.323 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T08:25:18.324 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T08:25:18.330 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T08:25:18.330 DEBUG:teuthology.orchestra.run.smithi176:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T08:25:18.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T08:25:18.388 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T08:25:18.394 DEBUG:teuthology.orchestra.run.smithi176:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T08:25:18.454 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T08:25:18.461 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T08:25:18.467 DEBUG:teuthology.orchestra.run.smithi176:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T08:25:18.519 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-06-06T08:25:18.521 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-06-06T08:25:18.524 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-06-06T08:25:18.525 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-06-06T08:25:18.529 INFO:teuthology.orchestra.run.smithi176.stdout:check_obsoletes = 1 2024-06-06T08:25:18.531 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum clean all 2024-06-06T08:25:18.750 INFO:teuthology.orchestra.run.smithi203.stdout:45 files removed 2024-06-06T08:25:18.753 INFO:teuthology.orchestra.run.smithi176.stdout:45 files removed 2024-06-06T08:25:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:45 files removed 2024-06-06T08:25:18.777 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T08:25:18.780 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T08:25:18.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T08:25:19.443 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 231 kB/s | 83 kB 00:00 2024-06-06T08:25:19.445 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-06-06T08:25:19.453 INFO:teuthology.orchestra.run.smithi176.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-06-06T08:25:19.777 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 41 kB/s | 11 kB 00:00 2024-06-06T08:25:19.795 INFO:teuthology.orchestra.run.smithi176.stdout:ceph noarch packages 39 kB/s | 11 kB 00:00 2024-06-06T08:25:19.854 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-06-06T08:25:20.205 INFO:teuthology.orchestra.run.smithi176.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-06-06T08:25:20.220 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-06-06T08:25:20.263 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-06-06T08:25:20.773 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-06-06T08:25:20.840 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-06-06T08:25:20.981 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-06-06T08:25:22.659 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-06-06T08:25:22.684 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-06-06T08:25:22.866 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-06-06T08:25:27.051 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.2 MB 00:00 2024-06-06T08:25:27.137 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.2 MB 00:00 2024-06-06T08:25:27.241 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.2 MB 00:00 2024-06-06T08:25:28.669 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 17 kB 00:00 2024-06-06T08:25:28.685 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 17 kB 00:00 2024-06-06T08:25:29.150 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 17 kB 00:00 2024-06-06T08:25:30.498 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-06-06T08:25:30.538 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-06-06T08:25:30.586 INFO:teuthology.orchestra.run.smithi176.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-06-06T08:25:35.719 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-06-06T08:25:35.969 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-06T08:25:36.067 INFO:teuthology.orchestra.run.smithi176.stdout:lab-extras 5.3 kB/s | 1.7 kB 00:00 2024-06-06T08:25:38.062 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T08:25:38.063 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T08:25:38.074 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T08:25:38.075 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T08:25:38.076 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T08:25:38.076 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T08:25:38.078 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T08:25:38.080 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T08:25:38.081 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T08:25:38.145 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-06-06T08:25:38.161 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-06-06T08:25:38.161 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-06-06T08:25:38.161 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T08:25:38.162 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T08:25:38.163 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-06-06T08:25:38.164 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T08:25:38.165 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T08:25:38.166 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T08:25:38.167 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T08:25:38.168 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T08:25:38.169 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T08:25:38.170 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T08:25:38.171 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T08:25:38.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T08:25:38.173 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-06-06T08:25:38.174 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T08:25:38.174 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:25:38.174 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-06-06T08:25:38.174 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-06-06T08:25:38.174 INFO:teuthology.orchestra.run.smithi203.stdout:Install 148 Packages 2024-06-06T08:25:38.174 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 11 Packages 2024-06-06T08:25:38.174 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:25:38.174 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 225 M 2024-06-06T08:25:38.175 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-06-06T08:25:38.435 INFO:teuthology.orchestra.run.smithi176.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T08:25:38.435 INFO:teuthology.orchestra.run.smithi176.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T08:25:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T08:25:38.448 INFO:teuthology.orchestra.run.smithi176.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T08:25:38.448 INFO:teuthology.orchestra.run.smithi176.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T08:25:38.449 INFO:teuthology.orchestra.run.smithi176.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T08:25:38.451 INFO:teuthology.orchestra.run.smithi176.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T08:25:38.453 INFO:teuthology.orchestra.run.smithi176.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T08:25:38.454 INFO:teuthology.orchestra.run.smithi176.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T08:25:38.463 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T08:25:38.464 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T08:25:38.475 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T08:25:38.476 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T08:25:38.476 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T08:25:38.477 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T08:25:38.483 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T08:25:38.483 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T08:25:38.483 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T08:25:38.518 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-06-06T08:25:38.532 INFO:teuthology.orchestra.run.smithi176.stdout:======================================================================================= 2024-06-06T08:25:38.532 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout:======================================================================================= 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T08:25:38.533 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T08:25:38.534 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T08:25:38.535 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrading: 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T08:25:38.536 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T08:25:38.537 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T08:25:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T08:25:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T08:25:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T08:25:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T08:25:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T08:25:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T08:25:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T08:25:38.538 INFO:teuthology.orchestra.run.smithi176.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T08:25:38.539 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T08:25:38.540 INFO:teuthology.orchestra.run.smithi176.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T08:25:38.541 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T08:25:38.542 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T08:25:38.543 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T08:25:38.544 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T08:25:38.545 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout:Installing weak dependencies: 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:25:38.546 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-06-06T08:25:38.547 INFO:teuthology.orchestra.run.smithi176.stdout:======================================================================================= 2024-06-06T08:25:38.547 INFO:teuthology.orchestra.run.smithi176.stdout:Install 148 Packages 2024-06-06T08:25:38.547 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrade 11 Packages 2024-06-06T08:25:38.547 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:25:38.547 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 225 M 2024-06-06T08:25:38.547 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-06-06T08:25:38.549 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-06-06T08:25:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-06-06T08:25:38.565 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-06-06T08:25:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-06-06T08:25:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T08:25:38.566 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T08:25:38.567 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T08:25:38.568 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T08:25:38.569 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T08:25:38.570 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T08:25:38.571 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T08:25:38.572 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T08:25:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T08:25:38.574 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T08:25:38.575 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T08:25:38.576 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-06-06T08:25:38.577 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-06-06T08:25:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:Install 148 Packages 2024-06-06T08:25:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 11 Packages 2024-06-06T08:25:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:25:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 225 M 2024-06-06T08:25:38.578 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-06-06T08:25:38.778 INFO:teuthology.orchestra.run.smithi203.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-06-06T08:25:39.212 INFO:teuthology.orchestra.run.smithi203.stdout:(2/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 7.8 MB/s | 5.4 MB 00:00 2024-06-06T08:25:39.212 INFO:teuthology.orchestra.run.smithi027.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-06-06T08:25:39.226 INFO:teuthology.orchestra.run.smithi176.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-06-06T08:25:39.279 INFO:teuthology.orchestra.run.smithi203.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 2.1 MB/s | 145 kB 00:00 2024-06-06T08:25:39.579 INFO:teuthology.orchestra.run.smithi203.stdout:(4/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 890 kB 00:00 2024-06-06T08:25:39.630 INFO:teuthology.orchestra.run.smithi203.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 6.7 MB/s | 2.4 MB 00:00 2024-06-06T08:25:40.201 INFO:teuthology.orchestra.run.smithi176.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 912 kB/s | 890 kB 00:00 2024-06-06T08:25:40.276 INFO:teuthology.orchestra.run.smithi176.stdout:(3/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 4.0 MB/s | 5.4 MB 00:01 2024-06-06T08:25:40.377 INFO:teuthology.orchestra.run.smithi176.stdout:(4/159): ceph-immutable-object-cache-19.0.0-294 827 kB/s | 145 kB 00:00 2024-06-06T08:25:40.513 INFO:teuthology.orchestra.run.smithi203.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-06-06T08:25:40.596 INFO:teuthology.orchestra.run.smithi027.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 643 kB/s | 890 kB 00:01 2024-06-06T08:25:40.685 INFO:teuthology.orchestra.run.smithi176.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 5.8 MB/s | 2.4 MB 00:00 2024-06-06T08:25:40.738 INFO:teuthology.orchestra.run.smithi027.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.0 MB/s | 145 kB 00:00 2024-06-06T08:25:40.755 INFO:teuthology.orchestra.run.smithi203.stdout:(7/159): ceph-common-19.0.0-2946.g3d89accf.el9. 9.4 MB/s | 21 MB 00:02 2024-06-06T08:25:40.831 INFO:teuthology.orchestra.run.smithi203.stdout:(8/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-06-06T08:25:40.906 INFO:teuthology.orchestra.run.smithi203.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 336 kB/s | 25 kB 00:00 2024-06-06T08:25:41.327 INFO:teuthology.orchestra.run.smithi176.stdout:(6/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 7.3 MB/s | 4.7 MB 00:00 2024-06-06T08:25:41.619 INFO:teuthology.orchestra.run.smithi176.stdout:(7/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.2 MB/s | 1.5 MB 00:01 2024-06-06T08:25:42.063 INFO:teuthology.orchestra.run.smithi027.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-06-06T08:25:42.905 INFO:teuthology.orchestra.run.smithi027.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-06-06T08:25:43.689 INFO:teuthology.orchestra.run.smithi203.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 3.5 MB/s | 10 MB 00:02 2024-06-06T08:25:43.848 INFO:teuthology.orchestra.run.smithi203.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.4 MB/s | 221 kB 00:00 2024-06-06T08:25:43.931 INFO:teuthology.orchestra.run.smithi203.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 325 kB/s | 27 kB 00:00 2024-06-06T08:25:44.148 INFO:teuthology.orchestra.run.smithi203.stdout:(13/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 4.6 MB/s | 17 MB 00:03 2024-06-06T08:25:44.174 INFO:teuthology.orchestra.run.smithi203.stdout:(14/159): libcephfs-devel-19.0.0-2946.g3d89accf 131 kB/s | 32 kB 00:00 2024-06-06T08:25:44.266 INFO:teuthology.orchestra.run.smithi203.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.7 MB/s | 163 kB 00:00 2024-06-06T08:25:44.324 INFO:teuthology.orchestra.run.smithi203.stdout:(16/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.2 MB/s | 747 kB 00:00 2024-06-06T08:25:44.330 INFO:teuthology.orchestra.run.smithi027.stdout:(6/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 673 kB/s | 2.4 MB 00:03 2024-06-06T08:25:44.350 INFO:teuthology.orchestra.run.smithi203.stdout:(17/159): librados-devel-19.0.0-2946.g3d89accf. 1.5 MB/s | 127 kB 00:00 2024-06-06T08:25:44.433 INFO:teuthology.orchestra.run.smithi203.stdout:(18/159): libradosstriper1-19.0.0-2946.g3d89acc 4.5 MB/s | 501 kB 00:00 2024-06-06T08:25:44.534 INFO:teuthology.orchestra.run.smithi203.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 451 kB/s | 45 kB 00:00 2024-06-06T08:25:44.617 INFO:teuthology.orchestra.run.smithi203.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.6 MB/s | 135 kB 00:00 2024-06-06T08:25:44.709 INFO:teuthology.orchestra.run.smithi203.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.7 MB/s | 162 kB 00:00 2024-06-06T08:25:44.794 INFO:teuthology.orchestra.run.smithi176.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 3.6 MB/s | 21 MB 00:05 2024-06-06T08:25:44.801 INFO:teuthology.orchestra.run.smithi203.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.4 MB/s | 321 kB 00:00 2024-06-06T08:25:44.844 INFO:teuthology.orchestra.run.smithi176.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 506 kB/s | 25 kB 00:00 2024-06-06T08:25:44.855 INFO:teuthology.orchestra.run.smithi027.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 2.4 MB/s | 4.7 MB 00:01 2024-06-06T08:25:44.893 INFO:teuthology.orchestra.run.smithi203.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.2 MB/s | 300 kB 00:00 2024-06-06T08:25:44.985 INFO:teuthology.orchestra.run.smithi203.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.1 MB/s | 99 kB 00:00 2024-06-06T08:25:45.069 INFO:teuthology.orchestra.run.smithi203.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-06T08:25:45.095 INFO:teuthology.orchestra.run.smithi176.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 4.5 MB/s | 17 MB 00:03 2024-06-06T08:25:45.195 INFO:teuthology.orchestra.run.smithi176.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.2 MB/s | 221 kB 00:00 2024-06-06T08:25:45.287 INFO:teuthology.orchestra.run.smithi176.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 295 kB/s | 27 kB 00:00 2024-06-06T08:25:45.378 INFO:teuthology.orchestra.run.smithi176.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 346 kB/s | 32 kB 00:00 2024-06-06T08:25:45.628 INFO:teuthology.orchestra.run.smithi203.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 5.5 MB/s | 3.1 MB 00:00 2024-06-06T08:25:45.628 INFO:teuthology.orchestra.run.smithi176.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.9 MB/s | 747 kB 00:00 2024-06-06T08:25:45.712 INFO:teuthology.orchestra.run.smithi176.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.9 MB/s | 163 kB 00:00 2024-06-06T08:25:45.728 INFO:teuthology.orchestra.run.smithi203.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.7 MB/s | 171 kB 00:00 2024-06-06T08:25:45.771 INFO:teuthology.orchestra.run.smithi176.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 2.1 MB/s | 127 kB 00:00 2024-06-06T08:25:45.854 INFO:teuthology.orchestra.run.smithi203.stdout:(28/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.5 MB/s | 5.3 MB 00:01 2024-06-06T08:25:45.879 INFO:teuthology.orchestra.run.smithi203.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2946.g 211 kB/s | 32 kB 00:00 2024-06-06T08:25:45.888 INFO:teuthology.orchestra.run.smithi176.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 4.2 MB/s | 501 kB 00:00 2024-06-06T08:25:45.920 INFO:teuthology.orchestra.run.smithi203.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.1 MB/s | 142 kB 00:00 2024-06-06T08:25:46.546 INFO:teuthology.orchestra.run.smithi203.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 5.3 MB/s | 3.5 MB 00:00 2024-06-06T08:25:46.663 INFO:teuthology.orchestra.run.smithi203.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.1 MB/s | 249 kB 00:00 2024-06-06T08:25:46.763 INFO:teuthology.orchestra.run.smithi203.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 492 kB/s | 49 kB 00:00 2024-06-06T08:25:46.864 INFO:teuthology.orchestra.run.smithi203.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 164 kB/s | 16 kB 00:00 2024-06-06T08:25:46.973 INFO:teuthology.orchestra.run.smithi203.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.4 MB/s | 264 kB 00:00 2024-06-06T08:25:47.173 INFO:teuthology.orchestra.run.smithi203.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 3.7 MB/s | 763 kB 00:00 2024-06-06T08:25:47.248 INFO:teuthology.orchestra.run.smithi203.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 243 kB/s | 18 kB 00:00 2024-06-06T08:25:47.287 INFO:teuthology.orchestra.run.smithi203.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:47.296 INFO:teuthology.orchestra.run.smithi176.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.8 MB/s | 5.3 MB 00:01 2024-06-06T08:25:47.355 INFO:teuthology.orchestra.run.smithi176.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 775 kB/s | 45 kB 00:00 2024-06-06T08:25:47.390 INFO:teuthology.orchestra.run.smithi203.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-06-06T08:25:47.391 INFO:teuthology.orchestra.run.smithi203.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:47.430 INFO:teuthology.orchestra.run.smithi176.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-06T08:25:47.498 INFO:teuthology.orchestra.run.smithi203.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-06-06T08:25:47.499 INFO:teuthology.orchestra.run.smithi203.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:47.514 INFO:teuthology.orchestra.run.smithi176.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.9 MB/s | 162 kB 00:00 2024-06-06T08:25:47.540 INFO:teuthology.orchestra.run.smithi203.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-06-06T08:25:47.583 INFO:teuthology.orchestra.run.smithi203.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-06-06T08:25:47.639 INFO:teuthology.orchestra.run.smithi176.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 2.5 MB/s | 321 kB 00:00 2024-06-06T08:25:47.658 INFO:teuthology.orchestra.run.smithi203.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-06-06T08:25:47.731 INFO:teuthology.orchestra.run.smithi176.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.2 MB/s | 300 kB 00:00 2024-06-06T08:25:47.789 INFO:teuthology.orchestra.run.smithi176.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.7 MB/s | 99 kB 00:00 2024-06-06T08:25:47.840 INFO:teuthology.orchestra.run.smithi176.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-06T08:25:47.940 INFO:teuthology.orchestra.run.smithi176.stdout:(26/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 1.6 MB/s | 10 MB 00:06 2024-06-06T08:25:48.075 INFO:teuthology.orchestra.run.smithi203.stdout:(43/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 6.7 MB/s | 48 MB 00:07 2024-06-06T08:25:48.082 INFO:teuthology.orchestra.run.smithi176.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.2 MB/s | 171 kB 00:00 2024-06-06T08:25:48.100 INFO:teuthology.orchestra.run.smithi203.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 241 kB/s | 106 kB 00:00 2024-06-06T08:25:48.149 INFO:teuthology.orchestra.run.smithi176.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 474 kB/s | 32 kB 00:00 2024-06-06T08:25:48.201 INFO:teuthology.orchestra.run.smithi203.stdout:(45/159): ceph-mgr-diskprediction-local-19.0.0- 3.2 MB/s | 7.4 MB 00:02 2024-06-06T08:25:48.225 INFO:teuthology.orchestra.run.smithi176.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.9 MB/s | 142 kB 00:00 2024-06-06T08:25:48.226 INFO:teuthology.orchestra.run.smithi203.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 997 kB/s | 125 kB 00:00 2024-06-06T08:25:48.227 INFO:teuthology.orchestra.run.smithi203.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T08:25:48.263 INFO:teuthology.orchestra.run.smithi027.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 3.0 MB/s | 10 MB 00:03 2024-06-06T08:25:48.285 INFO:teuthology.orchestra.run.smithi203.stdout:(47/159): python3-pycparser-2.20-6.el9.noarch.r 647 kB/s | 135 kB 00:00 2024-06-06T08:25:48.286 INFO:teuthology.orchestra.run.smithi203.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:48.308 INFO:teuthology.orchestra.run.smithi176.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 6.6 MB/s | 3.1 MB 00:00 2024-06-06T08:25:48.322 INFO:teuthology.orchestra.run.smithi027.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 434 kB/s | 25 kB 00:00 2024-06-06T08:25:48.352 INFO:teuthology.orchestra.run.smithi203.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.4 MB/s | 215 kB 00:00 2024-06-06T08:25:48.353 INFO:teuthology.orchestra.run.smithi203.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:48.386 INFO:teuthology.orchestra.run.smithi203.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-06-06T08:25:48.419 INFO:teuthology.orchestra.run.smithi203.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 656 kB/s | 22 kB 00:00 2024-06-06T08:25:48.453 INFO:teuthology.orchestra.run.smithi203.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 943 kB 00:00 2024-06-06T08:25:48.478 INFO:teuthology.orchestra.run.smithi203.stdout:(52/159): boost-program-options-1.75.0-8.el9.x8 848 kB/s | 107 kB 00:00 2024-06-06T08:25:48.503 INFO:teuthology.orchestra.run.smithi203.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-06-06T08:25:48.529 INFO:teuthology.orchestra.run.smithi203.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-06-06T08:25:48.554 INFO:teuthology.orchestra.run.smithi203.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-06-06T08:25:48.579 INFO:teuthology.orchestra.run.smithi203.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-06-06T08:25:48.605 INFO:teuthology.orchestra.run.smithi203.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-06T08:25:48.630 INFO:teuthology.orchestra.run.smithi203.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-06-06T08:25:48.672 INFO:teuthology.orchestra.run.smithi203.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-06-06T08:25:48.706 INFO:teuthology.orchestra.run.smithi203.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 582 kB 00:00 2024-06-06T08:25:48.731 INFO:teuthology.orchestra.run.smithi203.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 432 kB/s | 25 kB 00:00 2024-06-06T08:25:48.756 INFO:teuthology.orchestra.run.smithi203.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-06-06T08:25:48.782 INFO:teuthology.orchestra.run.smithi203.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-06T08:25:48.807 INFO:teuthology.orchestra.run.smithi203.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-06T08:25:48.832 INFO:teuthology.orchestra.run.smithi203.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-06-06T08:25:48.883 INFO:teuthology.orchestra.run.smithi203.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-06-06T08:25:48.942 INFO:teuthology.orchestra.run.smithi203.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-06-06T08:25:49.009 INFO:teuthology.orchestra.run.smithi203.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-06-06T08:25:49.075 INFO:teuthology.orchestra.run.smithi203.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-06-06T08:25:49.101 INFO:teuthology.orchestra.run.smithi203.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-06-06T08:25:49.134 INFO:teuthology.orchestra.run.smithi203.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 911 kB 00:00 2024-06-06T08:25:49.168 INFO:teuthology.orchestra.run.smithi203.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-06-06T08:25:49.185 INFO:teuthology.orchestra.run.smithi203.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-06-06T08:25:49.219 INFO:teuthology.orchestra.run.smithi203.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-06-06T08:25:49.284 INFO:teuthology.orchestra.run.smithi176.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-06-06T08:25:49.302 INFO:teuthology.orchestra.run.smithi203.stdout:(75/159): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-06-06T08:25:49.327 INFO:teuthology.orchestra.run.smithi203.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-06-06T08:25:49.342 INFO:teuthology.orchestra.run.smithi176.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 4.2 MB/s | 249 kB 00:00 2024-06-06T08:25:49.353 INFO:teuthology.orchestra.run.smithi203.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-06-06T08:25:49.370 INFO:teuthology.orchestra.run.smithi203.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 610 kB/s | 26 kB 00:00 2024-06-06T08:25:49.387 INFO:teuthology.orchestra.run.smithi203.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 569 kB/s | 19 kB 00:00 2024-06-06T08:25:49.392 INFO:teuthology.orchestra.run.smithi176.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 988 kB/s | 49 kB 00:00 2024-06-06T08:25:49.421 INFO:teuthology.orchestra.run.smithi203.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-06T08:25:49.446 INFO:teuthology.orchestra.run.smithi203.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-06-06T08:25:49.451 INFO:teuthology.orchestra.run.smithi176.stdout:(34/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 2.9 MB/s | 3.5 MB 00:01 2024-06-06T08:25:49.476 INFO:teuthology.orchestra.run.smithi176.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 197 kB/s | 16 kB 00:00 2024-06-06T08:25:49.530 INFO:teuthology.orchestra.run.smithi203.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-06-06T08:25:49.555 INFO:teuthology.orchestra.run.smithi203.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-06-06T08:25:49.559 INFO:teuthology.orchestra.run.smithi176.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.4 MB/s | 264 kB 00:00 2024-06-06T08:25:49.580 INFO:teuthology.orchestra.run.smithi203.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-06-06T08:25:49.606 INFO:teuthology.orchestra.run.smithi203.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-06T08:25:49.610 INFO:teuthology.orchestra.run.smithi176.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 5.6 MB/s | 763 kB 00:00 2024-06-06T08:25:49.631 INFO:teuthology.orchestra.run.smithi203.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-06-06T08:25:49.635 INFO:teuthology.orchestra.run.smithi176.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 243 kB/s | 18 kB 00:00 2024-06-06T08:25:49.656 INFO:teuthology.orchestra.run.smithi203.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-06-06T08:25:49.682 INFO:teuthology.orchestra.run.smithi203.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-06T08:25:49.693 INFO:teuthology.orchestra.run.smithi176.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:49.699 INFO:teuthology.orchestra.run.smithi176.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:49.707 INFO:teuthology.orchestra.run.smithi203.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-06T08:25:49.732 INFO:teuthology.orchestra.run.smithi203.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-06T08:25:49.758 INFO:teuthology.orchestra.run.smithi203.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-06T08:25:49.819 INFO:teuthology.orchestra.run.smithi176.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 385 kB/s | 80 kB 00:00 2024-06-06T08:25:49.823 INFO:teuthology.orchestra.run.smithi176.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:49.833 INFO:teuthology.orchestra.run.smithi203.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-06-06T08:25:49.859 INFO:teuthology.orchestra.run.smithi203.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-06-06T08:25:49.884 INFO:teuthology.orchestra.run.smithi203.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-06-06T08:25:49.885 INFO:teuthology.orchestra.run.smithi176.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-06-06T08:25:49.909 INFO:teuthology.orchestra.run.smithi203.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-06T08:25:49.919 INFO:teuthology.orchestra.run.smithi176.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-06-06T08:25:49.945 INFO:teuthology.orchestra.run.smithi176.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-06-06T08:25:49.995 INFO:teuthology.orchestra.run.smithi176.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-06-06T08:25:50.020 INFO:teuthology.orchestra.run.smithi176.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-06-06T08:25:50.035 INFO:teuthology.orchestra.run.smithi203.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 239 kB/s | 36 kB 00:00 2024-06-06T08:25:50.045 INFO:teuthology.orchestra.run.smithi176.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-06-06T08:25:50.071 INFO:teuthology.orchestra.run.smithi176.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-06T08:25:50.072 INFO:teuthology.orchestra.run.smithi176.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T08:25:50.101 INFO:teuthology.orchestra.run.smithi203.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-06-06T08:25:50.104 INFO:teuthology.orchestra.run.smithi176.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-06-06T08:25:50.105 INFO:teuthology.orchestra.run.smithi176.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:50.126 INFO:teuthology.orchestra.run.smithi203.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-06-06T08:25:50.146 INFO:teuthology.orchestra.run.smithi176.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 9.0 MB/s | 689 kB 00:00 2024-06-06T08:25:50.147 INFO:teuthology.orchestra.run.smithi176.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:50.148 INFO:teuthology.orchestra.run.smithi027.stdout:(10/159): ceph-common-19.0.0-2946.g3d89accf.el9 1.9 MB/s | 21 MB 00:11 2024-06-06T08:25:50.152 INFO:teuthology.orchestra.run.smithi203.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-06T08:25:50.177 INFO:teuthology.orchestra.run.smithi203.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-06T08:25:50.192 INFO:teuthology.orchestra.run.smithi176.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-06-06T08:25:50.203 INFO:teuthology.orchestra.run.smithi203.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-06T08:25:50.214 INFO:teuthology.orchestra.run.smithi176.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-06-06T08:25:50.228 INFO:teuthology.orchestra.run.smithi203.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-06-06T08:25:50.229 INFO:teuthology.orchestra.run.smithi203.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:50.229 INFO:teuthology.orchestra.run.smithi203.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:50.230 INFO:teuthology.orchestra.run.smithi176.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 564 kB/s | 22 kB 00:00 2024-06-06T08:25:50.264 INFO:teuthology.orchestra.run.smithi176.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-06T08:25:50.270 INFO:teuthology.orchestra.run.smithi203.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 333 kB/s | 22 kB 00:00 2024-06-06T08:25:50.281 INFO:teuthology.orchestra.run.smithi027.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.6 MB/s | 221 kB 00:00 2024-06-06T08:25:50.281 INFO:teuthology.orchestra.run.smithi176.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-06-06T08:25:50.312 INFO:teuthology.orchestra.run.smithi203.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-06-06T08:25:50.314 INFO:teuthology.orchestra.run.smithi176.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-06-06T08:25:50.348 INFO:teuthology.orchestra.run.smithi027.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 405 kB/s | 27 kB 00:00 2024-06-06T08:25:50.373 INFO:teuthology.orchestra.run.smithi176.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-06-06T08:25:50.379 INFO:teuthology.orchestra.run.smithi203.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 4.9 MB/s | 548 kB 00:00 2024-06-06T08:25:50.390 INFO:teuthology.orchestra.run.smithi176.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-06-06T08:25:50.404 INFO:teuthology.orchestra.run.smithi203.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 787 kB/s | 19 kB 00:00 2024-06-06T08:25:50.406 INFO:teuthology.orchestra.run.smithi027.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 545 kB/s | 32 kB 00:00 2024-06-06T08:25:50.424 INFO:teuthology.orchestra.run.smithi176.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-06T08:25:50.438 INFO:teuthology.orchestra.run.smithi203.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-06-06T08:25:50.449 INFO:teuthology.orchestra.run.smithi176.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-06-06T08:25:50.463 INFO:teuthology.orchestra.run.smithi203.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-06T08:25:50.483 INFO:teuthology.orchestra.run.smithi176.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-06-06T08:25:50.488 INFO:teuthology.orchestra.run.smithi203.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-06T08:25:50.516 INFO:teuthology.orchestra.run.smithi176.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-06-06T08:25:50.524 INFO:teuthology.orchestra.run.smithi203.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2024-06-06T08:25:50.542 INFO:teuthology.orchestra.run.smithi176.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-06-06T08:25:50.567 INFO:teuthology.orchestra.run.smithi176.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-06-06T08:25:50.592 INFO:teuthology.orchestra.run.smithi176.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-06T08:25:50.617 INFO:teuthology.orchestra.run.smithi176.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-06T08:25:50.643 INFO:teuthology.orchestra.run.smithi176.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-06-06T08:25:50.648 INFO:teuthology.orchestra.run.smithi027.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 3.0 MB/s | 747 kB 00:00 2024-06-06T08:25:50.700 INFO:teuthology.orchestra.run.smithi203.stdout:(111/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-06-06T08:25:50.725 INFO:teuthology.orchestra.run.smithi203.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 756 kB/s | 151 kB 00:00 2024-06-06T08:25:50.749 INFO:teuthology.orchestra.run.smithi027.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.6 MB/s | 163 kB 00:00 2024-06-06T08:25:50.760 INFO:teuthology.orchestra.run.smithi176.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-06-06T08:25:50.807 INFO:teuthology.orchestra.run.smithi027.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 2.1 MB/s | 127 kB 00:00 2024-06-06T08:25:50.809 INFO:teuthology.orchestra.run.smithi203.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 9.5 MB/s | 548 kB 00:00 2024-06-06T08:25:50.827 INFO:teuthology.orchestra.run.smithi176.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-06-06T08:25:50.834 INFO:teuthology.orchestra.run.smithi203.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-06T08:25:50.859 INFO:teuthology.orchestra.run.smithi203.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-06T08:25:50.877 INFO:teuthology.orchestra.run.smithi176.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-06-06T08:25:50.885 INFO:teuthology.orchestra.run.smithi203.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 601 kB/s | 15 kB 00:00 2024-06-06T08:25:50.910 INFO:teuthology.orchestra.run.smithi203.stdout:(117/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-06-06T08:25:50.910 INFO:teuthology.orchestra.run.smithi176.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-06-06T08:25:50.935 INFO:teuthology.orchestra.run.smithi203.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-06-06T08:25:50.936 INFO:teuthology.orchestra.run.smithi176.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-06T08:25:50.969 INFO:teuthology.orchestra.run.smithi203.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-06-06T08:25:50.991 INFO:teuthology.orchestra.run.smithi027.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 2.7 MB/s | 501 kB 00:00 2024-06-06T08:25:50.994 INFO:teuthology.orchestra.run.smithi203.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-06-06T08:25:50.995 INFO:teuthology.orchestra.run.smithi176.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-06-06T08:25:51.019 INFO:teuthology.orchestra.run.smithi203.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-06-06T08:25:51.078 INFO:teuthology.orchestra.run.smithi176.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-06-06T08:25:51.086 INFO:teuthology.orchestra.run.smithi203.stdout:(122/159): libarrow-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-06-06T08:25:51.103 INFO:teuthology.orchestra.run.smithi176.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 808 kB/s | 87 kB 00:00 2024-06-06T08:25:51.112 INFO:teuthology.orchestra.run.smithi203.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-06-06T08:25:51.137 INFO:teuthology.orchestra.run.smithi203.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-06-06T08:25:51.162 INFO:teuthology.orchestra.run.smithi176.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-06-06T08:25:51.163 INFO:teuthology.orchestra.run.smithi203.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-06-06T08:25:51.188 INFO:teuthology.orchestra.run.smithi203.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-06-06T08:25:51.212 INFO:teuthology.orchestra.run.smithi176.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-06-06T08:25:51.213 INFO:teuthology.orchestra.run.smithi203.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-06-06T08:25:51.238 INFO:teuthology.orchestra.run.smithi203.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-06-06T08:25:51.263 INFO:teuthology.orchestra.run.smithi203.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-06T08:25:51.279 INFO:teuthology.orchestra.run.smithi176.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-06-06T08:25:51.289 INFO:teuthology.orchestra.run.smithi203.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-06T08:25:51.304 INFO:teuthology.orchestra.run.smithi176.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-06-06T08:25:51.314 INFO:teuthology.orchestra.run.smithi203.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-06T08:25:51.330 INFO:teuthology.orchestra.run.smithi176.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-06T08:25:51.355 INFO:teuthology.orchestra.run.smithi176.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-06-06T08:25:51.356 INFO:teuthology.orchestra.run.smithi203.stdout:(132/159): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-06-06T08:25:51.372 INFO:teuthology.orchestra.run.smithi176.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-06-06T08:25:51.390 INFO:teuthology.orchestra.run.smithi203.stdout:(133/159): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-06-06T08:25:51.415 INFO:teuthology.orchestra.run.smithi203.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-06-06T08:25:51.440 INFO:teuthology.orchestra.run.smithi203.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-06-06T08:25:51.465 INFO:teuthology.orchestra.run.smithi203.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-06-06T08:25:51.491 INFO:teuthology.orchestra.run.smithi203.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-06-06T08:25:51.516 INFO:teuthology.orchestra.run.smithi203.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-06T08:25:51.541 INFO:teuthology.orchestra.run.smithi203.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-06-06T08:25:51.567 INFO:teuthology.orchestra.run.smithi203.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-06-06T08:25:51.592 INFO:teuthology.orchestra.run.smithi203.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-06T08:25:51.642 INFO:teuthology.orchestra.run.smithi203.stdout:(142/159): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-06-06T08:25:51.668 INFO:teuthology.orchestra.run.smithi203.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-06-06T08:25:51.693 INFO:teuthology.orchestra.run.smithi203.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-06-06T08:25:51.718 INFO:teuthology.orchestra.run.smithi203.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-06-06T08:25:51.743 INFO:teuthology.orchestra.run.smithi203.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-06T08:25:51.769 INFO:teuthology.orchestra.run.smithi203.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-06-06T08:25:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:(81/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 7.0 MB/s | 48 MB 00:06 2024-06-06T08:25:51.806 INFO:teuthology.orchestra.run.smithi176.stdout:(82/159): python3-markupsafe-1.1.1-12.el9.x86_6 77 kB/s | 35 kB 00:00 2024-06-06T08:25:51.819 INFO:teuthology.orchestra.run.smithi203.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-06T08:25:51.856 INFO:teuthology.orchestra.run.smithi176.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-06-06T08:25:51.932 INFO:teuthology.orchestra.run.smithi176.stdout:(84/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.1 MB/s | 5.1 MB 00:00 2024-06-06T08:25:51.948 INFO:teuthology.orchestra.run.smithi176.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 842 kB/s | 77 kB 00:00 2024-06-06T08:25:51.973 INFO:teuthology.orchestra.run.smithi176.stdout:(86/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-06-06T08:25:51.991 INFO:teuthology.orchestra.run.smithi176.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 716 kB/s | 42 kB 00:00 2024-06-06T08:25:52.016 INFO:teuthology.orchestra.run.smithi176.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-06-06T08:25:52.028 INFO:teuthology.orchestra.run.smithi203.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-06-06T08:25:52.041 INFO:teuthology.orchestra.run.smithi176.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-06-06T08:25:52.058 INFO:teuthology.orchestra.run.smithi176.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-06-06T08:25:52.083 INFO:teuthology.orchestra.run.smithi176.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.9 MB/s | 279 kB 00:00 2024-06-06T08:25:52.095 INFO:teuthology.orchestra.run.smithi203.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-06T08:25:52.109 INFO:teuthology.orchestra.run.smithi176.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 799 kB/s | 54 kB 00:00 2024-06-06T08:25:52.126 INFO:teuthology.orchestra.run.smithi176.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 996 kB/s | 42 kB 00:00 2024-06-06T08:25:52.142 INFO:teuthology.orchestra.run.smithi176.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-06-06T08:25:52.145 INFO:teuthology.orchestra.run.smithi203.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 833 kB/s | 41 kB 00:00 2024-06-06T08:25:52.193 INFO:teuthology.orchestra.run.smithi176.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-06-06T08:25:52.195 INFO:teuthology.orchestra.run.smithi203.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 4.8 MB/s | 248 kB 00:00 2024-06-06T08:25:52.210 INFO:teuthology.orchestra.run.smithi176.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-06-06T08:25:52.235 INFO:teuthology.orchestra.run.smithi176.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-06-06T08:25:52.245 INFO:teuthology.orchestra.run.smithi203.stdout:(153/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-06-06T08:25:52.261 INFO:teuthology.orchestra.run.smithi176.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-06T08:25:52.270 INFO:teuthology.orchestra.run.smithi203.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 489 kB/s | 37 kB 00:00 2024-06-06T08:25:52.291 INFO:teuthology.orchestra.run.smithi027.stdout:(18/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.1 MB/s | 17 MB 00:07 2024-06-06T08:25:52.313 INFO:teuthology.orchestra.run.smithi176.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 742 kB/s | 58 kB 00:00 2024-06-06T08:25:52.336 INFO:teuthology.orchestra.run.smithi176.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-06-06T08:25:52.337 INFO:teuthology.orchestra.run.smithi203.stdout:(155/159): librados2-19.0.0-2946.g3d89accf.el9. 5.7 MB/s | 3.4 MB 00:00 2024-06-06T08:25:52.350 INFO:teuthology.orchestra.run.smithi027.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 775 kB/s | 45 kB 00:00 2024-06-06T08:25:52.361 INFO:teuthology.orchestra.run.smithi176.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-06-06T08:25:52.363 INFO:teuthology.orchestra.run.smithi203.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.8 MB/s | 260 kB 00:00 2024-06-06T08:25:52.386 INFO:teuthology.orchestra.run.smithi176.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-06-06T08:25:52.387 INFO:teuthology.orchestra.run.smithi176.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:52.388 INFO:teuthology.orchestra.run.smithi176.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:52.388 INFO:teuthology.orchestra.run.smithi203.stdout:(157/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.4 MB/s | 207 kB 00:00 2024-06-06T08:25:52.417 INFO:teuthology.orchestra.run.smithi027.stdout:(20/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.7 MB/s | 5.3 MB 00:01 2024-06-06T08:25:52.442 INFO:teuthology.orchestra.run.smithi027.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 1.4 MB/s | 135 kB 00:00 2024-06-06T08:25:52.445 INFO:teuthology.orchestra.run.smithi176.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-06-06T08:25:52.475 INFO:teuthology.orchestra.run.smithi027.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.7 MB/s | 162 kB 00:00 2024-06-06T08:25:52.522 INFO:teuthology.orchestra.run.smithi203.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-06-06T08:25:52.543 INFO:teuthology.orchestra.run.smithi027.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 3.1 MB/s | 321 kB 00:00 2024-06-06T08:25:52.555 INFO:teuthology.orchestra.run.smithi203.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 1.3 MB 00:00 2024-06-06T08:25:52.561 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-06-06T08:25:52.562 INFO:teuthology.orchestra.run.smithi203.stdout:Total 16 MB/s | 225 MB 00:14 2024-06-06T08:25:52.576 INFO:teuthology.orchestra.run.smithi027.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.9 MB/s | 300 kB 00:00 2024-06-06T08:25:52.601 INFO:teuthology.orchestra.run.smithi027.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.7 MB/s | 99 kB 00:00 2024-06-06T08:25:52.604 INFO:teuthology.orchestra.run.smithi176.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-06-06T08:25:52.637 INFO:teuthology.orchestra.run.smithi176.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 MB/s | 548 kB 00:00 2024-06-06T08:25:52.643 INFO:teuthology.orchestra.run.smithi027.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.2 MB/s | 85 kB 00:00 2024-06-06T08:25:52.662 INFO:teuthology.orchestra.run.smithi176.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-06-06T08:25:52.688 INFO:teuthology.orchestra.run.smithi176.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-06-06T08:25:52.710 INFO:teuthology.orchestra.run.smithi027.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.5 MB/s | 171 kB 00:00 2024-06-06T08:25:52.713 INFO:teuthology.orchestra.run.smithi176.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-06T08:25:52.747 INFO:teuthology.orchestra.run.smithi176.stdout:(109/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-06-06T08:25:52.769 INFO:teuthology.orchestra.run.smithi027.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 543 kB/s | 32 kB 00:00 2024-06-06T08:25:52.772 INFO:teuthology.orchestra.run.smithi176.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 831 kB/s | 49 kB 00:00 2024-06-06T08:25:52.831 INFO:teuthology.orchestra.run.smithi176.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 4.8 MB/s | 151 kB 00:00 2024-06-06T08:25:52.836 INFO:teuthology.orchestra.run.smithi027.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.1 MB/s | 142 kB 00:00 2024-06-06T08:25:52.864 INFO:teuthology.orchestra.run.smithi176.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-06-06T08:25:52.906 INFO:teuthology.orchestra.run.smithi176.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-06-06T08:25:52.931 INFO:teuthology.orchestra.run.smithi176.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-06T08:25:52.957 INFO:teuthology.orchestra.run.smithi176.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-06T08:25:52.982 INFO:teuthology.orchestra.run.smithi176.stdout:(116/159): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-06-06T08:25:53.017 INFO:teuthology.orchestra.run.smithi176.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 254 kB/s | 15 kB 00:00 2024-06-06T08:25:53.066 INFO:teuthology.orchestra.run.smithi176.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-06-06T08:25:53.100 INFO:teuthology.orchestra.run.smithi176.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-06-06T08:25:53.133 INFO:teuthology.orchestra.run.smithi176.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-06-06T08:25:53.159 INFO:teuthology.orchestra.run.smithi176.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-06-06T08:25:53.192 INFO:teuthology.orchestra.run.smithi176.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-06-06T08:25:53.228 INFO:teuthology.orchestra.run.smithi027.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 4.9 MB/s | 3.1 MB 00:00 2024-06-06T08:25:53.251 INFO:teuthology.orchestra.run.smithi176.stdout:(123/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 4.4 MB 00:00 2024-06-06T08:25:53.276 INFO:teuthology.orchestra.run.smithi176.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-06-06T08:25:53.301 INFO:teuthology.orchestra.run.smithi176.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-06-06T08:25:53.327 INFO:teuthology.orchestra.run.smithi176.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-06-06T08:25:53.352 INFO:teuthology.orchestra.run.smithi176.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-06-06T08:25:53.377 INFO:teuthology.orchestra.run.smithi176.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-06-06T08:25:53.403 INFO:teuthology.orchestra.run.smithi176.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-06T08:25:53.436 INFO:teuthology.orchestra.run.smithi176.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-06-06T08:25:53.470 INFO:teuthology.orchestra.run.smithi176.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-06-06T08:25:53.504 INFO:teuthology.orchestra.run.smithi176.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-06-06T08:25:53.529 INFO:teuthology.orchestra.run.smithi176.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-06-06T08:25:53.554 INFO:teuthology.orchestra.run.smithi176.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-06-06T08:25:53.579 INFO:teuthology.orchestra.run.smithi176.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-06-06T08:25:53.604 INFO:teuthology.orchestra.run.smithi176.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-06-06T08:25:53.621 INFO:teuthology.orchestra.run.smithi176.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 736 kB/s | 31 kB 00:00 2024-06-06T08:25:53.638 INFO:teuthology.orchestra.run.smithi176.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-06-06T08:25:53.662 INFO:teuthology.orchestra.run.smithi027.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 4.3 MB/s | 3.5 MB 00:00 2024-06-06T08:25:53.680 INFO:teuthology.orchestra.run.smithi176.stdout:(139/159): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-06-06T08:25:53.697 INFO:teuthology.orchestra.run.smithi176.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-06-06T08:25:53.722 INFO:teuthology.orchestra.run.smithi176.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-06-06T08:25:53.739 INFO:teuthology.orchestra.run.smithi176.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-06-06T08:25:53.762 INFO:teuthology.orchestra.run.smithi027.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.4 MB/s | 249 kB 00:00 2024-06-06T08:25:53.764 INFO:teuthology.orchestra.run.smithi176.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-06-06T08:25:53.790 INFO:teuthology.orchestra.run.smithi176.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-06-06T08:25:53.792 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-06-06T08:25:53.815 INFO:teuthology.orchestra.run.smithi176.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-06-06T08:25:53.829 INFO:teuthology.orchestra.run.smithi027.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 738 kB/s | 49 kB 00:00 2024-06-06T08:25:53.840 INFO:teuthology.orchestra.run.smithi176.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-06T08:25:53.866 INFO:teuthology.orchestra.run.smithi176.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-06-06T08:25:53.887 INFO:teuthology.orchestra.run.smithi027.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 282 kB/s | 16 kB 00:00 2024-06-06T08:25:53.933 INFO:teuthology.orchestra.run.smithi176.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-06-06T08:25:53.953 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-06-06T08:25:53.953 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-06-06T08:25:54.005 INFO:teuthology.orchestra.run.smithi027.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.2 MB/s | 264 kB 00:00 2024-06-06T08:25:54.117 INFO:teuthology.orchestra.run.smithi176.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-06-06T08:25:54.192 INFO:teuthology.orchestra.run.smithi176.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-06-06T08:25:54.242 INFO:teuthology.orchestra.run.smithi176.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 831 kB/s | 41 kB 00:00 2024-06-06T08:25:54.255 INFO:teuthology.orchestra.run.smithi027.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 3.0 MB/s | 763 kB 00:00 2024-06-06T08:25:54.330 INFO:teuthology.orchestra.run.smithi027.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 243 kB/s | 18 kB 00:00 2024-06-06T08:25:54.367 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:54.368 INFO:teuthology.orchestra.run.smithi176.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 1.9 MB/s | 248 kB 00:00 2024-06-06T08:25:54.426 INFO:teuthology.orchestra.run.smithi176.stdout:(153/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-06-06T08:25:54.451 INFO:teuthology.orchestra.run.smithi176.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 439 kB/s | 37 kB 00:00 2024-06-06T08:25:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.1 MB/s | 260 kB 00:00 2024-06-06T08:25:54.560 INFO:teuthology.orchestra.run.smithi176.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-06-06T08:25:54.585 INFO:teuthology.orchestra.run.smithi176.stdout:(157/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.3 MB/s | 207 kB 00:00 2024-06-06T08:25:54.598 INFO:teuthology.orchestra.run.smithi027.stdout:(38/159): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-06-06T08:25:54.640 INFO:teuthology.orchestra.run.smithi027.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 260 kB/s | 80 kB 00:00 2024-06-06T08:25:54.640 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:54.642 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:54.669 INFO:teuthology.orchestra.run.smithi176.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-06-06T08:25:54.757 INFO:teuthology.orchestra.run.smithi027.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2024-06-06T08:25:54.799 INFO:teuthology.orchestra.run.smithi027.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-06-06T08:25:54.824 INFO:teuthology.orchestra.run.smithi027.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-06-06T08:25:54.866 INFO:teuthology.orchestra.run.smithi027.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-06-06T08:25:54.916 INFO:teuthology.orchestra.run.smithi027.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-06-06T08:25:54.941 INFO:teuthology.orchestra.run.smithi027.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-06-06T08:25:54.967 INFO:teuthology.orchestra.run.smithi027.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-06T08:25:54.968 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T08:25:54.977 INFO:teuthology.orchestra.run.smithi176.stdout:(159/159): librados2-19.0.0-2946.g3d89accf.el9. 3.0 MB/s | 3.4 MB 00:01 2024-06-06T08:25:54.983 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-06-06T08:25:54.984 INFO:teuthology.orchestra.run.smithi176.stdout:Total 14 MB/s | 225 MB 00:16 2024-06-06T08:25:54.992 INFO:teuthology.orchestra.run.smithi027.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-06-06T08:25:54.993 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:55.109 INFO:teuthology.orchestra.run.smithi027.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-06-06T08:25:55.151 INFO:teuthology.orchestra.run.smithi027.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-06-06T08:25:55.152 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:55.152 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:55.193 INFO:teuthology.orchestra.run.smithi027.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-06-06T08:25:55.218 INFO:teuthology.orchestra.run.smithi027.stdout:(51/159): boost-program-options-1.75.0-8.el9.x8 980 kB/s | 107 kB 00:00 2024-06-06T08:25:55.244 INFO:teuthology.orchestra.run.smithi027.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-06T08:25:55.269 INFO:teuthology.orchestra.run.smithi027.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-06-06T08:25:55.303 INFO:teuthology.orchestra.run.smithi027.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-06-06T08:25:55.353 INFO:teuthology.orchestra.run.smithi027.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 27 MB/s | 3.0 MB 00:00 2024-06-06T08:25:55.378 INFO:teuthology.orchestra.run.smithi027.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-06-06T08:25:55.403 INFO:teuthology.orchestra.run.smithi027.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-06T08:25:55.420 INFO:teuthology.orchestra.run.smithi027.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-06T08:25:55.454 INFO:teuthology.orchestra.run.smithi027.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-06-06T08:25:55.496 INFO:teuthology.orchestra.run.smithi027.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 582 kB 00:00 2024-06-06T08:25:55.513 INFO:teuthology.orchestra.run.smithi027.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 433 kB/s | 25 kB 00:00 2024-06-06T08:25:55.530 INFO:teuthology.orchestra.run.smithi027.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-06-06T08:25:55.555 INFO:teuthology.orchestra.run.smithi027.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-06-06T08:25:55.580 INFO:teuthology.orchestra.run.smithi027.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-06T08:25:55.614 INFO:teuthology.orchestra.run.smithi027.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 599 kB/s | 35 kB 00:00 2024-06-06T08:25:55.761 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-06-06T08:25:55.762 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-06-06T08:25:56.057 INFO:teuthology.orchestra.run.smithi027.stdout:(66/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 6.2 MB/s | 48 MB 00:07 2024-06-06T08:25:56.157 INFO:teuthology.orchestra.run.smithi027.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 8.2 MB/s | 4.7 MB 00:00 2024-06-06T08:25:56.205 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-06-06T08:25:56.224 INFO:teuthology.orchestra.run.smithi027.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-06-06T08:25:56.265 INFO:teuthology.orchestra.run.smithi027.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-06-06T08:25:56.299 INFO:teuthology.orchestra.run.smithi027.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-06-06T08:25:56.349 INFO:teuthology.orchestra.run.smithi027.stdout:(71/159): postgresql-13.14-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-06-06T08:25:56.364 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-06-06T08:25:56.365 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-06-06T08:25:56.383 INFO:teuthology.orchestra.run.smithi027.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-06-06T08:25:56.408 INFO:teuthology.orchestra.run.smithi027.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-06-06T08:25:56.492 INFO:teuthology.orchestra.run.smithi027.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-06-06T08:25:56.517 INFO:teuthology.orchestra.run.smithi027.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 564 kB/s | 61 kB 00:00 2024-06-06T08:25:56.543 INFO:teuthology.orchestra.run.smithi027.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-06-06T08:25:56.568 INFO:teuthology.orchestra.run.smithi027.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-06T08:25:56.593 INFO:teuthology.orchestra.run.smithi027.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-06T08:25:56.618 INFO:teuthology.orchestra.run.smithi027.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-06-06T08:25:56.644 INFO:teuthology.orchestra.run.smithi027.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-06-06T08:25:56.669 INFO:teuthology.orchestra.run.smithi027.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-06-06T08:25:56.713 INFO:teuthology.orchestra.run.smithi027.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-06T08:25:56.786 INFO:teuthology.orchestra.run.smithi027.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-06-06T08:25:56.811 INFO:teuthology.orchestra.run.smithi027.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-06-06T08:25:56.837 INFO:teuthology.orchestra.run.smithi027.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-06T08:25:56.862 INFO:teuthology.orchestra.run.smithi027.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-06-06T08:25:56.888 INFO:teuthology.orchestra.run.smithi027.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-06-06T08:25:56.913 INFO:teuthology.orchestra.run.smithi027.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-06T08:25:56.938 INFO:teuthology.orchestra.run.smithi027.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-06T08:25:56.963 INFO:teuthology.orchestra.run.smithi027.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-06T08:25:56.988 INFO:teuthology.orchestra.run.smithi027.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-06T08:25:57.064 INFO:teuthology.orchestra.run.smithi027.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-06-06T08:25:57.089 INFO:teuthology.orchestra.run.smithi027.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 414 kB/s | 42 kB 00:00 2024-06-06T08:25:57.106 INFO:teuthology.orchestra.run.smithi027.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-06T08:25:57.132 INFO:teuthology.orchestra.run.smithi027.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-06-06T08:25:57.149 INFO:teuthology.orchestra.run.smithi027.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 857 kB/s | 36 kB 00:00 2024-06-06T08:25:57.165 INFO:teuthology.orchestra.run.smithi027.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-06-06T08:25:57.199 INFO:teuthology.orchestra.run.smithi027.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-06T08:25:57.224 INFO:teuthology.orchestra.run.smithi027.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-06-06T08:25:57.250 INFO:teuthology.orchestra.run.smithi027.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-06T08:25:57.275 INFO:teuthology.orchestra.run.smithi027.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-06T08:25:57.300 INFO:teuthology.orchestra.run.smithi027.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-06-06T08:25:57.301 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:57.301 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T08:25:57.334 INFO:teuthology.orchestra.run.smithi027.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 381 kB/s | 22 kB 00:00 2024-06-06T08:25:57.368 INFO:teuthology.orchestra.run.smithi027.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-06-06T08:25:57.426 INFO:teuthology.orchestra.run.smithi027.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-06-06T08:25:57.451 INFO:teuthology.orchestra.run.smithi027.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 868 kB/s | 19 kB 00:00 2024-06-06T08:25:57.485 INFO:teuthology.orchestra.run.smithi027.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-06-06T08:25:57.510 INFO:teuthology.orchestra.run.smithi027.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-06T08:25:57.536 INFO:teuthology.orchestra.run.smithi027.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-06T08:25:57.561 INFO:teuthology.orchestra.run.smithi027.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-06-06T08:25:57.587 INFO:teuthology.orchestra.run.smithi027.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-06-06T08:25:57.637 INFO:teuthology.orchestra.run.smithi027.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-06-06T08:25:57.687 INFO:teuthology.orchestra.run.smithi027.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-06-06T08:25:57.712 INFO:teuthology.orchestra.run.smithi027.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-06T08:25:57.738 INFO:teuthology.orchestra.run.smithi027.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-06T08:25:57.905 INFO:teuthology.orchestra.run.smithi027.stdout:(116/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-06-06T08:25:57.930 INFO:teuthology.orchestra.run.smithi027.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 78 kB/s | 15 kB 00:00 2024-06-06T08:25:58.022 INFO:teuthology.orchestra.run.smithi027.stdout:(118/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-06-06T08:25:58.055 INFO:teuthology.orchestra.run.smithi027.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.6 MB/s | 358 kB 00:00 2024-06-06T08:25:58.081 INFO:teuthology.orchestra.run.smithi027.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.3 MB/s | 303 kB 00:00 2024-06-06T08:25:58.132 INFO:teuthology.orchestra.run.smithi027.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-06-06T08:25:58.157 INFO:teuthology.orchestra.run.smithi027.stdout:(122/159): python3-cheroot-8.6.0-4.el9.noarch.r 683 kB/s | 172 kB 00:00 2024-06-06T08:25:58.178 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-06-06T08:25:58.179 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-06-06T08:25:58.182 INFO:teuthology.orchestra.run.smithi027.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-06-06T08:25:58.208 INFO:teuthology.orchestra.run.smithi027.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-06-06T08:25:58.232 INFO:teuthology.orchestra.run.smithi027.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-06-06T08:25:58.258 INFO:teuthology.orchestra.run.smithi027.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-06-06T08:25:58.283 INFO:teuthology.orchestra.run.smithi027.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-06-06T08:25:58.309 INFO:teuthology.orchestra.run.smithi027.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-06-06T08:25:58.334 INFO:teuthology.orchestra.run.smithi027.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-06T08:25:58.359 INFO:teuthology.orchestra.run.smithi027.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-06T08:25:58.409 INFO:teuthology.orchestra.run.smithi027.stdout:(131/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-06-06T08:25:58.426 INFO:teuthology.orchestra.run.smithi027.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 865 kB/s | 58 kB 00:00 2024-06-06T08:25:58.443 INFO:teuthology.orchestra.run.smithi027.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 989 kB/s | 16 kB 00:00 2024-06-06T08:25:58.477 INFO:teuthology.orchestra.run.smithi027.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-06-06T08:25:58.510 INFO:teuthology.orchestra.run.smithi027.stdout:(135/159): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-06-06T08:25:58.527 INFO:teuthology.orchestra.run.smithi027.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-06-06T08:25:58.545 INFO:teuthology.orchestra.run.smithi027.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 917 kB/s | 31 kB 00:00 2024-06-06T08:25:58.561 INFO:teuthology.orchestra.run.smithi027.stdout:(138/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 764 kB/s | 90 kB 00:00 2024-06-06T08:25:58.587 INFO:teuthology.orchestra.run.smithi027.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-06-06T08:25:58.612 INFO:teuthology.orchestra.run.smithi027.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-06-06T08:25:58.637 INFO:teuthology.orchestra.run.smithi027.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-06T08:25:58.663 INFO:teuthology.orchestra.run.smithi027.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-06-06T08:25:58.680 INFO:teuthology.orchestra.run.smithi027.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-06-06T08:25:58.705 INFO:teuthology.orchestra.run.smithi027.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-06-06T08:25:58.730 INFO:teuthology.orchestra.run.smithi027.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-06-06T08:25:58.756 INFO:teuthology.orchestra.run.smithi027.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-06T08:25:58.789 INFO:teuthology.orchestra.run.smithi027.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-06-06T08:25:58.839 INFO:teuthology.orchestra.run.smithi027.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-06-06T08:25:59.048 INFO:teuthology.orchestra.run.smithi027.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-06-06T08:25:59.107 INFO:teuthology.orchestra.run.smithi027.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-06-06T08:25:59.149 INFO:teuthology.orchestra.run.smithi027.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 996 kB/s | 41 kB 00:00 2024-06-06T08:25:59.191 INFO:teuthology.orchestra.run.smithi027.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-06T08:25:59.241 INFO:teuthology.orchestra.run.smithi027.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 736 kB/s | 37 kB 00:00 2024-06-06T08:25:59.283 INFO:teuthology.orchestra.run.smithi027.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-06T08:25:59.342 INFO:teuthology.orchestra.run.smithi027.stdout:(155/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-06-06T08:25:59.367 INFO:teuthology.orchestra.run.smithi027.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.0 MB/s | 260 kB 00:00 2024-06-06T08:25:59.426 INFO:teuthology.orchestra.run.smithi027.stdout:(157/159): librados2-19.0.0-2946.g3d89accf.el9. 5.1 MB/s | 3.4 MB 00:00 2024-06-06T08:25:59.492 INFO:teuthology.orchestra.run.smithi027.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-06-06T08:25:59.551 INFO:teuthology.orchestra.run.smithi027.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-06-06T08:25:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-06-06T08:25:59.557 INFO:teuthology.orchestra.run.smithi027.stdout:Total 11 MB/s | 225 MB 00:20 2024-06-06T08:26:00.823 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-06-06T08:26:01.001 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-06-06T08:26:01.001 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-06-06T08:26:02.163 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-06-06T08:26:02.235 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T08:26:02.273 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T08:26:02.326 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T08:26:02.360 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T08:26:02.386 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T08:26:02.428 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T08:26:02.749 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T08:26:02.853 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T08:26:02.958 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T08:26:02.996 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T08:26:03.071 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T08:26:03.100 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T08:26:03.133 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T08:26:03.282 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-06-06T08:26:03.282 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-06-06T08:26:03.332 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T08:26:03.381 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T08:26:03.410 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T08:26:03.497 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T08:26:03.519 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T08:26:03.565 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T08:26:03.587 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T08:26:03.608 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T08:26:03.795 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T08:26:03.920 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T08:26:04.084 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T08:26:04.128 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T08:26:04.226 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T08:26:04.321 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T08:26:04.520 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T08:26:04.559 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-06-06T08:26:04.628 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T08:26:04.631 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T08:26:04.669 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T08:26:04.720 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T08:26:04.751 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T08:26:04.786 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T08:26:04.815 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T08:26:04.850 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T08:26:04.885 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T08:26:04.890 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T08:26:04.916 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T08:26:04.928 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T08:26:04.961 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T08:26:04.967 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T08:26:04.997 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T08:26:05.073 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T08:26:05.128 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T08:26:05.159 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T08:26:05.202 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T08:26:05.277 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T08:26:05.307 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T08:26:05.313 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T08:26:05.356 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T08:26:05.385 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T08:26:05.417 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T08:26:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T08:26:05.500 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T08:26:05.522 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T08:26:05.551 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T08:26:05.573 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T08:26:05.626 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T08:26:05.655 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T08:26:05.688 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T08:26:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T08:26:05.760 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T08:26:05.862 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T08:26:05.921 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T08:26:05.949 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T08:26:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T08:26:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T08:26:06.128 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T08:26:06.150 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T08:26:06.172 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T08:26:06.285 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T08:26:06.367 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T08:26:06.372 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T08:26:06.402 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T08:26:06.438 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T08:26:06.479 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T08:26:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T08:26:06.517 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T08:26:06.558 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T08:26:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T08:26:06.707 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T08:26:06.709 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T08:26:06.770 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T08:26:06.810 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T08:26:06.824 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T08:26:06.875 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T08:26:06.896 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T08:26:06.928 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T08:26:06.964 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T08:26:07.011 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T08:26:07.047 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T08:26:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T08:26:07.090 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T08:26:07.132 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T08:26:07.194 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T08:26:07.275 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T08:26:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T08:26:07.326 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T08:26:07.331 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T08:26:07.366 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T08:26:07.384 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T08:26:07.439 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T08:26:07.473 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T08:26:07.478 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T08:26:07.517 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T08:26:07.518 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T08:26:07.548 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T08:26:07.557 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T08:26:07.583 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T08:26:07.585 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T08:26:07.607 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T08:26:07.653 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T08:26:07.681 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T08:26:07.707 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T08:26:07.738 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T08:26:07.773 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T08:26:07.810 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T08:26:07.849 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T08:26:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T08:26:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T08:26:07.932 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T08:26:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T08:26:08.009 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T08:26:08.062 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T08:26:08.137 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T08:26:08.144 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T08:26:08.178 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T08:26:08.218 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T08:26:08.258 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T08:26:08.270 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T08:26:08.287 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T08:26:08.322 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T08:26:08.323 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T08:26:08.355 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T08:26:08.398 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T08:26:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T08:26:08.522 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T08:26:08.578 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T08:26:08.609 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T08:26:08.657 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T08:26:08.763 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T08:26:08.810 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T08:26:08.847 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T08:26:08.926 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T08:26:08.956 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T08:26:08.992 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T08:26:09.041 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T08:26:09.071 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T08:26:09.103 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T08:26:09.251 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T08:26:09.325 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T08:26:09.380 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T08:26:09.453 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T08:26:09.484 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T08:26:09.520 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T08:26:09.526 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T08:26:09.567 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T08:26:09.574 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T08:26:09.603 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T08:26:09.607 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T08:26:09.646 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T08:26:09.688 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T08:26:09.705 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T08:26:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-06-06T08:26:09.762 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T08:26:09.807 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T08:26:09.815 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T08:26:09.822 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T08:26:09.857 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T08:26:09.870 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T08:26:09.885 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T08:26:09.914 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T08:26:09.927 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T08:26:09.933 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T08:26:09.973 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T08:26:09.998 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T08:26:10.009 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T08:26:10.049 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T08:26:10.064 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T08:26:10.088 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T08:26:10.103 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T08:26:10.114 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T08:26:10.133 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T08:26:10.211 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T08:26:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T08:26:10.322 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T08:26:10.407 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T08:26:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T08:26:10.496 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T08:26:10.524 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T08:26:10.524 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T08:26:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T08:26:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T08:26:10.649 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T08:26:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T08:26:10.676 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T08:26:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T08:26:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T08:26:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T08:26:10.840 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T08:26:10.850 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T08:26:10.954 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T08:26:10.964 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T08:26:11.048 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T08:26:11.075 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T08:26:11.096 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T08:26:11.123 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T08:26:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T08:26:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T08:26:11.229 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T08:26:11.257 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T08:26:11.297 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T08:26:11.320 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T08:26:11.341 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T08:26:11.349 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T08:26:11.369 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T08:26:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T08:26:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T08:26:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T08:26:11.582 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T08:26:11.829 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T08:26:12.039 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T08:26:12.196 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T08:26:12.214 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T08:26:12.266 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T08:26:12.270 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T08:26:12.298 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T08:26:12.314 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T08:26:12.331 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T08:26:12.363 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T08:26:12.376 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T08:26:12.396 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T08:26:12.434 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T08:26:12.495 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T08:26:12.543 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T08:26:12.588 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T08:26:12.644 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T08:26:12.652 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T08:26:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T08:26:12.694 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T08:26:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T08:26:12.761 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T08:26:12.768 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T08:26:12.799 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T08:26:12.835 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T08:26:12.856 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T08:26:12.900 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T08:26:12.901 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T08:26:12.930 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T08:26:12.942 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T08:26:13.017 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T08:26:13.017 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T08:26:13.059 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T08:26:13.102 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T08:26:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T08:26:13.161 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T08:26:13.235 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T08:26:13.294 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T08:26:13.306 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T08:26:13.323 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T08:26:13.347 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T08:26:13.367 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T08:26:13.418 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T08:26:13.445 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T08:26:13.450 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T08:26:13.477 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T08:26:13.521 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T08:26:13.549 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T08:26:13.615 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T08:26:13.659 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T08:26:13.665 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T08:26:13.742 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T08:26:13.779 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T08:26:13.891 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T08:26:13.943 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T08:26:14.512 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T08:26:14.608 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T08:26:14.641 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T08:26:14.679 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T08:26:14.719 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T08:26:14.757 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T08:26:14.801 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T08:26:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T08:26:15.008 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T08:26:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T08:26:15.103 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T08:26:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T08:26:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T08:26:15.203 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T08:26:15.277 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T08:26:15.312 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T08:26:15.349 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T08:26:15.397 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T08:26:15.431 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T08:26:15.458 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T08:26:15.475 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T08:26:15.500 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T08:26:15.517 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T08:26:15.573 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T08:26:15.604 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T08:26:15.649 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T08:26:15.669 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T08:26:15.710 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T08:26:15.746 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T08:26:15.752 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T08:26:15.779 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T08:26:15.851 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T08:26:15.885 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T08:26:15.927 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T08:26:15.958 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T08:26:15.982 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T08:26:16.068 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T08:26:16.122 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T08:26:16.161 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T08:26:16.208 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T08:26:16.232 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T08:26:16.240 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T08:26:16.262 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T08:26:16.268 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T08:26:16.311 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T08:26:16.315 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T08:26:16.350 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T08:26:16.357 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T08:26:16.390 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T08:26:16.420 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T08:26:16.454 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T08:26:16.460 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T08:26:16.460 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T08:26:16.460 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T08:26:16.460 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:16.487 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T08:26:16.492 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T08:26:16.529 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T08:26:16.563 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T08:26:16.663 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T08:26:16.723 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T08:26:16.759 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T08:26:16.807 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T08:26:16.918 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T08:26:16.969 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T08:26:17.761 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T08:26:17.807 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T08:26:17.849 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T08:26:17.957 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T08:26:18.006 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T08:26:18.051 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T08:26:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T08:26:18.186 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T08:26:18.371 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T08:26:18.399 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T08:26:18.493 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T08:26:18.527 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T08:26:18.567 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T08:26:19.111 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T08:26:19.146 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T08:26:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T08:26:19.240 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T08:26:19.240 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T08:26:19.240 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T08:26:19.241 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:19.272 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T08:26:19.327 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T08:26:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T08:26:20.770 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T08:26:20.806 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T08:26:20.839 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T08:26:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T08:26:21.210 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T08:26:21.302 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T08:26:21.332 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T08:26:21.369 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T08:26:21.866 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T08:26:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T08:26:21.943 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T08:26:21.973 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T08:26:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T08:26:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T08:26:24.988 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T08:26:25.023 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T08:26:25.118 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T08:26:25.119 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T08:26:25.119 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T08:26:25.119 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:25.149 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T08:26:28.144 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T08:26:28.144 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-06-06T08:26:28.144 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-06-06T08:26:28.144 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-06-06T08:26:28.144 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-06-06T08:26:28.144 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-06-06T08:26:28.144 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-06-06T08:26:28.145 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-06-06T08:26:28.145 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:28.322 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T08:26:28.339 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T08:26:28.340 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:28.340 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T08:26:28.340 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:29.834 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T08:26:29.860 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T08:26:29.861 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:29.861 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T08:26:29.861 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T08:26:29.861 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T08:26:29.861 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:30.085 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T08:26:30.108 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T08:26:30.108 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:30.108 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T08:26:30.108 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T08:26:30.108 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T08:26:30.108 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:30.544 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T08:26:30.567 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T08:26:30.567 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:30.567 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T08:26:30.567 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T08:26:30.567 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T08:26:30.567 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:30.682 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T08:26:30.731 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T08:26:30.770 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T08:26:30.770 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /sys 2024-06-06T08:26:30.770 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /proc 2024-06-06T08:26:30.770 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /mnt 2024-06-06T08:26:30.770 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /var/tmp 2024-06-06T08:26:30.770 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /home 2024-06-06T08:26:30.770 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /root 2024-06-06T08:26:30.770 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /tmp 2024-06-06T08:26:30.770 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:30.786 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T08:26:30.934 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T08:26:30.948 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T08:26:30.949 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:30.949 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T08:26:30.949 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:30.971 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T08:26:31.020 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T08:26:31.098 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T08:26:31.186 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T08:26:31.327 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T08:26:31.371 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T08:26:31.403 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T08:26:31.431 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T08:26:31.892 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T08:26:31.921 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T08:26:31.961 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T08:26:32.023 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T08:26:32.053 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T08:26:32.437 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T08:26:32.463 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T08:26:32.463 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:32.463 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T08:26:32.463 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T08:26:32.463 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T08:26:32.463 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:32.696 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T08:26:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T08:26:32.718 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:32.718 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T08:26:32.718 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T08:26:32.718 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T08:26:32.718 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:33.091 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T08:26:33.121 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T08:26:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T08:26:33.186 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T08:26:33.186 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:33.186 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T08:26:33.186 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T08:26:33.186 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T08:26:33.186 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:33.284 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T08:26:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T08:26:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T08:26:33.573 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T08:26:33.622 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T08:26:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T08:26:33.788 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T08:26:33.927 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T08:26:33.972 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T08:26:34.004 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T08:26:34.032 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T08:26:34.089 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T08:26:34.114 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T08:26:34.215 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T08:26:34.377 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T08:26:34.416 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T08:26:34.439 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T08:26:34.439 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:34.439 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T08:26:34.439 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T08:26:34.440 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T08:26:34.440 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:34.484 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T08:26:34.492 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T08:26:34.503 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T08:26:34.522 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T08:26:34.554 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T08:26:34.609 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T08:26:34.615 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T08:26:34.645 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T08:26:34.665 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T08:26:35.563 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T08:26:35.603 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T08:26:35.626 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T08:26:35.627 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:35.627 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T08:26:35.627 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T08:26:35.627 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T08:26:35.627 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:35.670 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T08:26:35.676 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T08:26:35.693 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T08:26:35.693 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:35.693 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T08:26:35.693 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:35.705 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T08:26:35.738 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T08:26:35.761 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T08:26:35.761 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:35.761 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T08:26:35.761 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T08:26:35.761 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T08:26:35.761 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:36.064 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T08:26:36.086 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T08:26:36.087 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:36.087 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T08:26:36.087 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T08:26:36.087 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T08:26:36.087 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:36.670 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T08:26:36.697 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T08:26:36.799 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T08:26:36.960 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T08:26:37.001 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T08:26:37.024 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T08:26:37.024 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:37.024 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T08:26:37.025 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T08:26:37.025 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T08:26:37.025 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:37.084 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T08:26:37.095 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T08:26:37.192 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T08:26:37.257 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T08:26:37.528 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T08:26:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-06-06T08:26:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-06-06T08:26:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-06-06T08:26:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-06-06T08:26:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-06-06T08:26:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-06-06T08:26:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-06-06T08:26:37.529 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:37.706 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T08:26:37.724 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T08:26:37.724 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:37.724 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T08:26:37.724 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:38.156 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T08:26:38.206 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T08:26:38.230 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T08:26:38.230 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:38.230 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T08:26:38.230 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T08:26:38.230 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T08:26:38.230 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:38.270 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T08:26:38.292 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T08:26:38.292 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:38.292 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T08:26:38.292 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T08:26:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T08:26:38.360 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:38.360 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T08:26:38.360 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T08:26:38.360 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T08:26:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:38.663 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T08:26:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T08:26:38.686 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:38.686 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T08:26:38.686 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T08:26:38.687 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T08:26:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:39.260 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T08:26:39.288 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T08:26:39.288 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:39.288 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T08:26:39.288 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T08:26:39.289 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T08:26:39.289 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:39.527 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T08:26:39.551 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T08:26:39.552 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:39.552 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T08:26:39.552 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T08:26:39.552 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T08:26:39.552 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:40.028 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T08:26:40.052 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T08:26:40.052 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:40.052 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T08:26:40.052 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T08:26:40.052 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T08:26:40.053 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:40.167 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T08:26:40.225 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T08:26:40.290 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T08:26:40.487 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T08:26:40.537 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T08:26:40.609 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T08:26:40.619 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T08:26:40.651 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T08:26:40.691 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T08:26:40.724 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T08:26:40.725 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T08:26:40.751 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T08:26:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T08:26:40.921 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T08:26:40.933 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T08:26:40.955 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T08:26:40.979 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T08:26:40.981 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T08:26:41.014 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T08:26:41.054 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T08:26:41.084 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T08:26:41.139 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T08:26:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T08:26:41.436 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T08:26:41.467 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T08:26:41.533 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T08:26:41.559 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T08:26:41.743 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T08:26:41.788 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T08:26:41.808 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T08:26:42.653 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T08:26:42.656 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T08:26:42.694 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T08:26:42.744 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T08:26:42.785 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T08:26:42.866 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T08:26:42.906 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T08:26:42.931 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T08:26:42.956 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T08:26:42.983 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T08:26:42.983 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T08:26:43.068 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T08:26:43.101 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T08:26:43.101 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T08:26:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T08:26:43.169 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T08:26:43.203 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T08:26:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T08:26:43.208 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T08:26:43.208 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T08:26:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T08:26:43.297 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T08:26:43.325 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T08:26:43.396 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T08:26:43.396 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T08:26:43.480 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T08:26:43.503 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T08:26:43.505 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T08:26:43.545 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T08:26:43.570 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T08:26:43.578 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T08:26:43.618 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T08:26:43.648 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T08:26:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T08:26:43.710 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T08:26:43.746 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T08:26:43.855 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T08:26:44.031 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T08:26:44.063 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T08:26:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T08:26:44.088 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:44.088 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T08:26:44.088 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T08:26:44.088 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T08:26:44.088 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:44.138 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T08:26:44.150 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T08:26:44.252 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T08:26:44.264 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T08:26:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T08:26:44.314 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T08:26:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T08:26:45.146 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T08:26:45.254 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T08:26:45.261 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T08:26:45.291 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T08:26:45.296 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T08:26:45.316 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T08:26:45.316 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:45.316 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T08:26:45.316 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T08:26:45.317 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T08:26:45.317 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:45.375 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T08:26:45.376 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T08:26:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T08:26:45.400 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:45.400 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T08:26:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:45.408 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T08:26:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T08:26:45.459 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T08:26:45.459 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T08:26:45.484 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T08:26:45.484 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:45.484 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T08:26:45.485 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T08:26:45.485 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T08:26:45.485 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:45.494 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T08:26:45.494 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T08:26:45.578 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T08:26:45.611 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T08:26:45.611 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T08:26:45.695 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T08:26:45.720 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T08:26:45.725 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T08:26:45.725 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T08:26:45.802 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T08:26:45.826 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T08:26:45.827 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T08:26:45.827 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T08:26:45.827 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T08:26:45.827 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T08:26:45.827 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:26:45.906 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T08:26:45.906 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T08:26:45.990 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T08:26:46.015 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T08:26:46.079 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T08:26:49.436 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T08:26:49.436 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T08:26:49.436 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T08:26:49.436 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T08:26:49.436 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T08:26:49.436 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T08:26:49.436 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T08:26:49.436 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T08:26:49.436 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T08:26:49.437 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T08:26:49.438 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T08:26:49.438 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T08:26:49.438 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T08:26:49.438 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T08:26:49.438 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T08:26:49.438 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T08:26:49.450 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T08:26:49.450 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T08:26:49.450 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T08:26:49.450 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T08:26:49.450 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T08:26:49.450 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T08:26:49.450 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T08:26:49.450 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T08:26:49.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T08:26:49.452 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T08:26:49.453 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T08:26:49.454 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T08:26:49.455 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T08:26:49.456 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T08:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T08:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T08:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T08:26:49.460 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T08:26:49.461 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T08:26:49.461 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T08:26:49.461 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T08:26:49.461 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T08:26:49.461 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T08:26:50.605 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T08:26:50.647 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T08:26:50.705 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T08:26:50.736 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T08:26:50.764 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T08:26:50.969 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T08:26:51.018 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T08:26:51.054 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T08:26:51.091 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T08:26:51.121 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T08:26:51.175 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T08:26:51.509 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T08:26:51.509 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T08:26:51.509 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T08:26:51.510 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T08:26:51.511 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T08:26:51.512 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T08:26:51.513 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T08:26:51.513 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T08:26:51.524 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T08:26:51.524 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T08:26:51.524 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T08:26:51.524 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T08:26:51.524 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T08:26:51.524 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T08:26:51.525 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T08:26:51.526 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T08:26:51.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T08:26:51.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T08:26:51.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T08:26:51.530 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T08:26:51.531 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T08:26:51.532 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T08:26:51.533 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T08:26:51.534 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T08:26:51.745 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T08:26:51.781 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T08:26:51.803 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T08:26:52.245 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T08:26:52.245 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:52.245 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-06-06T08:26:52.246 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.248 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.249 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.250 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.250 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T08:26:52.250 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T08:26:52.250 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T08:26:52.250 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T08:26:52.250 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T08:26:52.250 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.250 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.250 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T08:26:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T08:26:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T08:26:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T08:26:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T08:26:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T08:26:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T08:26:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T08:26:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:26:52.258 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-06-06T08:26:52.706 DEBUG:teuthology.parallel:result is None 2024-06-06T08:26:52.714 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T08:26:52.803 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T08:26:52.846 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T08:26:52.957 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T08:26:52.998 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T08:26:53.022 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T08:26:53.048 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T08:26:53.074 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T08:26:53.074 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T08:26:53.159 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T08:26:53.192 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T08:26:53.192 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T08:26:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T08:26:53.318 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T08:26:53.324 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T08:26:53.324 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T08:26:53.512 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T08:26:53.512 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T08:26:53.580 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T08:26:53.604 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T08:26:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T08:26:54.068 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T08:26:54.068 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:54.068 INFO:teuthology.orchestra.run.smithi176.stdout:Upgraded: 2024-06-06T08:26:54.068 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T08:26:54.068 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T08:26:54.068 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T08:26:54.068 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T08:26:54.068 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T08:26:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.070 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:26:54.071 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.072 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.073 INFO:teuthology.orchestra.run.smithi176.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T08:26:54.074 INFO:teuthology.orchestra.run.smithi176.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T08:26:54.075 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T08:26:54.077 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.077 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.077 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.077 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T08:26:54.077 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T08:26:54.077 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T08:26:54.077 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T08:26:54.077 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T08:26:54.078 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T08:26:54.079 INFO:teuthology.orchestra.run.smithi176.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.080 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T08:26:54.081 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T08:26:54.082 INFO:teuthology.orchestra.run.smithi176.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T08:26:54.083 INFO:teuthology.orchestra.run.smithi176.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T08:26:54.083 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T08:26:54.083 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T08:26:54.083 INFO:teuthology.orchestra.run.smithi176.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T08:26:54.083 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-06-06T08:26:54.083 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-06-06T08:26:54.500 DEBUG:teuthology.parallel:result is None 2024-06-06T08:26:58.996 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T08:26:58.996 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T08:26:58.996 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T08:26:58.997 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T08:26:58.998 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T08:26:58.999 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T08:26:58.999 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T08:26:58.999 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T08:26:59.012 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T08:26:59.013 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T08:26:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T08:26:59.015 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T08:26:59.016 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T08:26:59.017 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T08:26:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T08:26:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T08:26:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T08:26:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T08:26:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T08:26:59.023 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T08:26:59.023 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T08:26:59.023 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T08:26:59.023 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T08:26:59.023 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T08:26:59.023 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T08:26:59.023 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T08:27:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T08:27:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:27:01.766 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-06-06T08:27:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T08:27:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T08:27:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T08:27:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T08:27:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T08:27:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T08:27:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T08:27:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T08:27:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T08:27:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T08:27:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T08:27:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T08:27:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T08:27:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T08:27:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:27:01.779 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-06-06T08:27:02.189 DEBUG:teuthology.parallel:result is None 2024-06-06T08:27:02.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:27:02.343 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T08:27:02.366 INFO:teuthology.orchestra.run.smithi027.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T08:27:02.367 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T08:27:02.367 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T08:27:02.368 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:27:02.508 DEBUG:teuthology.orchestra.run.smithi176:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T08:27:02.533 INFO:teuthology.orchestra.run.smithi176.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T08:27:02.533 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T08:27:02.533 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T08:27:02.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:27:02.683 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T08:27:02.708 INFO:teuthology.orchestra.run.smithi203.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T08:27:02.709 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T08:27:02.709 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T08:27:02.712 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-06T08:27:02.712 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:27:02.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T08:27:02.746 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:27:02.746 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T08:27:02.777 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:27:02.777 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T08:27:02.808 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-06T08:27:02.809 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:27:02.809 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T08:27:02.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T08:27:02.911 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:27:02.911 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T08:27:02.940 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T08:27:03.007 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:27:03.007 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T08:27:03.036 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T08:27:03.103 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-06T08:27:03.104 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:27:03.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T08:27:03.135 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T08:27:03.206 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:27:03.206 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T08:27:03.234 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T08:27:03.301 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:27:03.301 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T08:27:03.329 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T08:27:03.397 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-06T08:27:03.397 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:27:03.397 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T08:27:03.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T08:27:03.498 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:27:03.498 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T08:27:03.526 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T08:27:03.593 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:27:03.593 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T08:27:03.621 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T08:27:03.689 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-06T08:27:03.781 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-06-06T08:27:03.781 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:27:03.782 INFO:tasks.cephadm:Cluster fsid is 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:27:03.782 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-06T08:27:03.782 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.27', 'mon.b': '172.21.15.176', 'mon.c': '172.21.15.203'} 2024-06-06T08:27:03.783 INFO:tasks.cephadm:First mon is mon.a on smithi027 2024-06-06T08:27:03.783 INFO:tasks.cephadm:First mgr is x 2024-06-06T08:27:03.783 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-06T08:27:03.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-06-06T08:27:03.815 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname $(hostname -s) 2024-06-06T08:27:03.843 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2024-06-06T08:27:03.871 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-06T08:27:03.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:27:04.122 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-06T08:27:04.263 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:27:04.264 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-06T08:27:04.264 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T08:27:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 08:27 /home/ubuntu/cephtest/cephadm 2024-06-06T08:27:04.664 DEBUG:teuthology.orchestra.run.smithi176:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T08:27:05.220 INFO:teuthology.orchestra.run.smithi176.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 08:27 /home/ubuntu/cephtest/cephadm 2024-06-06T08:27:05.221 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T08:27:05.782 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 08:27 /home/ubuntu/cephtest/cephadm 2024-06-06T08:27:05.783 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T08:27:05.804 DEBUG:teuthology.orchestra.run.smithi176:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T08:27:05.822 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T08:27:05.860 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-06T08:27:05.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T08:27:05.863 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T08:27:05.866 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T08:27:06.754 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T08:27:06.765 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T08:27:06.766 INFO:teuthology.orchestra.run.smithi176.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T08:27:35.617 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-06-06T08:27:35.618 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T08:27:35.618 INFO:teuthology.orchestra.run.smithi176.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T08:27:35.618 INFO:teuthology.orchestra.run.smithi176.stdout: "repo_digests": [ 2024-06-06T08:27:35.618 INFO:teuthology.orchestra.run.smithi176.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T08:27:35.618 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-06-06T08:27:35.618 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-06-06T08:27:38.626 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-06-06T08:27:38.626 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T08:27:38.626 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T08:27:38.626 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2024-06-06T08:27:38.626 INFO:teuthology.orchestra.run.smithi203.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T08:27:38.627 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2024-06-06T08:27:38.627 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-06-06T08:27:42.930 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:27:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T08:27:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T08:27:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-06-06T08:27:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T08:27:42.930 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:27:42.930 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:27:42.956 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-06-06T08:27:43.000 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /etc/ceph 2024-06-06T08:27:43.034 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2024-06-06T08:27:43.067 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-06-06T08:27:43.093 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 777 /etc/ceph 2024-06-06T08:27:43.119 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2024-06-06T08:27:43.147 INFO:tasks.cephadm:Writing seed config... 2024-06-06T08:27:43.148 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-06T08:27:43.148 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-06T08:27:43.148 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-06T08:27:43.148 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-06T08:27:43.148 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-06T08:27:43.148 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-06T08:27:43.148 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-06T08:27:43.149 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-06T08:27:43.150 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-06T08:27:43.151 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-06-06T08:27:43.152 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:27:43.152 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-06T08:27:43.168 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8da40ada-23de-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-06T08:27:43.169 DEBUG:teuthology.orchestra.run.smithi027:mon.a> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.a.service 2024-06-06T08:27:43.212 DEBUG:teuthology.orchestra.run.smithi027:mgr.x> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-06T08:27:43.254 INFO:tasks.cephadm:Bootstrapping... 2024-06-06T08:27:43.254 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.27 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:27:43.452 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-06-06T08:27:43.452 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', '8da40ada-23de-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.27', '--skip-admin-label'] 2024-06-06T08:27:43.453 INFO:teuthology.orchestra.run.smithi027.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-06T08:27:43.453 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying podman|docker is present... 2024-06-06T08:27:43.478 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.0.2 2024-06-06T08:27:43.478 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying lvm2 is present... 2024-06-06T08:27:43.478 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying time synchronization is in place... 2024-06-06T08:27:43.486 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T08:27:43.487 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T08:27:43.493 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T08:27:43.493 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-06-06T08:27:43.499 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-06-06T08:27:43.506 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-06-06T08:27:43.506 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-06-06T08:27:43.506 INFO:teuthology.orchestra.run.smithi027.stdout:Repeating the final host check... 2024-06-06T08:27:43.530 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.0.2 2024-06-06T08:27:43.530 INFO:teuthology.orchestra.run.smithi027.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-06T08:27:43.530 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl is present 2024-06-06T08:27:43.530 INFO:teuthology.orchestra.run.smithi027.stdout:lvcreate is present 2024-06-06T08:27:43.537 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T08:27:43.537 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T08:27:43.543 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T08:27:43.543 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-06-06T08:27:43.550 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-06-06T08:27:43.556 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-06-06T08:27:43.557 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-06-06T08:27:43.557 INFO:teuthology.orchestra.run.smithi027.stdout:Host looks OK 2024-06-06T08:27:43.557 INFO:teuthology.orchestra.run.smithi027.stdout:Cluster fsid: 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:27:43.557 INFO:teuthology.orchestra.run.smithi027.stdout:Acquiring lock 140175364416896 on /run/cephadm/8da40ada-23de-11ef-bc9b-c7b262605968.lock 2024-06-06T08:27:43.557 INFO:teuthology.orchestra.run.smithi027.stdout:Lock 140175364416896 acquired on /run/cephadm/8da40ada-23de-11ef-bc9b-c7b262605968.lock 2024-06-06T08:27:43.558 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 3300 ... 2024-06-06T08:27:43.558 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 6789 ... 2024-06-06T08:27:43.558 INFO:teuthology.orchestra.run.smithi027.stdout:Base mon IP(s) is [172.21.15.27:3300, 172.21.15.27:6789], mon addrv is [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-06-06T08:27:43.561 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.27 metric 100 2024-06-06T08:27:43.562 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.27 metric 100 2024-06-06T08:27:43.564 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-06T08:27:43.564 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-06T08:27:43.564 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-06T08:27:43.567 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-06T08:27:43.567 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-06T08:27:43.567 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T08:27:43.567 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-06T08:27:43.567 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-06-06T08:27:43.567 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T08:27:43.568 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-06-06T08:27:43.568 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-06-06T08:27:43.568 INFO:teuthology.orchestra.run.smithi027.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-06T08:27:43.568 INFO:teuthology.orchestra.run.smithi027.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-06T08:27:43.569 INFO:teuthology.orchestra.run.smithi027.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T08:27:44.075 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T08:27:44.075 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T08:27:44.075 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Getting image source signatures 2024-06-06T08:27:44.075 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-06T08:27:44.076 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-06T08:27:44.076 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T08:27:44.076 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-06T08:27:44.988 INFO:teuthology.orchestra.run.smithi027.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T08:27:44.988 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T08:27:44.988 INFO:teuthology.orchestra.run.smithi027.stdout:Extracting ceph user uid/gid from container image... 2024-06-06T08:27:45.943 INFO:teuthology.orchestra.run.smithi027.stdout:stat: stdout 167 167 2024-06-06T08:27:45.944 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial keys... 2024-06-06T08:27:46.817 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQACc2Fmbc+4DxAAwbgnzKm5dv661vdaSdr1jQ== 2024-06-06T08:27:47.664 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQADc2FmxVy1BxAARBby3DoOOpQ71BYp8wikPA== 2024-06-06T08:27:48.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQADc2FmLULPORAAU6ciEjjsBLJti9aPKxUU6Q== 2024-06-06T08:27:48.488 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial monmap... 2024-06-06T08:27:55.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T08:27:55.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-06T08:27:55.630 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:27:55.630 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T08:27:55.630 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool for a [v2:172.21.15.27:3300,v1:172.21.15.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T08:27:55.630 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-06-06T08:27:55.630 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: set fsid to 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:27:55.630 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T08:27:55.630 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:27:55.631 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mon... 2024-06-06T08:28:02.000 INFO:teuthology.orchestra.run.smithi027.stdout:create mon.a on 2024-06-06T08:28:02.218 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-06T08:28:02.370 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-06T08:28:02.565 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8da40ada-23de-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-8da40ada-23de-11ef-bc9b-c7b262605968.target. 2024-06-06T08:28:02.565 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8da40ada-23de-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-8da40ada-23de-11ef-bc9b-c7b262605968.target. 2024-06-06T08:28:03.262 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.a 2024-06-06T08:28:03.262 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-06T08:28:03.433 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8da40ada-23de-11ef-bc9b-c7b262605968.target.wants/ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-8da40ada-23de-11ef-bc9b-c7b262605968@.service. 2024-06-06T08:28:03.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:03 smithi027 podman[31945]: 2024-06-06 08:28:03.578355782 +0000 UTC m=+0.017733310 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:28:03.915 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T08:28:03.915 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-06-06T08:28:03.922 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T08:28:03.922 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-06-06T08:28:03.922 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-06-06T08:28:03.922 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T08:28:03.923 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon to start... 2024-06-06T08:28:03.923 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon... 2024-06-06T08:28:04.044 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:03 smithi027 podman[31945]: 2024-06-06 08:28:03.827877859 +0000 UTC m=+0.267255394 container init 3e046ed9618e9a54505bfac809035767ca42249f980b1ee11a963906b06093b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-06T08:28:04.044 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:03 smithi027 podman[31945]: 2024-06-06 08:28:03.830950555 +0000 UTC m=+0.270328075 container start 3e046ed9618e9a54505bfac809035767ca42249f980b1ee11a963906b06093b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T08:28:04.044 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:03 smithi027 ceph-mon[31964]: mkfs 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:04.044 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:03 smithi027 ceph-mon[31964]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-06T08:28:04.044 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:03 smithi027 bash[31945]: 3e046ed9618e9a54505bfac809035767ca42249f980b1ee11a963906b06093b6 2024-06-06T08:28:04.044 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:03 smithi027 systemd[1]: Started Ceph mon.a for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:28:04.985 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout cluster: 2024-06-06T08:28:04.985 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout id: 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:04.985 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-06T08:28:04.985 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:04.985 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout services: 2024-06-06T08:28:04.985 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.56819s) 2024-06-06T08:28:04.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-06T08:28:04.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-06T08:28:04.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:04.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout data: 2024-06-06T08:28:04.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-06T08:28:04.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-06T08:28:04.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-06T08:28:04.986 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pgs: 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.358+0000 7f2ec0831700 1 Processor -- start 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.358+0000 7f2ec0831700 1 -- start start 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.358+0000 7f2ec0831700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.358+0000 7f2ec0831700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc107080 con 0x7f2ebc106690 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.359+0000 7f2eba59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.359+0000 7f2eba59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56170/0 (socket says 172.21.15.27:56170) 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.359+0000 7f2eba59c700 1 -- 172.21.15.27:0/4082147754 learned_addr learned my addr 172.21.15.27:0/4082147754 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.359+0000 7f2eba59c700 1 -- 172.21.15.27:0/4082147754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc107890 con 0x7f2ebc106690 2024-06-06T08:28:04.987 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.359+0000 7f2eba59c700 1 --2- 172.21.15.27:0/4082147754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc106ab0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009e80 tx=0x7f2eac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bad01cec827977 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2eb959a700 1 -- 172.21.15.27:0/4082147754 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2eac017070 con 0x7f2ebc106690 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2eb959a700 1 -- 172.21.15.27:0/4082147754 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eac003a20 con 0x7f2ebc106690 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2eb959a700 1 -- 172.21.15.27:0/4082147754 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2eac003d20 con 0x7f2ebc106690 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2ec0831700 1 -- 172.21.15.27:0/4082147754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 msgr2=0x7f2ebc106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2ec0831700 1 --2- 172.21.15.27:0/4082147754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc106ab0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009e80 tx=0x7f2eac00b330 comp rx=0 tx=0).stop 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2ec0831700 1 -- 172.21.15.27:0/4082147754 shutdown_connections 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2ec0831700 1 --2- 172.21.15.27:0/4082147754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc106ab0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2ec0831700 1 -- 172.21.15.27:0/4082147754 >> 172.21.15.27:0/4082147754 conn(0x7f2ebc101c70 msgr2=0x7f2ebc104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2ec0831700 1 -- 172.21.15.27:0/4082147754 shutdown_connections 2024-06-06T08:28:04.988 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.360+0000 7f2ec0831700 1 -- 172.21.15.27:0/4082147754 wait complete. 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.361+0000 7f2ec0831700 1 Processor -- start 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.361+0000 7f2ec0831700 1 -- start start 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.361+0000 7f2ec0831700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.361+0000 7f2ec0831700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc07b1d0 con 0x7f2ebc106690 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.361+0000 7f2eba59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2eba59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56174/0 (socket says 172.21.15.27:56174) 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2eba59c700 1 -- 172.21.15.27:0/1321481553 learned_addr learned my addr 172.21.15.27:0/1321481553 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2eba59c700 1 -- 172.21.15.27:0/1321481553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc104020 con 0x7f2ebc106690 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2eba59c700 1 --2- 172.21.15.27:0/1321481553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc103ae0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2eac004000 tx=0x7f2eac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2eb37fe700 1 -- 172.21.15.27:0/1321481553 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2eac022020 con 0x7f2ebc106690 2024-06-06T08:28:04.989 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2eb37fe700 1 -- 172.21.15.27:0/1321481553 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eac018040 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2ec0831700 1 -- 172.21.15.27:0/1321481553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ebc102210 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2eb37fe700 1 -- 172.21.15.27:0/1321481553 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2eac017070 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2ec0831700 1 -- 172.21.15.27:0/1321481553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ebc102750 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2eb37fe700 1 -- 172.21.15.27:0/1321481553 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2eac025020 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.362+0000 7f2eb37fe700 1 -- 172.21.15.27:0/1321481553 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2eac00f050 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.363+0000 7f2ec0831700 1 -- 172.21.15.27:0/1321481553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ebc063af0 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.366+0000 7f2eb37fe700 1 -- 172.21.15.27:0/1321481553 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2eac02c030 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.438+0000 7f2ec0831700 1 -- 172.21.15.27:0/1321481553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f2ebc194de0 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.438+0000 7f2eb37fe700 1 -- 172.21.15.27:0/1321481553 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f2eac01e480 con 0x7f2ebc106690 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.439+0000 7f2ec0831700 1 -- 172.21.15.27:0/1321481553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 msgr2=0x7f2ebc103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.439+0000 7f2ec0831700 1 --2- 172.21.15.27:0/1321481553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc103ae0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2eac004000 tx=0x7f2eac004ad0 comp rx=0 tx=0).stop 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.439+0000 7f2ec0831700 1 -- 172.21.15.27:0/1321481553 shutdown_connections 2024-06-06T08:28:04.990 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.439+0000 7f2ec0831700 1 --2- 172.21.15.27:0/1321481553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ebc106690 0x7f2ebc103ae0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:04.991 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.439+0000 7f2ec0831700 1 -- 172.21.15.27:0/1321481553 >> 172.21.15.27:0/1321481553 conn(0x7f2ebc101c70 msgr2=0x7f2ebc10b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:04.991 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.439+0000 7f2ec0831700 1 -- 172.21.15.27:0/1321481553 shutdown_connections 2024-06-06T08:28:04.991 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:04.439+0000 7f2ec0831700 1 -- 172.21.15.27:0/1321481553 wait complete. 2024-06-06T08:28:04.991 INFO:teuthology.orchestra.run.smithi027.stdout:mon is available 2024-06-06T08:28:04.991 INFO:teuthology.orchestra.run.smithi027.stdout:Assimilating anything we can from ceph.conf... 2024-06-06T08:28:05.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-06T08:28:05.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: monmap epoch 1 2024-06-06T08:28:05.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:05.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: last_changed 2024-06-06T08:27:49.610814+0000 2024-06-06T08:28:05.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: created 2024-06-06T08:27:49.610814+0000 2024-06-06T08:28:05.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: min_mon_release 19 (squid) 2024-06-06T08:28:05.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: election_strategy: 1 2024-06-06T08:28:05.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-06-06T08:28:05.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: fsmap 2024-06-06T08:28:05.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: osdmap e1: 0 total, 0 up, 0 in 2024-06-06T08:28:05.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: mgrmap e1: no daemons active 2024-06-06T08:28:05.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:04 smithi027 ceph-mon[31964]: from='client.? 172.21.15.27:0/1321481553' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-06T08:28:06.039 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:06.039 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T08:28:06.040 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.408+0000 7f5eec67b700 1 Processor -- start 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.408+0000 7f5eec67b700 1 -- start start 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.408+0000 7f5eec67b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee410ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.408+0000 7f5eec67b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee407c720 con 0x7f5ee4108a40 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.409+0000 7f5eea417700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee410ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.409+0000 7f5eea417700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee410ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56176/0 (socket says 172.21.15.27:56176) 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.409+0000 7f5eea417700 1 -- 172.21.15.27:0/364637685 learned_addr learned my addr 172.21.15.27:0/364637685 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.409+0000 7f5eea417700 1 -- 172.21.15.27:0/364637685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee410b370 con 0x7f5ee4108a40 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.409+0000 7f5eea417700 1 --2- 172.21.15.27:0/364637685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee410ae30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80098a0 tx=0x7f5ed8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fcd1bcba68afd7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:06.041 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.409+0000 7f5ee9415700 1 -- 172.21.15.27:0/364637685 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed8004160 con 0x7f5ee4108a40 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.409+0000 7f5ee9415700 1 -- 172.21.15.27:0/364637685 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed8004300 con 0x7f5ee4108a40 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.409+0000 7f5ee9415700 1 -- 172.21.15.27:0/364637685 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed80036a0 con 0x7f5ee4108a40 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.410+0000 7f5eec67b700 1 -- 172.21.15.27:0/364637685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 msgr2=0x7f5ee410ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.410+0000 7f5eec67b700 1 --2- 172.21.15.27:0/364637685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee410ae30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80098a0 tx=0x7f5ed8009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.410+0000 7f5eec67b700 1 -- 172.21.15.27:0/364637685 shutdown_connections 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.410+0000 7f5eec67b700 1 --2- 172.21.15.27:0/364637685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee410ae30 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.410+0000 7f5eec67b700 1 -- 172.21.15.27:0/364637685 >> 172.21.15.27:0/364637685 conn(0x7f5ee41023f0 msgr2=0x7f5ee4104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.410+0000 7f5eec67b700 1 -- 172.21.15.27:0/364637685 shutdown_connections 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.410+0000 7f5eec67b700 1 -- 172.21.15.27:0/364637685 wait complete. 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.410+0000 7f5eec67b700 1 Processor -- start 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.410+0000 7f5eec67b700 1 -- start start 2024-06-06T08:28:06.042 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5eec67b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee41082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5eec67b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee410b7e0 con 0x7f5ee4108a40 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5eea417700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee41082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5eea417700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee41082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56190/0 (socket says 172.21.15.27:56190) 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5eea417700 1 -- 172.21.15.27:0/2930130021 learned_addr learned my addr 172.21.15.27:0/2930130021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5eea417700 1 -- 172.21.15.27:0/2930130021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee4108800 con 0x7f5ee4108a40 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5eea417700 1 --2- 172.21.15.27:0/2930130021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee41082c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8000c00 tx=0x7f5ed8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5ed77fe700 1 -- 172.21.15.27:0/2930130021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed8003db0 con 0x7f5ee4108a40 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5ed77fe700 1 -- 172.21.15.27:0/2930130021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed8011630 con 0x7f5ee4108a40 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5eec67b700 1 -- 172.21.15.27:0/2930130021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee41069f0 con 0x7f5ee4108a40 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5ed77fe700 1 -- 172.21.15.27:0/2930130021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed8011950 con 0x7f5ee4108a40 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.411+0000 7f5eec67b700 1 -- 172.21.15.27:0/2930130021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee4106f30 con 0x7f5ee4108a40 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.412+0000 7f5ed77fe700 1 -- 172.21.15.27:0/2930130021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ed8011af0 con 0x7f5ee4108a40 2024-06-06T08:28:06.043 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.412+0000 7f5ed77fe700 1 -- 172.21.15.27:0/2930130021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ed801c450 con 0x7f5ee4108a40 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.412+0000 7f5eec67b700 1 -- 172.21.15.27:0/2930130021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee4063af0 con 0x7f5ee4108a40 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.415+0000 7f5ed77fe700 1 -- 172.21.15.27:0/2930130021 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5ed8032020 con 0x7f5ee4108a40 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.485+0000 7f5eec67b700 1 -- 172.21.15.27:0/2930130021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5ee4195050 con 0x7f5ee4108a40 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.488+0000 7f5ed77fe700 1 -- 172.21.15.27:0/2930130021 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f5ed8004260 con 0x7f5ee4108a40 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.489+0000 7f5ed77fe700 1 -- 172.21.15.27:0/2930130021 <== mon.0 v2:172.21.15.27:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ed801b450 con 0x7f5ee4108a40 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.489+0000 7f5eec67b700 1 -- 172.21.15.27:0/2930130021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 msgr2=0x7f5ee41082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.489+0000 7f5eec67b700 1 --2- 172.21.15.27:0/2930130021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee41082c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8000c00 tx=0x7f5ed8004750 comp rx=0 tx=0).stop 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.490+0000 7f5eec67b700 1 -- 172.21.15.27:0/2930130021 shutdown_connections 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.490+0000 7f5eec67b700 1 --2- 172.21.15.27:0/2930130021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee4108a40 0x7f5ee41082c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.490+0000 7f5eec67b700 1 -- 172.21.15.27:0/2930130021 >> 172.21.15.27:0/2930130021 conn(0x7f5ee41023f0 msgr2=0x7f5ee4104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.490+0000 7f5eec67b700 1 -- 172.21.15.27:0/2930130021 shutdown_connections 2024-06-06T08:28:06.044 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:05.490+0000 7f5eec67b700 1 -- 172.21.15.27:0/2930130021 wait complete. 2024-06-06T08:28:06.045 INFO:teuthology.orchestra.run.smithi027.stdout:Generating new minimal ceph.conf... 2024-06-06T08:28:06.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:05 smithi027 ceph-mon[31964]: from='client.? 172.21.15.27:0/2930130021' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-06T08:28:06.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:05 smithi027 ceph-mon[31964]: from='client.? 172.21.15.27:0/2930130021' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-06T08:28:07.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.445+0000 7f157e410700 1 Processor -- start 2024-06-06T08:28:07.069 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.445+0000 7f157e410700 1 -- start start 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.445+0000 7f157e410700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f1578108440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.445+0000 7f157e410700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1578108a10 con 0x7f1578108020 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.445+0000 7f1577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f1578108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.445+0000 7f1577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f1578108440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56206/0 (socket says 172.21.15.27:56206) 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.445+0000 7f1577fff700 1 -- 172.21.15.27:0/2047003880 learned_addr learned my addr 172.21.15.27:0/2047003880 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.446+0000 7f1577fff700 1 -- 172.21.15.27:0/2047003880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15781091d0 con 0x7f1578108020 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.446+0000 7f1577fff700 1 --2- 172.21.15.27:0/2047003880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f1578108440 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f15680098a0 tx=0x7f1568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d981e28a58d62c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.446+0000 7f1576ffd700 1 -- 172.21.15.27:0/2047003880 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1568004160 con 0x7f1578108020 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.446+0000 7f1576ffd700 1 -- 172.21.15.27:0/2047003880 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1568004300 con 0x7f1578108020 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.446+0000 7f1576ffd700 1 -- 172.21.15.27:0/2047003880 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1568011b50 con 0x7f1578108020 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.447+0000 7f157e410700 1 -- 172.21.15.27:0/2047003880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 msgr2=0x7f1578108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:07.070 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.447+0000 7f157e410700 1 --2- 172.21.15.27:0/2047003880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f1578108440 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f15680098a0 tx=0x7f1568009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.447+0000 7f157e410700 1 -- 172.21.15.27:0/2047003880 shutdown_connections 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.447+0000 7f157e410700 1 --2- 172.21.15.27:0/2047003880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f1578108440 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.447+0000 7f157e410700 1 -- 172.21.15.27:0/2047003880 >> 172.21.15.27:0/2047003880 conn(0x7f1578103620 msgr2=0x7f1578105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.447+0000 7f157e410700 1 -- 172.21.15.27:0/2047003880 shutdown_connections 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.447+0000 7f157e410700 1 -- 172.21.15.27:0/2047003880 wait complete. 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.447+0000 7f157e410700 1 Processor -- start 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.447+0000 7f157e410700 1 -- start start 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f157e410700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f157807e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f157e410700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f157810c580 con 0x7f1578108020 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f1577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f157807e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:07.071 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f1577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f157807e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56212/0 (socket says 172.21.15.27:56212) 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f1577fff700 1 -- 172.21.15.27:0/4231025093 learned_addr learned my addr 172.21.15.27:0/4231025093 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f1577fff700 1 -- 172.21.15.27:0/4231025093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157807e960 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f1577fff700 1 --2- 172.21.15.27:0/4231025093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f157807e420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1568000c00 tx=0x7f1568011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f15757fa700 1 -- 172.21.15.27:0/4231025093 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1568003b20 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f15757fa700 1 -- 172.21.15.27:0/4231025093 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1568003cc0 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f157e410700 1 -- 172.21.15.27:0/4231025093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157807cb50 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f15757fa700 1 -- 172.21.15.27:0/4231025093 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1568024910 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.448+0000 7f157e410700 1 -- 172.21.15.27:0/4231025093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157807d090 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.449+0000 7f15757fa700 1 -- 172.21.15.27:0/4231025093 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1568022020 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.449+0000 7f15757fa700 1 -- 172.21.15.27:0/4231025093 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1568011e00 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.449+0000 7f157e410700 1 -- 172.21.15.27:0/4231025093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1578063af0 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.452+0000 7f15757fa700 1 -- 172.21.15.27:0/4231025093 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f156801acd0 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.523+0000 7f157e410700 1 -- 172.21.15.27:0/4231025093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f157807d3d0 con 0x7f1578108020 2024-06-06T08:28:07.072 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.523+0000 7f15757fa700 1 -- 172.21.15.27:0/4231025093 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f1568032030 con 0x7f1578108020 2024-06-06T08:28:07.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.524+0000 7f157e410700 1 -- 172.21.15.27:0/4231025093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 msgr2=0x7f157807e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:07.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.524+0000 7f157e410700 1 --2- 172.21.15.27:0/4231025093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f157807e420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1568000c00 tx=0x7f1568011c10 comp rx=0 tx=0).stop 2024-06-06T08:28:07.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.524+0000 7f157e410700 1 -- 172.21.15.27:0/4231025093 shutdown_connections 2024-06-06T08:28:07.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.524+0000 7f157e410700 1 --2- 172.21.15.27:0/4231025093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1578108020 0x7f157807e420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:07.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.524+0000 7f157e410700 1 -- 172.21.15.27:0/4231025093 >> 172.21.15.27:0/4231025093 conn(0x7f1578103620 msgr2=0x7f1578104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:07.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.524+0000 7f157e410700 1 -- 172.21.15.27:0/4231025093 shutdown_connections 2024-06-06T08:28:07.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:06.524+0000 7f157e410700 1 -- 172.21.15.27:0/4231025093 wait complete. 2024-06-06T08:28:07.073 INFO:teuthology.orchestra.run.smithi027.stdout:Restarting the monitor... 2024-06-06T08:28:07.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:06 smithi027 ceph-mon[31964]: from='client.? 172.21.15.27:0/4231025093' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:28:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:07 smithi027 systemd[1]: Stopping Ceph mon.a for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:28:08.398 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:08 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[31960]: 2024-06-06T08:28:08.041+0000 7fce5e637700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:28:08.398 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:08 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[31960]: 2024-06-06T08:28:08.041+0000 7fce5e637700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-06T08:28:08.398 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:08 smithi027 podman[32211]: 2024-06-06 08:28:08.165381091 +0000 UTC m=+0.920235934 container died 3e046ed9618e9a54505bfac809035767ca42249f980b1ee11a963906b06093b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-06T08:28:08.689 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:08 smithi027 podman[32211]: 2024-06-06 08:28:08.465243073 +0000 UTC m=+1.220097911 container cleanup 3e046ed9618e9a54505bfac809035767ca42249f980b1ee11a963906b06093b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:28:08.689 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:08 smithi027 bash[32211]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a 2024-06-06T08:28:08.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:08 smithi027 podman[32225]: 2024-06-06 08:28:08.691347131 +0000 UTC m=+0.524839805 container remove 3e046ed9618e9a54505bfac809035767ca42249f980b1ee11a963906b06093b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-06T08:28:08.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:08 smithi027 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-06T08:28:08.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:08 smithi027 systemd[1]: Stopped Ceph mon.a for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:28:08.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:08 smithi027 systemd[1]: Starting Ceph mon.a for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:28:09.296 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 podman[32316]: 2024-06-06 08:28:08.935958163 +0000 UTC m=+0.017663538 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:28:09.296 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 podman[32316]: 2024-06-06 08:28:09.05906194 +0000 UTC m=+0.140767306 container create ddea9b723a2d2154bf44eb00b3345d1b0260a202d628926c929137d265f4843a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-06-06T08:28:09.296 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 podman[32316]: 2024-06-06 08:28:09.269842366 +0000 UTC m=+0.351547731 container init ddea9b723a2d2154bf44eb00b3345d1b0260a202d628926c929137d265f4843a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-06T08:28:09.296 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 podman[32316]: 2024-06-06 08:28:09.273852983 +0000 UTC m=+0.355558358 container start ddea9b723a2d2154bf44eb00b3345d1b0260a202d628926c929137d265f4843a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:28:09.347 INFO:teuthology.orchestra.run.smithi027.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: set uid:gid to 167:167 (ceph:ceph) 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: pidfile_write: ignore empty --pid-file 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: load: jerasure load: lrc 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: RocksDB version: 7.9.2 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Git sha 0 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: DB SUMMARY 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: DB Session ID: N7CHWSAX12PG7J3DZ3TS 2024-06-06T08:28:09.552 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: CURRENT file: CURRENT 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: IDENTITY file: IDENTITY 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105874 ; 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.error_if_exists: 0 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.create_if_missing: 0 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.paranoid_checks: 1 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.env: 0x55edba6a8c60 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.fs: PosixFileSystem 2024-06-06T08:28:09.553 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.info_log: 0x55edbc14bb20 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_file_opening_threads: 16 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.statistics: (nil) 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.use_fsync: 0 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_log_file_size: 0 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.keep_log_file_num: 1000 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.recycle_log_file_num: 0 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.allow_fallocate: 1 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.allow_mmap_reads: 0 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.allow_mmap_writes: 0 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.use_direct_reads: 0 2024-06-06T08:28:09.554 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.create_missing_column_families: 0 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.db_log_dir: 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.wal_dir: 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.advise_random_on_open: 1 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.db_write_buffer_size: 0 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.write_buffer_manager: 0x55edbc11c1e0 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-06T08:28:09.555 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.rate_limiter: (nil) 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.wal_recovery_mode: 2 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.enable_thread_tracking: 0 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.enable_pipelined_write: 0 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.unordered_write: 0 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.row_cache: None 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.wal_filter: None 2024-06-06T08:28:09.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.allow_ingest_behind: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.two_write_queues: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.manual_wal_flush: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.wal_compression: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.atomic_flush: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.log_readahead_size: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.best_efforts_recovery: 0 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-06T08:28:09.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.allow_data_in_errors: 0 2024-06-06T08:28:09.558 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.db_host_id: __hostname__ 2024-06-06T08:28:09.558 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_background_jobs: 2 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_background_compactions: -1 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_subcompactions: 1 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_total_wal_size: 0 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-06T08:28:09.559 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_open_files: -1 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bytes_per_sync: 0 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_readahead_size: 0 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_background_flushes: -1 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Compression algorithms supported: 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: kZSTD supported: 0 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: kXpressCompression supported: 0 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: kLZ4HCCompression supported: 1 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: kZlibCompression supported: 1 2024-06-06T08:28:09.560 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: kSnappyCompression supported: 1 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: kLZ4Compression supported: 1 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: kBZip2Compression supported: 0 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.merge_operator: 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_filter: None 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_filter_factory: None 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.sst_partitioner_factory: None 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-06T08:28:09.562 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55edbc14bc40) 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_type: 0 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_index_type: 0 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_shortening: 1 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: checksum: 4 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: no_block_cache: 0 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache: 0x55edbc14d090 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_options: 2024-06-06T08:28:09.563 INFO:journalctl@ceph.mon.a.smithi027.stdout: capacity : 536870912 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_shard_bits : 4 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: strict_capacity_limit : 0 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_compressed: (nil) 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: persistent_cache: (nil) 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size: 4096 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size_deviation: 10 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_restart_interval: 16 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_block_restart_interval: 1 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: metadata_block_size: 4096 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: partition_filters: 0 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: use_delta_encoding: 1 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: filter_policy: bloomfilter 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: whole_key_filtering: 1 2024-06-06T08:28:09.564 INFO:journalctl@ceph.mon.a.smithi027.stdout: verify_compression: 0 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout: format_version: 5 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_index_compression: 1 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_align: 0 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout: max_auto_readahead_size: 262144 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout: prepopulate_block_cache: 0 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout: initial_auto_readahead_size: 8192 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.write_buffer_size: 33554432 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_write_buffer_number: 2 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression: NoCompression 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression: Disabled 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.prefix_extractor: nullptr 2024-06-06T08:28:09.565 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.num_levels: 7 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-06T08:28:09.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression_opts.level: 32767 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression_opts.strategy: 0 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression_opts.enabled: false 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-06T08:28:09.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.target_file_size_base: 67108864 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-06T08:28:09.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.arena_block_size: 1048576 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.disable_auto_compactions: 0 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-06T08:28:09.569 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.table_properties_collectors: 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.inplace_update_support: 0 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.bloom_locality: 0 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.max_successive_merges: 0 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-06T08:28:09.570 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.paranoid_file_checks: 0 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.force_consistency_checks: 1 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.report_bg_io_stats: 0 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.ttl: 2592000 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.enable_blob_files: false 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.min_blob_size: 0 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.blob_file_size: 268435456 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-06T08:28:09.571 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.blob_file_starting_level: 0 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1aef7ae2-b200-4e0d-99aa-91e205be1583 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717662489304321, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717662489305350, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100924, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98833, "index_size": 309, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14439, "raw_average_key_size": 49, "raw_value_size": 90960, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717662489, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1aef7ae2-b200-4e0d-99aa-91e205be1583", "db_session_id": "N7CHWSAX12PG7J3DZ3TS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717662489305434, "job": 1, "event": "recovery_finished"} 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-06T08:28:09.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55edbc23c000 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: DB pointer 0x55edbc226000 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** DB Stats ** 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-06-06T08:28:09.573 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: L0 2/0 100.39 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: Sum 2/0 100.39 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-06T08:28:09.574 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative compaction: 0.00 GB write, 30.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval compaction: 0.00 GB write, 30.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache BinnedLRUCache@0x55edbc14d090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: starting mon.a rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:09.575 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mon.a@-1(???) e1 preinit fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mon.a@-1(???).mds e1 new map 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mon.a@-1(???).mds e1 print_map 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: e1 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: legacy client fscid: -1 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout: No filesystems configured 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T08:28:09.576 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 bash[32316]: ddea9b723a2d2154bf44eb00b3345d1b0260a202d628926c929137d265f4843a 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 systemd[1]: Started Ceph mon.a for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: monmap epoch 1 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: last_changed 2024-06-06T08:27:49.610814+0000 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: created 2024-06-06T08:27:49.610814+0000 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: min_mon_release 19 (squid) 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: election_strategy: 1 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: fsmap 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: osdmap e1: 0 total, 0 up, 0 in 2024-06-06T08:28:09.577 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:09 smithi027 ceph-mon[32335]: mgrmap e1: no daemons active 2024-06-06T08:28:10.379 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.746+0000 7f1e55e6f700 1 Processor -- start 2024-06-06T08:28:10.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.746+0000 7f1e55e6f700 1 -- start start 2024-06-06T08:28:10.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.747+0000 7f1e55e6f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5007cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:10.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.747+0000 7f1e55e6f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5007d0d0 con 0x7f1e5007e6e0 2024-06-06T08:28:10.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.747+0000 7f1e4f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5007cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:10.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.747+0000 7f1e4f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5007cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56228/0 (socket says 172.21.15.27:56228) 2024-06-06T08:28:10.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.747+0000 7f1e4f7fe700 1 -- 172.21.15.27:0/108770282 learned_addr learned my addr 172.21.15.27:0/108770282 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:10.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.747+0000 7f1e4f7fe700 1 -- 172.21.15.27:0/108770282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5007d250 con 0x7f1e5007e6e0 2024-06-06T08:28:10.384 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e4f7fe700 1 --2- 172.21.15.27:0/108770282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5007cb90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1e400098a0 tx=0x7f1e40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce259ad3551c5f95 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e4e7fc700 1 -- 172.21.15.27:0/108770282 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e40004160 con 0x7f1e5007e6e0 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e4e7fc700 1 -- 172.21.15.27:0/108770282 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1e40004300 con 0x7f1e5007e6e0 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e4e7fc700 1 -- 172.21.15.27:0/108770282 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e40011b50 con 0x7f1e5007e6e0 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e55e6f700 1 -- 172.21.15.27:0/108770282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 msgr2=0x7f1e5007cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e55e6f700 1 --2- 172.21.15.27:0/108770282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5007cb90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1e400098a0 tx=0x7f1e40009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e55e6f700 1 -- 172.21.15.27:0/108770282 shutdown_connections 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e55e6f700 1 --2- 172.21.15.27:0/108770282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5007cb90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e55e6f700 1 -- 172.21.15.27:0/108770282 >> 172.21.15.27:0/108770282 conn(0x7f1e50103620 msgr2=0x7f1e50105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e55e6f700 1 -- 172.21.15.27:0/108770282 shutdown_connections 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.748+0000 7f1e55e6f700 1 -- 172.21.15.27:0/108770282 wait complete. 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.749+0000 7f1e55e6f700 1 Processor -- start 2024-06-06T08:28:10.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.749+0000 7f1e55e6f700 1 -- start start 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.749+0000 7f1e55e6f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.749+0000 7f1e55e6f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5010e5f0 con 0x7f1e5007e6e0 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e4f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e4f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5019e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56242/0 (socket says 172.21.15.27:56242) 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e4f7fe700 1 -- 172.21.15.27:0/328533729 learned_addr learned my addr 172.21.15.27:0/328533729 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e4f7fe700 1 -- 172.21.15.27:0/328533729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5019e6e0 con 0x7f1e5007e6e0 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e4f7fe700 1 --2- 172.21.15.27:0/328533729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5019e1a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1e40011b60 tx=0x7f1e40011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e4cff9700 1 -- 172.21.15.27:0/328533729 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e40003b20 con 0x7f1e5007e6e0 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e4cff9700 1 -- 172.21.15.27:0/328533729 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1e40003cc0 con 0x7f1e5007e6e0 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e55e6f700 1 -- 172.21.15.27:0/328533729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5019e970 con 0x7f1e5007e6e0 2024-06-06T08:28:10.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e4cff9700 1 -- 172.21.15.27:0/328533729 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e40024840 con 0x7f1e5007e6e0 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e55e6f700 1 -- 172.21.15.27:0/328533729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e501a1660 con 0x7f1e5007e6e0 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.750+0000 7f1e4cff9700 1 -- 172.21.15.27:0/328533729 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e40022020 con 0x7f1e5007e6e0 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.751+0000 7f1e4cff9700 1 -- 172.21.15.27:0/328533729 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1e40011e00 con 0x7f1e5007e6e0 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.751+0000 7f1e55e6f700 1 -- 172.21.15.27:0/328533729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e50063af0 con 0x7f1e5007e6e0 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.754+0000 7f1e4cff9700 1 -- 172.21.15.27:0/328533729 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1e40043dd0 con 0x7f1e5007e6e0 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.823+0000 7f1e55e6f700 1 -- 172.21.15.27:0/328533729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f1e501a1830 con 0x7f1e5007e6e0 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.828+0000 7f1e4cff9700 1 -- 172.21.15.27:0/328533729 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f1e40032020 con 0x7f1e5007e6e0 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.829+0000 7f1e55e6f700 1 -- 172.21.15.27:0/328533729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 msgr2=0x7f1e5019e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.829+0000 7f1e55e6f700 1 --2- 172.21.15.27:0/328533729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5019e1a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1e40011b60 tx=0x7f1e40011c40 comp rx=0 tx=0).stop 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.829+0000 7f1e55e6f700 1 -- 172.21.15.27:0/328533729 shutdown_connections 2024-06-06T08:28:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.829+0000 7f1e55e6f700 1 --2- 172.21.15.27:0/328533729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e5007e6e0 0x7f1e5019e1a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:10.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.829+0000 7f1e55e6f700 1 -- 172.21.15.27:0/328533729 >> 172.21.15.27:0/328533729 conn(0x7f1e50103620 msgr2=0x7f1e501041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:10.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.829+0000 7f1e55e6f700 1 -- 172.21.15.27:0/328533729 shutdown_connections 2024-06-06T08:28:10.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:09.829+0000 7f1e55e6f700 1 -- 172.21.15.27:0/328533729 wait complete. 2024-06-06T08:28:10.388 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-06T08:28:10.409 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:28:10.409 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mgr... 2024-06-06T08:28:10.410 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-06T08:28:10.410 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-06T08:28:10.675 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T08:28:10.676 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-06T08:28:10.828 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8da40ada-23de-11ef-bc9b-c7b262605968.target.wants/ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-8da40ada-23de-11ef-bc9b-c7b262605968@.service. 2024-06-06T08:28:10.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:10 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/328533729' entity='client.admin' 2024-06-06T08:28:10.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:10 smithi027 systemd[1]: Starting Ceph mgr.x for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:28:11.347 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:11 smithi027 podman[32552]: 2024-06-06 08:28:11.063655349 +0000 UTC m=+0.109384466 container create e42a772f7b3d3c767939c76a22f6c558d37807f78ec1d1c8bc1ffbf6c2bc8b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-06T08:28:11.348 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:11 smithi027 podman[32552]: 2024-06-06 08:28:10.972614703 +0000 UTC m=+0.018343836 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T08:28:11.348 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:11 smithi027 podman[32552]: 2024-06-06 08:28:11.249323721 +0000 UTC m=+0.295052847 container init e42a772f7b3d3c767939c76a22f6c558d37807f78ec1d1c8bc1ffbf6c2bc8b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-06T08:28:11.348 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:11 smithi027 podman[32552]: 2024-06-06 08:28:11.252082747 +0000 UTC m=+0.297811878 container start e42a772f7b3d3c767939c76a22f6c558d37807f78ec1d1c8bc1ffbf6c2bc8b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-06-06T08:28:11.348 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:11 smithi027 bash[32552]: e42a772f7b3d3c767939c76a22f6c558d37807f78ec1d1c8bc1ffbf6c2bc8b50 2024-06-06T08:28:11.362 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T08:28:11.362 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-06-06T08:28:11.370 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T08:28:11.370 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-06-06T08:28:11.370 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-06-06T08:28:11.370 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T08:28:11.379 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T08:28:11.379 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-06-06T08:28:11.386 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T08:28:11.387 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-06-06T08:28:11.387 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-06-06T08:28:11.387 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-06T08:28:11.387 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr to start... 2024-06-06T08:28:11.387 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr... 2024-06-06T08:28:11.606 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:11 smithi027 systemd[1]: Started Ceph mgr.x for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:28:11.607 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:11 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:11.410+0000 7fe93b574200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T08:28:11.903 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:11 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:11.608+0000 7fe93b574200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:28:11.903 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:11 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:11.716+0000 7fe93b574200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T08:28:12.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:11 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/588492496' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T08:28:12.206 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:12 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:12.054+0000 7fe93b574200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T08:28:12.206 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:12 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:12.147+0000 7fe93b574200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T08:28:12.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T08:28:12.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T08:28:12.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T08:28:12.478 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T08:28:12.486 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T08:28:12.486 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T08:28:12.486 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T08:28:12.486 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:12.486 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T08:28:12.486 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T08:28:12.486 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T08:28:12.486 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T08:28:12.486 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:12.487 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T08:28:03.872228+0000", 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T08:28:12.488 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.823+0000 7f600a46e700 1 Processor -- start 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.823+0000 7f600a46e700 1 -- start start 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.823+0000 7f600a46e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f6004106c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.823+0000 7f600a46e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f600407be90 con 0x7f6004106830 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.824+0000 7f6003fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f6004106c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.824+0000 7f6003fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f6004106c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37046/0 (socket says 172.21.15.27:37046) 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.824+0000 7f6003fff700 1 -- 172.21.15.27:0/1965276963 learned_addr learned my addr 172.21.15.27:0/1965276963 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.824+0000 7f6003fff700 1 -- 172.21.15.27:0/1965276963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6004107190 con 0x7f6004106830 2024-06-06T08:28:12.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.824+0000 7f6003fff700 1 --2- 172.21.15.27:0/1965276963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f6004106c50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40098a0 tx=0x7f5ff4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a6c64623aef15f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.824+0000 7f6002ffd700 1 -- 172.21.15.27:0/1965276963 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ff4004160 con 0x7f6004106830 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.824+0000 7f6002ffd700 1 -- 172.21.15.27:0/1965276963 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ff4004300 con 0x7f6004106830 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.824+0000 7f6002ffd700 1 -- 172.21.15.27:0/1965276963 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ff4011b50 con 0x7f6004106830 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.825+0000 7f600a46e700 1 -- 172.21.15.27:0/1965276963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 msgr2=0x7f6004106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.825+0000 7f600a46e700 1 --2- 172.21.15.27:0/1965276963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f6004106c50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40098a0 tx=0x7f5ff4009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.825+0000 7f600a46e700 1 -- 172.21.15.27:0/1965276963 shutdown_connections 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.825+0000 7f600a46e700 1 --2- 172.21.15.27:0/1965276963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f6004106c50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.825+0000 7f600a46e700 1 -- 172.21.15.27:0/1965276963 >> 172.21.15.27:0/1965276963 conn(0x7f6004102470 msgr2=0x7f6004104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.825+0000 7f600a46e700 1 -- 172.21.15.27:0/1965276963 shutdown_connections 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.825+0000 7f600a46e700 1 -- 172.21.15.27:0/1965276963 wait complete. 2024-06-06T08:28:12.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.826+0000 7f600a46e700 1 Processor -- start 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.826+0000 7f600a46e700 1 -- start start 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.826+0000 7f600a46e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f600419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.826+0000 7f600a46e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6004108220 con 0x7f6004106830 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.826+0000 7f6003fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f600419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.826+0000 7f6003fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f600419e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37060/0 (socket says 172.21.15.27:37060) 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.826+0000 7f6003fff700 1 -- 172.21.15.27:0/588492496 learned_addr learned my addr 172.21.15.27:0/588492496 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.826+0000 7f6003fff700 1 -- 172.21.15.27:0/588492496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600419e680 con 0x7f6004106830 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.827+0000 7f6003fff700 1 --2- 172.21.15.27:0/588492496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f600419e140 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4011670 tx=0x7f5ff4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.827+0000 7f60017fa700 1 -- 172.21.15.27:0/588492496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ff40039c0 con 0x7f6004106830 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.827+0000 7f600a46e700 1 -- 172.21.15.27:0/588492496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600419e910 con 0x7f6004106830 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.827+0000 7f60017fa700 1 -- 172.21.15.27:0/588492496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ff4011e50 con 0x7f6004106830 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.827+0000 7f60017fa700 1 -- 172.21.15.27:0/588492496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ff401a5c0 con 0x7f6004106830 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.827+0000 7f600a46e700 1 -- 172.21.15.27:0/588492496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60041a1600 con 0x7f6004106830 2024-06-06T08:28:12.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.827+0000 7f60017fa700 1 -- 172.21.15.27:0/588492496 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ff401ac50 con 0x7f6004106830 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.827+0000 7f60017fa700 1 -- 172.21.15.27:0/588492496 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ff4019700 con 0x7f6004106830 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.828+0000 7f600a46e700 1 -- 172.21.15.27:0/588492496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6004063af0 con 0x7f6004106830 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.830+0000 7f60017fa700 1 -- 172.21.15.27:0/588492496 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5ff4003b60 con 0x7f6004106830 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.904+0000 7f600a46e700 1 -- 172.21.15.27:0/588492496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f60040008d0 con 0x7f6004106830 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.905+0000 7f60017fa700 1 -- 172.21.15.27:0/588492496 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5ff40199a0 con 0x7f6004106830 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.906+0000 7f600a46e700 1 -- 172.21.15.27:0/588492496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 msgr2=0x7f600419e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.906+0000 7f600a46e700 1 --2- 172.21.15.27:0/588492496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f600419e140 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4011670 tx=0x7f5ff4011b60 comp rx=0 tx=0).stop 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.906+0000 7f600a46e700 1 -- 172.21.15.27:0/588492496 shutdown_connections 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.906+0000 7f600a46e700 1 --2- 172.21.15.27:0/588492496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6004106830 0x7f600419e140 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.906+0000 7f600a46e700 1 -- 172.21.15.27:0/588492496 >> 172.21.15.27:0/588492496 conn(0x7f6004102470 msgr2=0x7f6004194d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.906+0000 7f600a46e700 1 -- 172.21.15.27:0/588492496 shutdown_connections 2024-06-06T08:28:12.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:11.906+0000 7f600a46e700 1 -- 172.21.15.27:0/588492496 wait complete. 2024-06-06T08:28:12.493 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (1/15)... 2024-06-06T08:28:13.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:13 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:13.417+0000 7fe93b574200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T08:28:13.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:13 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:13.593+0000 7fe93b574200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T08:28:14.067 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:13 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:13.769+0000 7fe93b574200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T08:28:14.068 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:14.069+0000 7fe93b574200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T08:28:14.349 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:14.168+0000 7fe93b574200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T08:28:14.601 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:14.383+0000 7fe93b574200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T08:28:14.852 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:14.602+0000 7fe93b574200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:28:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:15 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/418300131' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T08:28:15.627 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:15 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:15.255+0000 7fe93b574200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T08:28:15.630 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:15.633 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-06-06T08:28:15.633 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:28:15.633 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T08:28:15.633 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T08:28:15.633 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T08:28:15.633 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T08:28:15.633 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T08:28:15.634 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T08:28:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T08:28:15.642 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T08:28:15.642 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T08:28:15.642 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T08:28:15.642 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T08:28:15.642 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T08:28:15.642 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:15.643 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T08:28:03.872228+0000", 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.955+0000 7f2b66ded700 1 Processor -- start 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.955+0000 7f2b66ded700 1 -- start start 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.955+0000 7f2b66ded700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6010ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.955+0000 7f2b66ded700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6007be90 con 0x7f2b60108a40 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.956+0000 7f2b64b89700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6010ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:15.644 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.956+0000 7f2b64b89700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6010ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37068/0 (socket says 172.21.15.27:37068) 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.956+0000 7f2b64b89700 1 -- 172.21.15.27:0/2019204406 learned_addr learned my addr 172.21.15.27:0/2019204406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.956+0000 7f2b64b89700 1 -- 172.21.15.27:0/2019204406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6010b370 con 0x7f2b60108a40 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.956+0000 7f2b64b89700 1 --2- 172.21.15.27:0/2019204406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6010ae30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b540098a0 tx=0x7f2b54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451f5c6b075e4f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.956+0000 7f2b5f7fe700 1 -- 172.21.15.27:0/2019204406 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b54004160 con 0x7f2b60108a40 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.956+0000 7f2b5f7fe700 1 -- 172.21.15.27:0/2019204406 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2b54004300 con 0x7f2b60108a40 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.956+0000 7f2b5f7fe700 1 -- 172.21.15.27:0/2019204406 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b54011b50 con 0x7f2b60108a40 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.957+0000 7f2b66ded700 1 -- 172.21.15.27:0/2019204406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 msgr2=0x7f2b6010ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.957+0000 7f2b66ded700 1 --2- 172.21.15.27:0/2019204406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6010ae30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b540098a0 tx=0x7f2b54009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.957+0000 7f2b66ded700 1 -- 172.21.15.27:0/2019204406 shutdown_connections 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.957+0000 7f2b66ded700 1 --2- 172.21.15.27:0/2019204406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6010ae30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:15.645 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.957+0000 7f2b66ded700 1 -- 172.21.15.27:0/2019204406 >> 172.21.15.27:0/2019204406 conn(0x7f2b60102470 msgr2=0x7f2b60104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.957+0000 7f2b66ded700 1 -- 172.21.15.27:0/2019204406 shutdown_connections 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.957+0000 7f2b66ded700 1 -- 172.21.15.27:0/2019204406 wait complete. 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.958+0000 7f2b66ded700 1 Processor -- start 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.958+0000 7f2b66ded700 1 -- start start 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.958+0000 7f2b66ded700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.958+0000 7f2b66ded700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6010c390 con 0x7f2b60108a40 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.958+0000 7f2b64b89700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.958+0000 7f2b64b89700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37076/0 (socket says 172.21.15.27:37076) 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.958+0000 7f2b64b89700 1 -- 172.21.15.27:0/418300131 learned_addr learned my addr 172.21.15.27:0/418300131 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.958+0000 7f2b64b89700 1 -- 172.21.15.27:0/418300131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6019e690 con 0x7f2b60108a40 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.958+0000 7f2b64b89700 1 --2- 172.21.15.27:0/418300131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6019e150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b54011670 tx=0x7f2b54011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.959+0000 7f2b5dffb700 1 -- 172.21.15.27:0/418300131 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b540039c0 con 0x7f2b60108a40 2024-06-06T08:28:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.959+0000 7f2b5dffb700 1 -- 172.21.15.27:0/418300131 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2b54011e50 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.959+0000 7f2b5dffb700 1 -- 172.21.15.27:0/418300131 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b5401a5c0 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.959+0000 7f2b66ded700 1 -- 172.21.15.27:0/418300131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6019e920 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.959+0000 7f2b66ded700 1 -- 172.21.15.27:0/418300131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b601a1610 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.959+0000 7f2b5dffb700 1 -- 172.21.15.27:0/418300131 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2b5401ac00 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.959+0000 7f2b5dffb700 1 -- 172.21.15.27:0/418300131 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2b54019700 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.960+0000 7f2b66ded700 1 -- 172.21.15.27:0/418300131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b60063af0 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:14.962+0000 7f2b5dffb700 1 -- 172.21.15.27:0/418300131 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2b54003b60 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:15.036+0000 7f2b66ded700 1 -- 172.21.15.27:0/418300131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2b601a17e0 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:15.037+0000 7f2b5dffb700 1 -- 172.21.15.27:0/418300131 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2b54019900 con 0x7f2b60108a40 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:15.038+0000 7f2b66ded700 1 -- 172.21.15.27:0/418300131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 msgr2=0x7f2b6019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:15.038+0000 7f2b66ded700 1 --2- 172.21.15.27:0/418300131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6019e150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b54011670 tx=0x7f2b54011b60 comp rx=0 tx=0).stop 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:15.038+0000 7f2b66ded700 1 -- 172.21.15.27:0/418300131 shutdown_connections 2024-06-06T08:28:15.647 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:15.038+0000 7f2b66ded700 1 --2- 172.21.15.27:0/418300131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b60108a40 0x7f2b6019e150 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:15.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:15.038+0000 7f2b66ded700 1 -- 172.21.15.27:0/418300131 >> 172.21.15.27:0/418300131 conn(0x7f2b60102470 msgr2=0x7f2b60103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:15.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:15.038+0000 7f2b66ded700 1 -- 172.21.15.27:0/418300131 shutdown_connections 2024-06-06T08:28:15.648 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:15.038+0000 7f2b66ded700 1 -- 172.21.15.27:0/418300131 wait complete. 2024-06-06T08:28:15.648 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (2/15)... 2024-06-06T08:28:15.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:15 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:15.709+0000 7fe93b574200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T08:28:15.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:15 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:15.811+0000 7fe93b574200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T08:28:16.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:16 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:16.022+0000 7fe93b574200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T08:28:16.914 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:16 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:16.568+0000 7fe93b574200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T08:28:16.915 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:16 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:16.663+0000 7fe93b574200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T08:28:17.206 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:16 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:16.915+0000 7fe93b574200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T08:28:17.206 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:17.055+0000 7fe93b574200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T08:28:17.206 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:17.153+0000 7fe93b574200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T08:28:17.584 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:17.274+0000 7fe93b574200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T08:28:17.584 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:17.585+0000 7fe93b574200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T08:28:18.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:18 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1663694675' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T08:28:18.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:18 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:18.245+0000 7fe93b574200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T08:28:18.758 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:18 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:18.477+0000 7fe93b574200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T08:28:18.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:18.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T08:28:18.765 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T08:28:18.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T08:28:18.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T08:28:03.872228+0000", 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.033+0000 7f1488ea8700 1 Processor -- start 2024-06-06T08:28:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.033+0000 7f1488ea8700 1 -- start start 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.034+0000 7f1488ea8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148407cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.034+0000 7f1488ea8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f148407d1c0 con 0x7f148407e790 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.034+0000 7f148259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148407cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.034+0000 7f148259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148407cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37086/0 (socket says 172.21.15.27:37086) 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.034+0000 7f148259c700 1 -- 172.21.15.27:0/3973286778 learned_addr learned my addr 172.21.15.27:0/3973286778 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.034+0000 7f148259c700 1 -- 172.21.15.27:0/3973286778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148407d340 con 0x7f148407e790 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.034+0000 7f148259c700 1 --2- 172.21.15.27:0/3973286778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148407cbf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f14740098a0 tx=0x7f1474009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bbf0abde7bff1a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f148159a700 1 -- 172.21.15.27:0/3973286778 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1474004160 con 0x7f148407e790 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f148159a700 1 -- 172.21.15.27:0/3973286778 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1474004300 con 0x7f148407e790 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f148159a700 1 -- 172.21.15.27:0/3973286778 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1474011b50 con 0x7f148407e790 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f1488ea8700 1 -- 172.21.15.27:0/3973286778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 msgr2=0x7f148407cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f1488ea8700 1 --2- 172.21.15.27:0/3973286778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148407cbf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f14740098a0 tx=0x7f1474009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:18.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f1488ea8700 1 -- 172.21.15.27:0/3973286778 shutdown_connections 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f1488ea8700 1 --2- 172.21.15.27:0/3973286778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148407cbf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f1488ea8700 1 -- 172.21.15.27:0/3973286778 >> 172.21.15.27:0/3973286778 conn(0x7f1484103550 msgr2=0x7f1484105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f1488ea8700 1 -- 172.21.15.27:0/3973286778 shutdown_connections 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.035+0000 7f1488ea8700 1 -- 172.21.15.27:0/3973286778 wait complete. 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.036+0000 7f1488ea8700 1 Processor -- start 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.036+0000 7f1488ea8700 1 -- start start 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.036+0000 7f1488ea8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.036+0000 7f1488ea8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f148410e5d0 con 0x7f148407e790 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f148259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f148259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148419e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37092/0 (socket says 172.21.15.27:37092) 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f148259c700 1 -- 172.21.15.27:0/1663694675 learned_addr learned my addr 172.21.15.27:0/1663694675 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f148259c700 1 -- 172.21.15.27:0/1663694675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148419e6f0 con 0x7f148407e790 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f148259c700 1 --2- 172.21.15.27:0/1663694675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148419e1b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1474011670 tx=0x7f1474011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:18.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f14737fe700 1 -- 172.21.15.27:0/1663694675 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14740039c0 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f14737fe700 1 -- 172.21.15.27:0/1663694675 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1474011e50 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f14737fe700 1 -- 172.21.15.27:0/1663694675 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f147401a5c0 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f1488ea8700 1 -- 172.21.15.27:0/1663694675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148419e980 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.037+0000 7f1488ea8700 1 -- 172.21.15.27:0/1663694675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14841a1670 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.038+0000 7f14737fe700 1 -- 172.21.15.27:0/1663694675 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f147401ac50 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.038+0000 7f14737fe700 1 -- 172.21.15.27:0/1663694675 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1474019ae0 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.038+0000 7f1488ea8700 1 -- 172.21.15.27:0/1663694675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1484063af0 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.041+0000 7f14737fe700 1 -- 172.21.15.27:0/1663694675 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1474003b60 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.114+0000 7f1488ea8700 1 -- 172.21.15.27:0/1663694675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f14841a1840 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.115+0000 7f14737fe700 1 -- 172.21.15.27:0/1663694675 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1474019ce0 con 0x7f148407e790 2024-06-06T08:28:18.778 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.116+0000 7f1488ea8700 1 -- 172.21.15.27:0/1663694675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 msgr2=0x7f148419e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:18.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.116+0000 7f1488ea8700 1 --2- 172.21.15.27:0/1663694675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148419e1b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1474011670 tx=0x7f1474011b60 comp rx=0 tx=0).stop 2024-06-06T08:28:18.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.116+0000 7f1488ea8700 1 -- 172.21.15.27:0/1663694675 shutdown_connections 2024-06-06T08:28:18.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.116+0000 7f1488ea8700 1 --2- 172.21.15.27:0/1663694675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148407e790 0x7f148419e1b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:18.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.116+0000 7f1488ea8700 1 -- 172.21.15.27:0/1663694675 >> 172.21.15.27:0/1663694675 conn(0x7f1484103550 msgr2=0x7f1484104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:18.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.116+0000 7f1488ea8700 1 -- 172.21.15.27:0/1663694675 shutdown_connections 2024-06-06T08:28:18.779 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:18.116+0000 7f1488ea8700 1 -- 172.21.15.27:0/1663694675 wait complete. 2024-06-06T08:28:18.779 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (3/15)... 2024-06-06T08:28:19.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: Activating manager daemon x 2024-06-06T08:28:19.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: mgrmap e2: x(active, starting, since 0.00338113s) 2024-06-06T08:28:19.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T08:28:19.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T08:28:19.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T08:28:19.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:28:19.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T08:28:19.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: Manager daemon x is now available 2024-06-06T08:28:19.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T08:28:19.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T08:28:19.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' 2024-06-06T08:28:19.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' 2024-06-06T08:28:19.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:19 smithi027 ceph-mon[32335]: from='mgr.14100 172.21.15.27:0/290617189' entity='mgr.x' 2024-06-06T08:28:20.884 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:20 smithi027 ceph-mon[32335]: mgrmap e3: x(active, since 1.00728s) 2024-06-06T08:28:21.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:21 smithi027 ceph-mon[32335]: mgrmap e4: x(active, since 2s) 2024-06-06T08:28:21.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:21 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4131738240' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T08:28:22.220 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:22.221 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T08:28:22.222 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T08:28:22.223 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T08:28:03.872228+0000", 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.161+0000 7f55549ae700 1 Processor -- start 2024-06-06T08:28:22.224 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.161+0000 7f55549ae700 1 -- start start 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.161+0000 7f55549ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f555010ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.161+0000 7f55549ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f555007c760 con 0x7f5550108a40 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.161+0000 7f554e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f555010ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.161+0000 7f554e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f555010ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44480/0 (socket says 172.21.15.27:44480) 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.161+0000 7f554e59c700 1 -- 172.21.15.27:0/778293161 learned_addr learned my addr 172.21.15.27:0/778293161 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.162+0000 7f554e59c700 1 -- 172.21.15.27:0/778293161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555010b370 con 0x7f5550108a40 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.162+0000 7f554e59c700 1 --2- 172.21.15.27:0/778293161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f555010ae30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f55400098a0 tx=0x7f5540009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63ed0dd730d145e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.162+0000 7f554dd9b700 1 -- 172.21.15.27:0/778293161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5540004160 con 0x7f5550108a40 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.162+0000 7f554dd9b700 1 -- 172.21.15.27:0/778293161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5540004300 con 0x7f5550108a40 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.162+0000 7f554dd9b700 1 -- 172.21.15.27:0/778293161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5540011b50 con 0x7f5550108a40 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.163+0000 7f55549ae700 1 -- 172.21.15.27:0/778293161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 msgr2=0x7f555010ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.163+0000 7f55549ae700 1 --2- 172.21.15.27:0/778293161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f555010ae30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f55400098a0 tx=0x7f5540009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:22.225 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.163+0000 7f55549ae700 1 -- 172.21.15.27:0/778293161 shutdown_connections 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.163+0000 7f55549ae700 1 --2- 172.21.15.27:0/778293161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f555010ae30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.163+0000 7f55549ae700 1 -- 172.21.15.27:0/778293161 >> 172.21.15.27:0/778293161 conn(0x7f55501024f0 msgr2=0x7f5550104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.163+0000 7f55549ae700 1 -- 172.21.15.27:0/778293161 shutdown_connections 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.163+0000 7f55549ae700 1 -- 172.21.15.27:0/778293161 wait complete. 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.164+0000 7f55549ae700 1 Processor -- start 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.164+0000 7f55549ae700 1 -- start start 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.164+0000 7f55549ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f55501082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.164+0000 7f55549ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f555010c3a0 con 0x7f5550108a40 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.164+0000 7f554e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f55501082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.164+0000 7f554e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f55501082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44488/0 (socket says 172.21.15.27:44488) 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.164+0000 7f554e59c700 1 -- 172.21.15.27:0/4131738240 learned_addr learned my addr 172.21.15.27:0/4131738240 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.165+0000 7f554e59c700 1 -- 172.21.15.27:0/4131738240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5550108800 con 0x7f5550108a40 2024-06-06T08:28:22.226 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.165+0000 7f554e59c700 1 --2- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f55501082c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5540011b60 tx=0x7f5540011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.165+0000 7f5547fff700 1 -- 172.21.15.27:0/4131738240 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5540003b20 con 0x7f5550108a40 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.165+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55501069f0 con 0x7f5550108a40 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.165+0000 7f5547fff700 1 -- 172.21.15.27:0/4131738240 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5540003cc0 con 0x7f5550108a40 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.165+0000 7f5547fff700 1 -- 172.21.15.27:0/4131738240 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5540024840 con 0x7f5550108a40 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.165+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5550106f30 con 0x7f5550108a40 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.166+0000 7f5547fff700 1 -- 172.21.15.27:0/4131738240 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5540022020 con 0x7f5550108a40 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.166+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5550063af0 con 0x7f5550108a40 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.166+0000 7f5547fff700 1 --2- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f553c039000 0x7f553c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.166+0000 7f5547fff700 1 -- 172.21.15.27:0/4131738240 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5540011e00 con 0x7f5550108a40 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.166+0000 7f5545d9b700 1 --2- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f553c039000 0x7f553c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.167+0000 7f5545d9b700 1 --2- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f553c039000 0x7f553c03b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5538006fd0 tx=0x7f5538006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.172+0000 7f5547fff700 1 -- 172.21.15.27:0/4131738240 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55400172b0 con 0x7f5550108a40 2024-06-06T08:28:22.227 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.462+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f55501071b0 con 0x7f5550108a40 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.462+0000 7f5547fff700 1 -- 172.21.15.27:0/4131738240 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f554000fdd0 con 0x7f5550108a40 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f553c039000 msgr2=0x7f553c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 --2- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f553c039000 0x7f553c03b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5538006fd0 tx=0x7f5538006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 msgr2=0x7f55501082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 --2- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f55501082c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5540011b60 tx=0x7f5540011c40 comp rx=0 tx=0).stop 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 shutdown_connections 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 --2- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f553c039000 0x7f553c03b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 --2- 172.21.15.27:0/4131738240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5550108a40 0x7f55501082c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 >> 172.21.15.27:0/4131738240 conn(0x7f55501024f0 msgr2=0x7f5550102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 shutdown_connections 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:21.465+0000 7f55549ae700 1 -- 172.21.15.27:0/4131738240 wait complete. 2024-06-06T08:28:22.228 INFO:teuthology.orchestra.run.smithi027.stdout:mgr is available 2024-06-06T08:28:23.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:22 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2133362886' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-06T08:28:23.371 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:23.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T08:28:23.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:23.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T08:28:23.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T08:28:23.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T08:28:23.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T08:28:23.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:23.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T08:28:23.372 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T08:28:23.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-06-06T08:28:23.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T08:28:23.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T08:28:23.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T08:28:23.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T08:28:23.373 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T08:28:23.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T08:28:23.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.627+0000 7f9e7a00a700 1 Processor -- start 2024-06-06T08:28:23.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.627+0000 7f9e7a00a700 1 -- start start 2024-06-06T08:28:23.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.627+0000 7f9e7a00a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e741083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:23.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.627+0000 7f9e7a00a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741089c0 con 0x7f9e74107fd0 2024-06-06T08:28:23.374 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.628+0000 7f9e737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e741083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:23.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.628+0000 7f9e737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e741083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44504/0 (socket says 172.21.15.27:44504) 2024-06-06T08:28:23.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.628+0000 7f9e737fe700 1 -- 172.21.15.27:0/2698904672 learned_addr learned my addr 172.21.15.27:0/2698904672 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:23.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.628+0000 7f9e737fe700 1 -- 172.21.15.27:0/2698904672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e741091d0 con 0x7f9e74107fd0 2024-06-06T08:28:23.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.628+0000 7f9e737fe700 1 --2- 172.21.15.27:0/2698904672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e741083f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9e640098a0 tx=0x7f9e64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10f1494a5e7575b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:23.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.628+0000 7f9e727fc700 1 -- 172.21.15.27:0/2698904672 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e64004160 con 0x7f9e74107fd0 2024-06-06T08:28:23.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.628+0000 7f9e727fc700 1 -- 172.21.15.27:0/2698904672 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e64004300 con 0x7f9e74107fd0 2024-06-06T08:28:23.375 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.628+0000 7f9e727fc700 1 -- 172.21.15.27:0/2698904672 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e64011b50 con 0x7f9e74107fd0 2024-06-06T08:28:23.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.629+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2698904672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 msgr2=0x7f9e741083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:23.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.629+0000 7f9e7a00a700 1 --2- 172.21.15.27:0/2698904672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e741083f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9e640098a0 tx=0x7f9e64009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:23.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.629+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2698904672 shutdown_connections 2024-06-06T08:28:23.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.629+0000 7f9e7a00a700 1 --2- 172.21.15.27:0/2698904672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e741083f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:23.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.629+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2698904672 >> 172.21.15.27:0/2698904672 conn(0x7f9e741035d0 msgr2=0x7f9e741059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:23.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.629+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2698904672 shutdown_connections 2024-06-06T08:28:23.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.629+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2698904672 wait complete. 2024-06-06T08:28:23.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.630+0000 7f9e7a00a700 1 Processor -- start 2024-06-06T08:28:23.376 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.630+0000 7f9e7a00a700 1 -- start start 2024-06-06T08:28:23.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.630+0000 7f9e7a00a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e7419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:23.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.630+0000 7f9e7a00a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7410c580 con 0x7f9e74107fd0 2024-06-06T08:28:23.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.630+0000 7f9e737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e7419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:23.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.630+0000 7f9e737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e7419c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44520/0 (socket says 172.21.15.27:44520) 2024-06-06T08:28:23.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.630+0000 7f9e737fe700 1 -- 172.21.15.27:0/2133362886 learned_addr learned my addr 172.21.15.27:0/2133362886 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:23.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.630+0000 7f9e737fe700 1 -- 172.21.15.27:0/2133362886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7419c580 con 0x7f9e74107fd0 2024-06-06T08:28:23.377 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.631+0000 7f9e737fe700 1 --2- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e7419c040 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009fb0 tx=0x7f9e64011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:23.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.631+0000 7f9e70ff9700 1 -- 172.21.15.27:0/2133362886 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e640038b0 con 0x7f9e74107fd0 2024-06-06T08:28:23.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.631+0000 7f9e70ff9700 1 -- 172.21.15.27:0/2133362886 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e64011e50 con 0x7f9e74107fd0 2024-06-06T08:28:23.378 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.631+0000 7f9e70ff9700 1 -- 172.21.15.27:0/2133362886 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e64019550 con 0x7f9e74107fd0 2024-06-06T08:28:23.381 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.631+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7419c810 con 0x7f9e74107fd0 2024-06-06T08:28:23.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.631+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7419f500 con 0x7f9e74107fd0 2024-06-06T08:28:23.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.632+0000 7f9e70ff9700 1 -- 172.21.15.27:0/2133362886 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f9e64019850 con 0x7f9e74107fd0 2024-06-06T08:28:23.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.632+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e74063af0 con 0x7f9e74107fd0 2024-06-06T08:28:23.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.632+0000 7f9e70ff9700 1 --2- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9e5c038fb0 0x7f9e5c03b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:23.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.632+0000 7f9e70ff9700 1 -- 172.21.15.27:0/2133362886 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e6404d1b0 con 0x7f9e74107fd0 2024-06-06T08:28:23.382 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.632+0000 7f9e72ffd700 1 --2- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9e5c038fb0 0x7f9e5c03b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.632+0000 7f9e72ffd700 1 --2- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9e5c038fb0 0x7f9e5c03b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006fd0 tx=0x7f9e68006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.637+0000 7f9e70ff9700 1 -- 172.21.15.27:0/2133362886 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9e64023c90 con 0x7f9e74107fd0 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.840+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9e7419f6d0 con 0x7f9e74107fd0 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.841+0000 7f9e70ff9700 1 -- 172.21.15.27:0/2133362886 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f9e7419f6d0 con 0x7f9e74107fd0 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9e5c038fb0 msgr2=0x7f9e5c03b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 --2- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9e5c038fb0 0x7f9e5c03b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006fd0 tx=0x7f9e68006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 msgr2=0x7f9e7419c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 --2- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e7419c040 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009fb0 tx=0x7f9e64011bf0 comp rx=0 tx=0).stop 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 shutdown_connections 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 --2- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9e5c038fb0 0x7f9e5c03b470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 --2- 172.21.15.27:0/2133362886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e74107fd0 0x7f9e7419c040 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 >> 172.21.15.27:0/2133362886 conn(0x7f9e741035d0 msgr2=0x7f9e741041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 shutdown_connections 2024-06-06T08:28:23.383 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:22.844+0000 7f9e7a00a700 1 -- 172.21.15.27:0/2133362886 wait complete. 2024-06-06T08:28:23.384 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling cephadm module... 2024-06-06T08:28:24.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:24 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2692159970' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-06T08:28:24.444 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:24 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: ignoring --setuser ceph since I am not root 2024-06-06T08:28:24.444 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:24 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: ignoring --setgroup ceph since I am not root 2024-06-06T08:28:24.444 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:24 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:24.329+0000 7f3c3cb94200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T08:28:24.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:24 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:24.522+0000 7f3c3cb94200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:28:24.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:24 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:24.629+0000 7f3c3cb94200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T08:28:24.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.829+0000 7f9d06fa3700 1 Processor -- start 2024-06-06T08:28:24.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.829+0000 7f9d06fa3700 1 -- start start 2024-06-06T08:28:24.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.829+0000 7f9d06fa3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:24.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.829+0000 7f9d06fa3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0007c910 con 0x7f9d0007de90 2024-06-06T08:28:24.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.829+0000 7f9d04d3f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:24.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.829+0000 7f9d04d3f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44536/0 (socket says 172.21.15.27:44536) 2024-06-06T08:28:24.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.829+0000 7f9d04d3f700 1 -- 172.21.15.27:0/86570012 learned_addr learned my addr 172.21.15.27:0/86570012 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:24.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.830+0000 7f9d04d3f700 1 -- 172.21.15.27:0/86570012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0007ca90 con 0x7f9d0007de90 2024-06-06T08:28:24.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.830+0000 7f9d04d3f700 1 --2- 172.21.15.27:0/86570012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0007c340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40098a0 tx=0x7f9cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a959a11e515da792 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.830+0000 7f9cff7fe700 1 -- 172.21.15.27:0/86570012 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cf4004160 con 0x7f9d0007de90 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.830+0000 7f9cff7fe700 1 -- 172.21.15.27:0/86570012 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9cf4004300 con 0x7f9d0007de90 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.830+0000 7f9cff7fe700 1 -- 172.21.15.27:0/86570012 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cf4011b50 con 0x7f9d0007de90 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.831+0000 7f9d06fa3700 1 -- 172.21.15.27:0/86570012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 msgr2=0x7f9d0007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.831+0000 7f9d06fa3700 1 --2- 172.21.15.27:0/86570012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0007c340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40098a0 tx=0x7f9cf4009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.831+0000 7f9d06fa3700 1 -- 172.21.15.27:0/86570012 shutdown_connections 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.831+0000 7f9d06fa3700 1 --2- 172.21.15.27:0/86570012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0007c340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.831+0000 7f9d06fa3700 1 -- 172.21.15.27:0/86570012 >> 172.21.15.27:0/86570012 conn(0x7f9d00103570 msgr2=0x7f9d001059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.831+0000 7f9d06fa3700 1 -- 172.21.15.27:0/86570012 shutdown_connections 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.831+0000 7f9d06fa3700 1 -- 172.21.15.27:0/86570012 wait complete. 2024-06-06T08:28:24.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.831+0000 7f9d06fa3700 1 Processor -- start 2024-06-06T08:28:24.739 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.831+0000 7f9d06fa3700 1 -- start start 2024-06-06T08:28:24.739 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9d06fa3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:24.739 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9d06fa3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0010e560 con 0x7f9d0007de90 2024-06-06T08:28:24.739 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9d04d3f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:24.739 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9d04d3f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0019e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44544/0 (socket says 172.21.15.27:44544) 2024-06-06T08:28:24.739 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9d04d3f700 1 -- 172.21.15.27:0/2692159970 learned_addr learned my addr 172.21.15.27:0/2692159970 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:24.739 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9d04d3f700 1 -- 172.21.15.27:0/2692159970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0019e660 con 0x7f9d0007de90 2024-06-06T08:28:24.739 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9d04d3f700 1 --2- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0019e120 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4011670 tx=0x7f9cf4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:24.739 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9cfdffb700 1 -- 172.21.15.27:0/2692159970 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cf40039c0 con 0x7f9d0007de90 2024-06-06T08:28:24.740 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9cfdffb700 1 -- 172.21.15.27:0/2692159970 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9cf4011e50 con 0x7f9d0007de90 2024-06-06T08:28:24.740 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0019e8f0 con 0x7f9d0007de90 2024-06-06T08:28:24.740 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.832+0000 7f9cfdffb700 1 -- 172.21.15.27:0/2692159970 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cf401a5c0 con 0x7f9d0007de90 2024-06-06T08:28:24.740 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.833+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d001a15e0 con 0x7f9d0007de90 2024-06-06T08:28:24.740 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.833+0000 7f9cfdffb700 1 -- 172.21.15.27:0/2692159970 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f9cf401a870 con 0x7f9d0007de90 2024-06-06T08:28:24.740 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.834+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d00063af0 con 0x7f9d0007de90 2024-06-06T08:28:24.740 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.834+0000 7f9cfdffb700 1 --2- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9cf0039000 0x7f9cf003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:24.740 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.834+0000 7f9cfdffb700 1 -- 172.21.15.27:0/2692159970 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9cf404d5a0 con 0x7f9d0007de90 2024-06-06T08:28:24.740 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.834+0000 7f9cfffff700 1 --2- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9cf0039000 0x7f9cf003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.834+0000 7f9cfffff700 1 --2- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9cf0039000 0x7f9cf003b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006fd0 tx=0x7f9cec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:23.839+0000 7f9cfdffb700 1 -- 172.21.15.27:0/2692159970 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9cf402cc00 con 0x7f9d0007de90 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.104+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f9d001a17b0 con 0x7f9d0007de90 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.156+0000 7f9cfdffb700 1 -- 172.21.15.27:0/2692159970 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f9d001a17b0 con 0x7f9d0007de90 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.157+0000 7f9cfdffb700 1 -- 172.21.15.27:0/2692159970 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f9cf40248f0 con 0x7f9d0007de90 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.158+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9cf0039000 msgr2=0x7f9cf003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.158+0000 7f9d06fa3700 1 --2- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9cf0039000 0x7f9cf003b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006fd0 tx=0x7f9cec006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.158+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 msgr2=0x7f9d0019e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.158+0000 7f9d06fa3700 1 --2- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0019e120 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4011670 tx=0x7f9cf4011b60 comp rx=0 tx=0).stop 2024-06-06T08:28:24.741 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.158+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 shutdown_connections 2024-06-06T08:28:24.742 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.158+0000 7f9d06fa3700 1 --2- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f9cf0039000 0x7f9cf003b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:24.742 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.158+0000 7f9d06fa3700 1 --2- 172.21.15.27:0/2692159970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0007de90 0x7f9d0019e120 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:24.742 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.159+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 >> 172.21.15.27:0/2692159970 conn(0x7f9d00103570 msgr2=0x7f9d00104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:24.742 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.159+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 shutdown_connections 2024-06-06T08:28:24.742 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:24.159+0000 7f9d06fa3700 1 -- 172.21.15.27:0/2692159970 wait complete. 2024-06-06T08:28:24.980 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:24 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:24.962+0000 7f3c3cb94200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T08:28:25.382 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:25 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2692159970' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-06T08:28:25.382 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:25 smithi027 ceph-mon[32335]: mgrmap e5: x(active, since 5s) 2024-06-06T08:28:25.382 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:25 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:25.054+0000 7f3c3cb94200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T08:28:26.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-06-06T08:28:26.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-06T08:28:26.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T08:28:26.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T08:28:26.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T08:28:26.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-06-06T08:28:26.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.176+0000 7fe939d6c700 1 Processor -- start 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.176+0000 7fe939d6c700 1 -- start start 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.176+0000 7fe939d6c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe934108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.176+0000 7fe939d6c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe934108960 con 0x7fe934107f70 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.176+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe934108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.176+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe934108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44574/0 (socket says 172.21.15.27:44574) 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.176+0000 7fe9337fe700 1 -- 172.21.15.27:0/3154697130 learned_addr learned my addr 172.21.15.27:0/3154697130 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.177+0000 7fe9337fe700 1 -- 172.21.15.27:0/3154697130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe934109170 con 0x7fe934107f70 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.177+0000 7fe9337fe700 1 --2- 172.21.15.27:0/3154697130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe934108390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9240098a0 tx=0x7fe924009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bea47473433f885 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.177+0000 7fe9327fc700 1 -- 172.21.15.27:0/3154697130 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924004160 con 0x7fe934107f70 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.177+0000 7fe9327fc700 1 -- 172.21.15.27:0/3154697130 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe924004300 con 0x7fe934107f70 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.177+0000 7fe9327fc700 1 -- 172.21.15.27:0/3154697130 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924011b50 con 0x7fe934107f70 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.178+0000 7fe939d6c700 1 -- 172.21.15.27:0/3154697130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 msgr2=0x7fe934108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:26.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.178+0000 7fe939d6c700 1 --2- 172.21.15.27:0/3154697130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe934108390 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9240098a0 tx=0x7fe924009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.178+0000 7fe939d6c700 1 -- 172.21.15.27:0/3154697130 shutdown_connections 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.178+0000 7fe939d6c700 1 --2- 172.21.15.27:0/3154697130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe934108390 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.178+0000 7fe939d6c700 1 -- 172.21.15.27:0/3154697130 >> 172.21.15.27:0/3154697130 conn(0x7fe9341034f0 msgr2=0x7fe934105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.178+0000 7fe939d6c700 1 -- 172.21.15.27:0/3154697130 shutdown_connections 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.178+0000 7fe939d6c700 1 -- 172.21.15.27:0/3154697130 wait complete. 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.178+0000 7fe939d6c700 1 Processor -- start 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.179+0000 7fe939d6c700 1 -- start start 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.179+0000 7fe939d6c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe93419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.179+0000 7fe939d6c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410c520 con 0x7fe934107f70 2024-06-06T08:28:26.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.179+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe93419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.179+0000 7fe9337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe93419bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44590/0 (socket says 172.21.15.27:44590) 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.179+0000 7fe9337fe700 1 -- 172.21.15.27:0/684994314 learned_addr learned my addr 172.21.15.27:0/684994314 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.179+0000 7fe9337fe700 1 -- 172.21.15.27:0/684994314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93419c4d0 con 0x7fe934107f70 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.179+0000 7fe9337fe700 1 --2- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe93419bf90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe924000c00 tx=0x7fe924011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.180+0000 7fe930ff9700 1 -- 172.21.15.27:0/684994314 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924003b20 con 0x7fe934107f70 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.180+0000 7fe930ff9700 1 -- 172.21.15.27:0/684994314 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe924003cc0 con 0x7fe934107f70 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.180+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93419c760 con 0x7fe934107f70 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.180+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93419f450 con 0x7fe934107f70 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.180+0000 7fe930ff9700 1 -- 172.21.15.27:0/684994314 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924024910 con 0x7fe934107f70 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.181+0000 7fe930ff9700 1 -- 172.21.15.27:0/684994314 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fe924022020 con 0x7fe934107f70 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.181+0000 7fe930ff9700 1 --2- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7fe91c039050 0x7fe91c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.181+0000 7fe930ff9700 1 -- 172.21.15.27:0/684994314 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe924011e00 con 0x7fe934107f70 2024-06-06T08:28:26.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.181+0000 7fe932ffd700 1 -- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7fe91c039050 msgr2=0x7fe91c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/897818490 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.181+0000 7fe932ffd700 1 --2- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7fe91c039050 0x7fe91c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.181+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe934063af0 con 0x7fe934107f70 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.186+0000 7fe930ff9700 1 -- 172.21.15.27:0/684994314 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe92400b3e0 con 0x7fe934107f70 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.382+0000 7fe932ffd700 1 -- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7fe91c039050 msgr2=0x7fe91c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/897818490 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.382+0000 7fe932ffd700 1 --2- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7fe91c039050 0x7fe91c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.465+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe93419f620 con 0x7fe934107f70 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.466+0000 7fe930ff9700 1 -- 172.21.15.27:0/684994314 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fe92404cc00 con 0x7fe934107f70 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.474+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7fe91c039050 msgr2=0x7fe91c03b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.474+0000 7fe939d6c700 1 --2- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7fe91c039050 0x7fe91c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.474+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 msgr2=0x7fe93419bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.474+0000 7fe939d6c700 1 --2- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe93419bf90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe924000c00 tx=0x7fe924011c10 comp rx=0 tx=0).stop 2024-06-06T08:28:26.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.475+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 shutdown_connections 2024-06-06T08:28:26.106 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.475+0000 7fe939d6c700 1 --2- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7fe91c039050 0x7fe91c03b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:26.106 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.475+0000 7fe939d6c700 1 --2- 172.21.15.27:0/684994314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe934107f70 0x7fe93419bf90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:26.106 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.475+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 >> 172.21.15.27:0/684994314 conn(0x7fe9341034f0 msgr2=0x7fe9341040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:26.106 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.475+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 shutdown_connections 2024-06-06T08:28:26.106 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:25.475+0000 7fe939d6c700 1 -- 172.21.15.27:0/684994314 wait complete. 2024-06-06T08:28:26.106 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-06-06T08:28:26.106 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 5... 2024-06-06T08:28:26.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:26 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/684994314' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T08:28:26.453 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:26 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:26.328+0000 7f3c3cb94200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T08:28:26.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:26 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:26.510+0000 7f3c3cb94200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T08:28:26.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:26 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:26.689+0000 7f3c3cb94200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T08:28:27.087 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:26 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:26.991+0000 7f3c3cb94200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T08:28:27.087 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:27 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:27.087+0000 7f3c3cb94200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T08:28:27.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:27 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:27.298+0000 7f3c3cb94200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T08:28:27.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:27 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:27.501+0000 7f3c3cb94200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:28:28.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:28 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:28.135+0000 7f3c3cb94200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T08:28:28.887 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:28 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:28.576+0000 7f3c3cb94200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T08:28:28.887 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:28 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:28.677+0000 7f3c3cb94200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T08:28:28.887 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:28 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:28.888+0000 7f3c3cb94200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T08:28:29.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:29 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:29.435+0000 7f3c3cb94200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T08:28:29.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:29 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:29.531+0000 7f3c3cb94200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T08:28:30.143 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:29 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:29.785+0000 7f3c3cb94200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T08:28:30.143 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:29 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:29.924+0000 7f3c3cb94200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T08:28:30.143 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:30.023+0000 7f3c3cb94200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T08:28:30.143 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:30.144+0000 7f3c3cb94200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T08:28:30.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:30.455+0000 7f3c3cb94200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T08:28:31.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:31.085+0000 7f3c3cb94200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T08:28:31.364 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:31.312+0000 7f3c3cb94200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T08:28:31.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: Active manager daemon x restarted 2024-06-06T08:28:31.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: Activating manager daemon x 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: osdmap e2: 0 total, 0 up, 0 in 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: mgrmap e6: x(active, starting, since 0.00343652s) 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: Manager daemon x is now available 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:28:31.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:31 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T08:28:32.648 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:32 smithi027 ceph-mon[32335]: Found migration_current of "None". Setting to last migration. 2024-06-06T08:28:32.648 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:32 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T08:28:32.649 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:32 smithi027 ceph-mon[32335]: mgrmap e7: x(active, since 1.00537s) 2024-06-06T08:28:32.907 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.558+0000 7f5a241b7700 1 Processor -- start 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.559+0000 7f5a241b7700 1 -- start start 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.559+0000 7f5a241b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c108400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.559+0000 7f5a241b7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1089d0 con 0x7f5a1c107fe0 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.559+0000 7f5a21f53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c108400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.559+0000 7f5a21f53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c108400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44594/0 (socket says 172.21.15.27:44594) 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.559+0000 7f5a21f53700 1 -- 172.21.15.27:0/2641906480 learned_addr learned my addr 172.21.15.27:0/2641906480 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:32.908 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.559+0000 7f5a21f53700 1 -- 172.21.15.27:0/2641906480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c1091e0 con 0x7f5a1c107fe0 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.559+0000 7f5a21f53700 1 --2- 172.21.15.27:0/2641906480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c108400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098a0 tx=0x7f5a18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28dcf9b0e3a66d61 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.560+0000 7f5a20f51700 1 -- 172.21.15.27:0/2641906480 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a18004160 con 0x7f5a1c107fe0 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.560+0000 7f5a20f51700 1 -- 172.21.15.27:0/2641906480 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a18004300 con 0x7f5a1c107fe0 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.560+0000 7f5a20f51700 1 -- 172.21.15.27:0/2641906480 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a18011b50 con 0x7f5a1c107fe0 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.560+0000 7f5a241b7700 1 -- 172.21.15.27:0/2641906480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 msgr2=0x7f5a1c108400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.560+0000 7f5a241b7700 1 --2- 172.21.15.27:0/2641906480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c108400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098a0 tx=0x7f5a18009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.560+0000 7f5a241b7700 1 -- 172.21.15.27:0/2641906480 shutdown_connections 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.560+0000 7f5a241b7700 1 --2- 172.21.15.27:0/2641906480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c108400 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.560+0000 7f5a241b7700 1 -- 172.21.15.27:0/2641906480 >> 172.21.15.27:0/2641906480 conn(0x7f5a1c1035a0 msgr2=0x7f5a1c1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.561+0000 7f5a241b7700 1 -- 172.21.15.27:0/2641906480 shutdown_connections 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.561+0000 7f5a241b7700 1 -- 172.21.15.27:0/2641906480 wait complete. 2024-06-06T08:28:32.909 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.561+0000 7f5a241b7700 1 Processor -- start 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.561+0000 7f5a241b7700 1 -- start start 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.561+0000 7f5a241b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.561+0000 7f5a241b7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c10c590 con 0x7f5a1c107fe0 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a21f53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a21f53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c19bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44598/0 (socket says 172.21.15.27:44598) 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a21f53700 1 -- 172.21.15.27:0/3260600587 learned_addr learned my addr 172.21.15.27:0/3260600587 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a21f53700 1 -- 172.21.15.27:0/3260600587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c19c520 con 0x7f5a1c107fe0 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a21f53700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c19bfe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5a18000c00 tx=0x7f5a18011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a18003b20 con 0x7f5a1c107fe0 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a18003cc0 con 0x7f5a1c107fe0 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a241b7700 1 -- 172.21.15.27:0/3260600587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c19c7b0 con 0x7f5a1c107fe0 2024-06-06T08:28:32.910 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a18024910 con 0x7f5a1c107fe0 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.562+0000 7f5a241b7700 1 -- 172.21.15.27:0/3260600587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c19f4a0 con 0x7f5a1c107fe0 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.563+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f5a18022020 con 0x7f5a1c107fe0 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.564+0000 7f5a12ffd700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 0x7f5a0803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.564+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 --> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5a0803bc20 con 0x7f5a08039050 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.564+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5a18011e00 con 0x7f5a1c107fe0 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.564+0000 7f5a21752700 1 -- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 msgr2=0x7f5a0803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/897818490 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.564+0000 7f5a21752700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 0x7f5a0803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.765+0000 7f5a21752700 1 -- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 msgr2=0x7f5a0803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/897818490 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:26.765+0000 7f5a21752700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 0x7f5a0803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:27.166+0000 7f5a21752700 1 -- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 msgr2=0x7f5a0803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/897818490 2024-06-06T08:28:32.911 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:27.166+0000 7f5a21752700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 0x7f5a0803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:27.968+0000 7f5a21752700 1 -- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 msgr2=0x7f5a0803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/897818490 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:27.968+0000 7f5a21752700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 0x7f5a0803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:29.569+0000 7f5a21752700 1 -- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 msgr2=0x7f5a0803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/897818490 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:29.569+0000 7f5a21752700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 0x7f5a0803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:31.316+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f5a1801acd0 con 0x7f5a1c107fe0 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:31.316+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 msgr2=0x7f5a0803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:31.316+0000 7f5a12ffd700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 0x7f5a0803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.316+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f5a1804ea50 con 0x7f5a1c107fe0 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.317+0000 7f5a12ffd700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f5a0803c980 0x7f5a0803ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.317+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5a1800af30 con 0x7f5a0803c980 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.317+0000 7f5a21752700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f5a0803c980 0x7f5a0803ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.318+0000 7f5a21752700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f5a0803c980 0x7f5a0803ed70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c003a10 tx=0x7f5a0c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:32.912 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.319+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 <== mgr.14120 v2:172.21.15.27:6800/2182206949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f5a1800af30 con 0x7f5a0803c980 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.326+0000 7f5a241b7700 1 -- 172.21.15.27:0/3260600587 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5a1c10c3e0 con 0x7f5a0803c980 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.326+0000 7f5a12ffd700 1 -- 172.21.15.27:0/3260600587 <== mgr.14120 v2:172.21.15.27:6800/2182206949 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5a1c10c3e0 con 0x7f5a0803c980 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 -- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f5a0803c980 msgr2=0x7f5a0803ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f5a0803c980 0x7f5a0803ed70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c003a10 tx=0x7f5a0c009290 comp rx=0 tx=0).stop 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 -- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 msgr2=0x7f5a1c19bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c19bfe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5a18000c00 tx=0x7f5a18011c10 comp rx=0 tx=0).stop 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 -- 172.21.15.27:0/3260600587 shutdown_connections 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f5a0803c980 0x7f5a0803ed70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a1c107fe0 0x7f5a1c19bfe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 --2- 172.21.15.27:0/3260600587 >> [v2:172.21.15.27:6800/897818490,v1:172.21.15.27:6801/897818490] conn(0x7f5a08039050 0x7f5a0803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 -- 172.21.15.27:0/3260600587 >> 172.21.15.27:0/3260600587 conn(0x7f5a1c1035a0 msgr2=0x7f5a1c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 -- 172.21.15.27:0/3260600587 shutdown_connections 2024-06-06T08:28:32.913 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:32.327+0000 7f5a241b7700 1 -- 172.21.15.27:0/3260600587 wait complete. 2024-06-06T08:28:32.914 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 5 is available 2024-06-06T08:28:32.914 INFO:teuthology.orchestra.run.smithi027.stdout:Setting orchestrator backend to cephadm... 2024-06-06T08:28:33.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:33 smithi027 ceph-mon[32335]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-06T08:28:33.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:33 smithi027 ceph-mon[32335]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-06T08:28:34.120 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.315+0000 7f70b3778700 1 Processor -- start 2024-06-06T08:28:34.120 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.316+0000 7f70b3778700 1 -- start start 2024-06-06T08:28:34.120 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.316+0000 7f70b3778700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac106c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:34.120 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.316+0000 7f70b3778700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac07c760 con 0x7f70ac106810 2024-06-06T08:28:34.120 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.316+0000 7f70b1514700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac106c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:34.120 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.316+0000 7f70b1514700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac106c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55614/0 (socket says 172.21.15.27:55614) 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.316+0000 7f70b1514700 1 -- 172.21.15.27:0/939168930 learned_addr learned my addr 172.21.15.27:0/939168930 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.316+0000 7f70b1514700 1 -- 172.21.15.27:0/939168930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac107170 con 0x7f70ac106810 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.316+0000 7f70b1514700 1 --2- 172.21.15.27:0/939168930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac106c30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70a80098a0 tx=0x7f70a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cc278c25426741 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70a3fff700 1 -- 172.21.15.27:0/939168930 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70a8004160 con 0x7f70ac106810 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70a3fff700 1 -- 172.21.15.27:0/939168930 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f70a8004300 con 0x7f70ac106810 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70a3fff700 1 -- 172.21.15.27:0/939168930 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70a8011b50 con 0x7f70ac106810 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70b3778700 1 -- 172.21.15.27:0/939168930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 msgr2=0x7f70ac106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70b3778700 1 --2- 172.21.15.27:0/939168930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac106c30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f70a80098a0 tx=0x7f70a8009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70b3778700 1 -- 172.21.15.27:0/939168930 shutdown_connections 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70b3778700 1 --2- 172.21.15.27:0/939168930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac106c30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70b3778700 1 -- 172.21.15.27:0/939168930 >> 172.21.15.27:0/939168930 conn(0x7f70ac102450 msgr2=0x7f70ac104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:34.121 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70b3778700 1 -- 172.21.15.27:0/939168930 shutdown_connections 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.317+0000 7f70b3778700 1 -- 172.21.15.27:0/939168930 wait complete. 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.318+0000 7f70b3778700 1 Processor -- start 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.318+0000 7f70b3778700 1 -- start start 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.318+0000 7f70b3778700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.318+0000 7f70b3778700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac108200 con 0x7f70ac106810 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.318+0000 7f70b1514700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.318+0000 7f70b1514700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac19e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55624/0 (socket says 172.21.15.27:55624) 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.318+0000 7f70b1514700 1 -- 172.21.15.27:0/269177238 learned_addr learned my addr 172.21.15.27:0/269177238 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.319+0000 7f70b1514700 1 -- 172.21.15.27:0/269177238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac19e670 con 0x7f70ac106810 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.319+0000 7f70b1514700 1 --2- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac19e130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f70a8011b60 tx=0x7f70a8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.319+0000 7f70a27fc700 1 -- 172.21.15.27:0/269177238 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70a8003b20 con 0x7f70ac106810 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.319+0000 7f70a27fc700 1 -- 172.21.15.27:0/269177238 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f70a8003cc0 con 0x7f70ac106810 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.319+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac19e900 con 0x7f70ac106810 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.319+0000 7f70a27fc700 1 -- 172.21.15.27:0/269177238 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70a8024840 con 0x7f70ac106810 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.319+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac1a15f0 con 0x7f70ac106810 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.320+0000 7f70a27fc700 1 -- 172.21.15.27:0/269177238 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f70a8022020 con 0x7f70ac106810 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.320+0000 7f70a27fc700 1 --2- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7098038f80 0x7f709803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.320+0000 7f70a27fc700 1 -- 172.21.15.27:0/269177238 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f70a8011e00 con 0x7f70ac106810 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.320+0000 7f70b0d13700 1 --2- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7098038f80 0x7f709803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.320+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ac063af0 con 0x7f70ac106810 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.321+0000 7f70b0d13700 1 --2- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7098038f80 0x7f709803b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f709c006fd0 tx=0x7f709c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.326+0000 7f70a27fc700 1 -- 172.21.15.27:0/269177238 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f70a800ba30 con 0x7f70ac106810 2024-06-06T08:28:34.123 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.368+0000 7f70a27fc700 1 -- 172.21.15.27:0/269177238 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f70a801a960 con 0x7f70ac106810 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.581+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f70ac02df80 con 0x7f7098038f80 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.589+0000 7f70a27fc700 1 -- 172.21.15.27:0/269177238 <== mgr.14120 v2:172.21.15.27:6800/2182206949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f70ac02df80 con 0x7f7098038f80 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.591+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7098038f80 msgr2=0x7f709803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.591+0000 7f70b3778700 1 --2- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7098038f80 0x7f709803b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f709c006fd0 tx=0x7f709c006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.591+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 msgr2=0x7f70ac19e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.591+0000 7f70b3778700 1 --2- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac19e130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f70a8011b60 tx=0x7f70a8011c40 comp rx=0 tx=0).stop 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.591+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 shutdown_connections 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.591+0000 7f70b3778700 1 --2- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7098038f80 0x7f709803b440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.591+0000 7f70b3778700 1 --2- 172.21.15.27:0/269177238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70ac106810 0x7f70ac19e130 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:34.124 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.591+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 >> 172.21.15.27:0/269177238 conn(0x7f70ac102450 msgr2=0x7f70ac194d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:34.125 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.592+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 shutdown_connections 2024-06-06T08:28:34.125 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:33.592+0000 7f70b3778700 1 -- 172.21.15.27:0/269177238 wait complete. 2024-06-06T08:28:34.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:34 smithi027 ceph-mon[32335]: mgrmap e8: x(active, since 2s) 2024-06-06T08:28:34.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:34 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:34.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:34 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:34.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:34 smithi027 ceph-mon[32335]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:28:34.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:34 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:34.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:34 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:28:35.419 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-06T08:28:35.419 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.536+0000 7faa10de2700 1 Processor -- start 2024-06-06T08:28:35.419 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.537+0000 7faa10de2700 1 -- start start 2024-06-06T08:28:35.419 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.537+0000 7faa10de2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c10ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:35.419 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.537+0000 7faa10de2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c07be50 con 0x7faa0c108a80 2024-06-06T08:28:35.419 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.537+0000 7faa0a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c10ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:35.419 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.537+0000 7faa0a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c10ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55636/0 (socket says 172.21.15.27:55636) 2024-06-06T08:28:35.419 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.537+0000 7faa0a59c700 1 -- 172.21.15.27:0/3055352923 learned_addr learned my addr 172.21.15.27:0/3055352923 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.537+0000 7faa0a59c700 1 -- 172.21.15.27:0/3055352923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c10b3b0 con 0x7faa0c108a80 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.537+0000 7faa0a59c700 1 --2- 172.21.15.27:0/3055352923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c10ae70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0098a0 tx=0x7fa9fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f972b297d04970 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa09d9b700 1 -- 172.21.15.27:0/3055352923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9fc004160 con 0x7faa0c108a80 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa09d9b700 1 -- 172.21.15.27:0/3055352923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9fc004300 con 0x7faa0c108a80 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa09d9b700 1 -- 172.21.15.27:0/3055352923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9fc011b50 con 0x7faa0c108a80 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa10de2700 1 -- 172.21.15.27:0/3055352923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 msgr2=0x7faa0c10ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa10de2700 1 --2- 172.21.15.27:0/3055352923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c10ae70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0098a0 tx=0x7fa9fc009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa10de2700 1 -- 172.21.15.27:0/3055352923 shutdown_connections 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa10de2700 1 --2- 172.21.15.27:0/3055352923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c10ae70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa10de2700 1 -- 172.21.15.27:0/3055352923 >> 172.21.15.27:0/3055352923 conn(0x7faa0c102470 msgr2=0x7faa0c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa10de2700 1 -- 172.21.15.27:0/3055352923 shutdown_connections 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.538+0000 7faa10de2700 1 -- 172.21.15.27:0/3055352923 wait complete. 2024-06-06T08:28:35.420 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.539+0000 7faa10de2700 1 Processor -- start 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.539+0000 7faa10de2700 1 -- start start 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.539+0000 7faa10de2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c108300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.539+0000 7faa10de2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c10c370 con 0x7faa0c108a80 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.539+0000 7faa0a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c108300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.539+0000 7faa0a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c108300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55640/0 (socket says 172.21.15.27:55640) 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.539+0000 7faa0a59c700 1 -- 172.21.15.27:0/3298826045 learned_addr learned my addr 172.21.15.27:0/3298826045 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.540+0000 7faa0a59c700 1 -- 172.21.15.27:0/3298826045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c108840 con 0x7faa0c108a80 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.540+0000 7faa0a59c700 1 --2- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c108300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc011670 tx=0x7fa9fc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.540+0000 7faa037fe700 1 -- 172.21.15.27:0/3298826045 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9fc0039c0 con 0x7faa0c108a80 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.540+0000 7faa037fe700 1 -- 172.21.15.27:0/3298826045 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9fc011e50 con 0x7faa0c108a80 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.540+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c106a30 con 0x7faa0c108a80 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.540+0000 7faa037fe700 1 -- 172.21.15.27:0/3298826045 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9fc01a5c0 con 0x7faa0c108a80 2024-06-06T08:28:35.421 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.540+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c106f70 con 0x7faa0c108a80 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.541+0000 7faa037fe700 1 -- 172.21.15.27:0/3298826045 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa9fc01a870 con 0x7faa0c108a80 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.541+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c063af0 con 0x7faa0c108a80 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.541+0000 7faa037fe700 1 --2- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa9f8039050 0x7fa9f803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.541+0000 7faa037fe700 1 -- 172.21.15.27:0/3298826045 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa9fc04d310 con 0x7faa0c108a80 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.541+0000 7faa03fff700 1 --2- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa9f8039050 0x7fa9f803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.542+0000 7faa03fff700 1 --2- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa9f8039050 0x7fa9f803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4006fd0 tx=0x7fa9f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.547+0000 7faa037fe700 1 -- 172.21.15.27:0/3298826045 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa9fc019c70 con 0x7faa0c108a80 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.753+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7faa0c1077d0 con 0x7fa9f8039050 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.754+0000 7faa037fe700 1 -- 172.21.15.27:0/3298826045 <== mgr.14120 v2:172.21.15.27:6800/2182206949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7faa0c1077d0 con 0x7fa9f8039050 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa9f8039050 msgr2=0x7fa9f803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 --2- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa9f8039050 0x7fa9f803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4006fd0 tx=0x7fa9f4006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 msgr2=0x7faa0c108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:35.422 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 --2- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c108300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc011670 tx=0x7fa9fc011b60 comp rx=0 tx=0).stop 2024-06-06T08:28:35.423 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 shutdown_connections 2024-06-06T08:28:35.423 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 --2- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa9f8039050 0x7fa9f803b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:35.423 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 --2- 172.21.15.27:0/3298826045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa0c108a80 0x7faa0c108300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:35.423 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 >> 172.21.15.27:0/3298826045 conn(0x7faa0c102470 msgr2=0x7faa0c102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:35.423 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 shutdown_connections 2024-06-06T08:28:35.423 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:34.756+0000 7faa10de2700 1 -- 172.21.15.27:0/3298826045 wait complete. 2024-06-06T08:28:35.423 INFO:teuthology.orchestra.run.smithi027.stdout:Generating ssh key... 2024-06-06T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:35 smithi027 ceph-mon[32335]: [06/Jun/2024:08:28:34] ENGINE Bus STARTING 2024-06-06T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:35 smithi027 ceph-mon[32335]: [06/Jun/2024:08:28:34] ENGINE Serving on http://172.21.15.27:8765 2024-06-06T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:35 smithi027 ceph-mon[32335]: [06/Jun/2024:08:28:34] ENGINE Serving on https://172.21.15.27:7150 2024-06-06T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:35 smithi027 ceph-mon[32335]: [06/Jun/2024:08:28:34] ENGINE Bus STARTED 2024-06-06T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:35 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:35 smithi027 ceph-mon[32335]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:28:36.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: Generating public/private rsa key pair. 2024-06-06T08:28:36.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: Your identification has been saved in /tmp/tmp1l21el8g/key. 2024-06-06T08:28:36.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: Your public key has been saved in /tmp/tmp1l21el8g/key.pub. 2024-06-06T08:28:36.371 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: The key fingerprint is: 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: SHA256:4BW/inDvhMEKBkysvyD/I8b4g2snTwdsvncXySEXLuk ceph-8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: The key's randomart image is: 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: +---[RSA 3072]----+ 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: |.. . | 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: |o. + | 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: |o. . + o | 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: |... o * + . | 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: | .o+. * S + | 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: |o.+..+ E = | 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: |.* +..o + . | 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: |ooO.+. + . | 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: |o+=*o.. o | 2024-06-06T08:28:36.372 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: +----[SHA256]-----+ 2024-06-06T08:28:36.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-mon[32335]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:28:36.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-mon[32335]: Generating ssh key... 2024-06-06T08:28:36.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:36.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:36 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:36.850 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.863+0000 7f7dff145700 1 Processor -- start 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.863+0000 7f7dff145700 1 -- start start 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.863+0000 7f7dff145700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df8108440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.863+0000 7f7dff145700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8108a10 con 0x7f7df8108020 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.864+0000 7f7dfcee1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df8108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.864+0000 7f7dfcee1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df8108440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55642/0 (socket says 172.21.15.27:55642) 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.864+0000 7f7dfcee1700 1 -- 172.21.15.27:0/117855584 learned_addr learned my addr 172.21.15.27:0/117855584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.864+0000 7f7dfcee1700 1 -- 172.21.15.27:0/117855584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df81091d0 con 0x7f7df8108020 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.864+0000 7f7dfcee1700 1 --2- 172.21.15.27:0/117855584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df8108440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0098a0 tx=0x7f7dec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795b669cd9f006d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.864+0000 7f7df77fe700 1 -- 172.21.15.27:0/117855584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dec004160 con 0x7f7df8108020 2024-06-06T08:28:36.851 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.864+0000 7f7df77fe700 1 -- 172.21.15.27:0/117855584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7dec004300 con 0x7f7df8108020 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.864+0000 7f7df77fe700 1 -- 172.21.15.27:0/117855584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dec011b50 con 0x7f7df8108020 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.865+0000 7f7dff145700 1 -- 172.21.15.27:0/117855584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 msgr2=0x7f7df8108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.865+0000 7f7dff145700 1 --2- 172.21.15.27:0/117855584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df8108440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0098a0 tx=0x7f7dec009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.865+0000 7f7dff145700 1 -- 172.21.15.27:0/117855584 shutdown_connections 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.865+0000 7f7dff145700 1 --2- 172.21.15.27:0/117855584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df8108440 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.865+0000 7f7dff145700 1 -- 172.21.15.27:0/117855584 >> 172.21.15.27:0/117855584 conn(0x7f7df8103620 msgr2=0x7f7df8105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.865+0000 7f7dff145700 1 -- 172.21.15.27:0/117855584 shutdown_connections 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.865+0000 7f7dff145700 1 -- 172.21.15.27:0/117855584 wait complete. 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.866+0000 7f7dff145700 1 Processor -- start 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.866+0000 7f7dff145700 1 -- start start 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.866+0000 7f7dff145700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.866+0000 7f7dff145700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7df810c580 con 0x7f7df8108020 2024-06-06T08:28:36.852 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.866+0000 7f7dfcee1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.866+0000 7f7dfcee1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df819bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55648/0 (socket says 172.21.15.27:55648) 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.866+0000 7f7dfcee1700 1 -- 172.21.15.27:0/545434735 learned_addr learned my addr 172.21.15.27:0/545434735 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.866+0000 7f7dfcee1700 1 -- 172.21.15.27:0/545434735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df819c500 con 0x7f7df8108020 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.866+0000 7f7dfcee1700 1 --2- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df819bfc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7dec011670 tx=0x7f7dec011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.867+0000 7f7df5ffb700 1 -- 172.21.15.27:0/545434735 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dec0039c0 con 0x7f7df8108020 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.867+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df819c790 con 0x7f7df8108020 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.867+0000 7f7df5ffb700 1 -- 172.21.15.27:0/545434735 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7dec011e50 con 0x7f7df8108020 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.867+0000 7f7df5ffb700 1 -- 172.21.15.27:0/545434735 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dec01a5c0 con 0x7f7df8108020 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.867+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df819f480 con 0x7f7df8108020 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.868+0000 7f7df5ffb700 1 -- 172.21.15.27:0/545434735 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7dec024430 con 0x7f7df8108020 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.868+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df8063af0 con 0x7f7df8108020 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.868+0000 7f7df5ffb700 1 --2- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7de8039000 0x7f7de803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:36.853 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.868+0000 7f7df5ffb700 1 -- 172.21.15.27:0/545434735 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7dec04d480 con 0x7f7df8108020 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.868+0000 7f7df7fff700 1 --2- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7de8039000 0x7f7de803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.869+0000 7f7df7fff700 1 --2- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7de8039000 0x7f7de803b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7de4006fd0 tx=0x7f7de4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:35.873+0000 7f7df5ffb700 1 -- 172.21.15.27:0/545434735 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7dec02d3f0 con 0x7f7df8108020 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.081+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7df8104890 con 0x7f7de8039000 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.308+0000 7f7df5ffb700 1 -- 172.21.15.27:0/545434735 <== mgr.14120 v2:172.21.15.27:6800/2182206949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7df8104890 con 0x7f7de8039000 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7de8039000 msgr2=0x7f7de803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 --2- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7de8039000 0x7f7de803b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7de4006fd0 tx=0x7f7de4006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 msgr2=0x7f7df819bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 --2- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df819bfc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7dec011670 tx=0x7f7dec011b60 comp rx=0 tx=0).stop 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 shutdown_connections 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 --2- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f7de8039000 0x7f7de803b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 --2- 172.21.15.27:0/545434735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df8108020 0x7f7df819bfc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:36.854 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 >> 172.21.15.27:0/545434735 conn(0x7f7df8103620 msgr2=0x7f7df8104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:36.855 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 shutdown_connections 2024-06-06T08:28:36.855 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:36.310+0000 7f7dff145700 1 -- 172.21.15.27:0/545434735 wait complete. 2024-06-06T08:28:37.965 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN2w5SzjTE2hZXcsQjyC5WMYUZ83Q06jlZCh81b8C3hpflU77tI5tj7lzMgdVUviaTMr3bfhqJmfegufNjsRx0rGUV+c7ALSCyPdBeLLkRC9USxZnI6xJTmR01EwDKoNDRzP7jSW1Lv5MGO1IyvwyyKv4Wptw1XozilO/o+HC5cBMuT/vXw5fUL10e5GPoONz2C2Q6b+Aa1XqpxZ6KE1kty5W5G/kS6jcZD9u6ZqWdWvKRUo8RNwh2jNzrQTSUepRa5til6NGgdQruyA47TNwfbUUYldz49NT3f4MC/FiBBBfUhnQ+dVLzyCTZZDu6UHaM+V/dAP0fDahemFt9OvTD79X45jjx6haVArrGTglPn5Q72rIZugi1wasYv/5dcSbxMvGn1Q6cyI/+tutJumRlr1m7Ag3xruasLUefwyoshV3IYUGdBJp9F8kzbe9k86ZPYUUTwJ+yCbYGtKXUFHMltd8Nfk6Bj5ezZwM6x/vVbF/xhGILLCHTmosNxLts/sM= ceph-8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:28:37.965 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.215+0000 7f640efa3700 1 Processor -- start 2024-06-06T08:28:37.965 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.215+0000 7f640efa3700 1 -- start start 2024-06-06T08:28:37.965 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.215+0000 7f640efa3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f6408108440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:37.965 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.215+0000 7f640efa3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6408108a10 con 0x7f6408108020 2024-06-06T08:28:37.965 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.215+0000 7f640cd3f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f6408108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.215+0000 7f640cd3f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f6408108440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55654/0 (socket says 172.21.15.27:55654) 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.215+0000 7f640cd3f700 1 -- 172.21.15.27:0/1968779751 learned_addr learned my addr 172.21.15.27:0/1968779751 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f640cd3f700 1 -- 172.21.15.27:0/1968779751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64081091d0 con 0x7f6408108020 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f640cd3f700 1 --2- 172.21.15.27:0/1968779751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f6408108440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0098a0 tx=0x7f63fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedb893ec2b45e4c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f64077fe700 1 -- 172.21.15.27:0/1968779751 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63fc004160 con 0x7f6408108020 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f64077fe700 1 -- 172.21.15.27:0/1968779751 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63fc004300 con 0x7f6408108020 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f64077fe700 1 -- 172.21.15.27:0/1968779751 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63fc011b50 con 0x7f6408108020 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f640efa3700 1 -- 172.21.15.27:0/1968779751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 msgr2=0x7f6408108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f640efa3700 1 --2- 172.21.15.27:0/1968779751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f6408108440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0098a0 tx=0x7f63fc009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f640efa3700 1 -- 172.21.15.27:0/1968779751 shutdown_connections 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f640efa3700 1 --2- 172.21.15.27:0/1968779751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f6408108440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:37.966 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.216+0000 7f640efa3700 1 -- 172.21.15.27:0/1968779751 >> 172.21.15.27:0/1968779751 conn(0x7f6408103620 msgr2=0x7f6408105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.217+0000 7f640efa3700 1 -- 172.21.15.27:0/1968779751 shutdown_connections 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.217+0000 7f640efa3700 1 -- 172.21.15.27:0/1968779751 wait complete. 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.217+0000 7f640efa3700 1 Processor -- start 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.217+0000 7f640efa3700 1 -- start start 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.217+0000 7f640efa3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f640807e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.217+0000 7f640efa3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f640810c580 con 0x7f6408108020 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.217+0000 7f640cd3f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f640807e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.217+0000 7f640cd3f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f640807e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55664/0 (socket says 172.21.15.27:55664) 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.217+0000 7f640cd3f700 1 -- 172.21.15.27:0/1184939300 learned_addr learned my addr 172.21.15.27:0/1184939300 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.218+0000 7f640cd3f700 1 -- 172.21.15.27:0/1184939300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640807e960 con 0x7f6408108020 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.218+0000 7f640cd3f700 1 --2- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f640807e420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63fc000c00 tx=0x7f63fc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.218+0000 7f6405ffb700 1 -- 172.21.15.27:0/1184939300 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63fc003b20 con 0x7f6408108020 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.218+0000 7f6405ffb700 1 -- 172.21.15.27:0/1184939300 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63fc003cc0 con 0x7f6408108020 2024-06-06T08:28:37.967 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.218+0000 7f6405ffb700 1 -- 172.21.15.27:0/1184939300 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63fc024910 con 0x7f6408108020 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.218+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640807cb50 con 0x7f6408108020 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.218+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640807d090 con 0x7f6408108020 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.219+0000 7f6405ffb700 1 -- 172.21.15.27:0/1184939300 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f63fc022020 con 0x7f6408108020 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.219+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6408063af0 con 0x7f6408108020 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.219+0000 7f6405ffb700 1 --2- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f63f80390a0 0x7f63f803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.219+0000 7f6405ffb700 1 -- 172.21.15.27:0/1184939300 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f63fc011e00 con 0x7f6408108020 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.220+0000 7f6407fff700 1 --2- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f63f80390a0 0x7f63f803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.220+0000 7f6407fff700 1 --2- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f63f80390a0 0x7f63f803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f63f4006fd0 tx=0x7f63f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.224+0000 7f6405ffb700 1 -- 172.21.15.27:0/1184939300 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f63fc01ad80 con 0x7f6408108020 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.434+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f640807d8f0 con 0x7f63f80390a0 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.434+0000 7f6405ffb700 1 -- 172.21.15.27:0/1184939300 <== mgr.14120 v2:172.21.15.27:6800/2182206949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f640807d8f0 con 0x7f63f80390a0 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f63f80390a0 msgr2=0x7f63f803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 --2- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f63f80390a0 0x7f63f803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f63f4006fd0 tx=0x7f63f4006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:37.968 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 msgr2=0x7f640807e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 --2- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f640807e420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63fc000c00 tx=0x7f63fc011c10 comp rx=0 tx=0).stop 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 shutdown_connections 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 --2- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f63f80390a0 0x7f63f803b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 --2- 172.21.15.27:0/1184939300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6408108020 0x7f640807e420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 >> 172.21.15.27:0/1184939300 conn(0x7f6408103620 msgr2=0x7f6408104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 shutdown_connections 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:37.437+0000 7f640efa3700 1 -- 172.21.15.27:0/1184939300 wait complete. 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:Adding key to root@localhost authorized_keys... 2024-06-06T08:28:37.969 INFO:teuthology.orchestra.run.smithi027.stdout:Adding host smithi027... 2024-06-06T08:28:38.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:38 smithi027 ceph-mon[32335]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:28:39.516 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:39 smithi027 ceph-mon[32335]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:28:39.517 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:39 smithi027 ceph-mon[32335]: Deploying cephadm binary to smithi027 2024-06-06T08:28:43.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Added host 'smithi027' with addr '172.21.15.27' 2024-06-06T08:28:43.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.369+0000 7fa3d4ba1700 1 Processor -- start 2024-06-06T08:28:43.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.369+0000 7fa3d4ba1700 1 -- start start 2024-06-06T08:28:43.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.369+0000 7fa3d4ba1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d0108210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:43.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.369+0000 7fa3d4ba1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d01087e0 con 0x7fa3d0107df0 2024-06-06T08:28:43.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.369+0000 7fa3cf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d0108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:43.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.369+0000 7fa3cf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d0108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55674/0 (socket says 172.21.15.27:55674) 2024-06-06T08:28:43.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.369+0000 7fa3cf7fe700 1 -- 172.21.15.27:0/2792558730 learned_addr learned my addr 172.21.15.27:0/2792558730 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:43.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.369+0000 7fa3cf7fe700 1 -- 172.21.15.27:0/2792558730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d0108ff0 con 0x7fa3d0107df0 2024-06-06T08:28:43.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.369+0000 7fa3cf7fe700 1 --2- 172.21.15.27:0/2792558730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d0108210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00098a0 tx=0x7fa3c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732c21fe6028cd73 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:43.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3ce7fc700 1 -- 172.21.15.27:0/2792558730 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3c0004160 con 0x7fa3d0107df0 2024-06-06T08:28:43.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3ce7fc700 1 -- 172.21.15.27:0/2792558730 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3c0004300 con 0x7fa3d0107df0 2024-06-06T08:28:43.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3ce7fc700 1 -- 172.21.15.27:0/2792558730 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3c0011b50 con 0x7fa3d0107df0 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/2792558730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 msgr2=0x7fa3d0108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3d4ba1700 1 --2- 172.21.15.27:0/2792558730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d0108210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00098a0 tx=0x7fa3c0009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/2792558730 shutdown_connections 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3d4ba1700 1 --2- 172.21.15.27:0/2792558730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d0108210 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/2792558730 >> 172.21.15.27:0/2792558730 conn(0x7fa3d0103390 msgr2=0x7fa3d01057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/2792558730 shutdown_connections 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.370+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/2792558730 wait complete. 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.371+0000 7fa3d4ba1700 1 Processor -- start 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.371+0000 7fa3d4ba1700 1 -- start start 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.371+0000 7fa3d4ba1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3d4ba1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d010c3a0 con 0x7fa3d0107df0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3cf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3cf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d019be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55690/0 (socket says 172.21.15.27:55690) 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3cf7fe700 1 -- 172.21.15.27:0/4084142830 learned_addr learned my addr 172.21.15.27:0/4084142830 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3cf7fe700 1 -- 172.21.15.27:0/4084142830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d019c3b0 con 0x7fa3d0107df0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3cf7fe700 1 --2- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d019be70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0011850 tx=0x7fa3c0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3ccff9700 1 -- 172.21.15.27:0/4084142830 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3c0003af0 con 0x7fa3d0107df0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d019c640 con 0x7fa3d0107df0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3ccff9700 1 -- 172.21.15.27:0/4084142830 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3c0020070 con 0x7fa3d0107df0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3ccff9700 1 -- 172.21.15.27:0/4084142830 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3c00195a0 con 0x7fa3d0107df0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.372+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d019f330 con 0x7fa3d0107df0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.373+0000 7fa3ccff9700 1 -- 172.21.15.27:0/4084142830 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa3c0019850 con 0x7fa3d0107df0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.373+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d0063af0 con 0x7fa3d0107df0 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.373+0000 7fa3ccff9700 1 --2- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa3b80390a0 0x7fa3b803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.373+0000 7fa3ccff9700 1 -- 172.21.15.27:0/4084142830 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa3c0011e00 con 0x7fa3d0107df0 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.374+0000 7fa3ceffd700 1 --2- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa3b80390a0 0x7fa3b803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.374+0000 7fa3ceffd700 1 --2- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa3b80390a0 0x7fa3b803b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4006fd0 tx=0x7fa3c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.378+0000 7fa3ccff9700 1 -- 172.21.15.27:0/4084142830 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa3c0003c90 con 0x7fa3d0107df0 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:38.589+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}) -- 0x7fa3d0104680 con 0x7fa3b80390a0 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.851+0000 7fa3ccff9700 1 -- 172.21.15.27:0/4084142830 <== mgr.14120 v2:172.21.15.27:6800/2182206949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa3d0104680 con 0x7fa3b80390a0 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.854+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa3b80390a0 msgr2=0x7fa3b803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.854+0000 7fa3d4ba1700 1 --2- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa3b80390a0 0x7fa3b803b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4006fd0 tx=0x7fa3c4006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.854+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 msgr2=0x7fa3d019be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.854+0000 7fa3d4ba1700 1 --2- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d019be70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0011850 tx=0x7fa3c0011c10 comp rx=0 tx=0).stop 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.854+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 shutdown_connections 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.854+0000 7fa3d4ba1700 1 --2- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fa3b80390a0 0x7fa3b803b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.854+0000 7fa3d4ba1700 1 --2- 172.21.15.27:0/4084142830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3d0107df0 0x7fa3d019be70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:43.469 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.854+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 >> 172.21.15.27:0/4084142830 conn(0x7fa3d0103390 msgr2=0x7fa3d0103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:43.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.855+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 shutdown_connections 2024-06-06T08:28:43.470 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:42.855+0000 7fa3d4ba1700 1 -- 172.21.15.27:0/4084142830 wait complete. 2024-06-06T08:28:43.470 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mon service... 2024-06-06T08:28:44.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:43 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:44.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:43 smithi027 ceph-mon[32335]: Added host smithi027 2024-06-06T08:28:44.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:43 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:28:45.400 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:45 smithi027 ceph-mon[32335]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:28:45.400 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:45 smithi027 ceph-mon[32335]: Saving service mon spec with placement count:5 2024-06-06T08:28:45.400 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:45 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.891+0000 7f311e1db700 1 Processor -- start 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.891+0000 7f311e1db700 1 -- start start 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.891+0000 7f311e1db700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f31181072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.891+0000 7f311e1db700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f31181077f0 con 0x7f3118106e90 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.892+0000 7f31177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f31181072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.892+0000 7f31177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f31181072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41316/0 (socket says 172.21.15.27:41316) 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.892+0000 7f31177fe700 1 -- 172.21.15.27:0/463382989 learned_addr learned my addr 172.21.15.27:0/463382989 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.892+0000 7f31177fe700 1 -- 172.21.15.27:0/463382989 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3118108080 con 0x7f3118106e90 2024-06-06T08:28:45.821 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.892+0000 7f31177fe700 1 --2- 172.21.15.27:0/463382989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f31181072b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3108009e80 tx=0x7f310800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af6848fd1191378 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.892+0000 7f31167fc700 1 -- 172.21.15.27:0/463382989 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3108017070 con 0x7f3118106e90 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.892+0000 7f31167fc700 1 -- 172.21.15.27:0/463382989 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f310800b8c0 con 0x7f3118106e90 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.892+0000 7f31167fc700 1 -- 172.21.15.27:0/463382989 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31080045c0 con 0x7f3118106e90 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.893+0000 7f311e1db700 1 -- 172.21.15.27:0/463382989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 msgr2=0x7f31181072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.893+0000 7f311e1db700 1 --2- 172.21.15.27:0/463382989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f31181072b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3108009e80 tx=0x7f310800b330 comp rx=0 tx=0).stop 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.893+0000 7f311e1db700 1 -- 172.21.15.27:0/463382989 shutdown_connections 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.893+0000 7f311e1db700 1 --2- 172.21.15.27:0/463382989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f31181072b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.893+0000 7f311e1db700 1 -- 172.21.15.27:0/463382989 >> 172.21.15.27:0/463382989 conn(0x7f31181024c0 msgr2=0x7f31181048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.893+0000 7f311e1db700 1 -- 172.21.15.27:0/463382989 shutdown_connections 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.893+0000 7f311e1db700 1 -- 172.21.15.27:0/463382989 wait complete. 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.893+0000 7f311e1db700 1 Processor -- start 2024-06-06T08:28:45.822 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.894+0000 7f311e1db700 1 -- start start 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.894+0000 7f311e1db700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f311819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.894+0000 7f311e1db700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f311810b340 con 0x7f3118106e90 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.894+0000 7f31177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f311819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.894+0000 7f31177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f311819ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41324/0 (socket says 172.21.15.27:41324) 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.894+0000 7f31177fe700 1 -- 172.21.15.27:0/2463140809 learned_addr learned my addr 172.21.15.27:0/2463140809 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.894+0000 7f31177fe700 1 -- 172.21.15.27:0/2463140809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311819b3c0 con 0x7f3118106e90 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.894+0000 7f31177fe700 1 --2- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f311819ae80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3108009530 tx=0x7f310800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.895+0000 7f3114ff9700 1 -- 172.21.15.27:0/2463140809 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3108020020 con 0x7f3118106e90 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.895+0000 7f3114ff9700 1 -- 172.21.15.27:0/2463140809 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3108003b50 con 0x7f3118106e90 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.895+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311819b650 con 0x7f3118106e90 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.895+0000 7f3114ff9700 1 -- 172.21.15.27:0/2463140809 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3108017040 con 0x7f3118106e90 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.895+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311819e340 con 0x7f3118106e90 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.895+0000 7f3114ff9700 1 -- 172.21.15.27:0/2463140809 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3108023020 con 0x7f3118106e90 2024-06-06T08:28:45.823 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.896+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f311810b4c0 con 0x7f3118106e90 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.896+0000 7f3114ff9700 1 --2- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f31040390a0 0x7f310403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.896+0000 7f3114ff9700 1 -- 172.21.15.27:0/2463140809 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f310800f050 con 0x7f3118106e90 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.896+0000 7f3116ffd700 1 --2- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f31040390a0 0x7f310403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.896+0000 7f3116ffd700 1 --2- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f31040390a0 0x7f310403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3100006fd0 tx=0x7f3100006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:43.901+0000 7f3114ff9700 1 -- 172.21.15.27:0/2463140809 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f310802a020 con 0x7f3118106e90 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.124+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f31181036c0 con 0x7f31040390a0 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.130+0000 7f3114ff9700 1 -- 172.21.15.27:0/2463140809 <== mgr.14120 v2:172.21.15.27:6800/2182206949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f31181036c0 con 0x7f31040390a0 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f31040390a0 msgr2=0x7f310403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 --2- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f31040390a0 0x7f310403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3100006fd0 tx=0x7f3100006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 msgr2=0x7f311819ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 --2- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f311819ae80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3108009530 tx=0x7f310800bd50 comp rx=0 tx=0).stop 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 shutdown_connections 2024-06-06T08:28:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 --2- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f31040390a0 0x7f310403b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:45.825 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 --2- 172.21.15.27:0/2463140809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3118106e90 0x7f311819ae80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:45.825 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 >> 172.21.15.27:0/2463140809 conn(0x7f31181024c0 msgr2=0x7f3118102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:45.825 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 shutdown_connections 2024-06-06T08:28:45.825 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:44.132+0000 7f311e1db700 1 -- 172.21.15.27:0/2463140809 wait complete. 2024-06-06T08:28:45.825 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mgr service... 2024-06-06T08:28:46.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:46 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:46.812 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:46 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:47.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-06T08:28:47.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.246+0000 7fdba5626700 1 Processor -- start 2024-06-06T08:28:47.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.246+0000 7fdba5626700 1 -- start start 2024-06-06T08:28:47.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.247+0000 7fdba5626700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba007cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:47.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.247+0000 7fdba5626700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdba007d160 con 0x7fdba007e730 2024-06-06T08:28:47.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.247+0000 7fdb9effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba007cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.247+0000 7fdb9effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba007cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41334/0 (socket says 172.21.15.27:41334) 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.247+0000 7fdb9effd700 1 -- 172.21.15.27:0/2835585009 learned_addr learned my addr 172.21.15.27:0/2835585009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.247+0000 7fdb9effd700 1 -- 172.21.15.27:0/2835585009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba007d2e0 con 0x7fdba007e730 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.247+0000 7fdb9effd700 1 --2- 172.21.15.27:0/2835585009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba007cb90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb900098a0 tx=0x7fdb90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22420249ae74fae server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdb9dffb700 1 -- 172.21.15.27:0/2835585009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb90004160 con 0x7fdba007e730 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdb9dffb700 1 -- 172.21.15.27:0/2835585009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb90004300 con 0x7fdba007e730 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdb9dffb700 1 -- 172.21.15.27:0/2835585009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb90011b20 con 0x7fdba007e730 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdba5626700 1 -- 172.21.15.27:0/2835585009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 msgr2=0x7fdba007cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdba5626700 1 --2- 172.21.15.27:0/2835585009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba007cb90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb900098a0 tx=0x7fdb90009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdba5626700 1 -- 172.21.15.27:0/2835585009 shutdown_connections 2024-06-06T08:28:47.096 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdba5626700 1 --2- 172.21.15.27:0/2835585009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba007cb90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdba5626700 1 -- 172.21.15.27:0/2835585009 >> 172.21.15.27:0/2835585009 conn(0x7fdba0103510 msgr2=0x7fdba0105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdba5626700 1 -- 172.21.15.27:0/2835585009 shutdown_connections 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.248+0000 7fdba5626700 1 -- 172.21.15.27:0/2835585009 wait complete. 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.249+0000 7fdba5626700 1 Processor -- start 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.249+0000 7fdba5626700 1 -- start start 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.249+0000 7fdba5626700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba019e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.249+0000 7fdba5626700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdba010e630 con 0x7fdba007e730 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdb9effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba019e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdb9effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba019e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41346/0 (socket says 172.21.15.27:41346) 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdb9effd700 1 -- 172.21.15.27:0/2201584463 learned_addr learned my addr 172.21.15.27:0/2201584463 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdb9effd700 1 -- 172.21.15.27:0/2201584463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba019e650 con 0x7fdba007e730 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdb9effd700 1 --2- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba019e110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdb900117e0 tx=0x7fdb90011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:47.097 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdb97fff700 1 -- 172.21.15.27:0/2201584463 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb90003bf0 con 0x7fdba007e730 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba019e8e0 con 0x7fdba007e730 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdb97fff700 1 -- 172.21.15.27:0/2201584463 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9001a910 con 0x7fdba007e730 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdb97fff700 1 -- 172.21.15.27:0/2201584463 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb90024970 con 0x7fdba007e730 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.250+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba019ed40 con 0x7fdba007e730 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.251+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba0063af0 con 0x7fdba007e730 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.251+0000 7fdb97fff700 1 -- 172.21.15.27:0/2201584463 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdb90022020 con 0x7fdba007e730 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.252+0000 7fdb97fff700 1 --2- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fdb8c039000 0x7fdb8c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.252+0000 7fdb97fff700 1 -- 172.21.15.27:0/2201584463 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdb9004d370 con 0x7fdba007e730 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.252+0000 7fdb9e7fc700 1 --2- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fdb8c039000 0x7fdb8c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.252+0000 7fdb9e7fc700 1 --2- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fdb8c039000 0x7fdb8c03b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdb88006fd0 tx=0x7fdb88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.261+0000 7fdb97fff700 1 -- 172.21.15.27:0/2201584463 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdb90011e20 con 0x7fdba007e730 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.483+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdba0104730 con 0x7fdb8c039000 2024-06-06T08:28:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.488+0000 7fdb97fff700 1 -- 172.21.15.27:0/2201584463 <== mgr.14120 v2:172.21.15.27:6800/2182206949 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdba0104730 con 0x7fdb8c039000 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.490+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fdb8c039000 msgr2=0x7fdb8c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.490+0000 7fdba5626700 1 --2- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fdb8c039000 0x7fdb8c03b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdb88006fd0 tx=0x7fdb88006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.490+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 msgr2=0x7fdba019e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.490+0000 7fdba5626700 1 --2- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba019e110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdb900117e0 tx=0x7fdb90011c10 comp rx=0 tx=0).stop 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.491+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 shutdown_connections 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.491+0000 7fdba5626700 1 --2- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fdb8c039000 0x7fdb8c03b4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.491+0000 7fdba5626700 1 --2- 172.21.15.27:0/2201584463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba007e730 0x7fdba019e110 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.491+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 >> 172.21.15.27:0/2201584463 conn(0x7fdba0103510 msgr2=0x7fdba0104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.491+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 shutdown_connections 2024-06-06T08:28:47.099 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:46.491+0000 7fdba5626700 1 -- 172.21.15.27:0/2201584463 wait complete. 2024-06-06T08:28:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:47 smithi027 ceph-mon[32335]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:28:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:47 smithi027 ceph-mon[32335]: Saving service mgr spec with placement count:2 2024-06-06T08:28:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:47 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:48.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.516+0000 7f4eda0f5700 1 Processor -- start 2024-06-06T08:28:48.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.516+0000 7f4eda0f5700 1 -- start start 2024-06-06T08:28:48.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.517+0000 7f4eda0f5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed4106e60 0x7f4ed4107280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:48.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.517+0000 7f4eda0f5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41077c0 con 0x7f4ed4106e60 2024-06-06T08:28:48.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.517+0000 7f4ed37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed4106e60 0x7f4ed4107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:48.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.517+0000 7f4ed37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed4106e60 0x7f4ed4107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41356/0 (socket says 172.21.15.27:41356) 2024-06-06T08:28:48.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.517+0000 7f4ed37fe700 1 -- 172.21.15.27:0/407468337 learned_addr learned my addr 172.21.15.27:0/407468337 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:48.385 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.517+0000 7f4ed37fe700 1 -- 172.21.15.27:0/407468337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed4108050 con 0x7f4ed4106e60 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.517+0000 7f4ed37fe700 1 --2- 172.21.15.27:0/407468337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed4106e60 0x7f4ed4107280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009e80 tx=0x7f4ec400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4053d28d7d2c0c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.518+0000 7f4ed27fc700 1 -- 172.21.15.27:0/407468337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ec4017070 con 0x7f4ed4106e60 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.518+0000 7f4ed27fc700 1 -- 172.21.15.27:0/407468337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec400b8c0 con 0x7f4ed4106e60 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.518+0000 7f4ed27fc700 1 -- 172.21.15.27:0/407468337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ec40045c0 con 0x7f4ed4106e60 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.519+0000 7f4eda0f5700 1 -- 172.21.15.27:0/407468337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed4106e60 msgr2=0x7f4ed4107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.519+0000 7f4eda0f5700 1 --2- 172.21.15.27:0/407468337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed4106e60 0x7f4ed4107280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009e80 tx=0x7f4ec400b330 comp rx=0 tx=0).stop 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.519+0000 7f4eda0f5700 1 -- 172.21.15.27:0/407468337 shutdown_connections 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.519+0000 7f4eda0f5700 1 --2- 172.21.15.27:0/407468337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed4106e60 0x7f4ed4107280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.519+0000 7f4eda0f5700 1 -- 172.21.15.27:0/407468337 >> 172.21.15.27:0/407468337 conn(0x7f4ed4102470 msgr2=0x7f4ed4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.519+0000 7f4eda0f5700 1 -- 172.21.15.27:0/407468337 shutdown_connections 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.519+0000 7f4eda0f5700 1 -- 172.21.15.27:0/407468337 wait complete. 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.519+0000 7f4eda0f5700 1 Processor -- start 2024-06-06T08:28:48.386 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.520+0000 7f4eda0f5700 1 -- start start 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.520+0000 7f4eda0f5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed419e350 0x7f4ed419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.520+0000 7f4eda0f5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed407b880 con 0x7f4ed419e350 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.520+0000 7f4ed37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed419e350 0x7f4ed419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.520+0000 7f4ed37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed419e350 0x7f4ed419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41370/0 (socket says 172.21.15.27:41370) 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.520+0000 7f4ed37fe700 1 -- 172.21.15.27:0/1185300896 learned_addr learned my addr 172.21.15.27:0/1185300896 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.520+0000 7f4ed37fe700 1 -- 172.21.15.27:0/1185300896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed419ecb0 con 0x7f4ed419e350 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.521+0000 7f4ed37fe700 1 --2- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed419e350 0x7f4ed419e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400b7b0 tx=0x7f4ec4003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.521+0000 7f4ed0ff9700 1 -- 172.21.15.27:0/1185300896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ec4021020 con 0x7f4ed419e350 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.521+0000 7f4ed0ff9700 1 -- 172.21.15.27:0/1185300896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec400bba0 con 0x7f4ed419e350 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.521+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed419ef40 con 0x7f4ed419e350 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.521+0000 7f4ed0ff9700 1 -- 172.21.15.27:0/1185300896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ec4017040 con 0x7f4ed419e350 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.521+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed41a1aa0 con 0x7f4ed419e350 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.522+0000 7f4ed0ff9700 1 -- 172.21.15.27:0/1185300896 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4ec4024020 con 0x7f4ed419e350 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.522+0000 7f4ed0ff9700 1 --2- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f4ec0039050 0x7f4ec003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:48.387 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.522+0000 7f4ed0ff9700 1 -- 172.21.15.27:0/1185300896 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4ec400f050 con 0x7f4ed419e350 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.522+0000 7f4ed2ffd700 1 --2- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f4ec0039050 0x7f4ec003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.522+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed4063af0 con 0x7f4ed419e350 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.523+0000 7f4ed2ffd700 1 --2- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f4ec0039050 0x7f4ec003b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006fd0 tx=0x7f4ebc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.529+0000 7f4ed0ff9700 1 -- 172.21.15.27:0/1185300896 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4ec402b020 con 0x7f4ed419e350 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.746+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4ed41a20c0 con 0x7f4ed419e350 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.748+0000 7f4ed0ff9700 1 -- 172.21.15.27:0/1185300896 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4ec400bd50 con 0x7f4ed419e350 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.751+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f4ec0039050 msgr2=0x7f4ec003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.751+0000 7f4eda0f5700 1 --2- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f4ec0039050 0x7f4ec003b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006fd0 tx=0x7f4ebc006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.751+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed419e350 msgr2=0x7f4ed419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.751+0000 7f4eda0f5700 1 --2- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed419e350 0x7f4ed419e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400b7b0 tx=0x7f4ec4003d00 comp rx=0 tx=0).stop 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.751+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 shutdown_connections 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.751+0000 7f4eda0f5700 1 --2- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f4ec0039050 0x7f4ec003b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.751+0000 7f4eda0f5700 1 --2- 172.21.15.27:0/1185300896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ed419e350 0x7f4ed419e770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.751+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 >> 172.21.15.27:0/1185300896 conn(0x7f4ed4102470 msgr2=0x7f4ed4190460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:48.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.751+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 shutdown_connections 2024-06-06T08:28:48.389 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:47.752+0000 7f4eda0f5700 1 -- 172.21.15.27:0/1185300896 wait complete. 2024-06-06T08:28:48.830 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:48 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1185300896' entity='client.admin' 2024-06-06T08:28:49.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.848+0000 7fb420487700 1 Processor -- start 2024-06-06T08:28:49.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.848+0000 7fb420487700 1 -- start start 2024-06-06T08:28:49.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.848+0000 7fb420487700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb4181083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:49.902 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.848+0000 7fb420487700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181089c0 con 0x7fb418107fd0 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.849+0000 7fb41e223700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb4181083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.849+0000 7fb41e223700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb4181083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41384/0 (socket says 172.21.15.27:41384) 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.849+0000 7fb41e223700 1 -- 172.21.15.27:0/3588347408 learned_addr learned my addr 172.21.15.27:0/3588347408 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.849+0000 7fb41e223700 1 -- 172.21.15.27:0/3588347408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4181091d0 con 0x7fb418107fd0 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.849+0000 7fb41e223700 1 --2- 172.21.15.27:0/3588347408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb4181083f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb4140098a0 tx=0x7fb414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f2586f314f0648 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.849+0000 7fb41d221700 1 -- 172.21.15.27:0/3588347408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb414004160 con 0x7fb418107fd0 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.849+0000 7fb41d221700 1 -- 172.21.15.27:0/3588347408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb414004300 con 0x7fb418107fd0 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.849+0000 7fb41d221700 1 -- 172.21.15.27:0/3588347408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb414011b20 con 0x7fb418107fd0 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.850+0000 7fb420487700 1 -- 172.21.15.27:0/3588347408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 msgr2=0x7fb4181083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.850+0000 7fb420487700 1 --2- 172.21.15.27:0/3588347408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb4181083f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb4140098a0 tx=0x7fb414009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.850+0000 7fb420487700 1 -- 172.21.15.27:0/3588347408 shutdown_connections 2024-06-06T08:28:49.903 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.850+0000 7fb420487700 1 --2- 172.21.15.27:0/3588347408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb4181083f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.850+0000 7fb420487700 1 -- 172.21.15.27:0/3588347408 >> 172.21.15.27:0/3588347408 conn(0x7fb4181035d0 msgr2=0x7fb4181059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.850+0000 7fb420487700 1 -- 172.21.15.27:0/3588347408 shutdown_connections 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.850+0000 7fb420487700 1 -- 172.21.15.27:0/3588347408 wait complete. 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb420487700 1 Processor -- start 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb420487700 1 -- start start 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb420487700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb41819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb420487700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb41810c580 con 0x7fb418107fd0 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb41e223700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb41819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb41e223700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb41819bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41396/0 (socket says 172.21.15.27:41396) 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb41e223700 1 -- 172.21.15.27:0/1515924587 learned_addr learned my addr 172.21.15.27:0/1515924587 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb41e223700 1 -- 172.21.15.27:0/1515924587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41819c4e0 con 0x7fb418107fd0 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb41e223700 1 --2- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb41819bfa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4140117e0 tx=0x7fb414011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb40f7fe700 1 -- 172.21.15.27:0/1515924587 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb414003bf0 con 0x7fb418107fd0 2024-06-06T08:28:49.904 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb40f7fe700 1 -- 172.21.15.27:0/1515924587 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb41401a910 con 0x7fb418107fd0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb40f7fe700 1 -- 172.21.15.27:0/1515924587 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb414024970 con 0x7fb418107fd0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.851+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41819c770 con 0x7fb418107fd0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.852+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41819cbd0 con 0x7fb418107fd0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.852+0000 7fb40f7fe700 1 -- 172.21.15.27:0/1515924587 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb414022020 con 0x7fb418107fd0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.852+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb418063af0 con 0x7fb418107fd0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.853+0000 7fb40f7fe700 1 --2- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb404039050 0x7fb40403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.853+0000 7fb40f7fe700 1 -- 172.21.15.27:0/1515924587 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb41404d8d0 con 0x7fb418107fd0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.853+0000 7fb41da22700 1 --2- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb404039050 0x7fb40403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.853+0000 7fb41da22700 1 --2- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb404039050 0x7fb40403b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb408006fd0 tx=0x7fb408006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:48.858+0000 7fb40f7fe700 1 -- 172.21.15.27:0/1515924587 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb41402d910 con 0x7fb418107fd0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.064+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb41819f5a0 con 0x7fb418107fd0 2024-06-06T08:28:49.905 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.067+0000 7fb40f7fe700 1 -- 172.21.15.27:0/1515924587 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb41402d3f0 con 0x7fb418107fd0 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb404039050 msgr2=0x7fb40403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 --2- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb404039050 0x7fb40403b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb408006fd0 tx=0x7fb408006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 msgr2=0x7fb41819bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 --2- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb41819bfa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4140117e0 tx=0x7fb414011ba0 comp rx=0 tx=0).stop 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 shutdown_connections 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 --2- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb404039050 0x7fb40403b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 --2- 172.21.15.27:0/1515924587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb418107fd0 0x7fb41819bfa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 >> 172.21.15.27:0/1515924587 conn(0x7fb4181035d0 msgr2=0x7fb418104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 shutdown_connections 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:49.070+0000 7fb420487700 1 -- 172.21.15.27:0/1515924587 wait complete. 2024-06-06T08:28:49.906 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling the dashboard module... 2024-06-06T08:28:50.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:50 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1515924587' entity='client.admin' 2024-06-06T08:28:50.160 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:50 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:51.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:51 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1520604594' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-06T08:28:51.359 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:51 smithi027 ceph-mon[32335]: from='mgr.14120 172.21.15.27:0/1718514788' entity='mgr.x' 2024-06-06T08:28:51.359 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: ignoring --setuser ceph since I am not root 2024-06-06T08:28:51.359 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: ignoring --setgroup ceph since I am not root 2024-06-06T08:28:51.359 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:51.265+0000 7fc2a231f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T08:28:51.668 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:51.461+0000 7fc2a231f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:28:51.669 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:51.568+0000 7fc2a231f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T08:28:51.675 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.366+0000 7f660cf20700 1 Processor -- start 2024-06-06T08:28:51.675 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.366+0000 7f660cf20700 1 -- start start 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.366+0000 7f660cf20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f6608108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.366+0000 7f660cf20700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f66081089e0 con 0x7f6608107ff0 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.367+0000 7f660659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f6608108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.367+0000 7f660659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f6608108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49694/0 (socket says 172.21.15.27:49694) 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.367+0000 7f660659c700 1 -- 172.21.15.27:0/1530594661 learned_addr learned my addr 172.21.15.27:0/1530594661 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.367+0000 7f660659c700 1 -- 172.21.15.27:0/1530594661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66081091a0 con 0x7f6608107ff0 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.367+0000 7f660659c700 1 --2- 172.21.15.27:0/1530594661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f6608108410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f65f80098a0 tx=0x7f65f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2457c690ebfbffd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.367+0000 7f660559a700 1 -- 172.21.15.27:0/1530594661 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65f8004160 con 0x7f6608107ff0 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.367+0000 7f660559a700 1 -- 172.21.15.27:0/1530594661 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f8004300 con 0x7f6608107ff0 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.367+0000 7f660559a700 1 -- 172.21.15.27:0/1530594661 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65f8011b20 con 0x7f6608107ff0 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.368+0000 7f660cf20700 1 -- 172.21.15.27:0/1530594661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 msgr2=0x7f6608108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.368+0000 7f660cf20700 1 --2- 172.21.15.27:0/1530594661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f6608108410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f65f80098a0 tx=0x7f65f8009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:51.676 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.368+0000 7f660cf20700 1 -- 172.21.15.27:0/1530594661 shutdown_connections 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.368+0000 7f660cf20700 1 --2- 172.21.15.27:0/1530594661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f6608108410 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.368+0000 7f660cf20700 1 -- 172.21.15.27:0/1530594661 >> 172.21.15.27:0/1530594661 conn(0x7f66081035b0 msgr2=0x7f66081059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.368+0000 7f660cf20700 1 -- 172.21.15.27:0/1530594661 shutdown_connections 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.368+0000 7f660cf20700 1 -- 172.21.15.27:0/1530594661 wait complete. 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.369+0000 7f660cf20700 1 Processor -- start 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.369+0000 7f660cf20700 1 -- start start 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.369+0000 7f660cf20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f660807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.369+0000 7f660cf20700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f660810c550 con 0x7f6608107ff0 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.369+0000 7f660659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f660807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.369+0000 7f660659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f660807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49696/0 (socket says 172.21.15.27:49696) 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.369+0000 7f660659c700 1 -- 172.21.15.27:0/1520604594 learned_addr learned my addr 172.21.15.27:0/1520604594 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:51.677 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.370+0000 7f660659c700 1 -- 172.21.15.27:0/1520604594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660807e0d0 con 0x7f6608107ff0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.370+0000 7f660659c700 1 --2- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f660807db90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f65f80117e0 tx=0x7f65f8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.370+0000 7f65f77fe700 1 -- 172.21.15.27:0/1520604594 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65f8003bf0 con 0x7f6608107ff0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.370+0000 7f65f77fe700 1 -- 172.21.15.27:0/1520604594 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f801a910 con 0x7f6608107ff0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.370+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660807c2c0 con 0x7f6608107ff0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.370+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660807c780 con 0x7f6608107ff0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.370+0000 7f65f77fe700 1 -- 172.21.15.27:0/1520604594 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65f80249a0 con 0x7f6608107ff0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.371+0000 7f65f77fe700 1 -- 172.21.15.27:0/1520604594 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f65f8022020 con 0x7f6608107ff0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.371+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e8005360 con 0x7f6608107ff0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.371+0000 7f65f77fe700 1 --2- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f65f0039050 0x7f65f003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.371+0000 7f65f77fe700 1 -- 172.21.15.27:0/1520604594 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f65f804e340 con 0x7f6608107ff0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.371+0000 7f6605d9b700 1 --2- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f65f0039050 0x7f65f003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.372+0000 7f6605d9b700 1 --2- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f65f0039050 0x7f65f003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65fc006fd0 tx=0x7f65fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:51.678 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.377+0000 7f65f77fe700 1 -- 172.21.15.27:0/1520604594 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f65f8011e20 con 0x7f6608107ff0 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:50.651+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f65e8005650 con 0x7f6608107ff0 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.071+0000 7f65f77fe700 1 -- 172.21.15.27:0/1520604594 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f65f8019c70 con 0x7f6608107ff0 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.072+0000 7f65f77fe700 1 -- 172.21.15.27:0/1520604594 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f65f802d910 con 0x7f6608107ff0 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f65f0039050 msgr2=0x7f65f003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 --2- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f65f0039050 0x7f65f003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65fc006fd0 tx=0x7f65fc006e40 comp rx=0 tx=0).stop 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 msgr2=0x7f660807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 --2- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f660807db90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f65f80117e0 tx=0x7f65f8011c10 comp rx=0 tx=0).stop 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 shutdown_connections 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 --2- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7f65f0039050 0x7f65f003b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 --2- 172.21.15.27:0/1520604594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6608107ff0 0x7f660807db90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:51.679 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 >> 172.21.15.27:0/1520604594 conn(0x7f66081035b0 msgr2=0x7f66081040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:51.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 shutdown_connections 2024-06-06T08:28:51.680 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:51.074+0000 7f660cf20700 1 -- 172.21.15.27:0/1520604594 wait complete. 2024-06-06T08:28:51.922 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:51.900+0000 7fc2a231f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T08:28:52.206 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:51.993+0000 7fc2a231f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T08:28:52.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:52 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1520604594' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-06T08:28:52.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:52 smithi027 ceph-mon[32335]: mgrmap e9: x(active, since 19s) 2024-06-06T08:28:53.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:53 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2945016367' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T08:28:53.478 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:53 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:53.297+0000 7fc2a231f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T08:28:53.478 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:53 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:53.476+0000 7fc2a231f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T08:28:53.763 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:53 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:53.654+0000 7fc2a231f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T08:28:53.771 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-06-06T08:28:53.771 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-06T08:28:53.771 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T08:28:53.771 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T08:28:53.771 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T08:28:53.771 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-06-06T08:28:53.771 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.126+0000 7fb235fc5700 1 Processor -- start 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.126+0000 7fb235fc5700 1 -- start start 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.126+0000 7fb235fc5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb23010ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.126+0000 7fb235fc5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb23007c760 con 0x7fb230108a20 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.127+0000 7fb22f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb23010ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.127+0000 7fb22f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb23010ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49724/0 (socket says 172.21.15.27:49724) 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.127+0000 7fb22f7fe700 1 -- 172.21.15.27:0/2279811792 learned_addr learned my addr 172.21.15.27:0/2279811792 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.127+0000 7fb22f7fe700 1 -- 172.21.15.27:0/2279811792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23010b350 con 0x7fb230108a20 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.127+0000 7fb22f7fe700 1 --2- 172.21.15.27:0/2279811792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb23010ae10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2200098a0 tx=0x7fb220009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21aa6e6ef1c71d45 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.127+0000 7fb22e7fc700 1 -- 172.21.15.27:0/2279811792 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb220004160 con 0x7fb230108a20 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.127+0000 7fb22e7fc700 1 -- 172.21.15.27:0/2279811792 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb220004300 con 0x7fb230108a20 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.127+0000 7fb22e7fc700 1 -- 172.21.15.27:0/2279811792 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb220011b20 con 0x7fb230108a20 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.128+0000 7fb235fc5700 1 -- 172.21.15.27:0/2279811792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 msgr2=0x7fb23010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.128+0000 7fb235fc5700 1 --2- 172.21.15.27:0/2279811792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb23010ae10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2200098a0 tx=0x7fb220009bb0 comp rx=0 tx=0).stop 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.128+0000 7fb235fc5700 1 -- 172.21.15.27:0/2279811792 shutdown_connections 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.128+0000 7fb235fc5700 1 --2- 172.21.15.27:0/2279811792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb23010ae10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.128+0000 7fb235fc5700 1 -- 172.21.15.27:0/2279811792 >> 172.21.15.27:0/2279811792 conn(0x7fb230102450 msgr2=0x7fb230104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.128+0000 7fb235fc5700 1 -- 172.21.15.27:0/2279811792 shutdown_connections 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.128+0000 7fb235fc5700 1 -- 172.21.15.27:0/2279811792 wait complete. 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.129+0000 7fb235fc5700 1 Processor -- start 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.129+0000 7fb235fc5700 1 -- start start 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.129+0000 7fb235fc5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb2301082a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.129+0000 7fb235fc5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb23010c3c0 con 0x7fb230108a20 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.129+0000 7fb22f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb2301082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.129+0000 7fb22f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb2301082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49732/0 (socket says 172.21.15.27:49732) 2024-06-06T08:28:53.773 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.129+0000 7fb22f7fe700 1 -- 172.21.15.27:0/2945016367 learned_addr learned my addr 172.21.15.27:0/2945016367 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.130+0000 7fb22f7fe700 1 -- 172.21.15.27:0/2945016367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2301087e0 con 0x7fb230108a20 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.130+0000 7fb22f7fe700 1 --2- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb2301082a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2200117e0 tx=0x7fb220011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.130+0000 7fb22cff9700 1 -- 172.21.15.27:0/2945016367 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb220003bf0 con 0x7fb230108a20 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.130+0000 7fb22cff9700 1 -- 172.21.15.27:0/2945016367 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb22001a910 con 0x7fb230108a20 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.130+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2301069d0 con 0x7fb230108a20 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.130+0000 7fb22cff9700 1 -- 172.21.15.27:0/2945016367 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb220024970 con 0x7fb230108a20 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.130+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb230106e90 con 0x7fb230108a20 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.131+0000 7fb22cff9700 1 -- 172.21.15.27:0/2945016367 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb220022020 con 0x7fb230108a20 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.131+0000 7fb22cff9700 1 --2- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb21c039050 0x7fb21c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.131+0000 7fb22cff9700 1 -- 172.21.15.27:0/2945016367 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb22004e3f0 con 0x7fb230108a20 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.131+0000 7fb22effd700 1 -- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb21c039050 msgr2=0x7fb21c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2182206949 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.131+0000 7fb22effd700 1 --2- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb21c039050 0x7fb21c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T08:28:53.774 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.131+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb230063af0 con 0x7fb230108a20 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.136+0000 7fb22cff9700 1 -- 172.21.15.27:0/2945016367 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb220011e20 con 0x7fb230108a20 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.332+0000 7fb22effd700 1 -- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb21c039050 msgr2=0x7fb21c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2182206949 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.332+0000 7fb22effd700 1 --2- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb21c039050 0x7fb21c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.471+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb230107510 con 0x7fb230108a20 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.471+0000 7fb22cff9700 1 -- 172.21.15.27:0/2945016367 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb2200172b0 con 0x7fb230108a20 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb21c039050 msgr2=0x7fb21c03b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 --2- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb21c039050 0x7fb21c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 msgr2=0x7fb2301082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 --2- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb2301082a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2200117e0 tx=0x7fb220011c10 comp rx=0 tx=0).stop 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 shutdown_connections 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 --2- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb21c039050 0x7fb21c03b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 --2- 172.21.15.27:0/2945016367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb230108a20 0x7fb2301082a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:28:53.775 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 >> 172.21.15.27:0/2945016367 conn(0x7fb230102450 msgr2=0x7fb230102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:28:53.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 shutdown_connections 2024-06-06T08:28:53.776 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:52.474+0000 7fb235fc5700 1 -- 172.21.15.27:0/2945016367 wait complete. 2024-06-06T08:28:53.776 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-06-06T08:28:53.776 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 9... 2024-06-06T08:28:54.063 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:53 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:53.965+0000 7fc2a231f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T08:28:54.063 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:54 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:54.064+0000 7fc2a231f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T08:28:54.396 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:54 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:54.281+0000 7fc2a231f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T08:28:54.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:54 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:54.485+0000 7fc2a231f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:28:55.136 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:55 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:55.137+0000 7fc2a231f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T08:28:55.898 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:55 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:55.583+0000 7fc2a231f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T08:28:55.899 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:55 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:55.685+0000 7fc2a231f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T08:28:55.899 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:55 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:55.899+0000 7fc2a231f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T08:28:56.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:56 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:56.447+0000 7fc2a231f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T08:28:56.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:56 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:56.546+0000 7fc2a231f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T08:28:57.171 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:56 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:56.805+0000 7fc2a231f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T08:28:57.171 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:56 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:56.946+0000 7fc2a231f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T08:28:57.171 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:57 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:57.048+0000 7fc2a231f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T08:28:57.171 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:57 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:57.172+0000 7fc2a231f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T08:28:57.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:57 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:57.483+0000 7fc2a231f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T08:28:58.395 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:58.116+0000 7fc2a231f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T08:28:58.396 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:28:58.344+0000 7fc2a231f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T08:28:58.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: Active manager daemon x restarted 2024-06-06T08:28:58.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: Activating manager daemon x 2024-06-06T08:28:58.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: osdmap e3: 0 total, 0 up, 0 in 2024-06-06T08:28:58.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: mgrmap e10: x(active, starting, since 0.0034412s) 2024-06-06T08:28:58.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:28:58.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T08:28:58.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T08:28:58.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T08:28:58.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T08:28:58.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: Manager daemon x is now available 2024-06-06T08:28:58.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:28:58.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T08:28:59.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T08:28:59.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:59 smithi027 ceph-mon[32335]: [06/Jun/2024:08:28:59] ENGINE Bus STARTING 2024-06-06T08:28:59.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:59 smithi027 ceph-mon[32335]: [06/Jun/2024:08:28:59] ENGINE Serving on https://172.21.15.27:7150 2024-06-06T08:28:59.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:28:59 smithi027 ceph-mon[32335]: mgrmap e11: x(active, since 1.00585s) 2024-06-06T08:29:00.174 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.192+0000 7fb11df76700 1 Processor -- start 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.192+0000 7fb11df76700 1 -- start start 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.192+0000 7fb11df76700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11810ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.192+0000 7fb11df76700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb11807c760 con 0x7fb118108a20 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.193+0000 7fb1177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11810ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.193+0000 7fb1177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11810ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49742/0 (socket says 172.21.15.27:49742) 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.193+0000 7fb1177fe700 1 -- 172.21.15.27:0/2535518281 learned_addr learned my addr 172.21.15.27:0/2535518281 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.193+0000 7fb1177fe700 1 -- 172.21.15.27:0/2535518281 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11810b350 con 0x7fb118108a20 2024-06-06T08:29:00.175 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.193+0000 7fb1177fe700 1 --2- 172.21.15.27:0/2535518281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11810ae10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb1080098a0 tx=0x7fb108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbed472d3e06b034 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.193+0000 7fb1167fc700 1 -- 172.21.15.27:0/2535518281 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb108004160 con 0x7fb118108a20 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.193+0000 7fb1167fc700 1 -- 172.21.15.27:0/2535518281 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb108004300 con 0x7fb118108a20 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.193+0000 7fb1167fc700 1 -- 172.21.15.27:0/2535518281 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb108011b20 con 0x7fb118108a20 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.194+0000 7fb11df76700 1 -- 172.21.15.27:0/2535518281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 msgr2=0x7fb11810ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.194+0000 7fb11df76700 1 --2- 172.21.15.27:0/2535518281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11810ae10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb1080098a0 tx=0x7fb108009bb0 comp rx=0 tx=0).stop 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.194+0000 7fb11df76700 1 -- 172.21.15.27:0/2535518281 shutdown_connections 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.194+0000 7fb11df76700 1 --2- 172.21.15.27:0/2535518281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11810ae10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.194+0000 7fb11df76700 1 -- 172.21.15.27:0/2535518281 >> 172.21.15.27:0/2535518281 conn(0x7fb118102450 msgr2=0x7fb118104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.194+0000 7fb11df76700 1 -- 172.21.15.27:0/2535518281 shutdown_connections 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.194+0000 7fb11df76700 1 -- 172.21.15.27:0/2535518281 wait complete. 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.195+0000 7fb11df76700 1 Processor -- start 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.195+0000 7fb11df76700 1 -- start start 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.195+0000 7fb11df76700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11819bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:00.176 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.195+0000 7fb11df76700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb11810c3c0 con 0x7fb118108a20 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.195+0000 7fb1177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11819bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.195+0000 7fb1177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11819bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49746/0 (socket says 172.21.15.27:49746) 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.195+0000 7fb1177fe700 1 -- 172.21.15.27:0/2818189964 learned_addr learned my addr 172.21.15.27:0/2818189964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.196+0000 7fb1177fe700 1 -- 172.21.15.27:0/2818189964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11819c400 con 0x7fb118108a20 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.196+0000 7fb1177fe700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11819bec0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb1080117e0 tx=0x7fb108011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.196+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb108003ba0 con 0x7fb118108a20 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.196+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb10801a910 con 0x7fb118108a20 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.196+0000 7fb11df76700 1 -- 172.21.15.27:0/2818189964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11819c690 con 0x7fb118108a20 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.196+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1080248f0 con 0x7fb118108a20 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.196+0000 7fb11df76700 1 -- 172.21.15.27:0/2818189964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11819caf0 con 0x7fb118108a20 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.197+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb108022020 con 0x7fb118108a20 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.197+0000 7fb114ff9700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 0x7fb10403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.197+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 --> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb10403bcd0 con 0x7fb104039100 2024-06-06T08:29:00.177 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.197+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb10804d7e0 con 0x7fb118108a20 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.197+0000 7fb116ffd700 1 -- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 msgr2=0x7fb10403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2182206949 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.197+0000 7fb116ffd700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 0x7fb10403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.398+0000 7fb116ffd700 1 -- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 msgr2=0x7fb10403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2182206949 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.398+0000 7fb116ffd700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 0x7fb10403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.799+0000 7fb116ffd700 1 -- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 msgr2=0x7fb10403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2182206949 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:54.799+0000 7fb116ffd700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 0x7fb10403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:55.600+0000 7fb116ffd700 1 -- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 msgr2=0x7fb10403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2182206949 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:55.600+0000 7fb116ffd700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 0x7fb10403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:57.201+0000 7fb116ffd700 1 -- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 msgr2=0x7fb10403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/2182206949 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:57.201+0000 7fb116ffd700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 0x7fb10403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:58.347+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fb10802d3f0 con 0x7fb118108a20 2024-06-06T08:29:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:58.347+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 msgr2=0x7fb10403b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:58.347+0000 7fb114ff9700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 0x7fb10403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.348+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fb10804f2c0 con 0x7fb118108a20 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.349+0000 7fb114ff9700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb10403cbc0 0x7fb10403efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.349+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb108016e70 con 0x7fb10403cbc0 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.349+0000 7fb116ffd700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb10403cbc0 0x7fb10403efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.350+0000 7fb116ffd700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb10403cbc0 0x7fb10403efb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb100003a10 tx=0x7fb100009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.351+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fb108016e70 con 0x7fb10403cbc0 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 -- 172.21.15.27:0/2818189964 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb118063af0 con 0x7fb10403cbc0 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb114ff9700 1 -- 172.21.15.27:0/2818189964 <== mgr.14152 v2:172.21.15.27:6800/3763593724 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fb118063af0 con 0x7fb10403cbc0 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 -- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb10403cbc0 msgr2=0x7fb10403efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb10403cbc0 0x7fb10403efb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb100003a10 tx=0x7fb100009270 comp rx=0 tx=0).stop 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 -- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 msgr2=0x7fb11819bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11819bec0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb1080117e0 tx=0x7fb108011ba0 comp rx=0 tx=0).stop 2024-06-06T08:29:00.180 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 -- 172.21.15.27:0/2818189964 shutdown_connections 2024-06-06T08:29:00.180 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb10403cbc0 0x7fb10403efb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:00.180 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb118108a20 0x7fb11819bec0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:00.180 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 --2- 172.21.15.27:0/2818189964 >> [v2:172.21.15.27:6800/2182206949,v1:172.21.15.27:6801/2182206949] conn(0x7fb104039100 0x7fb10403b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:00.180 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.358+0000 7fb11df76700 1 -- 172.21.15.27:0/2818189964 >> 172.21.15.27:0/2818189964 conn(0x7fb118102450 msgr2=0x7fb118102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:00.180 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.359+0000 7fb11df76700 1 -- 172.21.15.27:0/2818189964 shutdown_connections 2024-06-06T08:29:00.180 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:28:59.359+0000 7fb11df76700 1 -- 172.21.15.27:0/2818189964 wait complete. 2024-06-06T08:29:00.180 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 9 is available 2024-06-06T08:29:00.180 INFO:teuthology.orchestra.run.smithi027.stdout:Generating a dashboard self-signed certificate... 2024-06-06T08:29:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:00 smithi027 ceph-mon[32335]: [06/Jun/2024:08:28:59] ENGINE Serving on http://172.21.15.27:8765 2024-06-06T08:29:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:00 smithi027 ceph-mon[32335]: [06/Jun/2024:08:28:59] ENGINE Bus STARTED 2024-06-06T08:29:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:00 smithi027 ceph-mon[32335]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-06T08:29:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:00 smithi027 ceph-mon[32335]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-06T08:29:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:01.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-06T08:29:01.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.626+0000 7feda9b20700 1 Processor -- start 2024-06-06T08:29:01.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.626+0000 7feda9b20700 1 -- start start 2024-06-06T08:29:01.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.626+0000 7feda9b20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda410ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:01.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.626+0000 7feda9b20700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feda407be50 con 0x7feda4108a80 2024-06-06T08:29:01.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.626+0000 7feda37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda410ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:01.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.626+0000 7feda37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda410ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41662/0 (socket says 172.21.15.27:41662) 2024-06-06T08:29:01.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.626+0000 7feda37fe700 1 -- 172.21.15.27:0/581370731 learned_addr learned my addr 172.21.15.27:0/581370731 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:01.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.627+0000 7feda37fe700 1 -- 172.21.15.27:0/581370731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda410b3b0 con 0x7feda4108a80 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.627+0000 7feda37fe700 1 --2- 172.21.15.27:0/581370731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda410ae70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fed940098a0 tx=0x7fed94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c4314fc7f2e779 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.627+0000 7feda27fc700 1 -- 172.21.15.27:0/581370731 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed94004160 con 0x7feda4108a80 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.627+0000 7feda27fc700 1 -- 172.21.15.27:0/581370731 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed94004300 con 0x7feda4108a80 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.627+0000 7feda27fc700 1 -- 172.21.15.27:0/581370731 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed94011b20 con 0x7feda4108a80 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.628+0000 7feda9b20700 1 -- 172.21.15.27:0/581370731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 msgr2=0x7feda410ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.628+0000 7feda9b20700 1 --2- 172.21.15.27:0/581370731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda410ae70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fed940098a0 tx=0x7fed94009bb0 comp rx=0 tx=0).stop 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.628+0000 7feda9b20700 1 -- 172.21.15.27:0/581370731 shutdown_connections 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.628+0000 7feda9b20700 1 --2- 172.21.15.27:0/581370731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda410ae70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.628+0000 7feda9b20700 1 -- 172.21.15.27:0/581370731 >> 172.21.15.27:0/581370731 conn(0x7feda4102470 msgr2=0x7feda4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.628+0000 7feda9b20700 1 -- 172.21.15.27:0/581370731 shutdown_connections 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.628+0000 7feda9b20700 1 -- 172.21.15.27:0/581370731 wait complete. 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.629+0000 7feda9b20700 1 Processor -- start 2024-06-06T08:29:01.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.629+0000 7feda9b20700 1 -- start start 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.629+0000 7feda9b20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.629+0000 7feda9b20700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feda410c3b0 con 0x7feda4108a80 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.629+0000 7feda37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.630+0000 7feda37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda419bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41666/0 (socket says 172.21.15.27:41666) 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.630+0000 7feda37fe700 1 -- 172.21.15.27:0/1576232350 learned_addr learned my addr 172.21.15.27:0/1576232350 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.630+0000 7feda37fe700 1 -- 172.21.15.27:0/1576232350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda419c420 con 0x7feda4108a80 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.630+0000 7feda37fe700 1 --2- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda419bee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fed940117e0 tx=0x7fed94011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.630+0000 7feda0ff9700 1 -- 172.21.15.27:0/1576232350 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed94003bf0 con 0x7feda4108a80 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.630+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda419c6b0 con 0x7feda4108a80 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.630+0000 7feda0ff9700 1 -- 172.21.15.27:0/1576232350 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9401a910 con 0x7feda4108a80 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.630+0000 7feda0ff9700 1 -- 172.21.15.27:0/1576232350 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed94024970 con 0x7feda4108a80 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.630+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda419cb10 con 0x7feda4108a80 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.631+0000 7feda0ff9700 1 -- 172.21.15.27:0/1576232350 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fed94022020 con 0x7feda4108a80 2024-06-06T08:29:01.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.631+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda4063af0 con 0x7feda4108a80 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.631+0000 7feda0ff9700 1 --2- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fed8c038f40 0x7fed8c03b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.631+0000 7feda0ff9700 1 -- 172.21.15.27:0/1576232350 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fed9404d370 con 0x7feda4108a80 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.632+0000 7feda2ffd700 1 --2- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fed8c038f40 0x7fed8c03b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.632+0000 7feda2ffd700 1 --2- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fed8c038f40 0x7fed8c03b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed98006fd0 tx=0x7fed98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.637+0000 7feda0ff9700 1 -- 172.21.15.27:0/1576232350 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed94011e20 con 0x7feda4108a80 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.848+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7feda41035d0 con 0x7fed8c038f40 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.910+0000 7feda0ff9700 1 -- 172.21.15.27:0/1576232350 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fed94024cd0 con 0x7feda4108a80 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.985+0000 7feda0ff9700 1 -- 172.21.15.27:0/1576232350 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7feda41035d0 con 0x7fed8c038f40 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.987+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fed8c038f40 msgr2=0x7fed8c03b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.987+0000 7feda9b20700 1 --2- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fed8c038f40 0x7fed8c03b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed98006fd0 tx=0x7fed98006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.987+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 msgr2=0x7feda419bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:01.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.987+0000 7feda9b20700 1 --2- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda419bee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fed940117e0 tx=0x7fed94011c10 comp rx=0 tx=0).stop 2024-06-06T08:29:01.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.987+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 shutdown_connections 2024-06-06T08:29:01.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.987+0000 7feda9b20700 1 --2- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fed8c038f40 0x7fed8c03b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:01.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.987+0000 7feda9b20700 1 --2- 172.21.15.27:0/1576232350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feda4108a80 0x7feda419bee0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:01.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.987+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 >> 172.21.15.27:0/1576232350 conn(0x7feda4102470 msgr2=0x7feda4102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:01.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.988+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 shutdown_connections 2024-06-06T08:29:01.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:00.988+0000 7feda9b20700 1 -- 172.21.15.27:0/1576232350 wait complete. 2024-06-06T08:29:01.550 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial admin user... 2024-06-06T08:29:01.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:01 smithi027 ceph-mon[32335]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:01.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:01 smithi027 ceph-mon[32335]: mgrmap e12: x(active, since 2s) 2024-06-06T08:29:01.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:01 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:01.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:01 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:01.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:01 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:03.140 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$xCx9dryFJzYP1Nb8vQDJ/.Nl6yatQcZh5TAnSQ.G3.bFlwCYQV3A6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717662542, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-06T08:29:03.140 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.958+0000 7fe067b3b700 1 Processor -- start 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.958+0000 7fe067b3b700 1 -- start start 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.958+0000 7fe067b3b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06010ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.958+0000 7fe067b3b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe06007c720 con 0x7fe060108a40 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.959+0000 7fe0658d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06010ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.959+0000 7fe0658d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06010ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41674/0 (socket says 172.21.15.27:41674) 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.959+0000 7fe0658d7700 1 -- 172.21.15.27:0/2191075591 learned_addr learned my addr 172.21.15.27:0/2191075591 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.959+0000 7fe0658d7700 1 -- 172.21.15.27:0/2191075591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06010b370 con 0x7fe060108a40 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.959+0000 7fe0658d7700 1 --2- 172.21.15.27:0/2191075591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06010ae30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0098a0 tx=0x7fe05c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80aa512a58f3a77 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.959+0000 7fe0648d5700 1 -- 172.21.15.27:0/2191075591 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe05c004160 con 0x7fe060108a40 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.959+0000 7fe0648d5700 1 -- 172.21.15.27:0/2191075591 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05c004300 con 0x7fe060108a40 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.959+0000 7fe0648d5700 1 -- 172.21.15.27:0/2191075591 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe05c011ad0 con 0x7fe060108a40 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.960+0000 7fe067b3b700 1 -- 172.21.15.27:0/2191075591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 msgr2=0x7fe06010ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:03.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.960+0000 7fe067b3b700 1 --2- 172.21.15.27:0/2191075591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06010ae30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0098a0 tx=0x7fe05c009bb0 comp rx=0 tx=0).stop 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.960+0000 7fe067b3b700 1 -- 172.21.15.27:0/2191075591 shutdown_connections 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.960+0000 7fe067b3b700 1 --2- 172.21.15.27:0/2191075591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06010ae30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.960+0000 7fe067b3b700 1 -- 172.21.15.27:0/2191075591 >> 172.21.15.27:0/2191075591 conn(0x7fe0601023f0 msgr2=0x7fe060104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.960+0000 7fe067b3b700 1 -- 172.21.15.27:0/2191075591 shutdown_connections 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.961+0000 7fe067b3b700 1 -- 172.21.15.27:0/2191075591 wait complete. 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.961+0000 7fe067b3b700 1 Processor -- start 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.961+0000 7fe067b3b700 1 -- start start 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.961+0000 7fe067b3b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.961+0000 7fe067b3b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe06010c3d0 con 0x7fe060108a40 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe0658d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe0658d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06019bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41680/0 (socket says 172.21.15.27:41680) 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe0658d7700 1 -- 172.21.15.27:0/3488397953 learned_addr learned my addr 172.21.15.27:0/3488397953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:03.142 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe0658d7700 1 -- 172.21.15.27:0/3488397953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06019c430 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe0658d7700 1 --2- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06019bef0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe05c011c30 tx=0x7fe05c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe056ffd700 1 -- 172.21.15.27:0/3488397953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe05c003b20 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06019c6c0 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe056ffd700 1 -- 172.21.15.27:0/3488397953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05c01a910 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe056ffd700 1 -- 172.21.15.27:0/3488397953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe05c019780 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.962+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06019cb20 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.963+0000 7fe056ffd700 1 -- 172.21.15.27:0/3488397953 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe05c019ab0 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.963+0000 7fe056ffd700 1 --2- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe04c0390f0 0x7fe04c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.963+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe060063af0 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.963+0000 7fe056ffd700 1 -- 172.21.15.27:0/3488397953 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe05c04e810 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.964+0000 7fe0650d6700 1 --2- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe04c0390f0 0x7fe04c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.964+0000 7fe0650d6700 1 --2- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe04c0390f0 0x7fe04c03b5b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe050006fd0 tx=0x7fe050006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:01.969+0000 7fe056ffd700 1 -- 172.21.15.27:0/3488397953 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe05c017ca0 con 0x7fe060108a40 2024-06-06T08:29:03.143 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.198+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe06019f530 con 0x7fe04c0390f0 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.444+0000 7fe056ffd700 1 -- 172.21.15.27:0/3488397953 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe06019f530 con 0x7fe04c0390f0 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe04c0390f0 msgr2=0x7fe04c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 --2- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe04c0390f0 0x7fe04c03b5b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe050006fd0 tx=0x7fe050006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 msgr2=0x7fe06019bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 --2- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06019bef0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe05c011c30 tx=0x7fe05c004280 comp rx=0 tx=0).stop 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 shutdown_connections 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 --2- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe04c0390f0 0x7fe04c03b5b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 --2- 172.21.15.27:0/3488397953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe060108a40 0x7fe06019bef0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 >> 172.21.15.27:0/3488397953 conn(0x7fe0601023f0 msgr2=0x7fe060102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 shutdown_connections 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:02.447+0000 7fe067b3b700 1 -- 172.21.15.27:0/3488397953 wait complete. 2024-06-06T08:29:03.144 INFO:teuthology.orchestra.run.smithi027.stdout:Fetching dashboard port number... 2024-06-06T08:29:03.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:03 smithi027 ceph-mon[32335]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:03.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:04.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:04 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/212237271' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-06T08:29:04.624 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 8443 2024-06-06T08:29:04.624 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.592+0000 7f9eabe2b700 1 Processor -- start 2024-06-06T08:29:04.624 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.592+0000 7f9eabe2b700 1 -- start start 2024-06-06T08:29:04.624 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.593+0000 7f9eabe2b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea410ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:04.624 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.593+0000 7f9eabe2b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea407c760 con 0x7f9ea4108a20 2024-06-06T08:29:04.624 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.593+0000 7f9ea9bc7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea410ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.593+0000 7f9ea9bc7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea410ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41692/0 (socket says 172.21.15.27:41692) 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.593+0000 7f9ea9bc7700 1 -- 172.21.15.27:0/1915436395 learned_addr learned my addr 172.21.15.27:0/1915436395 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.593+0000 7f9ea9bc7700 1 -- 172.21.15.27:0/1915436395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea410b350 con 0x7f9ea4108a20 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.593+0000 7f9ea9bc7700 1 --2- 172.21.15.27:0/1915436395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea410ae10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00098a0 tx=0x7f9ea0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cf8ac86204ca9c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.593+0000 7f9ea8bc5700 1 -- 172.21.15.27:0/1915436395 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0004160 con 0x7f9ea4108a20 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.594+0000 7f9ea8bc5700 1 -- 172.21.15.27:0/1915436395 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea0004300 con 0x7f9ea4108a20 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.594+0000 7f9ea8bc5700 1 -- 172.21.15.27:0/1915436395 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0011b20 con 0x7f9ea4108a20 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.594+0000 7f9eabe2b700 1 -- 172.21.15.27:0/1915436395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 msgr2=0x7f9ea410ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.594+0000 7f9eabe2b700 1 --2- 172.21.15.27:0/1915436395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea410ae10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00098a0 tx=0x7f9ea0009bb0 comp rx=0 tx=0).stop 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.594+0000 7f9eabe2b700 1 -- 172.21.15.27:0/1915436395 shutdown_connections 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.594+0000 7f9eabe2b700 1 --2- 172.21.15.27:0/1915436395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea410ae10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.594+0000 7f9eabe2b700 1 -- 172.21.15.27:0/1915436395 >> 172.21.15.27:0/1915436395 conn(0x7f9ea4102450 msgr2=0x7f9ea4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:04.625 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.594+0000 7f9eabe2b700 1 -- 172.21.15.27:0/1915436395 shutdown_connections 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.594+0000 7f9eabe2b700 1 -- 172.21.15.27:0/1915436395 wait complete. 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.595+0000 7f9eabe2b700 1 Processor -- start 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.595+0000 7f9eabe2b700 1 -- start start 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.595+0000 7f9eabe2b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea41082a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.595+0000 7f9eabe2b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea410c3c0 con 0x7f9ea4108a20 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.595+0000 7f9ea9bc7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea41082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.595+0000 7f9ea9bc7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea41082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41704/0 (socket says 172.21.15.27:41704) 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.595+0000 7f9ea9bc7700 1 -- 172.21.15.27:0/212237271 learned_addr learned my addr 172.21.15.27:0/212237271 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.596+0000 7f9ea9bc7700 1 -- 172.21.15.27:0/212237271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea41087e0 con 0x7f9ea4108a20 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.596+0000 7f9ea9bc7700 1 --2- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea41082a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00117e0 tx=0x7f9ea0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.596+0000 7f9e9affd700 1 -- 172.21.15.27:0/212237271 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0003bf0 con 0x7f9ea4108a20 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.596+0000 7f9e9affd700 1 -- 172.21.15.27:0/212237271 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea001a910 con 0x7f9ea4108a20 2024-06-06T08:29:04.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.596+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea41069d0 con 0x7f9ea4108a20 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.596+0000 7f9e9affd700 1 -- 172.21.15.27:0/212237271 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0024970 con 0x7f9ea4108a20 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.596+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea4106e90 con 0x7f9ea4108a20 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.597+0000 7f9e9affd700 1 -- 172.21.15.27:0/212237271 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f9ea0022020 con 0x7f9ea4108a20 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.597+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea4063af0 con 0x7f9ea4108a20 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.597+0000 7f9e9affd700 1 --2- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9e900390b0 0x7f9e9003b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.597+0000 7f9e9affd700 1 -- 172.21.15.27:0/212237271 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9ea004d470 con 0x7f9ea4108a20 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.597+0000 7f9ea93c6700 1 --2- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9e900390b0 0x7f9e9003b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.598+0000 7f9ea93c6700 1 --2- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9e900390b0 0x7f9e9003b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e94006fd0 tx=0x7f9e94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.602+0000 7f9e9affd700 1 -- 172.21.15.27:0/212237271 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ea0011e20 con 0x7f9ea4108a20 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.811+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f9ea4107110 con 0x7f9ea4108a20 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.811+0000 7f9e9affd700 1 -- 172.21.15.27:0/212237271 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f9ea0016260 con 0x7f9ea4108a20 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9e900390b0 msgr2=0x7f9e9003b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:04.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 --2- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9e900390b0 0x7f9e9003b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e94006fd0 tx=0x7f9e94006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:04.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 msgr2=0x7f9ea41082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:04.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 --2- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea41082a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00117e0 tx=0x7f9ea0011c10 comp rx=0 tx=0).stop 2024-06-06T08:29:04.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 shutdown_connections 2024-06-06T08:29:04.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 --2- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9e900390b0 0x7f9e9003b570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:04.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 --2- 172.21.15.27:0/212237271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea4108a20 0x7f9ea41082a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:04.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 >> 172.21.15.27:0/212237271 conn(0x7f9ea4102450 msgr2=0x7f9ea4102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:04.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 shutdown_connections 2024-06-06T08:29:04.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:03.814+0000 7f9eabe2b700 1 -- 172.21.15.27:0/212237271 wait complete. 2024-06-06T08:29:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T08:29:04.635 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-06-06T08:29:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T08:29:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-06-06T08:29:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-06-06T08:29:04.643 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-06T08:29:04.645 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph Dashboard is now available at: 2024-06-06T08:29:04.646 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:04.646 INFO:teuthology.orchestra.run.smithi027.stdout: URL: https://smithi027.front.sepia.ceph.com:8443/ 2024-06-06T08:29:04.646 INFO:teuthology.orchestra.run.smithi027.stdout: User: admin 2024-06-06T08:29:04.646 INFO:teuthology.orchestra.run.smithi027.stdout: Password: 90x9ohf2fo 2024-06-06T08:29:04.646 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:04.646 INFO:teuthology.orchestra.run.smithi027.stdout:Saving cluster configuration to /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config directory 2024-06-06T08:29:04.647 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling autotune for osd_memory_target 2024-06-06T08:29:05.564 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:05 smithi027 ceph-mon[32335]: mgrmap e13: x(active, since 6s) 2024-06-06T08:29:05.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.093+0000 7fd42e8d8700 1 Processor -- start 2024-06-06T08:29:05.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.093+0000 7fd42e8d8700 1 -- start start 2024-06-06T08:29:05.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.093+0000 7fd42e8d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd428109af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:05.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.093+0000 7fd42e8d8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd428107c60 con 0x7fd4281096d0 2024-06-06T08:29:05.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.093+0000 7fd427fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd428109af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.093+0000 7fd427fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd428109af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41710/0 (socket says 172.21.15.27:41710) 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.093+0000 7fd427fff700 1 -- 172.21.15.27:0/3307701669 learned_addr learned my addr 172.21.15.27:0/3307701669 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.093+0000 7fd427fff700 1 -- 172.21.15.27:0/3307701669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd428107de0 con 0x7fd4281096d0 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.094+0000 7fd427fff700 1 --2- 172.21.15.27:0/3307701669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd428109af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4180098a0 tx=0x7fd418009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845db0e234a8b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.094+0000 7fd426ffd700 1 -- 172.21.15.27:0/3307701669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd418004160 con 0x7fd4281096d0 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.094+0000 7fd426ffd700 1 -- 172.21.15.27:0/3307701669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd418004300 con 0x7fd4281096d0 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.094+0000 7fd426ffd700 1 -- 172.21.15.27:0/3307701669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd418011b20 con 0x7fd4281096d0 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.094+0000 7fd42e8d8700 1 -- 172.21.15.27:0/3307701669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 msgr2=0x7fd428109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.094+0000 7fd42e8d8700 1 --2- 172.21.15.27:0/3307701669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd428109af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4180098a0 tx=0x7fd418009bb0 comp rx=0 tx=0).stop 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.094+0000 7fd42e8d8700 1 -- 172.21.15.27:0/3307701669 shutdown_connections 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.094+0000 7fd42e8d8700 1 --2- 172.21.15.27:0/3307701669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd428109af0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:05.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.094+0000 7fd42e8d8700 1 -- 172.21.15.27:0/3307701669 >> 172.21.15.27:0/3307701669 conn(0x7fd4281035e0 msgr2=0x7fd428105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.095+0000 7fd42e8d8700 1 -- 172.21.15.27:0/3307701669 shutdown_connections 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.095+0000 7fd42e8d8700 1 -- 172.21.15.27:0/3307701669 wait complete. 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.095+0000 7fd42e8d8700 1 Processor -- start 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.095+0000 7fd42e8d8700 1 -- start start 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd42e8d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd42819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd42e8d8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd42810c5a0 con 0x7fd4281096d0 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd427fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd42819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd427fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd42819c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41714/0 (socket says 172.21.15.27:41714) 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd427fff700 1 -- 172.21.15.27:0/2675082642 learned_addr learned my addr 172.21.15.27:0/2675082642 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd427fff700 1 -- 172.21.15.27:0/2675082642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42819c590 con 0x7fd4281096d0 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd427fff700 1 --2- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd42819c050 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd4180117e0 tx=0x7fd418011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd4257fa700 1 -- 172.21.15.27:0/2675082642 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd418003bf0 con 0x7fd4281096d0 2024-06-06T08:29:05.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd4257fa700 1 -- 172.21.15.27:0/2675082642 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41801a910 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42819c820 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd4257fa700 1 -- 172.21.15.27:0/2675082642 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd418024970 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.096+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42819cc80 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.097+0000 7fd4257fa700 1 -- 172.21.15.27:0/2675082642 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd418022020 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.097+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd428063af0 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.098+0000 7fd4257fa700 1 --2- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd4100391a0 0x7fd41003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.098+0000 7fd4257fa700 1 -- 172.21.15.27:0/2675082642 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd41804d860 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.098+0000 7fd4277fe700 1 --2- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd4100391a0 0x7fd41003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.098+0000 7fd4277fe700 1 --2- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd4100391a0 0x7fd41003b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd41c006fd0 tx=0x7fd41c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.103+0000 7fd4257fa700 1 -- 172.21.15.27:0/2675082642 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd41800b3e0 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.306+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fd42819f510 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.307+0000 7fd4257fa700 1 -- 172.21.15.27:0/2675082642 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fd42819f510 con 0x7fd4281096d0 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd4100391a0 msgr2=0x7fd41003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:05.849 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 --2- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd4100391a0 0x7fd41003b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd41c006fd0 tx=0x7fd41c006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:05.850 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 msgr2=0x7fd42819c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:05.850 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 --2- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd42819c050 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd4180117e0 tx=0x7fd418011ba0 comp rx=0 tx=0).stop 2024-06-06T08:29:05.850 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 shutdown_connections 2024-06-06T08:29:05.850 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 --2- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd4100391a0 0x7fd41003b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:05.850 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 --2- 172.21.15.27:0/2675082642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4281096d0 0x7fd42819c050 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:05.850 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 >> 172.21.15.27:0/2675082642 conn(0x7fd4281035e0 msgr2=0x7fd428104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:05.850 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 shutdown_connections 2024-06-06T08:29:05.850 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:05.309+0000 7fd42e8d8700 1 -- 172.21.15.27:0/2675082642 wait complete. 2024-06-06T08:29:07.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.295+0000 7f06a5a47700 1 Processor -- start 2024-06-06T08:29:07.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.295+0000 7f06a5a47700 1 -- start start 2024-06-06T08:29:07.228 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.296+0000 7f06a5a47700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0108440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.296+0000 7f06a5a47700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06a0108a10 con 0x7f06a0108020 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.296+0000 7f069effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.296+0000 7f069effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0108440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41726/0 (socket says 172.21.15.27:41726) 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.296+0000 7f069effd700 1 -- 172.21.15.27:0/1101286588 learned_addr learned my addr 172.21.15.27:0/1101286588 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.296+0000 7f069effd700 1 -- 172.21.15.27:0/1101286588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a01091d0 con 0x7f06a0108020 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.296+0000 7f069effd700 1 --2- 172.21.15.27:0/1101286588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0108440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f06900098a0 tx=0x7f0690009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3773cb61488c6add server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f069dffb700 1 -- 172.21.15.27:0/1101286588 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0690004160 con 0x7f06a0108020 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f069dffb700 1 -- 172.21.15.27:0/1101286588 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0690004300 con 0x7f06a0108020 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f069dffb700 1 -- 172.21.15.27:0/1101286588 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0690011b20 con 0x7f06a0108020 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f06a5a47700 1 -- 172.21.15.27:0/1101286588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 msgr2=0x7f06a0108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f06a5a47700 1 --2- 172.21.15.27:0/1101286588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0108440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f06900098a0 tx=0x7f0690009bb0 comp rx=0 tx=0).stop 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f06a5a47700 1 -- 172.21.15.27:0/1101286588 shutdown_connections 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f06a5a47700 1 --2- 172.21.15.27:0/1101286588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0108440 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:07.229 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f06a5a47700 1 -- 172.21.15.27:0/1101286588 >> 172.21.15.27:0/1101286588 conn(0x7f06a0103620 msgr2=0x7f06a0105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f06a5a47700 1 -- 172.21.15.27:0/1101286588 shutdown_connections 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.297+0000 7f06a5a47700 1 -- 172.21.15.27:0/1101286588 wait complete. 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.298+0000 7f06a5a47700 1 Processor -- start 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.298+0000 7f06a5a47700 1 -- start start 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.298+0000 7f06a5a47700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.298+0000 7f06a5a47700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06a007c990 con 0x7f06a0108020 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.298+0000 7f069effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.299+0000 7f069effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41734/0 (socket says 172.21.15.27:41734) 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.299+0000 7f069effd700 1 -- 172.21.15.27:0/629059499 learned_addr learned my addr 172.21.15.27:0/629059499 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.299+0000 7f069effd700 1 -- 172.21.15.27:0/629059499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a019a280 con 0x7f06a0108020 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.299+0000 7f069effd700 1 --2- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0199d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06900117e0 tx=0x7f0690011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:07.230 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.299+0000 7f06a4a45700 1 -- 172.21.15.27:0/629059499 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0690003bf0 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.299+0000 7f06a4a45700 1 -- 172.21.15.27:0/629059499 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069001a910 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.299+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a019a510 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.299+0000 7f06a4a45700 1 -- 172.21.15.27:0/629059499 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06900249a0 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.299+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a019a970 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.300+0000 7f06a4a45700 1 -- 172.21.15.27:0/629059499 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0690022020 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.300+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a0063af0 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.300+0000 7f06a4a45700 1 --2- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f068c03d520 0x7f068c03f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.300+0000 7f06a4a45700 1 -- 172.21.15.27:0/629059499 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f069004d5b0 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.301+0000 7f069e7fc700 1 --2- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f068c03d520 0x7f068c03f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.301+0000 7f069e7fc700 1 --2- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f068c03d520 0x7f068c03f9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0688006fd0 tx=0x7f0688006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.306+0000 7f06a4a45700 1 -- 172.21.15.27:0/629059499 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0690011e20 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.631+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f06a00630a0 con 0x7f06a0108020 2024-06-06T08:29:07.231 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.634+0000 7f06a4a45700 1 -- 172.21.15.27:0/629059499 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f069000b3e0 con 0x7f06a0108020 2024-06-06T08:29:07.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f068c03d520 msgr2=0x7f068c03f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:07.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 --2- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f068c03d520 0x7f068c03f9e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0688006fd0 tx=0x7f0688006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:07.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 msgr2=0x7f06a0199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:07.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 --2- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0199d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06900117e0 tx=0x7f0690011c10 comp rx=0 tx=0).stop 2024-06-06T08:29:07.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 shutdown_connections 2024-06-06T08:29:07.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 --2- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f068c03d520 0x7f068c03f9e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:07.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 --2- 172.21.15.27:0/629059499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06a0108020 0x7f06a0199d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:07.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 >> 172.21.15.27:0/629059499 conn(0x7f06a0103620 msgr2=0x7f06a0104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:07.232 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 shutdown_connections 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-06-06T08:29:06.636+0000 7f06a5a47700 1 -- 172.21.15.27:0/629059499 wait complete. 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout:Or, if you are only running a single cluster on this host: 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: ceph telemetry on 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:07.233 INFO:teuthology.orchestra.run.smithi027.stdout:For more information see: 2024-06-06T08:29:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-06T08:29:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:07.234 INFO:teuthology.orchestra.run.smithi027.stdout:Bootstrap complete. 2024-06-06T08:29:07.270 INFO:tasks.cephadm:Fetching config... 2024-06-06T08:29:07.271 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:29:07.271 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-06T08:29:07.288 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-06T08:29:07.288 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:29:07.288 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-06T08:29:07.343 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-06T08:29:07.344 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:29:07.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-06T08:29:07.409 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-06T08:29:07.410 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:29:07.410 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-06T08:29:07.465 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-06T08:29:07.465 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN2w5SzjTE2hZXcsQjyC5WMYUZ83Q06jlZCh81b8C3hpflU77tI5tj7lzMgdVUviaTMr3bfhqJmfegufNjsRx0rGUV+c7ALSCyPdBeLLkRC9USxZnI6xJTmR01EwDKoNDRzP7jSW1Lv5MGO1IyvwyyKv4Wptw1XozilO/o+HC5cBMuT/vXw5fUL10e5GPoONz2C2Q6b+Aa1XqpxZ6KE1kty5W5G/kS6jcZD9u6ZqWdWvKRUo8RNwh2jNzrQTSUepRa5til6NGgdQruyA47TNwfbUUYldz49NT3f4MC/FiBBBfUhnQ+dVLzyCTZZDu6UHaM+V/dAP0fDahemFt9OvTD79X45jjx6haVArrGTglPn5Q72rIZugi1wasYv/5dcSbxMvGn1Q6cyI/+tutJumRlr1m7Ag3xruasLUefwyoshV3IYUGdBJp9F8kzbe9k86ZPYUUTwJ+yCbYGtKXUFHMltd8Nfk6Bj5ezZwM6x/vVbF/xhGILLCHTmosNxLts/sM= ceph-8da40ada-23de-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T08:29:07.547 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN2w5SzjTE2hZXcsQjyC5WMYUZ83Q06jlZCh81b8C3hpflU77tI5tj7lzMgdVUviaTMr3bfhqJmfegufNjsRx0rGUV+c7ALSCyPdBeLLkRC9USxZnI6xJTmR01EwDKoNDRzP7jSW1Lv5MGO1IyvwyyKv4Wptw1XozilO/o+HC5cBMuT/vXw5fUL10e5GPoONz2C2Q6b+Aa1XqpxZ6KE1kty5W5G/kS6jcZD9u6ZqWdWvKRUo8RNwh2jNzrQTSUepRa5til6NGgdQruyA47TNwfbUUYldz49NT3f4MC/FiBBBfUhnQ+dVLzyCTZZDu6UHaM+V/dAP0fDahemFt9OvTD79X45jjx6haVArrGTglPn5Q72rIZugi1wasYv/5dcSbxMvGn1Q6cyI/+tutJumRlr1m7Ag3xruasLUefwyoshV3IYUGdBJp9F8kzbe9k86ZPYUUTwJ+yCbYGtKXUFHMltd8Nfk6Bj5ezZwM6x/vVbF/xhGILLCHTmosNxLts/sM= ceph-8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:07.561 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN2w5SzjTE2hZXcsQjyC5WMYUZ83Q06jlZCh81b8C3hpflU77tI5tj7lzMgdVUviaTMr3bfhqJmfegufNjsRx0rGUV+c7ALSCyPdBeLLkRC9USxZnI6xJTmR01EwDKoNDRzP7jSW1Lv5MGO1IyvwyyKv4Wptw1XozilO/o+HC5cBMuT/vXw5fUL10e5GPoONz2C2Q6b+Aa1XqpxZ6KE1kty5W5G/kS6jcZD9u6ZqWdWvKRUo8RNwh2jNzrQTSUepRa5til6NGgdQruyA47TNwfbUUYldz49NT3f4MC/FiBBBfUhnQ+dVLzyCTZZDu6UHaM+V/dAP0fDahemFt9OvTD79X45jjx6haVArrGTglPn5Q72rIZugi1wasYv/5dcSbxMvGn1Q6cyI/+tutJumRlr1m7Ag3xruasLUefwyoshV3IYUGdBJp9F8kzbe9k86ZPYUUTwJ+yCbYGtKXUFHMltd8Nfk6Bj5ezZwM6x/vVbF/xhGILLCHTmosNxLts/sM= ceph-8da40ada-23de-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T08:29:07.600 INFO:teuthology.orchestra.run.smithi176.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN2w5SzjTE2hZXcsQjyC5WMYUZ83Q06jlZCh81b8C3hpflU77tI5tj7lzMgdVUviaTMr3bfhqJmfegufNjsRx0rGUV+c7ALSCyPdBeLLkRC9USxZnI6xJTmR01EwDKoNDRzP7jSW1Lv5MGO1IyvwyyKv4Wptw1XozilO/o+HC5cBMuT/vXw5fUL10e5GPoONz2C2Q6b+Aa1XqpxZ6KE1kty5W5G/kS6jcZD9u6ZqWdWvKRUo8RNwh2jNzrQTSUepRa5til6NGgdQruyA47TNwfbUUYldz49NT3f4MC/FiBBBfUhnQ+dVLzyCTZZDu6UHaM+V/dAP0fDahemFt9OvTD79X45jjx6haVArrGTglPn5Q72rIZugi1wasYv/5dcSbxMvGn1Q6cyI/+tutJumRlr1m7Ag3xruasLUefwyoshV3IYUGdBJp9F8kzbe9k86ZPYUUTwJ+yCbYGtKXUFHMltd8Nfk6Bj5ezZwM6x/vVbF/xhGILLCHTmosNxLts/sM= ceph-8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:07.612 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN2w5SzjTE2hZXcsQjyC5WMYUZ83Q06jlZCh81b8C3hpflU77tI5tj7lzMgdVUviaTMr3bfhqJmfegufNjsRx0rGUV+c7ALSCyPdBeLLkRC9USxZnI6xJTmR01EwDKoNDRzP7jSW1Lv5MGO1IyvwyyKv4Wptw1XozilO/o+HC5cBMuT/vXw5fUL10e5GPoONz2C2Q6b+Aa1XqpxZ6KE1kty5W5G/kS6jcZD9u6ZqWdWvKRUo8RNwh2jNzrQTSUepRa5til6NGgdQruyA47TNwfbUUYldz49NT3f4MC/FiBBBfUhnQ+dVLzyCTZZDu6UHaM+V/dAP0fDahemFt9OvTD79X45jjx6haVArrGTglPn5Q72rIZugi1wasYv/5dcSbxMvGn1Q6cyI/+tutJumRlr1m7Ag3xruasLUefwyoshV3IYUGdBJp9F8kzbe9k86ZPYUUTwJ+yCbYGtKXUFHMltd8Nfk6Bj5ezZwM6x/vVbF/xhGILLCHTmosNxLts/sM= ceph-8da40ada-23de-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T08:29:07.651 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN2w5SzjTE2hZXcsQjyC5WMYUZ83Q06jlZCh81b8C3hpflU77tI5tj7lzMgdVUviaTMr3bfhqJmfegufNjsRx0rGUV+c7ALSCyPdBeLLkRC9USxZnI6xJTmR01EwDKoNDRzP7jSW1Lv5MGO1IyvwyyKv4Wptw1XozilO/o+HC5cBMuT/vXw5fUL10e5GPoONz2C2Q6b+Aa1XqpxZ6KE1kty5W5G/kS6jcZD9u6ZqWdWvKRUo8RNwh2jNzrQTSUepRa5til6NGgdQruyA47TNwfbUUYldz49NT3f4MC/FiBBBfUhnQ+dVLzyCTZZDu6UHaM+V/dAP0fDahemFt9OvTD79X45jjx6haVArrGTglPn5Q72rIZugi1wasYv/5dcSbxMvGn1Q6cyI/+tutJumRlr1m7Ag3xruasLUefwyoshV3IYUGdBJp9F8kzbe9k86ZPYUUTwJ+yCbYGtKXUFHMltd8Nfk6Bj5ezZwM6x/vVbF/xhGILLCHTmosNxLts/sM= ceph-8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:07.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-06T08:29:07.908 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:29:07.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:07 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/629059499' entity='client.admin' 2024-06-06T08:29:10.266 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:10 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:10.266 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:10 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:10.266 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:10 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:29:10.269 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:10 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:10.269 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:10 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:10.270 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:10 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.282+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/3918657614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 msgr2=0x7f6ae4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.282+0000 7f6aeb1a8700 1 --2- 172.21.15.27:0/3918657614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 0x7f6ae4104e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009910 tx=0x7f6ad8009c20 comp rx=0 tx=0).stop 2024-06-06T08:29:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.282+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/3918657614 shutdown_connections 2024-06-06T08:29:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.282+0000 7f6aeb1a8700 1 --2- 172.21.15.27:0/3918657614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 0x7f6ae4104e30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.282+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/3918657614 >> 172.21.15.27:0/3918657614 conn(0x7f6ae40fc390 msgr2=0x7f6ae40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.282+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/3918657614 shutdown_connections 2024-06-06T08:29:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.282+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/3918657614 wait complete. 2024-06-06T08:29:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.283+0000 7f6aeb1a8700 1 Processor -- start 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.283+0000 7f6aeb1a8700 1 -- start start 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.283+0000 7f6aeb1a8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 0x7f6ae41022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.283+0000 7f6aeb1a8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4107f80 con 0x7f6ae4102a40 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.283+0000 7f6ae8f44700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 0x7f6ae41022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.283+0000 7f6ae8f44700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 0x7f6ae41022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60696/0 (socket says 172.21.15.27:60696) 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.283+0000 7f6ae8f44700 1 -- 172.21.15.27:0/2096091917 learned_addr learned my addr 172.21.15.27:0/2096091917 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.283+0000 7f6ae8f44700 1 -- 172.21.15.27:0/2096091917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae4102800 con 0x7f6ae4102a40 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.284+0000 7f6ae8f44700 1 --2- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 0x7f6ae41022c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8004fa0 tx=0x7f6ad8005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.284+0000 7f6ae1ffb700 1 -- 172.21.15.27:0/2096091917 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad8027070 con 0x7f6ae4102a40 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.284+0000 7f6ae1ffb700 1 -- 172.21.15.27:0/2096091917 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad800be40 con 0x7f6ae4102a40 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.284+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae41009f0 con 0x7f6ae4102a40 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.284+0000 7f6ae1ffb700 1 -- 172.21.15.27:0/2096091917 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ad800f460 con 0x7f6ae4102a40 2024-06-06T08:29:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.284+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae4100f30 con 0x7f6ae4102a40 2024-06-06T08:29:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.285+0000 7f6ae1ffb700 1 -- 172.21.15.27:0/2096091917 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6ad802a070 con 0x7f6ae4102a40 2024-06-06T08:29:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.285+0000 7f6ae1ffb700 1 --2- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ad4039150 0x7f6ad403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.285+0000 7f6ae1ffb700 1 -- 172.21.15.27:0/2096091917 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6ad801e030 con 0x7f6ae4102a40 2024-06-06T08:29:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.285+0000 7f6ae3fff700 1 --2- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ad4039150 0x7f6ad403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.285+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae4108700 con 0x7f6ae4102a40 2024-06-06T08:29:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.286+0000 7f6ae3fff700 1 --2- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ad4039150 0x7f6ad403b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0006fd0 tx=0x7f6ad0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.291+0000 7f6ae1ffb700 1 -- 172.21.15.27:0/2096091917 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ad8013070 con 0x7f6ae4102a40 2024-06-06T08:29:10.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.510+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f6ae41908f0 con 0x7f6ae4102a40 2024-06-06T08:29:10.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.512+0000 7f6ae1ffb700 1 -- 172.21.15.27:0/2096091917 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f6ad8035030 con 0x7f6ae4102a40 2024-06-06T08:29:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ad4039150 msgr2=0x7f6ad403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 --2- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ad4039150 0x7f6ad403b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0006fd0 tx=0x7f6ad0006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 msgr2=0x7f6ae41022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 --2- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 0x7f6ae41022c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8004fa0 tx=0x7f6ad8005080 comp rx=0 tx=0).stop 2024-06-06T08:29:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 shutdown_connections 2024-06-06T08:29:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 --2- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ad4039150 0x7f6ad403b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 --2- 172.21.15.27:0/2096091917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae4102a40 0x7f6ae41022c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 >> 172.21.15.27:0/2096091917 conn(0x7f6ae40fc390 msgr2=0x7f6ae40fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 shutdown_connections 2024-06-06T08:29:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:10.515+0000 7f6aeb1a8700 1 -- 172.21.15.27:0/2096091917 wait complete. 2024-06-06T08:29:11.081 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-06T08:29:11.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-06T08:29:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:29:11.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:11 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2096091917' entity='client.admin' 2024-06-06T08:29:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:13.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.298+0000 7f91a5ab0700 1 -- 172.21.15.27:0/377867276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 msgr2=0x7f91a0101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:13.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.298+0000 7f91a5ab0700 1 --2- 172.21.15.27:0/377867276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 0x7f91a0101230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9190009910 tx=0x7f9190009c20 comp rx=0 tx=0).stop 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.298+0000 7f91a5ab0700 1 -- 172.21.15.27:0/377867276 shutdown_connections 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.298+0000 7f91a5ab0700 1 --2- 172.21.15.27:0/377867276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 0x7f91a0101230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.298+0000 7f91a5ab0700 1 -- 172.21.15.27:0/377867276 >> 172.21.15.27:0/377867276 conn(0x7f91a00fc3d0 msgr2=0x7f91a00fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.298+0000 7f91a5ab0700 1 -- 172.21.15.27:0/377867276 shutdown_connections 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.298+0000 7f91a5ab0700 1 -- 172.21.15.27:0/377867276 wait complete. 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.298+0000 7f91a5ab0700 1 Processor -- start 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.299+0000 7f91a5ab0700 1 -- start start 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.299+0000 7f91a5ab0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 0x7f91a006ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.299+0000 7f91a5ab0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0106e20 con 0x7f91a0100e10 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.299+0000 7f919f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 0x7f91a006ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.299+0000 7f919f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 0x7f91a006ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60722/0 (socket says 172.21.15.27:60722) 2024-06-06T08:29:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.299+0000 7f919f7fe700 1 -- 172.21.15.27:0/891869941 learned_addr learned my addr 172.21.15.27:0/891869941 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:13.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.299+0000 7f919f7fe700 1 -- 172.21.15.27:0/891869941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a019a3f0 con 0x7f91a0100e10 2024-06-06T08:29:13.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.299+0000 7f919f7fe700 1 --2- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 0x7f91a006ad80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f91900052c0 tx=0x7f9190005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:13.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.300+0000 7f919d7fa700 1 -- 172.21.15.27:0/891869941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9190026070 con 0x7f91a0100e10 2024-06-06T08:29:13.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.300+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a019a5c0 con 0x7f91a0100e10 2024-06-06T08:29:13.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.300+0000 7f919d7fa700 1 -- 172.21.15.27:0/891869941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f919000bc30 con 0x7f91a0100e10 2024-06-06T08:29:13.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.300+0000 7f919d7fa700 1 -- 172.21.15.27:0/891869941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9190003b10 con 0x7f91a0100e10 2024-06-06T08:29:13.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.300+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a019a960 con 0x7f91a0100e10 2024-06-06T08:29:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.301+0000 7f919d7fa700 1 -- 172.21.15.27:0/891869941 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9190029070 con 0x7f91a0100e10 2024-06-06T08:29:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.301+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a01075a0 con 0x7f91a0100e10 2024-06-06T08:29:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.301+0000 7f919d7fa700 1 --2- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f918c0390f0 0x7f918c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.301+0000 7f919d7fa700 1 -- 172.21.15.27:0/891869941 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f919001e070 con 0x7f91a0100e10 2024-06-06T08:29:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.301+0000 7f9196ffd700 1 --2- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f918c0390f0 0x7f918c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.301+0000 7f9196ffd700 1 --2- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f918c0390f0 0x7f918c03b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9188006fd0 tx=0x7f9188006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:13.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.306+0000 7f919d7fa700 1 -- 172.21.15.27:0/891869941 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9190013070 con 0x7f91a0100e10 2024-06-06T08:29:13.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.522+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f91a002dee0 con 0x7f918c0390f0 2024-06-06T08:29:13.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.525+0000 7f919d7fa700 1 -- 172.21.15.27:0/891869941 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f91a002dee0 con 0x7f918c0390f0 2024-06-06T08:29:13.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f918c0390f0 msgr2=0x7f918c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:13.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 --2- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f918c0390f0 0x7f918c03b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9188006fd0 tx=0x7f9188006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:13.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 msgr2=0x7f91a006ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:13.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 --2- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 0x7f91a006ad80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f91900052c0 tx=0x7f9190005030 comp rx=0 tx=0).stop 2024-06-06T08:29:13.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 shutdown_connections 2024-06-06T08:29:13.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 --2- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f918c0390f0 0x7f918c03b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:13.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 --2- 172.21.15.27:0/891869941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91a0100e10 0x7f91a006ad80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:13.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 >> 172.21.15.27:0/891869941 conn(0x7f91a00fc3d0 msgr2=0x7f91a00fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:13.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 shutdown_connections 2024-06-06T08:29:13.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:13.527+0000 7f91a5ab0700 1 -- 172.21.15.27:0/891869941 wait complete. 2024-06-06T08:29:14.248 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi176 2024-06-06T08:29:14.248 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:29:14.248 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.conf 2024-06-06T08:29:14.267 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:29:14.268 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:14.325 INFO:tasks.cephadm:Adding host smithi176 to orchestrator... 2024-06-06T08:29:14.325 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch host add smithi176 2024-06-06T08:29:14.576 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:29:14.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:14 smithi027 ceph-mon[32335]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:14.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:14.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:14.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:14.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:14.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:14 smithi027 ceph-mon[32335]: Updating smithi027:/etc/ceph/ceph.conf 2024-06-06T08:29:14.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:14 smithi027 ceph-mon[32335]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:29:14.862 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:14 smithi027 ceph-mon[32335]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:15.922 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:15 smithi027 ceph-mon[32335]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:29:15.922 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:15.922 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:15.922 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:16.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.165+0000 7ff44fad6700 1 -- 172.21.15.27:0/80264862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 msgr2=0x7ff4481013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:16.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.165+0000 7ff44fad6700 1 --2- 172.21.15.27:0/80264862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 0x7ff4481013e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff444009910 tx=0x7ff444009c20 comp rx=0 tx=0).stop 2024-06-06T08:29:16.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.166+0000 7ff44fad6700 1 -- 172.21.15.27:0/80264862 shutdown_connections 2024-06-06T08:29:16.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.166+0000 7ff44fad6700 1 --2- 172.21.15.27:0/80264862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 0x7ff4481013e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:16.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.166+0000 7ff44fad6700 1 -- 172.21.15.27:0/80264862 >> 172.21.15.27:0/80264862 conn(0x7ff4480fc890 msgr2=0x7ff4480fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:16.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.166+0000 7ff44fad6700 1 -- 172.21.15.27:0/80264862 shutdown_connections 2024-06-06T08:29:16.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.166+0000 7ff44fad6700 1 -- 172.21.15.27:0/80264862 wait complete. 2024-06-06T08:29:16.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.166+0000 7ff44fad6700 1 Processor -- start 2024-06-06T08:29:16.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.166+0000 7ff44fad6700 1 -- start start 2024-06-06T08:29:16.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.166+0000 7ff44fad6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 0x7ff44819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:16.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.166+0000 7ff44fad6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810b3d0 con 0x7ff448101000 2024-06-06T08:29:16.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff44d872700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 0x7ff44819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:16.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff44d872700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 0x7ff44819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60736/0 (socket says 172.21.15.27:60736) 2024-06-06T08:29:16.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff44d872700 1 -- 172.21.15.27:0/3395717279 learned_addr learned my addr 172.21.15.27:0/3395717279 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:16.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff44d872700 1 -- 172.21.15.27:0/3395717279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44819fd60 con 0x7ff448101000 2024-06-06T08:29:16.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff44d872700 1 --2- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 0x7ff44819f820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff444005850 tx=0x7ff444005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:16.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff43affd700 1 -- 172.21.15.27:0/3395717279 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff444026070 con 0x7ff448101000 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4481a0050 con 0x7ff448101000 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff43affd700 1 -- 172.21.15.27:0/3395717279 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff44400bbd0 con 0x7ff448101000 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff43affd700 1 -- 172.21.15.27:0/3395717279 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff444003970 con 0x7ff448101000 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.167+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4481999e0 con 0x7ff448101000 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.168+0000 7ff43affd700 1 -- 172.21.15.27:0/3395717279 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff444029070 con 0x7ff448101000 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.168+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4481024e0 con 0x7ff448101000 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.168+0000 7ff43affd700 1 --2- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff434039150 0x7ff43403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.168+0000 7ff43affd700 1 -- 172.21.15.27:0/3395717279 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff44401e070 con 0x7ff448101000 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.169+0000 7ff44d071700 1 --2- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff434039150 0x7ff43403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:16.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.169+0000 7ff44d071700 1 --2- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff434039150 0x7ff43403b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006fd0 tx=0x7ff43c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.174+0000 7ff43affd700 1 -- 172.21.15.27:0/3395717279 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff444013070 con 0x7ff448101000 2024-06-06T08:29:16.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:16.386+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi176", "target": ["mon-mgr", ""]}) -- 0x7ff44819af10 con 0x7ff434039150 2024-06-06T08:29:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:17 smithi027 ceph-mon[32335]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi176", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:17 smithi027 ceph-mon[32335]: Deploying cephadm binary to smithi176 2024-06-06T08:29:19.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:19 smithi027 ceph-mon[32335]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.657+0000 7ff43affd700 1 -- 172.21.15.27:0/3395717279 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff44819af10 con 0x7ff434039150 2024-06-06T08:29:20.656 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi176' with addr '172.21.15.176' 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff434039150 msgr2=0x7ff43403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 --2- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff434039150 0x7ff43403b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006fd0 tx=0x7ff43c006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 msgr2=0x7ff44819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 --2- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 0x7ff44819f820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff444005850 tx=0x7ff444005030 comp rx=0 tx=0).stop 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 shutdown_connections 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 --2- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff434039150 0x7ff43403b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 --2- 172.21.15.27:0/3395717279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff448101000 0x7ff44819f820 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 >> 172.21.15.27:0/3395717279 conn(0x7ff4480fc890 msgr2=0x7ff44806a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 shutdown_connections 2024-06-06T08:29:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:20.660+0000 7ff44fad6700 1 -- 172.21.15.27:0/3395717279 wait complete. 2024-06-06T08:29:21.295 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T08:29:21.547 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:29:21.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:21 smithi027 ceph-mon[32335]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:21.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:21.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:21 smithi027 ceph-mon[32335]: Added host smithi176 2024-06-06T08:29:21.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:21.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:22.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:22 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.168+0000 7f8b6dba8700 1 -- 172.21.15.27:0/1566851654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 msgr2=0x7f8b681033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.168+0000 7f8b6dba8700 1 --2- 172.21.15.27:0/1566851654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 0x7f8b681033c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800b0a0 tx=0x7f8b5800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:29:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.168+0000 7f8b6dba8700 1 -- 172.21.15.27:0/1566851654 shutdown_connections 2024-06-06T08:29:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.168+0000 7f8b6dba8700 1 --2- 172.21.15.27:0/1566851654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 0x7f8b681033c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.168+0000 7f8b6dba8700 1 -- 172.21.15.27:0/1566851654 >> 172.21.15.27:0/1566851654 conn(0x7f8b680fc8e0 msgr2=0x7f8b680fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.168+0000 7f8b6dba8700 1 -- 172.21.15.27:0/1566851654 shutdown_connections 2024-06-06T08:29:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.168+0000 7f8b6dba8700 1 -- 172.21.15.27:0/1566851654 wait complete. 2024-06-06T08:29:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.169+0000 7f8b6dba8700 1 Processor -- start 2024-06-06T08:29:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.169+0000 7f8b6dba8700 1 -- start start 2024-06-06T08:29:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.169+0000 7f8b6dba8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 0x7f8b68100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.169+0000 7f8b6dba8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6810c540 con 0x7f8b68102fe0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.169+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 0x7f8b68100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.169+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 0x7f8b68100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41764/0 (socket says 172.21.15.27:41764) 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.169+0000 7f8b677fe700 1 -- 172.21.15.27:0/732893392 learned_addr learned my addr 172.21.15.27:0/732893392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.169+0000 7f8b677fe700 1 -- 172.21.15.27:0/732893392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b68102bc0 con 0x7f8b68102fe0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.169+0000 7f8b677fe700 1 --2- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 0x7f8b68100cd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b580062a0 tx=0x7f8b58003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.170+0000 7f8b64ff9700 1 -- 172.21.15.27:0/732893392 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b58017040 con 0x7f8b68102fe0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.170+0000 7f8b64ff9700 1 -- 172.21.15.27:0/732893392 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b58022070 con 0x7f8b68102fe0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.170+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b68101210 con 0x7f8b68102fe0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.170+0000 7f8b64ff9700 1 -- 172.21.15.27:0/732893392 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b5801d440 con 0x7f8b68102fe0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.170+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b681016d0 con 0x7f8b68102fe0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.171+0000 7f8b64ff9700 1 -- 172.21.15.27:0/732893392 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8b5801d730 con 0x7f8b68102fe0 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.171+0000 7f8b64ff9700 1 --2- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8b54039150 0x7f8b5403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:23.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.171+0000 7f8b64ff9700 1 -- 172.21.15.27:0/732893392 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8b5800f050 con 0x7f8b68102fe0 2024-06-06T08:29:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.171+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6806ab80 con 0x7f8b68102fe0 2024-06-06T08:29:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.171+0000 7f8b66ffd700 1 --2- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8b54039150 0x7f8b5403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.171+0000 7f8b66ffd700 1 --2- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8b54039150 0x7f8b5403b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8b50006fd0 tx=0x7f8b50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.176+0000 7f8b64ff9700 1 -- 172.21.15.27:0/732893392 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8b58007e20 con 0x7f8b68102fe0 2024-06-06T08:29:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.391+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b68102200 con 0x7f8b54039150 2024-06-06T08:29:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.392+0000 7f8b64ff9700 1 -- 172.21.15.27:0/732893392 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f8b68102200 con 0x7f8b54039150 2024-06-06T08:29:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:23.391 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.176", "hostname": "smithi176", "labels": [], "status": ""}] 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8b54039150 msgr2=0x7f8b5403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 --2- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8b54039150 0x7f8b5403b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8b50006fd0 tx=0x7f8b50006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 msgr2=0x7f8b68100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 --2- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 0x7f8b68100cd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b580062a0 tx=0x7f8b58003ce0 comp rx=0 tx=0).stop 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 shutdown_connections 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 --2- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8b54039150 0x7f8b5403b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 --2- 172.21.15.27:0/732893392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b68102fe0 0x7f8b68100cd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 >> 172.21.15.27:0/732893392 conn(0x7f8b680fc8e0 msgr2=0x7f8b68103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 shutdown_connections 2024-06-06T08:29:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:23.394+0000 7f8b6dba8700 1 -- 172.21.15.27:0/732893392 wait complete. 2024-06-06T08:29:23.942 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2024-06-06T08:29:23.942 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:29:23.943 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2024-06-06T08:29:23.960 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:29:23.961 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:24.017 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2024-06-06T08:29:24.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch host add smithi203 2024-06-06T08:29:24.042 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:23 smithi027 ceph-mon[32335]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:24.042 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:24.262 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:29:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:24 smithi027 ceph-mon[32335]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:29:25.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.810+0000 7f322d49d700 1 -- 172.21.15.27:0/2235421148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 msgr2=0x7f32281024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:25.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.810+0000 7f322d49d700 1 --2- 172.21.15.27:0/2235421148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 0x7f32281024e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3218009910 tx=0x7f3218009c20 comp rx=0 tx=0).stop 2024-06-06T08:29:25.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.810+0000 7f322d49d700 1 -- 172.21.15.27:0/2235421148 shutdown_connections 2024-06-06T08:29:25.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.810+0000 7f322d49d700 1 --2- 172.21.15.27:0/2235421148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 0x7f32281024e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:25.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.810+0000 7f322d49d700 1 -- 172.21.15.27:0/2235421148 >> 172.21.15.27:0/2235421148 conn(0x7f32280fd9d0 msgr2=0x7f32280ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:25.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.810+0000 7f322d49d700 1 -- 172.21.15.27:0/2235421148 shutdown_connections 2024-06-06T08:29:25.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.810+0000 7f322d49d700 1 -- 172.21.15.27:0/2235421148 wait complete. 2024-06-06T08:29:25.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.811+0000 7f322d49d700 1 Processor -- start 2024-06-06T08:29:25.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.811+0000 7f322d49d700 1 -- start start 2024-06-06T08:29:25.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.811+0000 7f322d49d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 0x7f322819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:25.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.811+0000 7f322d49d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f322810c4d0 con 0x7f3228102100 2024-06-06T08:29:25.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.811+0000 7f3226ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 0x7f322819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:25.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.811+0000 7f3226ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 0x7f322819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41788/0 (socket says 172.21.15.27:41788) 2024-06-06T08:29:25.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.811+0000 7f3226ffd700 1 -- 172.21.15.27:0/2571947863 learned_addr learned my addr 172.21.15.27:0/2571947863 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.812+0000 7f3226ffd700 1 -- 172.21.15.27:0/2571947863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322819dc20 con 0x7f3228102100 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.812+0000 7f3226ffd700 1 --2- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 0x7f322819d6e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3218005850 tx=0x7f3218005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.812+0000 7f320ffff700 1 -- 172.21.15.27:0/2571947863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3218026070 con 0x7f3228102100 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.812+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322819df10 con 0x7f3228102100 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.812+0000 7f320ffff700 1 -- 172.21.15.27:0/2571947863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321800bbd0 con 0x7f3228102100 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.812+0000 7f320ffff700 1 -- 172.21.15.27:0/2571947863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3218003970 con 0x7f3228102100 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.812+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32281978a0 con 0x7f3228102100 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.813+0000 7f320ffff700 1 -- 172.21.15.27:0/2571947863 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3218029070 con 0x7f3228102100 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.813+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32281035e0 con 0x7f3228102100 2024-06-06T08:29:25.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.813+0000 7f320ffff700 1 --2- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f32100390f0 0x7f321003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:25.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.813+0000 7f320ffff700 1 -- 172.21.15.27:0/2571947863 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f321801e070 con 0x7f3228102100 2024-06-06T08:29:25.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.813+0000 7f32267fc700 1 --2- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f32100390f0 0x7f321003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:25.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.814+0000 7f32267fc700 1 --2- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f32100390f0 0x7f321003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f321c006fd0 tx=0x7f321c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:25.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:25.818+0000 7f320ffff700 1 -- 172.21.15.27:0/2571947863 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3218013070 con 0x7f3228102100 2024-06-06T08:29:25.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:25 smithi027 ceph-mon[32335]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:26.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:26.033+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}) -- 0x7f3228198dd0 con 0x7f32100390f0 2024-06-06T08:29:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:26 smithi027 ceph-mon[32335]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:26 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:26 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:26 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:26 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:26 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:29:27.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:26 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:27.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:26 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:27 smithi027 ceph-mon[32335]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:27 smithi027 ceph-mon[32335]: Updating smithi176:/etc/ceph/ceph.conf 2024-06-06T08:29:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:27 smithi027 ceph-mon[32335]: Deploying cephadm binary to smithi203 2024-06-06T08:29:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:27 smithi027 ceph-mon[32335]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:29:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:27 smithi027 ceph-mon[32335]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:28 smithi027 ceph-mon[32335]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:29:30.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:29 smithi027 ceph-mon[32335]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:30.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.274+0000 7f320ffff700 1 -- 172.21.15.27:0/2571947863 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3228198dd0 con 0x7f32100390f0 2024-06-06T08:29:30.273 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi203' with addr '172.21.15.203' 2024-06-06T08:29:30.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f32100390f0 msgr2=0x7f321003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 --2- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f32100390f0 0x7f321003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f321c006fd0 tx=0x7f321c006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 msgr2=0x7f322819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 --2- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 0x7f322819d6e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3218005850 tx=0x7f3218005030 comp rx=0 tx=0).stop 2024-06-06T08:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 shutdown_connections 2024-06-06T08:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 --2- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f32100390f0 0x7f321003b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 --2- 172.21.15.27:0/2571947863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3228102100 0x7f322819d6e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 >> 172.21.15.27:0/2571947863 conn(0x7f32280fd9d0 msgr2=0x7f3228104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 shutdown_connections 2024-06-06T08:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:30.277+0000 7f322d49d700 1 -- 172.21.15.27:0/2571947863 wait complete. 2024-06-06T08:29:30.884 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:30.885 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:30.885 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:30.885 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:30.885 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:30.917 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T08:29:31.208 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:29:31.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:31 smithi027 ceph-mon[32335]: Added host smithi203 2024-06-06T08:29:31.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:31 smithi027 ceph-mon[32335]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:31.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:31.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.742+0000 7f00f6e69700 1 -- 172.21.15.27:0/1175814397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 msgr2=0x7f00f0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.742+0000 7f00f6e69700 1 --2- 172.21.15.27:0/1175814397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 0x7f00f0101420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009910 tx=0x7f00e0009c20 comp rx=0 tx=0).stop 2024-06-06T08:29:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.742+0000 7f00f6e69700 1 -- 172.21.15.27:0/1175814397 shutdown_connections 2024-06-06T08:29:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.742+0000 7f00f6e69700 1 --2- 172.21.15.27:0/1175814397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 0x7f00f0101420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.742+0000 7f00f6e69700 1 -- 172.21.15.27:0/1175814397 >> 172.21.15.27:0/1175814397 conn(0x7f00f00fc8f0 msgr2=0x7f00f00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.742+0000 7f00f6e69700 1 -- 172.21.15.27:0/1175814397 shutdown_connections 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.742+0000 7f00f6e69700 1 -- 172.21.15.27:0/1175814397 wait complete. 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00f6e69700 1 Processor -- start 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00f6e69700 1 -- start start 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00f6e69700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 0x7f00f019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00f6e69700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00f010b410 con 0x7f00f0101040 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00f4c05700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 0x7f00f019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00f4c05700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 0x7f00f019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39130/0 (socket says 172.21.15.27:39130) 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00f4c05700 1 -- 172.21.15.27:0/3716629526 learned_addr learned my addr 172.21.15.27:0/3716629526 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00f4c05700 1 -- 172.21.15.27:0/3716629526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f019dbc0 con 0x7f00f0101040 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00f4c05700 1 --2- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 0x7f00f019d680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005850 tx=0x7f00e0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.743+0000 7f00edffb700 1 -- 172.21.15.27:0/3716629526 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e0026070 con 0x7f00f0101040 2024-06-06T08:29:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.744+0000 7f00edffb700 1 -- 172.21.15.27:0/3716629526 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e000bbd0 con 0x7f00f0101040 2024-06-06T08:29:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.744+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f019deb0 con 0x7f00f0101040 2024-06-06T08:29:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.744+0000 7f00edffb700 1 -- 172.21.15.27:0/3716629526 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e0003970 con 0x7f00f0101040 2024-06-06T08:29:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.744+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f0197800 con 0x7f00f0101040 2024-06-06T08:29:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.745+0000 7f00edffb700 1 -- 172.21.15.27:0/3716629526 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f00e0029070 con 0x7f00f0101040 2024-06-06T08:29:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.745+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f0102520 con 0x7f00f0101040 2024-06-06T08:29:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.745+0000 7f00edffb700 1 --2- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00d80390f0 0x7f00d803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.745+0000 7f00edffb700 1 -- 172.21.15.27:0/3716629526 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f00e001e070 con 0x7f00f0101040 2024-06-06T08:29:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.745+0000 7f00effff700 1 --2- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00d80390f0 0x7f00d803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.745+0000 7f00effff700 1 --2- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00d80390f0 0x7f00d803b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006fd0 tx=0x7f00e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:32.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.750+0000 7f00edffb700 1 -- 172.21.15.27:0/3716629526 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f00e0013070 con 0x7f00f0101040 2024-06-06T08:29:32.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.964+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00f0198e40 con 0x7f00d80390f0 2024-06-06T08:29:32.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.965+0000 7f00edffb700 1 -- 172.21.15.27:0/3716629526 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f00f0198e40 con 0x7f00d80390f0 2024-06-06T08:29:32.964 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:29:32.964 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.176", "hostname": "smithi176", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2024-06-06T08:29:32.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00d80390f0 msgr2=0x7f00d803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:32.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 --2- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00d80390f0 0x7f00d803b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006fd0 tx=0x7f00e4006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:32.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 msgr2=0x7f00f019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:32.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 --2- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 0x7f00f019d680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005850 tx=0x7f00e0005030 comp rx=0 tx=0).stop 2024-06-06T08:29:32.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 shutdown_connections 2024-06-06T08:29:32.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 --2- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00d80390f0 0x7f00d803b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:32.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 --2- 172.21.15.27:0/3716629526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f0101040 0x7f00f019d680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:32.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 >> 172.21.15.27:0/3716629526 conn(0x7f00f00fc8f0 msgr2=0x7f00f006a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:32.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 shutdown_connections 2024-06-06T08:29:32.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:32.967+0000 7f00f6e69700 1 -- 172.21.15.27:0/3716629526 wait complete. 2024-06-06T08:29:33.553 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-06T08:29:33.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-06T08:29:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:29:34.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:33 smithi027 ceph-mon[32335]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:34.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:33 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:34.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:33 smithi027 ceph-mon[32335]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:29:35.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.327+0000 7f22ce451700 1 -- 172.21.15.27:0/1158485548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 msgr2=0x7f22c8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:35.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.327+0000 7f22ce451700 1 --2- 172.21.15.27:0/1158485548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 0x7f22c8101420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f22b8009910 tx=0x7f22b8009c20 comp rx=0 tx=0).stop 2024-06-06T08:29:35.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.327+0000 7f22ce451700 1 -- 172.21.15.27:0/1158485548 shutdown_connections 2024-06-06T08:29:35.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.327+0000 7f22ce451700 1 --2- 172.21.15.27:0/1158485548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 0x7f22c8101420 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:35.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.327+0000 7f22ce451700 1 -- 172.21.15.27:0/1158485548 >> 172.21.15.27:0/1158485548 conn(0x7f22c80768f0 msgr2=0x7f22c8076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:35.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.327+0000 7f22ce451700 1 -- 172.21.15.27:0/1158485548 shutdown_connections 2024-06-06T08:29:35.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.327+0000 7f22ce451700 1 -- 172.21.15.27:0/1158485548 wait complete. 2024-06-06T08:29:35.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.328+0000 7f22ce451700 1 Processor -- start 2024-06-06T08:29:35.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.328+0000 7f22ce451700 1 -- start start 2024-06-06T08:29:35.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.328+0000 7f22ce451700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 0x7f22c819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:35.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.328+0000 7f22ce451700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22c810b3c0 con 0x7f22c8101040 2024-06-06T08:29:35.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.328+0000 7f22c7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 0x7f22c819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:35.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.328+0000 7f22c7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 0x7f22c819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39152/0 (socket says 172.21.15.27:39152) 2024-06-06T08:29:35.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.328+0000 7f22c7fff700 1 -- 172.21.15.27:0/3129132196 learned_addr learned my addr 172.21.15.27:0/3129132196 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:29:35.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.329+0000 7f22c7fff700 1 -- 172.21.15.27:0/3129132196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c819cb50 con 0x7f22c8101040 2024-06-06T08:29:35.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.329+0000 7f22c7fff700 1 --2- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 0x7f22c819c610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22b8005850 tx=0x7f22b8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:35.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.329+0000 7f22c57fa700 1 -- 172.21.15.27:0/3129132196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22b8026070 con 0x7f22c8101040 2024-06-06T08:29:35.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.329+0000 7f22c57fa700 1 -- 172.21.15.27:0/3129132196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22b800bbd0 con 0x7f22c8101040 2024-06-06T08:29:35.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.329+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c819ce40 con 0x7f22c8101040 2024-06-06T08:29:35.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.329+0000 7f22c57fa700 1 -- 172.21.15.27:0/3129132196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22b8003970 con 0x7f22c8101040 2024-06-06T08:29:35.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.329+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c8196790 con 0x7f22c8101040 2024-06-06T08:29:35.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.330+0000 7f22c57fa700 1 -- 172.21.15.27:0/3129132196 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f22b8029070 con 0x7f22c8101040 2024-06-06T08:29:35.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.330+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c81024d0 con 0x7f22c8101040 2024-06-06T08:29:35.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.330+0000 7f22c57fa700 1 --2- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f22b40390f0 0x7f22b403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:35.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.330+0000 7f22c57fa700 1 -- 172.21.15.27:0/3129132196 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f22b801e070 con 0x7f22c8101040 2024-06-06T08:29:35.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.330+0000 7f22c77fe700 1 --2- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f22b40390f0 0x7f22b403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:35.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.331+0000 7f22c77fe700 1 --2- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f22b40390f0 0x7f22b403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f22b0006fd0 tx=0x7f22b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:35.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.335+0000 7f22c57fa700 1 -- 172.21.15.27:0/3129132196 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f22b8013070 con 0x7f22c8101040 2024-06-06T08:29:35.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.546+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f22c8063af0 con 0x7f22c8101040 2024-06-06T08:29:35.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.894+0000 7f22c57fa700 1 -- 172.21.15.27:0/3129132196 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f22b8030090 con 0x7f22c8101040 2024-06-06T08:29:35.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f22b40390f0 msgr2=0x7f22b403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:35.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 --2- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f22b40390f0 0x7f22b403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f22b0006fd0 tx=0x7f22b0006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:35.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 msgr2=0x7f22c819c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:35.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 --2- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 0x7f22c819c610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22b8005850 tx=0x7f22b8005030 comp rx=0 tx=0).stop 2024-06-06T08:29:35.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 shutdown_connections 2024-06-06T08:29:35.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 --2- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f22b40390f0 0x7f22b403b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:35.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 --2- 172.21.15.27:0/3129132196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22c8101040 0x7f22c819c610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:35.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 >> 172.21.15.27:0/3129132196 conn(0x7f22c80768f0 msgr2=0x7f22c8103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:35.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 shutdown_connections 2024-06-06T08:29:35.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:29:35.897+0000 7f22ce451700 1 -- 172.21.15.27:0/3129132196 wait complete. 2024-06-06T08:29:35.897 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-06-06T08:29:36.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:35 smithi027 ceph-mon[32335]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:36.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:35 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3129132196' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T08:29:36.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:36.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:36.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:36.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:36.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:29:36.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:36.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:36.575 INFO:tasks.cephadm:Adding mon.a on smithi027 2024-06-06T08:29:36.575 INFO:tasks.cephadm:Adding mon.b on smithi176 2024-06-06T08:29:36.575 INFO:tasks.cephadm:Adding mon.c on smithi203 2024-06-06T08:29:36.575 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi027:172.21.15.27=a;smithi176:172.21.15.176=b;smithi203:172.21.15.203=c' 2024-06-06T08:29:36.804 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:29:37.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:36 smithi027 ceph-mon[32335]: Updating smithi203:/etc/ceph/ceph.conf 2024-06-06T08:29:37.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:36 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3129132196' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T08:29:37.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:36 smithi027 ceph-mon[32335]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T08:29:37.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:36 smithi027 ceph-mon[32335]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:29:37.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:36 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:37.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:36 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:37.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:36 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:38.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:37 smithi027 ceph-mon[32335]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:38.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:37 smithi027 ceph-mon[32335]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:38.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:37 smithi027 ceph-mon[32335]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:29:38.262 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.262+0000 7f295392f700 1 -- 172.21.15.203:0/2119211405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 msgr2=0x7f294c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:38.262 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.262+0000 7f295392f700 1 --2- 172.21.15.203:0/2119211405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 0x7f294c1024e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2948009910 tx=0x7f2948009c20 comp rx=0 tx=0).stop 2024-06-06T08:29:38.262 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.262+0000 7f295392f700 1 -- 172.21.15.203:0/2119211405 shutdown_connections 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.262+0000 7f295392f700 1 --2- 172.21.15.203:0/2119211405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 0x7f294c1024e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.262+0000 7f295392f700 1 -- 172.21.15.203:0/2119211405 >> 172.21.15.203:0/2119211405 conn(0x7f294c0fd9d0 msgr2=0x7f294c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.262+0000 7f295392f700 1 -- 172.21.15.203:0/2119211405 shutdown_connections 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.262+0000 7f295392f700 1 -- 172.21.15.203:0/2119211405 wait complete. 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.263+0000 7f295392f700 1 Processor -- start 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.263+0000 7f295392f700 1 -- start start 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.263+0000 7f295392f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 0x7f294c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.263+0000 7f295392f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f294c10c4d0 con 0x7f294c102100 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f29516cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 0x7f294c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f29516cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 0x7f294c19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.203:51064/0 (socket says 172.21.15.203:51064) 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f29516cb700 1 -- 172.21.15.203:0/349514793 learned_addr learned my addr 172.21.15.203:0/349514793 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:29:38.263 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f29516cb700 1 -- 172.21.15.203:0/349514793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c19dbd0 con 0x7f294c102100 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f29516cb700 1 --2- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 0x7f294c19d690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2948005850 tx=0x7f2948005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f29427fc700 1 -- 172.21.15.203:0/349514793 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2948026070 con 0x7f294c102100 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c19dec0 con 0x7f294c102100 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f29427fc700 1 -- 172.21.15.203:0/349514793 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294800bbd0 con 0x7f294c102100 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f29427fc700 1 -- 172.21.15.203:0/349514793 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2948003970 con 0x7f294c102100 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.264+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c197810 con 0x7f294c102100 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.265+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294c1035e0 con 0x7f294c102100 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.265+0000 7f29427fc700 1 -- 172.21.15.203:0/349514793 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2948029070 con 0x7f294c102100 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.266+0000 7f29427fc700 1 --2- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29380390f0 0x7f293803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.266+0000 7f29427fc700 1 -- 172.21.15.203:0/349514793 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f294801e070 con 0x7f294c102100 2024-06-06T08:29:38.265 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.266+0000 7f2950eca700 1 --2- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29380390f0 0x7f293803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:38.266 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.266+0000 7f2950eca700 1 --2- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29380390f0 0x7f293803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f293c006fd0 tx=0x7f293c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:38.269 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.271+0000 7f29427fc700 1 -- 172.21.15.203:0/349514793 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2948013070 con 0x7f294c102100 2024-06-06T08:29:38.474 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.475+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi176:172.21.15.176=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}) -- 0x7f294c198e00 con 0x7f29380390f0 2024-06-06T08:29:38.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.482+0000 7f29427fc700 1 -- 172.21.15.203:0/349514793 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f294c198e00 con 0x7f29380390f0 2024-06-06T08:29:38.481 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29380390f0 msgr2=0x7f293803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 --2- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29380390f0 0x7f293803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f293c006fd0 tx=0x7f293c006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 msgr2=0x7f294c19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 --2- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 0x7f294c19d690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2948005850 tx=0x7f2948005030 comp rx=0 tx=0).stop 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 shutdown_connections 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 --2- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29380390f0 0x7f293803b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 --2- 172.21.15.203:0/349514793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f294c102100 0x7f294c19d690 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 >> 172.21.15.203:0/349514793 conn(0x7f294c0fd9d0 msgr2=0x7f294c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 shutdown_connections 2024-06-06T08:29:38.483 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:38.484+0000 7f295392f700 1 -- 172.21.15.203:0/349514793 wait complete. 2024-06-06T08:29:39.033 DEBUG:teuthology.orchestra.run.smithi176:mon.b> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.b.service 2024-06-06T08:29:39.036 DEBUG:teuthology.orchestra.run.smithi203:mon.c> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.c.service 2024-06-06T08:29:39.038 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T08:29:39.039 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T08:29:39.744 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:29:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi176:172.21.15.176=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:39.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi176:172.21.15.176=b;smithi203:172.21.15.203=c;count:3 2024-06-06T08:29:39.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:39.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:39.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:39.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:39.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:39.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:29:39.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:39.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:39 smithi027 ceph-mon[32335]: Deploying daemon mon.c on smithi203 2024-06-06T08:29:41.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:41 smithi027 ceph-mon[32335]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:42.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.183+0000 7fea67e8a700 1 -- 172.21.15.203:0/4152161187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 msgr2=0x7fea601024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:42.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.183+0000 7fea67e8a700 1 --2- 172.21.15.203:0/4152161187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 0x7fea601024e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009910 tx=0x7fea5c009c20 comp rx=0 tx=0).stop 2024-06-06T08:29:42.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.183+0000 7fea67e8a700 1 -- 172.21.15.203:0/4152161187 shutdown_connections 2024-06-06T08:29:42.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.183+0000 7fea67e8a700 1 --2- 172.21.15.203:0/4152161187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 0x7fea601024e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:42.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.183+0000 7fea67e8a700 1 -- 172.21.15.203:0/4152161187 >> 172.21.15.203:0/4152161187 conn(0x7fea600fd9d0 msgr2=0x7fea600ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:42.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.183+0000 7fea67e8a700 1 -- 172.21.15.203:0/4152161187 shutdown_connections 2024-06-06T08:29:42.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.183+0000 7fea67e8a700 1 -- 172.21.15.203:0/4152161187 wait complete. 2024-06-06T08:29:42.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.184+0000 7fea67e8a700 1 Processor -- start 2024-06-06T08:29:42.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.184+0000 7fea67e8a700 1 -- start start 2024-06-06T08:29:42.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.184+0000 7fea67e8a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 0x7fea60074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:42.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.184+0000 7fea67e8a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea6010c4d0 con 0x7fea60102100 2024-06-06T08:29:42.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.185+0000 7fea65c26700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 0x7fea60074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:42.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.185+0000 7fea65c26700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 0x7fea60074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.203:51076/0 (socket says 172.21.15.203:51076) 2024-06-06T08:29:42.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.185+0000 7fea65c26700 1 -- 172.21.15.203:0/3491562441 learned_addr learned my addr 172.21.15.203:0/3491562441 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:29:42.185 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.185+0000 7fea65c26700 1 -- 172.21.15.203:0/3491562441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea60076680 con 0x7fea60102100 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.185+0000 7fea65c26700 1 --2- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 0x7fea60074790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea5c005850 tx=0x7fea5c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.185+0000 7fea56ffd700 1 -- 172.21.15.203:0/3491562441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea5c026070 con 0x7fea60102100 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.186+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea60074d30 con 0x7fea60102100 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.186+0000 7fea56ffd700 1 -- 172.21.15.203:0/3491562441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c00bc30 con 0x7fea60102100 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.186+0000 7fea56ffd700 1 -- 172.21.15.203:0/3491562441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea5c003a20 con 0x7fea60102100 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.186+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea600751f0 con 0x7fea60102100 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.187+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea601035e0 con 0x7fea60102100 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.187+0000 7fea56ffd700 1 -- 172.21.15.203:0/3491562441 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fea5c029070 con 0x7fea60102100 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.187+0000 7fea56ffd700 1 --2- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea4c0390f0 0x7fea4c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.187+0000 7fea56ffd700 1 -- 172.21.15.203:0/3491562441 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fea5c01e070 con 0x7fea60102100 2024-06-06T08:29:42.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.187+0000 7fea65425700 1 --2- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea4c0390f0 0x7fea4c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:42.187 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.187+0000 7fea65425700 1 --2- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea4c0390f0 0x7fea4c03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fea50006fd0 tx=0x7fea50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:42.191 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.192+0000 7fea56ffd700 1 -- 172.21.15.203:0/3491562441 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea5c013070 con 0x7fea60102100 2024-06-06T08:29:42.500 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.500+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fea60075cc0 con 0x7fea60102100 2024-06-06T08:29:42.500 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.501+0000 7fea56ffd700 1 -- 172.21.15.203:0/3491562441 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fea5c030020 con 0x7fea60102100 2024-06-06T08:29:42.500 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:29:42.500 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","modified":"2024-06-06T08:27:49.610814Z","created":"2024-06-06T08:27:49.610814Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-06T08:29:42.502 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea4c0390f0 msgr2=0x7fea4c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 --2- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea4c0390f0 0x7fea4c03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fea50006fd0 tx=0x7fea50006e40 comp rx=0 tx=0).stop 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 msgr2=0x7fea60074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 --2- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 0x7fea60074790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea5c005850 tx=0x7fea5c005030 comp rx=0 tx=0).stop 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 shutdown_connections 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 --2- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea4c0390f0 0x7fea4c03b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 --2- 172.21.15.203:0/3491562441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea60102100 0x7fea60074790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 >> 172.21.15.203:0/3491562441 conn(0x7fea600fd9d0 msgr2=0x7fea60103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 shutdown_connections 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:42.504+0000 7fea67e8a700 1 -- 172.21.15.203:0/3491562441 wait complete. 2024-06-06T08:29:42.503 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-06-06T08:29:43.601 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).mds e1 new map 2024-06-06T08:29:43.601 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-06T08:29:43.601 INFO:journalctl@ceph.mon.c.smithi203.stdout: e1 2024-06-06T08:29:43.601 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T08:29:43.601 INFO:journalctl@ceph.mon.c.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T08:29:43.601 INFO:journalctl@ceph.mon.c.smithi203.stdout: legacy client fscid: -1 2024-06-06T08:29:43.601 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-06-06T08:29:43.601 INFO:journalctl@ceph.mon.c.smithi203.stdout: No filesystems configured 2024-06-06T08:29:43.601 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-06T08:29:43.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:43 smithi027 ceph-mon[32335]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:43 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/3491562441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T08:29:43.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:29:43.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Updating smithi176:/etc/ceph/ceph.conf 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Deploying cephadm binary to smithi203 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Added host smithi203 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3129132196' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Updating smithi203:/etc/ceph/ceph.conf 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3129132196' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi176:172.21.15.176=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi176:172.21.15.176=b;smithi203:172.21.15.203=c;count:3 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: Deploying daemon mon.c on smithi203 2024-06-06T08:29:43.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:43.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/3491562441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T08:29:43.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 ceph-mon[36579]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-06T08:29:43.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 bash[36560]: 276655564d2fc611c839927ffd967345da9739347f4d67cfb84a114381d4d734 2024-06-06T08:29:43.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:43 smithi203 systemd[1]: Started Ceph mon.c for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:29:44.188 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T08:29:44.189 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T08:29:44.424 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:29:48.741 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.742+0000 7f220b386700 1 -- 172.21.15.203:0/728610291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21f00035c0 msgr2=0x7f21f0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:48.741 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.742+0000 7f220b386700 1 --2- 172.21.15.203:0/728610291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21f00035c0 0x7f21f0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2204100db0 tx=0x7f21fc00feb0 comp rx=0 tx=0).stop 2024-06-06T08:29:48.741 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.742+0000 7f220b386700 1 -- 172.21.15.203:0/728610291 shutdown_connections 2024-06-06T08:29:48.741 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.742+0000 7f220b386700 1 --2- 172.21.15.203:0/728610291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21f00035c0 0x7f21f0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:48.741 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.742+0000 7f220b386700 1 -- 172.21.15.203:0/728610291 >> 172.21.15.203:0/728610291 conn(0x7f22040fc970 msgr2=0x7f22040fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:48.742 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.742+0000 7f220b386700 1 -- 172.21.15.203:0/728610291 shutdown_connections 2024-06-06T08:29:48.742 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.742+0000 7f220b386700 1 -- 172.21.15.203:0/728610291 wait complete. 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.743+0000 7f220b386700 1 Processor -- start 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.743+0000 7f220b386700 1 -- start start 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.743+0000 7f220b386700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 0x7f220419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.743+0000 7f220b386700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f220419dda0 0x7f2204197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f220b386700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2204108f10 con 0x7f220419dda0 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f220b386700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2204108d90 con 0x7f21f00035c0 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2209122700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 0x7f220419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2209122700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 0x7f220419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49768/0 (socket says 172.21.15.203:49768) 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2209122700 1 -- 172.21.15.203:0/2425837684 learned_addr learned my addr 172.21.15.203:0/2425837684 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2208921700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f220419dda0 0x7f2204197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:48.744 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2209122700 1 -- 172.21.15.203:0/2425837684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 msgr2=0x7f220419d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2209122700 1 -- 172.21.15.203:0/2425837684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 msgr2=0x7f220419d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2209122700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 0x7f220419d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2209122700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 0x7f220419d860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2208921700 1 -- 172.21.15.203:0/2425837684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 msgr2=0x7f220419d860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2208921700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 0x7f220419d860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2208921700 1 -- 172.21.15.203:0/2425837684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220419e3e0 con 0x7f220419dda0 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.744+0000 7f2208921700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f220419dda0 0x7f2204197a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f21f400b4d0 tx=0x7f21f400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.745+0000 7f21fa7fc700 1 -- 172.21.15.203:0/2425837684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f21f4011630 con 0x7f220419dda0 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.745+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2204197f60 con 0x7f220419dda0 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.745+0000 7f21fa7fc700 1 -- 172.21.15.203:0/2425837684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21f4011cb0 con 0x7f220419dda0 2024-06-06T08:29:48.745 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.745+0000 7f21fa7fc700 1 -- 172.21.15.203:0/2425837684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f21f4010e60 con 0x7f220419dda0 2024-06-06T08:29:48.746 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.745+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22041984f0 con 0x7f220419dda0 2024-06-06T08:29:48.746 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.746+0000 7f21fa7fc700 1 -- 172.21.15.203:0/2425837684 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f21f4018020 con 0x7f220419dda0 2024-06-06T08:29:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.746+0000 7f21fa7fc700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21e8038e30 0x7f21e803b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.746+0000 7f21fa7fc700 1 -- 172.21.15.203:0/2425837684 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f21f404bc40 con 0x7f220419dda0 2024-06-06T08:29:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.746+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22041025c0 con 0x7f220419dda0 2024-06-06T08:29:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.746+0000 7f2209122700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21e8038e30 0x7f21e803b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:48.747 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.747+0000 7f2209122700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21e8038e30 0x7f21e803b2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00fa60 tx=0x7f21fc005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:48.750 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:48.752+0000 7f21fa7fc700 1 -- 172.21.15.203:0/2425837684 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f21f401f020 con 0x7f220419dda0 2024-06-06T08:29:48.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: mon.a calling monitor election 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: mon.c calling monitor election 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T08:29:48.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: monmap epoch 2 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: last_changed 2024-06-06T08:29:43.608363+0000 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: created 2024-06-06T08:27:49.610814+0000 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: min_mon_release 19 (squid) 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: election_strategy: 1 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: fsmap 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: mgrmap e13: x(active, since 50s) 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: overall HEALTH_OK 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:48.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:48.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:29:48.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:48.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:29:48.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: mon.a calling monitor election 2024-06-06T08:29:48.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: mon.c calling monitor election 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: monmap epoch 2 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:48.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: last_changed 2024-06-06T08:29:43.608363+0000 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: created 2024-06-06T08:27:49.610814+0000 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: min_mon_release 19 (squid) 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: election_strategy: 1 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: fsmap 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: mgrmap e13: x(active, since 50s) 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: overall HEALTH_OK 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:48.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:48.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:29:48.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.032+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2204198f50 con 0x7f220419dda0 2024-06-06T08:29:49.031 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.033+0000 7f21fa7fc700 1 -- 172.21.15.203:0/2425837684 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f21f4018310 con 0x7f220419dda0 2024-06-06T08:29:49.031 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:29:49.032 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","modified":"2024-06-06T08:29:43.608363Z","created":"2024-06-06T08:27:49.610814Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-06T08:29:49.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21e8038e30 msgr2=0x7f21e803b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:49.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21e8038e30 0x7f21e803b2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00fa60 tx=0x7f21fc005c20 comp rx=0 tx=0).stop 2024-06-06T08:29:49.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f220419dda0 msgr2=0x7f2204197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:49.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f220419dda0 0x7f2204197a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f21f400b4d0 tx=0x7f21f400b890 comp rx=0 tx=0).stop 2024-06-06T08:29:49.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 shutdown_connections 2024-06-06T08:29:49.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21e8038e30 0x7f21e803b2f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:49.034 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21f00035c0 0x7f220419d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:49.035 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 --2- 172.21.15.203:0/2425837684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f220419dda0 0x7f2204197a20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:49.035 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 >> 172.21.15.203:0/2425837684 conn(0x7f22040fc970 msgr2=0x7f220406a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:49.035 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 shutdown_connections 2024-06-06T08:29:49.035 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:49.035+0000 7f220b386700 1 -- 172.21.15.203:0/2425837684 wait complete. 2024-06-06T08:29:49.035 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-06-06T08:29:49.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:49 smithi027 ceph-mon[32335]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:49.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:49 smithi027 ceph-mon[32335]: Deploying daemon mon.b on smithi176 2024-06-06T08:29:49.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:49 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/2425837684' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T08:29:49.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:49 smithi203 ceph-mon[36579]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:49 smithi203 ceph-mon[36579]: Deploying daemon mon.b on smithi176 2024-06-06T08:29:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:49 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/2425837684' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T08:29:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:50.587 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T08:29:50.587 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T08:29:50.820 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:29:51.938 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:51 smithi203 ceph-mon[36579]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:51.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:51 smithi027 ceph-mon[32335]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.329 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.328+0000 7fc0e5665700 1 -- 172.21.15.203:0/3847052038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e01031c0 msgr2=0x7fc0e01035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:52.329 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.328+0000 7fc0e5665700 1 --2- 172.21.15.203:0/3847052038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e01031c0 0x7fc0e01035a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009910 tx=0x7fc0d0009c20 comp rx=0 tx=0).stop 2024-06-06T08:29:52.329 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.328+0000 7fc0e5665700 1 -- 172.21.15.203:0/3847052038 shutdown_connections 2024-06-06T08:29:52.329 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.328+0000 7fc0e5665700 1 --2- 172.21.15.203:0/3847052038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e01031c0 0x7fc0e01035a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:52.329 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.328+0000 7fc0e5665700 1 -- 172.21.15.203:0/3847052038 >> 172.21.15.203:0/3847052038 conn(0x7fc0e00fc9d0 msgr2=0x7fc0e00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:52.329 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.328+0000 7fc0e5665700 1 -- 172.21.15.203:0/3847052038 shutdown_connections 2024-06-06T08:29:52.329 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.328+0000 7fc0e5665700 1 -- 172.21.15.203:0/3847052038 wait complete. 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.328+0000 7fc0e5665700 1 Processor -- start 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.328+0000 7fc0e5665700 1 -- start start 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0e5665700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 0x7fc0e0197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0e5665700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e01982f0 0x7fc0e019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0e5665700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010c640 con 0x7fc0e01982f0 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0e5665700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010c4c0 con 0x7fc0e01031c0 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 0x7fc0e0197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0deffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 0x7fc0e0197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49780/0 (socket says 172.21.15.203:49780) 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0deffd700 1 -- 172.21.15.203:0/1399360315 learned_addr learned my addr 172.21.15.203:0/1399360315 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0deffd700 1 -- 172.21.15.203:0/1399360315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 msgr2=0x7fc0e0197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0deffd700 1 -- 172.21.15.203:0/1399360315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 msgr2=0x7fc0e0197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T08:29:52.330 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0deffd700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 0x7fc0e0197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0de7fc700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e01982f0 0x7fc0e019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0deffd700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 0x7fc0e0197db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0de7fc700 1 -- 172.21.15.203:0/1399360315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 msgr2=0x7fc0e0197db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0de7fc700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 0x7fc0e0197db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0de7fc700 1 -- 172.21.15.203:0/1399360315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e0198930 con 0x7fc0e01982f0 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.329+0000 7fc0de7fc700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e01982f0 0x7fc0e019c5e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400b4d0 tx=0x7fc0d400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.330+0000 7fc0c7fff700 1 -- 172.21.15.203:0/1399360315 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc0d4011630 con 0x7fc0e01982f0 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.330+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e019cb20 con 0x7fc0e01982f0 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.330+0000 7fc0c7fff700 1 -- 172.21.15.203:0/1399360315 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d4011cb0 con 0x7fc0e01982f0 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.330+0000 7fc0c7fff700 1 -- 172.21.15.203:0/1399360315 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc0d4010e60 con 0x7fc0e01982f0 2024-06-06T08:29:52.331 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.330+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e019d080 con 0x7fc0e01982f0 2024-06-06T08:29:52.332 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.331+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e006af20 con 0x7fc0e01982f0 2024-06-06T08:29:52.332 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.331+0000 7fc0c7fff700 1 -- 172.21.15.203:0/1399360315 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc0d4018020 con 0x7fc0e01982f0 2024-06-06T08:29:52.332 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.331+0000 7fc0c7fff700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0c8038e30 0x7fc0c803b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:29:52.333 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.331+0000 7fc0c7fff700 1 -- 172.21.15.203:0/1399360315 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc0d404bb30 con 0x7fc0e01982f0 2024-06-06T08:29:52.333 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.331+0000 7fc0deffd700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0c8038e30 0x7fc0c803b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:29:52.333 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.331+0000 7fc0deffd700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0c8038e30 0x7fc0c803b2f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000b5c0 tx=0x7fc0d0009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:29:52.335 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.336+0000 7fc0c7fff700 1 -- 172.21.15.203:0/1399360315 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0d401f020 con 0x7fc0e01982f0 2024-06-06T08:29:52.431 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(???) e0 preinit fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:52.452 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.454+0000 7fc0c7fff700 1 -- 172.21.15.203:0/1399360315 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4010510 con 0x7fc0e01982f0 2024-06-06T08:29:52.613 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:52.614+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc0e019d390 con 0x7fc0e01982f0 2024-06-06T08:29:52.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 bash[36767]: e636c7c3892f72f6a915ae818401de738affce058b04e6c0d29c8296db1bf5f1 2024-06-06T08:29:52.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 systemd[1]: Started Ceph mon.b for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:29:52.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).mds e1 new map 2024-06-06T08:29:52.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).mds e1 print_map 2024-06-06T08:29:52.757 INFO:journalctl@ceph.mon.b.smithi176.stdout: e1 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout: legacy client fscid: -1 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout: 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout: No filesystems configured 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-06T08:29:52.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:52.759 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Updating smithi176:/etc/ceph/ceph.conf 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Deploying cephadm binary to smithi203 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Added host smithi203 2024-06-06T08:29:52.760 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3129132196' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:52.761 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Updating smithi203:/etc/ceph/ceph.conf 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3129132196' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:29:52.762 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.763 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi176:172.21.15.176=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:29:52.763 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi176:172.21.15.176=b;smithi203:172.21.15.203=c;count:3 2024-06-06T08:29:52.763 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.763 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:52.763 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:52.763 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:52.763 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Deploying daemon mon.c on smithi203 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/3491562441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.a calling monitor election 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:52.764 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:52.765 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.c calling monitor election 2024-06-06T08:29:52.765 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.765 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:52.765 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:52.765 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:52.765 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T08:29:52.765 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: monmap epoch 2 2024-06-06T08:29:52.765 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:52.765 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: last_changed 2024-06-06T08:29:43.608363+0000 2024-06-06T08:29:52.766 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: created 2024-06-06T08:27:49.610814+0000 2024-06-06T08:29:52.766 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: min_mon_release 19 (squid) 2024-06-06T08:29:52.766 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: election_strategy: 1 2024-06-06T08:29:52.766 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-06-06T08:29:52.766 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-06-06T08:29:52.766 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: fsmap 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mgrmap e13: x(active, since 50s) 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: overall HEALTH_OK 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: Deploying daemon mon.b on smithi176 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/2425837684' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T08:29:52.767 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:52.768 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:52.768 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:52 smithi176 ceph-mon[36786]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-06T08:29:57.463 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.464+0000 7fc0c7fff700 1 -- 172.21.15.203:0/1399360315 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7fc0d401a920 con 0x7fc0e01982f0 2024-06-06T08:29:57.463 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-06-06T08:29:57.464 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":3,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","modified":"2024-06-06T08:29:52.453549Z","created":"2024-06-06T08:27:49.610814Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-06-06T08:29:57.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0c8038e30 msgr2=0x7fc0c803b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:57.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0c8038e30 0x7fc0c803b2f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000b5c0 tx=0x7fc0d0009650 comp rx=0 tx=0).stop 2024-06-06T08:29:57.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e01982f0 msgr2=0x7fc0e019c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:29:57.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e01982f0 0x7fc0e019c5e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400b4d0 tx=0x7fc0d400b890 comp rx=0 tx=0).stop 2024-06-06T08:29:57.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 shutdown_connections 2024-06-06T08:29:57.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0c8038e30 0x7fc0c803b2f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:57.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e01031c0 0x7fc0e0197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:57.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 --2- 172.21.15.203:0/1399360315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0e01982f0 0x7fc0e019c5e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:29:57.468 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 >> 172.21.15.203:0/1399360315 conn(0x7fc0e00fc9d0 msgr2=0x7fc0e0104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:29:57.468 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 shutdown_connections 2024-06-06T08:29:57.468 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:29:57.468+0000 7fc0e5665700 1 -- 172.21.15.203:0/1399360315 wait complete. 2024-06-06T08:29:57.468 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 3 2024-06-06T08:29:57.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:57.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:29:57.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: mon.a calling monitor election 2024-06-06T08:29:57.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: mon.c calling monitor election 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/1399360315' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: mon.b calling monitor election 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: monmap epoch 3 2024-06-06T08:29:57.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: last_changed 2024-06-06T08:29:52.453549+0000 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: created 2024-06-06T08:27:49.610814+0000 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: min_mon_release 19 (squid) 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: election_strategy: 1 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: 2: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.b 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: fsmap 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: mgrmap e13: x(active, since 59s) 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: overall HEALTH_OK 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:57.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:57.743 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:57.743 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:57.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: mon.a calling monitor election 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: mon.c calling monitor election 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/1399360315' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: mon.b calling monitor election 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: monmap epoch 3 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: last_changed 2024-06-06T08:29:52.453549+0000 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: created 2024-06-06T08:27:49.610814+0000 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: min_mon_release 19 (squid) 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: election_strategy: 1 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: 2: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.b 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: fsmap 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T08:29:57.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: mgrmap e13: x(active, since 59s) 2024-06-06T08:29:57.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: overall HEALTH_OK 2024-06-06T08:29:57.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:57.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:57.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:57.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:58.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:58.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:29:58.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: mon.a calling monitor election 2024-06-06T08:29:58.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:58.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:29:58.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: mon.c calling monitor election 2024-06-06T08:29:58.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/1399360315' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T08:29:58.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: mon.b calling monitor election 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: monmap epoch 3 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: last_changed 2024-06-06T08:29:52.453549+0000 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: created 2024-06-06T08:27:49.610814+0000 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: min_mon_release 19 (squid) 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: election_strategy: 1 2024-06-06T08:29:58.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: 2: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.b 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: fsmap 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: mgrmap e13: x(active, since 59s) 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: overall HEALTH_OK 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:58.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:58.107 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-06T08:29:58.107 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-06T08:29:58.363 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:29:58.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:58.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:58.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:58.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:58.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:58.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:58.808 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:29:58.808 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:58.808 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:29:59.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:29:59.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:29:59.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:29:59.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: Updating smithi027:/etc/ceph/ceph.conf 2024-06-06T08:29:59.830 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: Updating smithi176:/etc/ceph/ceph.conf 2024-06-06T08:29:59.830 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: Updating smithi203:/etc/ceph/ceph.conf 2024-06-06T08:29:59.830 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.830 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.830 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.830 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:29:59.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:29:59.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T08:29:59.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:29:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:00.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.067+0000 7f799833d700 1 -- 172.21.15.27:0/940907643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 msgr2=0x7f7990108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:00.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.067+0000 7f799833d700 1 --2- 172.21.15.27:0/940907643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 0x7f7990108310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f798c009910 tx=0x7f798c009c20 comp rx=0 tx=0).stop 2024-06-06T08:30:00.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.067+0000 7f799833d700 1 -- 172.21.15.27:0/940907643 shutdown_connections 2024-06-06T08:30:00.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.067+0000 7f799833d700 1 --2- 172.21.15.27:0/940907643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 0x7f7990108310 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:00.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.067+0000 7f799833d700 1 -- 172.21.15.27:0/940907643 >> 172.21.15.27:0/940907643 conn(0x7f79900768f0 msgr2=0x7f7990076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:30:00.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.067+0000 7f799833d700 1 -- 172.21.15.27:0/940907643 shutdown_connections 2024-06-06T08:30:00.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.067+0000 7f799833d700 1 -- 172.21.15.27:0/940907643 wait complete. 2024-06-06T08:30:00.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f799833d700 1 Processor -- start 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f799833d700 1 -- start start 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f799833d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 0x7f799019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f799833d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f799019ddb0 0x7f7990197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f799833d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7990197f70 0x7f7990198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f799833d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f799010c470 con 0x7f7990107f30 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f799833d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f799010c2f0 con 0x7f7990197f70 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f799833d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f799010c5f0 con 0x7f799019ddb0 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f79960d9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 0x7f799019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f79960d9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 0x7f799019d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52454/0 (socket says 172.21.15.27:52454) 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.068+0000 7f79960d9700 1 -- 172.21.15.27:0/453446257 learned_addr learned my addr 172.21.15.27:0/453446257 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79958d8700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f799019ddb0 0x7f7990197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:00.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79960d9700 1 -- 172.21.15.27:0/453446257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f799019ddb0 msgr2=0x7f7990197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79960d9700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f799019ddb0 0x7f7990197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79968da700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7990197f70 0x7f7990198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79960d9700 1 -- 172.21.15.27:0/453446257 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7990197f70 msgr2=0x7f7990198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79960d9700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7990197f70 0x7f7990198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79960d9700 1 -- 172.21.15.27:0/453446257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79901a4740 con 0x7f7990107f30 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79960d9700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 0x7f799019d870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f798c00bdc0 tx=0x7f798c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79837fe700 1 -- 172.21.15.27:0/453446257 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c026070 con 0x7f7990107f30 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79837fe700 1 -- 172.21.15.27:0/453446257 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f798c0050d0 con 0x7f7990107f30 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79901a4970 con 0x7f7990107f30 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f79837fe700 1 -- 172.21.15.27:0/453446257 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c00fdf0 con 0x7f7990107f30 2024-06-06T08:30:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.069+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79901a4eb0 con 0x7f7990107f30 2024-06-06T08:30:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.070+0000 7f79837fe700 1 -- 172.21.15.27:0/453446257 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f798c029070 con 0x7f7990107f30 2024-06-06T08:30:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.070+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7990063af0 con 0x7f7990107f30 2024-06-06T08:30:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.071+0000 7f79837fe700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f797c039580 0x7f797c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.071+0000 7f79837fe700 1 -- 172.21.15.27:0/453446257 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f798c04cf90 con 0x7f7990107f30 2024-06-06T08:30:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.071+0000 7f79958d8700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f797c039580 0x7f797c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:00.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.071+0000 7f79958d8700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f797c039580 0x7f797c03ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7984009850 tx=0x7f7984008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.076+0000 7f79837fe700 1 -- 172.21.15.27:0/453446257 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f798c013070 con 0x7f7990107f30 2024-06-06T08:30:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: Updating smithi027:/etc/ceph/ceph.conf 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: Updating smithi176:/etc/ceph/ceph.conf 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: Updating smithi203:/etc/ceph/ceph.conf 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:30:00.242 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T08:30:00.242 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:29:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: Updating smithi027:/etc/ceph/ceph.conf 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: Updating smithi176:/etc/ceph/ceph.conf 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: Updating smithi203:/etc/ceph/ceph.conf 2024-06-06T08:30:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:30:00.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T08:30:00.259 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:29:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.295+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f79901a50e0 con 0x7f7990107f30 2024-06-06T08:30:00.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.296+0000 7f79837fe700 1 -- 172.21.15.27:0/453446257 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f798c030020 con 0x7f7990107f30 2024-06-06T08:30:00.294 INFO:teuthology.orchestra.run.smithi027.stdout:# minimal ceph.conf for 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:30:00.295 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-06-06T08:30:00.295 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:30:00.295 INFO:teuthology.orchestra.run.smithi027.stdout: mon_host = [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f797c039580 msgr2=0x7f797c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f797c039580 0x7f797c03ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7984009850 tx=0x7f7984008040 comp rx=0 tx=0).stop 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 msgr2=0x7f799019d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 0x7f799019d870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f798c00bdc0 tx=0x7f798c00bea0 comp rx=0 tx=0).stop 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 shutdown_connections 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f797c039580 0x7f797c03ba40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7990197f70 0x7f7990198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7990107f30 0x7f799019d870 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 --2- 172.21.15.27:0/453446257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f799019ddb0 0x7f7990197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 >> 172.21.15.27:0/453446257 conn(0x7f79900768f0 msgr2=0x7f79900fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:30:00.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 shutdown_connections 2024-06-06T08:30:00.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:00.298+0000 7f799833d700 1 -- 172.21.15.27:0/453446257 wait complete. 2024-06-06T08:30:01.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:00 smithi027 ceph-mon[32335]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:30:01.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:00 smithi027 ceph-mon[32335]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:30:01.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:00 smithi027 ceph-mon[32335]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:30:01.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:00 smithi027 ceph-mon[32335]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T08:30:01.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:00 smithi027 ceph-mon[32335]: Reconfiguring daemon mon.a on smithi027 2024-06-06T08:30:01.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:00 smithi027 ceph-mon[32335]: overall HEALTH_OK 2024-06-06T08:30:01.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:00 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/453446257' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:01.035 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-06T08:30:01.035 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:30:01.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T08:30:01.063 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:30:01.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:30:01.129 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:30:01.129 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T08:30:01.156 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:00 smithi176 ceph-mon[36786]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:30:01.156 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:00 smithi176 ceph-mon[36786]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:30:01.156 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:00 smithi176 ceph-mon[36786]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:30:01.156 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:00 smithi176 ceph-mon[36786]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T08:30:01.156 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:00 smithi176 ceph-mon[36786]: Reconfiguring daemon mon.a on smithi027 2024-06-06T08:30:01.156 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:00 smithi176 ceph-mon[36786]: overall HEALTH_OK 2024-06-06T08:30:01.156 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:00 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/453446257' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:01.161 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:30:01.161 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:30:01.230 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:30:01.230 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T08:30:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:00 smithi203 ceph-mon[36579]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:30:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:00 smithi203 ceph-mon[36579]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:30:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:00 smithi203 ceph-mon[36579]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:30:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:00 smithi203 ceph-mon[36579]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T08:30:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:00 smithi203 ceph-mon[36579]: Reconfiguring daemon mon.a on smithi027 2024-06-06T08:30:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:00 smithi203 ceph-mon[36579]: overall HEALTH_OK 2024-06-06T08:30:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:00 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/453446257' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:01.262 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:30:01.262 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:30:01.330 INFO:tasks.cephadm:Adding mgr.x on smithi027 2024-06-06T08:30:01.331 INFO:tasks.cephadm:Adding mgr.y on smithi176 2024-06-06T08:30:01.331 INFO:tasks.cephadm:Adding mgr.z on smithi203 2024-06-06T08:30:01.331 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi027=x;smithi176=y;smithi203=z' 2024-06-06T08:30:01.607 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:30:01.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:01 smithi027 ceph-mon[32335]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:01.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:01 smithi203 ceph-mon[36579]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:02.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:01 smithi176 ceph-mon[36786]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:03.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.057+0000 7fa991378700 1 -- 172.21.15.203:0/1727582318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98c100ef0 msgr2=0x7fa98c101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:03.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.057+0000 7fa991378700 1 --2- 172.21.15.203:0/1727582318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98c100ef0 0x7fa98c101330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa974009910 tx=0x7fa974009c20 comp rx=0 tx=0).stop 2024-06-06T08:30:03.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.057+0000 7fa991378700 1 -- 172.21.15.203:0/1727582318 shutdown_connections 2024-06-06T08:30:03.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.057+0000 7fa991378700 1 --2- 172.21.15.203:0/1727582318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98c100ef0 0x7fa98c101330 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:03.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.057+0000 7fa991378700 1 -- 172.21.15.203:0/1727582318 >> 172.21.15.203:0/1727582318 conn(0x7fa98c0fc500 msgr2=0x7fa98c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:30:03.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.057+0000 7fa991378700 1 -- 172.21.15.203:0/1727582318 shutdown_connections 2024-06-06T08:30:03.056 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.057+0000 7fa991378700 1 -- 172.21.15.203:0/1727582318 wait complete. 2024-06-06T08:30:03.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:03.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:03.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:02 smithi203 ceph-mon[36579]: Reconfiguring mon.b (monmap changed)... 2024-06-06T08:30:03.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:30:03.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T08:30:03.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:03.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:02 smithi203 ceph-mon[36579]: Reconfiguring daemon mon.b on smithi176 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.057+0000 7fa991378700 1 Processor -- start 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa991378700 1 -- start start 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa991378700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa98c100ef0 0x7fa98c19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa991378700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98c19e400 0x7fa98c1a3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa991378700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa98c19e880 0x7fa98c1a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa991378700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c106f30 con 0x7fa98c19e400 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa991378700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c106db0 con 0x7fa98c19e880 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa991378700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1070b0 con 0x7fa98c100ef0 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa98affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa98c100ef0 0x7fa98c19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa98affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa98c100ef0 0x7fa98c19dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:35328/0 (socket says 172.21.15.203:35328) 2024-06-06T08:30:03.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa98affd700 1 -- 172.21.15.203:0/3220413098 learned_addr learned my addr 172.21.15.203:0/3220413098 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa98a7fc700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98c19e400 0x7fa98c1a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa98b7fe700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa98c19e880 0x7fa98c1a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa98affd700 1 -- 172.21.15.203:0/3220413098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa98c19e880 msgr2=0x7fa98c1a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.058+0000 7fa98affd700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa98c19e880 0x7fa98c1a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa98affd700 1 -- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98c19e400 msgr2=0x7fa98c1a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa98affd700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98c19e400 0x7fa98c1a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa98affd700 1 -- 172.21.15.203:0/3220413098 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c1a62d0 con 0x7fa98c100ef0 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa98a7fc700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98c19e400 0x7fa98c1a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa98affd700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa98c100ef0 0x7fa98c19dec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa974004f80 tx=0x7fa974004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa983fff700 1 -- 172.21.15.203:0/3220413098 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974026070 con 0x7fa98c100ef0 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa983fff700 1 -- 172.21.15.203:0/3220413098 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa97400b750 con 0x7fa98c100ef0 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c1a6560 con 0x7fa98c100ef0 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa983fff700 1 -- 172.21.15.203:0/3220413098 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9740039b0 con 0x7fa98c100ef0 2024-06-06T08:30:03.061 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.059+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c1a6ac0 con 0x7fa98c100ef0 2024-06-06T08:30:03.062 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.060+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98c063af0 con 0x7fa98c100ef0 2024-06-06T08:30:03.062 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.060+0000 7fa983fff700 1 -- 172.21.15.203:0/3220413098 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa974029070 con 0x7fa98c100ef0 2024-06-06T08:30:03.062 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.060+0000 7fa983fff700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa978039580 0x7fa97803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:03.062 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.060+0000 7fa983fff700 1 -- 172.21.15.203:0/3220413098 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa97404cd30 con 0x7fa98c100ef0 2024-06-06T08:30:03.062 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.061+0000 7fa98a7fc700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa978039580 0x7fa97803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:03.062 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.061+0000 7fa98a7fc700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa978039580 0x7fa97803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009ec0 tx=0x7fa97c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:03.064 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.065+0000 7fa983fff700 1 -- 172.21.15.203:0/3220413098 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa974013070 con 0x7fa98c100ef0 2024-06-06T08:30:03.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:03.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:03.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:02 smithi176 ceph-mon[36786]: Reconfiguring mon.b (monmap changed)... 2024-06-06T08:30:03.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:30:03.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T08:30:03.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:03.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:02 smithi176 ceph-mon[36786]: Reconfiguring daemon mon.b on smithi176 2024-06-06T08:30:03.270 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.271+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi176=y;smithi203=z", "target": ["mon-mgr", ""]}) -- 0x7fa98c06adb0 con 0x7fa978039580 2024-06-06T08:30:03.276 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.277+0000 7fa983fff700 1 -- 172.21.15.203:0/3220413098 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa98c06adb0 con 0x7fa978039580 2024-06-06T08:30:03.276 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mgr update... 2024-06-06T08:30:03.278 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa978039580 msgr2=0x7fa97803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:03.278 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa978039580 0x7fa97803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009ec0 tx=0x7fa97c009210 comp rx=0 tx=0).stop 2024-06-06T08:30:03.278 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa98c100ef0 msgr2=0x7fa98c19dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:03.278 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa98c100ef0 0x7fa98c19dec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa974004f80 tx=0x7fa974004fb0 comp rx=0 tx=0).stop 2024-06-06T08:30:03.278 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 shutdown_connections 2024-06-06T08:30:03.279 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa978039580 0x7fa97803ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:03.279 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa98c19e880 0x7fa98c1a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:03.279 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa98c100ef0 0x7fa98c19dec0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:03.279 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 --2- 172.21.15.203:0/3220413098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa98c19e400 0x7fa98c1a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:03.279 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 >> 172.21.15.203:0/3220413098 conn(0x7fa98c0fc500 msgr2=0x7fa98c06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:30:03.279 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 shutdown_connections 2024-06-06T08:30:03.279 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:03.279+0000 7fa991378700 1 -- 172.21.15.203:0/3220413098 wait complete. 2024-06-06T08:30:03.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:03.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:03.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:02 smithi027 ceph-mon[32335]: Reconfiguring mon.b (monmap changed)... 2024-06-06T08:30:03.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:30:03.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T08:30:03.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:03.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:02 smithi027 ceph-mon[32335]: Reconfiguring daemon mon.b on smithi176 2024-06-06T08:30:03.832 DEBUG:teuthology.orchestra.run.smithi176:mgr.y> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-06T08:30:03.835 DEBUG:teuthology.orchestra.run.smithi203:mgr.z> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-06T08:30:03.837 INFO:tasks.cephadm:Deploying OSDs... 2024-06-06T08:30:03.837 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:30:03.837 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T08:30:03.858 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T08:30:03.858 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-06-06T08:30:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T08:30:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:03.917 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-06T08:30:03.918 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:03.918 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:03.918 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-06-06 08:29:10.587722786 +0000 2024-06-06T08:30:03.918 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-06-06 08:24:29.629763687 +0000 2024-06-06T08:30:03.918 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-06-06 08:24:29.629763687 +0000 2024-06-06T08:30:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-06-06 08:24:29.629763687 +0000 2024-06-06T08:30:03.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T08:30:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-06-06T08:30:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-06-06T08:30:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000206242 s, 2.5 MB/s 2024-06-06T08:30:03.989 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T08:30:04.049 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-06-06T08:30:04.106 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T08:30:04.109 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:04.110 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-06-06T08:30:04.110 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:04.110 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:04.110 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-06-06 08:29:10.616722059 +0000 2024-06-06T08:30:04.110 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-06-06 08:24:29.973755020 +0000 2024-06-06T08:30:04.110 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-06-06 08:24:29.973755020 +0000 2024-06-06T08:30:04.110 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-06-06 08:24:29.973755020 +0000 2024-06-06T08:30:04.110 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T08:30:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-06-06T08:30:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-06-06T08:30:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000183321 s, 2.8 MB/s 2024-06-06T08:30:04.175 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T08:30:04.233 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-06-06T08:30:04.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:03 smithi203 ceph-mon[36579]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:04.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:04.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:04.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:04.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:30:04.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T08:30:04.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:03 smithi176 ceph-mon[36786]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:30:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T08:30:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T08:30:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:04.291 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-06T08:30:04.291 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:04.291 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:04.291 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-06-06 08:29:10.643721383 +0000 2024-06-06T08:30:04.291 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-06-06 08:24:30.321746252 +0000 2024-06-06T08:30:04.291 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-06-06 08:24:30.321746252 +0000 2024-06-06T08:30:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-06-06 08:24:30.321746252 +0000 2024-06-06T08:30:04.292 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T08:30:04.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:03 smithi027 ceph-mon[32335]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:04.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:04.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:04.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:04.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T08:30:04.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T08:30:04.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:04.357 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-06-06T08:30:04.357 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-06-06T08:30:04.357 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000200375 s, 2.6 MB/s 2024-06-06T08:30:04.359 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T08:30:04.416 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-06-06T08:30:04.473 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T08:30:04.473 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:04.473 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-06-06T08:30:04.473 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:04.473 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:04.473 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-06-06 08:29:10.669720732 +0000 2024-06-06T08:30:04.473 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-06-06 08:24:30.672737409 +0000 2024-06-06T08:30:04.474 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-06-06 08:24:30.672737409 +0000 2024-06-06T08:30:04.474 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-06-06 08:24:30.672737409 +0000 2024-06-06T08:30:04.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T08:30:04.538 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-06-06T08:30:04.538 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-06-06T08:30:04.538 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000177884 s, 2.9 MB/s 2024-06-06T08:30:04.540 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T08:30:04.597 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-06-06T08:30:04.597 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T08:30:04.616 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T08:30:04.617 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_1 2024-06-06T08:30:04.675 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T08:30:04.675 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:04.675 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-06T08:30:04.675 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:04.675 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:04.675 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-06-06 08:29:51.885581927 +0000 2024-06-06T08:30:04.675 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-06-06 08:24:34.181384442 +0000 2024-06-06T08:30:04.675 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-06-06 08:24:34.181384442 +0000 2024-06-06T08:30:04.675 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-06-06 08:24:34.181384442 +0000 2024-06-06T08:30:04.675 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T08:30:04.741 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-06-06T08:30:04.741 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-06-06T08:30:04.741 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000287282 s, 1.8 MB/s 2024-06-06T08:30:04.742 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T08:30:04.800 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_2 2024-06-06T08:30:04.857 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T08:30:04.857 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:04.858 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-06T08:30:04.858 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:04.858 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:04.858 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-06-06 08:29:51.886581919 +0000 2024-06-06T08:30:04.858 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-06-06 08:24:34.540381960 +0000 2024-06-06T08:30:04.858 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-06-06 08:24:34.540381960 +0000 2024-06-06T08:30:04.858 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-06-06 08:24:34.540381960 +0000 2024-06-06T08:30:04.858 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T08:30:04.922 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-06-06T08:30:04.925 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-06-06T08:30:04.925 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000231121 s, 2.2 MB/s 2024-06-06T08:30:04.926 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T08:30:04.984 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_3 2024-06-06T08:30:05.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:04 smithi176 ceph-mon[36786]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi176=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:30:05.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:04 smithi176 ceph-mon[36786]: Saving service mgr spec with placement smithi027=x;smithi176=y;smithi203=z;count:3 2024-06-06T08:30:05.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:04 smithi176 ceph-mon[36786]: Reconfiguring mon.c (monmap changed)... 2024-06-06T08:30:05.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:04 smithi176 ceph-mon[36786]: Reconfiguring daemon mon.c on smithi203 2024-06-06T08:30:05.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:04 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:05.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:04 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:05.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:04 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:05.022 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T08:30:05.022 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:05.022 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-06T08:30:05.022 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:05.022 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:05.022 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-06-06 08:29:51.886581919 +0000 2024-06-06T08:30:05.022 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-06-06 08:24:34.864379720 +0000 2024-06-06T08:30:05.022 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-06-06 08:24:34.864379720 +0000 2024-06-06T08:30:05.023 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-06-06 08:24:34.864379720 +0000 2024-06-06T08:30:05.023 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T08:30:05.086 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-06-06T08:30:05.086 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-06-06T08:30:05.086 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000201802 s, 2.5 MB/s 2024-06-06T08:30:05.087 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T08:30:05.146 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_4 2024-06-06T08:30:05.201 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T08:30:05.201 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:05.201 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-06-06T08:30:05.201 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:05.201 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:05.201 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-06-06 08:29:51.886581919 +0000 2024-06-06T08:30:05.201 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-06-06 08:24:35.190377465 +0000 2024-06-06T08:30:05.201 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-06-06 08:24:35.190377465 +0000 2024-06-06T08:30:05.201 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-06-06 08:24:35.190377465 +0000 2024-06-06T08:30:05.202 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T08:30:05.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:04 smithi203 ceph-mon[36579]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi176=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:30:05.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:04 smithi203 ceph-mon[36579]: Saving service mgr spec with placement smithi027=x;smithi176=y;smithi203=z;count:3 2024-06-06T08:30:05.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:04 smithi203 ceph-mon[36579]: Reconfiguring mon.c (monmap changed)... 2024-06-06T08:30:05.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:04 smithi203 ceph-mon[36579]: Reconfiguring daemon mon.c on smithi203 2024-06-06T08:30:05.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:04 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:05.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:04 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:05.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:04 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:05.263 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-06-06T08:30:05.264 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-06-06T08:30:05.264 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000207395 s, 2.5 MB/s 2024-06-06T08:30:05.265 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T08:30:05.321 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-06-06T08:30:05.321 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T08:30:05.337 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T08:30:05.338 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-06-06T08:30:05.393 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T08:30:05.393 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:05.393 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-06-06T08:30:05.393 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:05.393 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:05.393 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-06-06 08:29:42.972555911 +0000 2024-06-06T08:30:05.393 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-06-06 08:24:38.663838610 +0000 2024-06-06T08:30:05.394 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-06-06 08:24:38.663838610 +0000 2024-06-06T08:30:05.394 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-06-06 08:24:38.663838610 +0000 2024-06-06T08:30:05.394 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T08:30:05.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:04 smithi027 ceph-mon[32335]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi176=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:30:05.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:04 smithi027 ceph-mon[32335]: Saving service mgr spec with placement smithi027=x;smithi176=y;smithi203=z;count:3 2024-06-06T08:30:05.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:04 smithi027 ceph-mon[32335]: Reconfiguring mon.c (monmap changed)... 2024-06-06T08:30:05.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:04 smithi027 ceph-mon[32335]: Reconfiguring daemon mon.c on smithi203 2024-06-06T08:30:05.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:04 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:05.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:04 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:05.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:04 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:05.457 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-06-06T08:30:05.457 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-06-06T08:30:05.457 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000275103 s, 1.9 MB/s 2024-06-06T08:30:05.458 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T08:30:05.515 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-06-06T08:30:05.570 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T08:30:05.570 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:05.570 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-06T08:30:05.570 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:05.570 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:05.570 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-06-06 08:29:42.972555911 +0000 2024-06-06T08:30:05.570 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-06-06 08:24:39.010830299 +0000 2024-06-06T08:30:05.570 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-06-06 08:24:39.010830299 +0000 2024-06-06T08:30:05.570 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-06-06 08:24:39.010830299 +0000 2024-06-06T08:30:05.571 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T08:30:05.633 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-06-06T08:30:05.633 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-06-06T08:30:05.633 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000187597 s, 2.7 MB/s 2024-06-06T08:30:05.635 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T08:30:05.691 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-06-06T08:30:05.746 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T08:30:05.746 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:05.746 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-06T08:30:05.746 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:05.746 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:05.746 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-06-06 08:29:42.972555911 +0000 2024-06-06T08:30:05.747 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-06-06 08:24:39.326822730 +0000 2024-06-06T08:30:05.747 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-06-06 08:24:39.326822730 +0000 2024-06-06T08:30:05.747 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-06-06 08:24:39.326822730 +0000 2024-06-06T08:30:05.747 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T08:30:05.809 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-06-06T08:30:05.809 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-06-06T08:30:05.809 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000188475 s, 2.7 MB/s 2024-06-06T08:30:05.810 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T08:30:05.866 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-06-06T08:30:05.922 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T08:30:05.922 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T08:30:05.922 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-06T08:30:05.922 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:30:05.922 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T08:30:05.922 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-06-06 08:29:42.973555887 +0000 2024-06-06T08:30:05.922 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-06-06 08:24:39.651814947 +0000 2024-06-06T08:30:05.923 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-06-06 08:24:39.651814947 +0000 2024-06-06T08:30:05.923 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-06-06 08:24:39.651814947 +0000 2024-06-06T08:30:05.923 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T08:30:05.987 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-06-06T08:30:05.987 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-06-06T08:30:05.987 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000185199 s, 2.8 MB/s 2024-06-06T08:30:05.988 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T08:30:06.045 INFO:tasks.cephadm:Deploying osd.0 on smithi027 with /dev/vg_nvme/lv_4... 2024-06-06T08:30:06.045 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T08:30:06.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:05 smithi203 ceph-mon[36579]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:06.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:05 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:06.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:05 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:06.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:05 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:06.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:05 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T08:30:06.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:05 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T08:30:06.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:05 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T08:30:06.119 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:05 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:06.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:05 smithi176 ceph-mon[36786]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:06.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:05 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:06.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:05 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:06.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:05 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:06.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:05 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T08:30:06.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:05 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T08:30:06.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:05 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T08:30:06.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:05 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:06.290 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:30:06.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:05 smithi027 ceph-mon[32335]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:06.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:05 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:06.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:05 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:06.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:05 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:06.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:05 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T08:30:06.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:05 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T08:30:06.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:05 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T08:30:06.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:05 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:07.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:06 smithi203 ceph-mon[36579]: Deploying daemon mgr.z on smithi203 2024-06-06T08:30:07.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:06 smithi176 ceph-mon[36786]: Deploying daemon mgr.z on smithi203 2024-06-06T08:30:07.342 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:06 smithi027 ceph-mon[32335]: Deploying daemon mgr.z on smithi203 2024-06-06T08:30:07.490 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:07 smithi203 systemd[1]: Starting Ceph mgr.z for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:30:07.866 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:07 smithi203 podman[38402]: 2024-06-06 08:30:07.595672966 +0000 UTC m=+0.110803168 container create a7190f80265abf2a453ac436f682ad3c34b8b8f7fa61167e9f7e1197952df9bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-06T08:30:07.866 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:07 smithi203 podman[38402]: 2024-06-06 08:30:07.501686472 +0000 UTC m=+0.016816685 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:30:07.866 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:07 smithi203 podman[38402]: 2024-06-06 08:30:07.839616368 +0000 UTC m=+0.354746575 container init a7190f80265abf2a453ac436f682ad3c34b8b8f7fa61167e9f7e1197952df9bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-06T08:30:07.866 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:07 smithi203 podman[38402]: 2024-06-06 08:30:07.842134474 +0000 UTC m=+0.357264687 container start a7190f80265abf2a453ac436f682ad3c34b8b8f7fa61167e9f7e1197952df9bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:30:07.866 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:07 smithi203 ceph-mgr[38421]: -- 172.21.15.203:0/2652216175 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f8904805a0 con 0x55f890455000 2024-06-06T08:30:08.185 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:07 smithi203 ceph-mon[36579]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:08.185 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:07 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:08.185 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:07 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:08.185 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:07 smithi203 bash[38402]: a7190f80265abf2a453ac436f682ad3c34b8b8f7fa61167e9f7e1197952df9bd 2024-06-06T08:30:08.186 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:07 smithi203 systemd[1]: Started Ceph mgr.z for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:30:08.186 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:07.998+0000 7f38fbf7e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T08:30:08.186 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:08.185+0000 7f38fbf7e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:30:08.214 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T08:30:08.214 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T08:30:08.214 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-06-06T08:30:08.214 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-06-06T08:30:08.215 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012024 s, 872 MB/s 2024-06-06T08:30:08.215 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-06-06T08:30:08.239 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:07 smithi027 ceph-mon[32335]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:08.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:07 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:08.240 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:07 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:08.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:07 smithi176 ceph-mon[36786]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:08.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:07 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:08.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:07 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:08.490 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:08.288+0000 7f38fbf7e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T08:30:08.778 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2024-06-06T08:30:08.990 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:08.598+0000 7f38fbf7e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T08:30:08.990 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:08.684+0000 7f38fbf7e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T08:30:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:30:09.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:09.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T08:30:09.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T08:30:09.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T08:30:09.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:09.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:09 smithi176 ceph-mon[36786]: Deploying daemon mgr.y on smithi176 2024-06-06T08:30:09.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:09.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T08:30:09.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T08:30:09.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T08:30:09.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:09.273 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:09 smithi027 ceph-mon[32335]: Deploying daemon mgr.y on smithi176 2024-06-06T08:30:09.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:09.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T08:30:09.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T08:30:09.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T08:30:09.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:09.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:09 smithi203 ceph-mon[36579]: Deploying daemon mgr.y on smithi176 2024-06-06T08:30:10.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:10 smithi176 ceph-mon[36786]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:10.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:10 smithi203 ceph-mon[36579]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:10.238 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:09 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:09.884+0000 7f38fbf7e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T08:30:10.238 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:10.058+0000 7f38fbf7e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T08:30:10.238 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:10.236+0000 7f38fbf7e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T08:30:10.300 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:10 smithi027 ceph-mon[32335]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:10.418 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:10 smithi176 ceph-mgr[38160]: -- 172.21.15.176:0/970562587 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bdfcd565a0 con 0x55bdfcd2ac00 2024-06-06T08:30:10.418 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:10 smithi176 bash[38140]: 26d069ee88a772a8e634b75c19d4b502a6d166684abb588671c50fa64aa16b37 2024-06-06T08:30:10.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.541+0000 7fda272c5700 1 -- 172.21.15.27:0/3716773258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda201013e0 msgr2=0x7fda20101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:10.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.541+0000 7fda272c5700 1 --2- 172.21.15.27:0/3716773258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda201013e0 0x7fda20101800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0099c0 tx=0x7fda1c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:30:10.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.542+0000 7fda272c5700 1 -- 172.21.15.27:0/3716773258 shutdown_connections 2024-06-06T08:30:10.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.542+0000 7fda272c5700 1 --2- 172.21.15.27:0/3716773258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda20102f80 0x7fda20109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:10.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.542+0000 7fda272c5700 1 --2- 172.21.15.27:0/3716773258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda201013e0 0x7fda20101800 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:10.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.542+0000 7fda272c5700 1 --2- 172.21.15.27:0/3716773258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda201025e0 0x7fda20102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:10.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.542+0000 7fda272c5700 1 -- 172.21.15.27:0/3716773258 >> 172.21.15.27:0/3716773258 conn(0x7fda200fc980 msgr2=0x7fda200fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:30:10.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.542+0000 7fda272c5700 1 -- 172.21.15.27:0/3716773258 shutdown_connections 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.542+0000 7fda272c5700 1 -- 172.21.15.27:0/3716773258 wait complete. 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.542+0000 7fda272c5700 1 Processor -- start 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.542+0000 7fda272c5700 1 -- start start 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda272c5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda201013e0 0x7fda201975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda272c5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda201025e0 0x7fda20197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda272c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda20102f80 0x7fda2019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda272c5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda2010cfc0 con 0x7fda20102f80 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda272c5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fda2010ce40 con 0x7fda201025e0 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda272c5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fda2010d140 con 0x7fda201013e0 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25061700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda201013e0 0x7fda201975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:10.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25862700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda20102f80 0x7fda2019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25061700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda201013e0 0x7fda201975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:36832/0 (socket says 172.21.15.27:36832) 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25862700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda20102f80 0x7fda2019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55560/0 (socket says 172.21.15.27:55560) 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25061700 1 -- 172.21.15.27:0/2297183649 learned_addr learned my addr 172.21.15.27:0/2297183649 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda24860700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda201025e0 0x7fda20197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25061700 1 -- 172.21.15.27:0/2297183649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda201025e0 msgr2=0x7fda20197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25061700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda201025e0 0x7fda20197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25061700 1 -- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda20102f80 msgr2=0x7fda2019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25061700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda20102f80 0x7fda2019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25061700 1 -- 172.21.15.27:0/2297183649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2019f250 con 0x7fda201013e0 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25862700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda20102f80 0x7fda2019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda25061700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda201013e0 0x7fda201975f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00fa00 tx=0x7fda1c00fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.543+0000 7fda127fc700 1 -- 172.21.15.27:0/2297183649 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c027070 con 0x7fda201013e0 2024-06-06T08:30:10.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.544+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2019f4e0 con 0x7fda201013e0 2024-06-06T08:30:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.544+0000 7fda127fc700 1 -- 172.21.15.27:0/2297183649 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda1c00fc50 con 0x7fda201013e0 2024-06-06T08:30:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.544+0000 7fda127fc700 1 -- 172.21.15.27:0/2297183649 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c0173f0 con 0x7fda201013e0 2024-06-06T08:30:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.544+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2019f9c0 con 0x7fda201013e0 2024-06-06T08:30:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.544+0000 7fda127fc700 1 -- 172.21.15.27:0/2297183649 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fda1c02a070 con 0x7fda201013e0 2024-06-06T08:30:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.545+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda20069180 con 0x7fda201013e0 2024-06-06T08:30:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.545+0000 7fda127fc700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fda0c039330 0x7fda0c03b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.545+0000 7fda127fc700 1 -- 172.21.15.27:0/2297183649 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fda1c04d040 con 0x7fda201013e0 2024-06-06T08:30:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.545+0000 7fda24860700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fda0c039330 0x7fda0c03b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:10.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.545+0000 7fda24860700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fda0c039330 0x7fda0c03b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fda14007c60 tx=0x7fda140073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:10.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.550+0000 7fda127fc700 1 -- 172.21.15.27:0/2297183649 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fda1c013070 con 0x7fda201013e0 2024-06-06T08:30:10.704 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:10 smithi176 systemd[1]: Started Ceph mgr.y for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:30:10.704 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:10.520+0000 7fd836da7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T08:30:10.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:10.773+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fda200628d0 con 0x7fda0c039330 2024-06-06T08:30:10.851 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:10.539+0000 7f38fbf7e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T08:30:10.852 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:10.636+0000 7f38fbf7e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T08:30:10.852 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:10.852+0000 7f38fbf7e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T08:30:11.006 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:10.717+0000 7fd836da7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:30:11.007 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:10.830+0000 7fd836da7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T08:30:11.235 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:11.077+0000 7f38fbf7e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:30:11.489 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:11.189+0000 7fd836da7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T08:30:11.489 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:11.297+0000 7fd836da7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T08:30:11.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:11.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:30:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:30:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:30:11.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:11.749 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:11.725+0000 7f38fbf7e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T08:30:11.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:11.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.749 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:11.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:30:11.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:30:11.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:30:11.750 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:30:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:12.456 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:12.161+0000 7f38fbf7e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T08:30:12.457 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:12.258+0000 7f38fbf7e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T08:30:12.457 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:12.456+0000 7f38fbf7e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T08:30:12.866 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:12.533+0000 7fd836da7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T08:30:12.866 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:12.698+0000 7fd836da7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T08:30:12.866 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:12.866+0000 7fd836da7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T08:30:13.148 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.148 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.148 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.148 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.148 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:13.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:12.975+0000 7f38fbf7e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T08:30:13.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:13.066+0000 7f38fbf7e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T08:30:13.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:13.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:13.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T08:30:13.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T08:30:13.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:13.440 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:13.149+0000 7fd836da7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T08:30:13.440 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:13.239+0000 7fd836da7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T08:30:13.440 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:13.441+0000 7fd836da7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T08:30:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T08:30:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T08:30:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:13.642 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:13.305+0000 7f38fbf7e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T08:30:13.642 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:13.436+0000 7f38fbf7e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T08:30:13.643 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:13.529+0000 7f38fbf7e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T08:30:13.643 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:13.642+0000 7f38fbf7e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T08:30:13.756 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:13.630+0000 7fd836da7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:30:14.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:13.942+0000 7f38fbf7e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T08:30:14.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-mon[36579]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T08:30:14.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-mon[36579]: Reconfiguring daemon mgr.x on smithi027 2024-06-06T08:30:14.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-mon[36579]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:14.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:14.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1885002581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70143ff8-83c2-42ee-b943-52fd8388c1c0"}]: dispatch 2024-06-06T08:30:14.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1885002581' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70143ff8-83c2-42ee-b943-52fd8388c1c0"}]': finished 2024-06-06T08:30:14.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-mon[36579]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T08:30:14.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:14.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2226914017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:30:14.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:14 smithi027 ceph-mon[32335]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T08:30:14.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:14 smithi027 ceph-mon[32335]: Reconfiguring daemon mgr.x on smithi027 2024-06-06T08:30:14.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:14 smithi027 ceph-mon[32335]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:14.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:14.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:14 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1885002581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70143ff8-83c2-42ee-b943-52fd8388c1c0"}]: dispatch 2024-06-06T08:30:14.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:14 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1885002581' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70143ff8-83c2-42ee-b943-52fd8388c1c0"}]': finished 2024-06-06T08:30:14.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:14 smithi027 ceph-mon[32335]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T08:30:14.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:14.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:14 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2226914017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:30:14.507 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:14.225+0000 7fd836da7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T08:30:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-mon[36786]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T08:30:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-mon[36786]: Reconfiguring daemon mgr.x on smithi027 2024-06-06T08:30:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-mon[36786]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:14.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:14.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1885002581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70143ff8-83c2-42ee-b943-52fd8388c1c0"}]: dispatch 2024-06-06T08:30:14.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1885002581' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70143ff8-83c2-42ee-b943-52fd8388c1c0"}]': finished 2024-06-06T08:30:14.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-mon[36786]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T08:30:14.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:14.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2226914017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:30:14.927 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:14.637+0000 7fd836da7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T08:30:14.928 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:14.732+0000 7fd836da7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T08:30:14.928 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:14.928+0000 7fd836da7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T08:30:14.990 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:14.534+0000 7f38fbf7e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T08:30:14.990 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:30:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:30:14.747+0000 7f38fbf7e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T08:30:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:15.137+0000 7fda127fc700 1 -- 172.21.15.27:0/2297183649 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fda1c016ca0 con 0x7fda201013e0 2024-06-06T08:30:15.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-mon[36786]: Standby manager daemon z started 2024-06-06T08:30:15.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T08:30:15.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:30:15.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T08:30:15.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:30:15.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:15 smithi027 ceph-mon[32335]: Standby manager daemon z started 2024-06-06T08:30:15.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:15 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T08:30:15.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:15 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:30:15.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:15 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T08:30:15.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:15 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:30:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:15 smithi203 ceph-mon[36579]: Standby manager daemon z started 2024-06-06T08:30:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:15 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T08:30:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:15 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:30:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:15 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T08:30:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:15 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.203:0/4244111969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:30:15.756 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:15.445+0000 7fd836da7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T08:30:15.757 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:15.534+0000 7fd836da7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T08:30:16.104 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:15.770+0000 7fd836da7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T08:30:16.104 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:15.899+0000 7fd836da7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T08:30:16.104 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:15.991+0000 7fd836da7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T08:30:16.104 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:16 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:16.105+0000 7fd836da7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T08:30:16.403 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:16 smithi176 ceph-mon[36786]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:16.403 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:16 smithi176 ceph-mon[36786]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-06T08:30:16.403 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T08:30:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:16 smithi027 ceph-mon[32335]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:16 smithi027 ceph-mon[32335]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-06T08:30:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T08:30:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:16 smithi203 ceph-mon[36579]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:16 smithi203 ceph-mon[36579]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-06T08:30:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T08:30:16.756 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:16 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:16.404+0000 7fd836da7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T08:30:17.257 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:16 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:16.993+0000 7fd836da7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T08:30:17.257 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:30:17 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:30:17.211+0000 7fd836da7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T08:30:18.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:18.145+0000 7fda127fc700 1 -- 172.21.15.27:0/2297183649 <== mon.1 v2:172.21.15.203:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fda1c02ab40 con 0x7fda201013e0 2024-06-06T08:30:18.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:18 smithi027 ceph-mon[32335]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:18.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:18 smithi027 ceph-mon[32335]: Standby manager daemon y started 2024-06-06T08:30:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:18 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T08:30:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:18 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:30:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:18 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T08:30:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:18 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:30:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:18 smithi203 ceph-mon[36579]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:18 smithi203 ceph-mon[36579]: Standby manager daemon y started 2024-06-06T08:30:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:18 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T08:30:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:18 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:30:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:18 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T08:30:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:18 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:30:18.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:18 smithi176 ceph-mon[36786]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:18.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:18 smithi176 ceph-mon[36786]: Standby manager daemon y started 2024-06-06T08:30:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:18 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T08:30:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:18 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:30:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:18 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T08:30:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:18 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.176:0/1512959668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:30:19.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:19 smithi027 ceph-mon[32335]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-06T08:30:19.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T08:30:19.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:19.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:19.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:19.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:19.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:19.254 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:19 smithi203 ceph-mon[36579]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-06T08:30:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T08:30:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:19.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:19 smithi176 ceph-mon[36786]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-06T08:30:19.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T08:30:19.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:19.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:19.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:19.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:19.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:19.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:20.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:20 smithi027 ceph-mon[32335]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:20.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:20 smithi203 ceph-mon[36579]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:20.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:20 smithi176 ceph-mon[36786]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:21.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:21 smithi176 ceph-mon[36786]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:21.420 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:21 smithi027 ceph-mon[32335]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:21.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:21 smithi203 ceph-mon[36579]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:22.407 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:22 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T08:30:22.407 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:22 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:22.407 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:22 smithi027 ceph-mon[32335]: Deploying daemon osd.0 on smithi027 2024-06-06T08:30:22.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:22 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T08:30:22.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:22 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:22.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:22 smithi203 ceph-mon[36579]: Deploying daemon osd.0 on smithi027 2024-06-06T08:30:22.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:22 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T08:30:22.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:22 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:22.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:22 smithi176 ceph-mon[36786]: Deploying daemon osd.0 on smithi027 2024-06-06T08:30:23.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:23 smithi027 ceph-mon[32335]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:23.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:23 smithi203 ceph-mon[36579]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:23.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:23 smithi176 ceph-mon[36786]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:25.625 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:25 smithi027 ceph-mon[32335]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:25.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:25 smithi203 ceph-mon[36579]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:25.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:25 smithi176 ceph-mon[36786]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:27 smithi027 ceph-mon[32335]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:28.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:27 smithi203 ceph-mon[36579]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:28.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:28.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:28.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:28.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:27 smithi176 ceph-mon[36786]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:28.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:30.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:30 smithi027 ceph-mon[32335]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:30.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:30.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:30.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:30.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:30.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:30 smithi203 ceph-mon[36579]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:30.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:30 smithi176 ceph-mon[36786]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:30.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:30.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:30.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:30.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:30.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.592+0000 7fda127fc700 1 -- 172.21.15.27:0/2297183649 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda200628d0 con 0x7fda0c039330 2024-06-06T08:30:30.591 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 0 on host 'smithi027' 2024-06-06T08:30:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fda0c039330 msgr2=0x7fda0c03b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fda0c039330 0x7fda0c03b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fda14007c60 tx=0x7fda140073d0 comp rx=0 tx=0).stop 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda201013e0 msgr2=0x7fda201975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda201013e0 0x7fda201975f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00fa00 tx=0x7fda1c00fa30 comp rx=0 tx=0).stop 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 shutdown_connections 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fda0c039330 0x7fda0c03b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda20102f80 0x7fda2019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda201013e0 0x7fda201975f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 --2- 172.21.15.27:0/2297183649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda201025e0 0x7fda20197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 >> 172.21.15.27:0/2297183649 conn(0x7fda200fc980 msgr2=0x7fda20105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 shutdown_connections 2024-06-06T08:30:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:30:30.595+0000 7fda272c5700 1 -- 172.21.15.27:0/2297183649 wait complete. 2024-06-06T08:30:31.255 DEBUG:teuthology.orchestra.run.smithi027:osd.0> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.0.service 2024-06-06T08:30:31.258 INFO:tasks.cephadm:Deploying osd.1 on smithi176 with /dev/vg_nvme/lv_4... 2024-06-06T08:30:31.258 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T08:30:31.502 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:30:31.808 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:31 smithi027 ceph-mon[32335]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:31.809 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:31.809 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:31.809 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:31.809 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:31 smithi027 ceph-mon[32335]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T08:30:31.856 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:31 smithi176 ceph-mon[36786]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:31.856 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:31.856 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:31.856 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:31.856 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:31 smithi176 ceph-mon[36786]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T08:30:31.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:31 smithi203 ceph-mon[36579]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:31.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:31.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:31.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:31.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:31 smithi203 ceph-mon[36579]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T08:30:32.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:32 smithi027 ceph-mon[32335]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T08:30:32.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:32 smithi027 ceph-mon[32335]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T08:30:32.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:32 smithi027 ceph-mon[32335]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:30:32.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:32 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:32.707 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:30:32 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0[46879]: 2024-06-06T08:30:32.563+0000 7f55e812d700 -1 osd.0 0 waiting for initial osdmap 2024-06-06T08:30:32.707 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:30:32 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0[46879]: 2024-06-06T08:30:32.566+0000 7f55e3f20700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:30:32.717 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:32 smithi176 ceph-mon[36786]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T08:30:32.717 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:32 smithi176 ceph-mon[36786]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T08:30:32.717 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:32 smithi176 ceph-mon[36786]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:30:32.717 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:32 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:32.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:32 smithi203 ceph-mon[36579]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T08:30:32.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:32 smithi203 ceph-mon[36579]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T08:30:32.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:32 smithi203 ceph-mon[36579]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:30:32.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:32 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T08:30:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T08:30:33.265 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-06-06T08:30:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-06-06T08:30:33.265 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121996 s, 860 MB/s 2024-06-06T08:30:33.266 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-06-06T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:33 smithi176 ceph-mon[36786]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:33 smithi176 ceph-mon[36786]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:33 smithi176 ceph-mon[36786]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:33 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:33 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:33.852 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_4 2024-06-06T08:30:33.920 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:33 smithi027 ceph-mon[32335]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:33.920 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:33 smithi027 ceph-mon[32335]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:30:33.920 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:33 smithi027 ceph-mon[32335]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T08:30:33.920 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:33 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:33.920 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:33 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:33.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:33 smithi203 ceph-mon[36579]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:33.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:33 smithi203 ceph-mon[36579]: from='osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:30:33.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:33 smithi203 ceph-mon[36579]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T08:30:33.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:33 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:33.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:33 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:34.095 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] boot 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:34.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:34.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:34.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:30:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:30:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] boot 2024-06-06T08:30:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T08:30:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:34.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:34.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:30:34.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:34.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:34.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:34.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: osd.0 [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] boot 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:35.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:35.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:35.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 -- 172.21.15.176:0/2743334027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 msgr2=0x7f43d8102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:35.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 --2- 172.21.15.176:0/2743334027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 0x7f43d8102910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f43c80099c0 tx=0x7f43c8009cd0 comp rx=0 tx=0).stop 2024-06-06T08:30:35.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 -- 172.21.15.176:0/2743334027 shutdown_connections 2024-06-06T08:30:35.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 --2- 172.21.15.176:0/2743334027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d8104090 0x7f43d810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:35.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 --2- 172.21.15.176:0/2743334027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 0x7f43d8102910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:35.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 --2- 172.21.15.176:0/2743334027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d81036f0 0x7f43d8103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:35.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 -- 172.21.15.176:0/2743334027 >> 172.21.15.176:0/2743334027 conn(0x7f43d80fda90 msgr2=0x7f43d80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 -- 172.21.15.176:0/2743334027 shutdown_connections 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 -- 172.21.15.176:0/2743334027 wait complete. 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 Processor -- start 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.588+0000 7f43de88e700 1 -- start start 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43de88e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 0x7f43d8076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43de88e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d81036f0 0x7f43d8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43de88e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8104090 0x7f43d8074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43de88e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43d810e200 con 0x7f43d8104090 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43de88e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f43d810e080 con 0x7f43d81024f0 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43de88e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f43d810e380 con 0x7f43d81036f0 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d7fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 0x7f43d8076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d7fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 0x7f43d8076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53484/0 (socket says 172.21.15.176:53484) 2024-06-06T08:30:35.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d7fff700 1 -- 172.21.15.176:0/2727836448 learned_addr learned my addr 172.21.15.176:0/2727836448 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d77fe700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d81036f0 0x7f43d8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43dce2b700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8104090 0x7f43d8074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d7fff700 1 -- 172.21.15.176:0/2727836448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d81036f0 msgr2=0x7f43d8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d7fff700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d81036f0 0x7f43d8074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d7fff700 1 -- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8104090 msgr2=0x7f43d8074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d7fff700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8104090 0x7f43d8074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d7fff700 1 -- 172.21.15.176:0/2727836448 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d8075670 con 0x7f43d81024f0 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43dce2b700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8104090 0x7f43d8074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d7fff700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 0x7f43d8076230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f43c80098e0 tx=0x7f43c80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:35.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d57fa700 1 -- 172.21.15.176:0/2727836448 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8027070 con 0x7f43d81024f0 2024-06-06T08:30:35.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43d57fa700 1 -- 172.21.15.176:0/2727836448 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c8022470 con 0x7f43d81024f0 2024-06-06T08:30:35.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.589+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d81a2a00 con 0x7f43d81024f0 2024-06-06T08:30:35.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.590+0000 7f43d57fa700 1 -- 172.21.15.176:0/2727836448 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c800f6c0 con 0x7f43d81024f0 2024-06-06T08:30:35.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.590+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d81a2ed0 con 0x7f43d81024f0 2024-06-06T08:30:35.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.590+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d8069180 con 0x7f43d81024f0 2024-06-06T08:30:35.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.592+0000 7f43d57fa700 1 -- 172.21.15.176:0/2727836448 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f43c802a070 con 0x7f43d81024f0 2024-06-06T08:30:35.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.592+0000 7f43d57fa700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f43c00a2c10 0x7f43c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:35.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.593+0000 7f43d57fa700 1 -- 172.21.15.176:0/2727836448 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f43c80ce890 con 0x7f43d81024f0 2024-06-06T08:30:35.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.593+0000 7f43d77fe700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f43c00a2c10 0x7f43c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:35.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.593+0000 7f43d77fe700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f43c00a2c10 0x7f43c00a50d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43cc006fd0 tx=0x7f43cc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:35.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.596+0000 7f43d57fa700 1 -- 172.21.15.176:0/2727836448 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f43c8013070 con 0x7f43d81024f0 2024-06-06T08:30:35.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:35 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi027 2024-06-06T08:30:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:35 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi027 to 17088M 2024-06-06T08:30:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:35 smithi176 ceph-mon[36786]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:35 smithi176 ceph-mon[36786]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T08:30:35.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:35.801+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f43d80628d0 con 0x7f43c00a2c10 2024-06-06T08:30:35.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:35 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi027 2024-06-06T08:30:35.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:35 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi027 to 17088M 2024-06-06T08:30:35.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:35 smithi027 ceph-mon[32335]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:35.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:35 smithi027 ceph-mon[32335]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T08:30:35.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:35 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi027 2024-06-06T08:30:35.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:35 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi027 to 17088M 2024-06-06T08:30:35.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:35 smithi203 ceph-mon[36579]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T08:30:35.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:35 smithi203 ceph-mon[36579]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T08:30:36.704 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:36 smithi176 ceph-mon[36786]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:30:36.705 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:36 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:30:36.705 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:36 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:30:36.705 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:36 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:36.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:36 smithi027 ceph-mon[32335]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:30:36.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:36 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:30:36.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:36 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:30:36.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:36 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:36.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:36 smithi203 ceph-mon[36579]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:30:36.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:36 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:30:36.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:36 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:30:36.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:36 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:37.891 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:37 smithi203 ceph-mon[36579]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:37.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:37 smithi027 ceph-mon[32335]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:38.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:37 smithi176 ceph-mon[36786]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:38.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:38 smithi027 ceph-mon[32335]: from='client.? 172.21.15.176:0/3618051387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab40e570-fc0d-4f20-840d-fb0e56483710"}]: dispatch 2024-06-06T08:30:38.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:38 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab40e570-fc0d-4f20-840d-fb0e56483710"}]: dispatch 2024-06-06T08:30:38.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:38 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab40e570-fc0d-4f20-840d-fb0e56483710"}]': finished 2024-06-06T08:30:38.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:38 smithi027 ceph-mon[32335]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T08:30:38.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:38.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:38 smithi203 ceph-mon[36579]: from='client.? 172.21.15.176:0/3618051387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab40e570-fc0d-4f20-840d-fb0e56483710"}]: dispatch 2024-06-06T08:30:38.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:38 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab40e570-fc0d-4f20-840d-fb0e56483710"}]: dispatch 2024-06-06T08:30:38.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:38 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab40e570-fc0d-4f20-840d-fb0e56483710"}]': finished 2024-06-06T08:30:38.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:38 smithi203 ceph-mon[36579]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T08:30:38.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:39.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:38 smithi176 ceph-mon[36786]: from='client.? 172.21.15.176:0/3618051387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab40e570-fc0d-4f20-840d-fb0e56483710"}]: dispatch 2024-06-06T08:30:39.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:38 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab40e570-fc0d-4f20-840d-fb0e56483710"}]: dispatch 2024-06-06T08:30:39.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:38 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab40e570-fc0d-4f20-840d-fb0e56483710"}]': finished 2024-06-06T08:30:39.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:38 smithi176 ceph-mon[36786]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T08:30:39.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:39.908 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:39 smithi176 ceph-mon[36786]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:39.908 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:39 smithi176 ceph-mon[36786]: from='client.? 172.21.15.176:0/2748703077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:30:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:39 smithi027 ceph-mon[32335]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:39 smithi027 ceph-mon[32335]: from='client.? 172.21.15.176:0/2748703077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:30:39.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:39 smithi203 ceph-mon[36579]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:39.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:39 smithi203 ceph-mon[36579]: from='client.? 172.21.15.176:0/2748703077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:30:41.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:41 smithi027 ceph-mon[32335]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:41.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:41 smithi203 ceph-mon[36579]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:42.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:41 smithi176 ceph-mon[36786]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:43.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:43 smithi027 ceph-mon[32335]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:43.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:43 smithi203 ceph-mon[36579]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:44.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:43 smithi176 ceph-mon[36786]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:45.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:45 smithi027 ceph-mon[32335]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:45.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T08:30:45.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:45.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:45 smithi203 ceph-mon[36579]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:45.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T08:30:45.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:46.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:45 smithi176 ceph-mon[36786]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:46.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T08:30:46.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:46.726 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:46 smithi176 ceph-mon[36786]: Deploying daemon osd.1 on smithi176 2024-06-06T08:30:46.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:46 smithi027 ceph-mon[32335]: Deploying daemon osd.1 on smithi176 2024-06-06T08:30:46.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:46 smithi203 ceph-mon[36579]: Deploying daemon osd.1 on smithi176 2024-06-06T08:30:47.932 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:47 smithi176 ceph-mon[36786]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:47.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:47 smithi027 ceph-mon[32335]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:47.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:47 smithi203 ceph-mon[36579]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:49.781 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:49 smithi176 ceph-mon[36786]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:49.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:49 smithi027 ceph-mon[32335]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:49 smithi203 ceph-mon[36579]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:51.874 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:51 smithi176 ceph-mon[36786]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:51.875 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:51.875 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:51.875 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:51.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:51 smithi027 ceph-mon[32335]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:51.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:51.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:51.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:51 smithi203 ceph-mon[36579]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:53 smithi176 ceph-mon[36786]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:53.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:53 smithi027 ceph-mon[32335]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:53.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:53.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:53.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:53.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:53.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:53.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:53 smithi203 ceph-mon[36579]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:53.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:53.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:53.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:53.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:53.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:54.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.120+0000 7f43d57fa700 1 -- 172.21.15.176:0/2727836448 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f43d80628d0 con 0x7f43c00a2c10 2024-06-06T08:30:54.119 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 1 on host 'smithi176' 2024-06-06T08:30:54.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f43c00a2c10 msgr2=0x7f43c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:54.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f43c00a2c10 0x7f43c00a50d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43cc006fd0 tx=0x7f43cc008040 comp rx=0 tx=0).stop 2024-06-06T08:30:54.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 msgr2=0x7f43d8076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:54.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 0x7f43d8076230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f43c80098e0 tx=0x7f43c80047b0 comp rx=0 tx=0).stop 2024-06-06T08:30:54.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 shutdown_connections 2024-06-06T08:30:54.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f43c00a2c10 0x7f43c00a50d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:54.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8104090 0x7f43d8074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:54.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43d81024f0 0x7f43d8076230 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:54.122 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 --2- 172.21.15.176:0/2727836448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43d81036f0 0x7f43d8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:54.122 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 >> 172.21.15.176:0/2727836448 conn(0x7f43d80fda90 msgr2=0x7f43d8106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:30:54.122 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 shutdown_connections 2024-06-06T08:30:54.122 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:30:54.123+0000 7f43de88e700 1 -- 172.21.15.176:0/2727836448 wait complete. 2024-06-06T08:30:54.723 DEBUG:teuthology.orchestra.run.smithi176:osd.1> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.1.service 2024-06-06T08:30:54.726 INFO:tasks.cephadm:Deploying osd.2 on smithi203 with /dev/vg_nvme/lv_4... 2024-06-06T08:30:54.726 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T08:30:54.967 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:30:55.366 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:55.366 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:55.366 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:55.366 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:55 smithi203 ceph-mon[36579]: from='osd.1 [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T08:30:55.367 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:55 smithi203 ceph-mon[36579]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T08:30:55.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:55.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:55.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:55.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:55 smithi027 ceph-mon[32335]: from='osd.1 [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T08:30:55.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:55 smithi027 ceph-mon[32335]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T08:30:55.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:55.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:30:55.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:55.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:55 smithi176 ceph-mon[36786]: from='osd.1 [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T08:30:55.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:55 smithi176 ceph-mon[36786]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T08:30:56.201 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:56 smithi176 ceph-mon[36786]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:56.202 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:56 smithi176 ceph-mon[36786]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T08:30:56.202 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:56 smithi176 ceph-mon[36786]: from='osd.1 [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:30:56.202 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:56 smithi176 ceph-mon[36786]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T08:30:56.202 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:56 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:56.202 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:56 smithi176 ceph-mon[36786]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:30:56.202 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:56 smithi203 ceph-mon[36579]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:56.202 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:56 smithi203 ceph-mon[36579]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T08:30:56.203 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:56 smithi203 ceph-mon[36579]: from='osd.1 [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:30:56.203 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:56 smithi203 ceph-mon[36579]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T08:30:56.203 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:56 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:56.203 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:56 smithi203 ceph-mon[36579]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:30:56.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:56 smithi027 ceph-mon[32335]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:56.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:56 smithi027 ceph-mon[32335]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T08:30:56.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:56 smithi027 ceph-mon[32335]: from='osd.1 [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:30:56.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:56 smithi027 ceph-mon[32335]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T08:30:56.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:56 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:56.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:56 smithi027 ceph-mon[32335]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:30:56.754 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T08:30:56.754 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T08:30:56.754 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-06-06T08:30:56.754 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-06-06T08:30:56.754 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014641 s, 716 MB/s 2024-06-06T08:30:56.754 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-06-06T08:30:57.007 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:30:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1[41353]: 2024-06-06T08:30:56.551+0000 7fec1e5c2700 -1 osd.1 0 waiting for initial osdmap 2024-06-06T08:30:57.007 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:30:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1[41353]: 2024-06-06T08:30:56.554+0000 7fec163ad700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:30:57.350 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:57 smithi203 ceph-mon[36579]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:30:57.350 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:57 smithi203 ceph-mon[36579]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T08:30:57.350 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:57.351 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:57.351 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:57 smithi203 ceph-mon[36579]: from='osd.1 ' entity='osd.1' 2024-06-06T08:30:57.351 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:57 smithi176 ceph-mon[36786]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:30:57.351 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:57 smithi176 ceph-mon[36786]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T08:30:57.351 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:57.351 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:57.351 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:57 smithi176 ceph-mon[36786]: from='osd.1 ' entity='osd.1' 2024-06-06T08:30:57.352 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_4 2024-06-06T08:30:57.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:57 smithi027 ceph-mon[32335]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:30:57.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:57 smithi027 ceph-mon[32335]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T08:30:57.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:57.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:57.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:57 smithi027 ceph-mon[32335]: from='osd.1 ' entity='osd.1' 2024-06-06T08:30:57.593 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:30:58.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:30:58.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:30:58.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:58.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:58.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: osd.1 [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] boot 2024-06-06T08:30:58.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T08:30:58.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:58.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:30:58.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:58.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:58.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:30:58.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:30:58.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:58.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: osd.1 [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] boot 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: osd.1 [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] boot 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:30:58.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.077+0000 7f1c86659700 1 -- 172.21.15.203:0/2423010483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c80104090 msgr2=0x7f1c8010a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.077+0000 7f1c86659700 1 --2- 172.21.15.203:0/2423010483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c80104090 0x7f1c8010a940 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800b0a0 tx=0x7f1c7800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.077+0000 7f1c86659700 1 -- 172.21.15.203:0/2423010483 shutdown_connections 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.078+0000 7f1c86659700 1 --2- 172.21.15.203:0/2423010483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c80104090 0x7f1c8010a940 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.078+0000 7f1c86659700 1 --2- 172.21.15.203:0/2423010483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c801024f0 0x7f1c80102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.078+0000 7f1c86659700 1 --2- 172.21.15.203:0/2423010483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c801036f0 0x7f1c80103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.078+0000 7f1c86659700 1 -- 172.21.15.203:0/2423010483 >> 172.21.15.203:0/2423010483 conn(0x7f1c800fda90 msgr2=0x7f1c800ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.078+0000 7f1c86659700 1 -- 172.21.15.203:0/2423010483 shutdown_connections 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.078+0000 7f1c86659700 1 -- 172.21.15.203:0/2423010483 wait complete. 2024-06-06T08:30:59.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.078+0000 7f1c86659700 1 Processor -- start 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.078+0000 7f1c86659700 1 -- start start 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c86659700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c801024f0 0x7f1c80198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c86659700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c801036f0 0x7f1c80198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c86659700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c80104090 0x7f1c8019fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c86659700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8010e0c0 con 0x7f1c80104090 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c86659700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8010df40 con 0x7f1c801024f0 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c86659700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8010e240 con 0x7f1c801036f0 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c801036f0 0x7f1c80198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c801036f0 0x7f1c80198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55900/0 (socket says 172.21.15.203:55900) 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7f7fe700 1 -- 172.21.15.203:0/285840507 learned_addr learned my addr 172.21.15.203:0/285840507 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7ffff700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c801024f0 0x7f1c80198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:59.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c84bf6700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c80104090 0x7f1c8019fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7f7fe700 1 -- 172.21.15.203:0/285840507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c801024f0 msgr2=0x7f1c80198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7f7fe700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c801024f0 0x7f1c80198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7f7fe700 1 -- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c80104090 msgr2=0x7f1c8019fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7f7fe700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c80104090 0x7f1c8019fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7f7fe700 1 -- 172.21.15.203:0/285840507 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c801a0180 con 0x7f1c801036f0 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7f7fe700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c801036f0 0x7f1c80198a60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000c9e0 tx=0x7f1c7000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7d7fa700 1 -- 172.21.15.203:0/285840507 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70007ac0 con 0x7f1c801036f0 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7d7fa700 1 -- 172.21.15.203:0/285840507 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c70007c60 con 0x7f1c801036f0 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c801a0470 con 0x7f1c801036f0 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.079+0000 7f1c7d7fa700 1 -- 172.21.15.203:0/285840507 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70018730 con 0x7f1c801036f0 2024-06-06T08:30:59.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.080+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c801a09b0 con 0x7f1c801036f0 2024-06-06T08:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.081+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c80069180 con 0x7f1c801036f0 2024-06-06T08:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.081+0000 7f1c7d7fa700 1 -- 172.21.15.203:0/285840507 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f1c700188d0 con 0x7f1c801036f0 2024-06-06T08:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.082+0000 7f1c7d7fa700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c6c0a2b90 0x7f1c6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:30:59.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.082+0000 7f1c7d7fa700 1 -- 172.21.15.203:0/285840507 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f1c700cd7d0 con 0x7f1c801036f0 2024-06-06T08:30:59.084 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.082+0000 7f1c7ffff700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c6c0a2b90 0x7f1c6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:30:59.084 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.083+0000 7f1c7ffff700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c6c0a2b90 0x7f1c6c0a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c80103550 tx=0x7f1c6800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:30:59.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.087+0000 7f1c7d7fa700 1 -- 172.21.15.203:0/285840507 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1c70014030 con 0x7f1c801036f0 2024-06-06T08:30:59.289 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:30:59.289+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1c800628d0 con 0x7f1c6c0a2b90 2024-06-06T08:30:59.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:59 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi176 2024-06-06T08:30:59.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:30:59 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi176 to 17088M 2024-06-06T08:30:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:59 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi176 2024-06-06T08:30:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:30:59 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi176 to 17088M 2024-06-06T08:30:59.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:59 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi176 2024-06-06T08:30:59.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:30:59 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi176 to 17088M 2024-06-06T08:31:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:00 smithi203 ceph-mon[36579]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:00 smithi203 ceph-mon[36579]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T08:31:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:31:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:31:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:00 smithi027 ceph-mon[32335]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:00 smithi027 ceph-mon[32335]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T08:31:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:31:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:31:00.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:00.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:00 smithi176 ceph-mon[36786]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:00.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:00 smithi176 ceph-mon[36786]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T08:31:00.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:31:00.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:31:00.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:01.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:01 smithi027 ceph-mon[32335]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:31:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:01 smithi203 ceph-mon[36579]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:31:01.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:01 smithi176 ceph-mon[36786]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:31:02.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:02 smithi027 ceph-mon[32335]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:02.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:02 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/865966199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c91227d-9be5-4fea-81bc-e6696441777f"}]: dispatch 2024-06-06T08:31:02.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:02 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c91227d-9be5-4fea-81bc-e6696441777f"}]: dispatch 2024-06-06T08:31:02.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:02 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c91227d-9be5-4fea-81bc-e6696441777f"}]': finished 2024-06-06T08:31:02.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:02 smithi027 ceph-mon[32335]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T08:31:02.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:02 smithi203 ceph-mon[36579]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:02 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/865966199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c91227d-9be5-4fea-81bc-e6696441777f"}]: dispatch 2024-06-06T08:31:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:02 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c91227d-9be5-4fea-81bc-e6696441777f"}]: dispatch 2024-06-06T08:31:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:02 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c91227d-9be5-4fea-81bc-e6696441777f"}]': finished 2024-06-06T08:31:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:02 smithi203 ceph-mon[36579]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T08:31:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:02.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:02 smithi176 ceph-mon[36786]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:02.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:02 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/865966199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c91227d-9be5-4fea-81bc-e6696441777f"}]: dispatch 2024-06-06T08:31:02.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:02 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c91227d-9be5-4fea-81bc-e6696441777f"}]: dispatch 2024-06-06T08:31:02.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:02 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c91227d-9be5-4fea-81bc-e6696441777f"}]': finished 2024-06-06T08:31:02.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:02 smithi176 ceph-mon[36786]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T08:31:02.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:03.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:03 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/3024271296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:31:03.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:03 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/3024271296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:31:03.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:03 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/3024271296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:31:04.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:04 smithi027 ceph-mon[32335]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:04 smithi203 ceph-mon[36579]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:04.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:04 smithi176 ceph-mon[36786]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:06.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:06 smithi203 ceph-mon[36579]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:06.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:06 smithi027 ceph-mon[32335]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:06.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:06 smithi176 ceph-mon[36786]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:08.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:08 smithi203 ceph-mon[36579]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:08.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:08 smithi027 ceph-mon[32335]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:08.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:08 smithi176 ceph-mon[36786]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:09.408 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T08:31:09.408 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:09.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T08:31:09.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:09.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T08:31:09.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:10.398 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:10 smithi203 ceph-mon[36579]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:10.398 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:10 smithi203 ceph-mon[36579]: Deploying daemon osd.2 on smithi203 2024-06-06T08:31:10.424 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:10 smithi176 ceph-mon[36786]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:10.424 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:10 smithi176 ceph-mon[36786]: Deploying daemon osd.2 on smithi203 2024-06-06T08:31:10.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:10 smithi027 ceph-mon[32335]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:10.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:10 smithi027 ceph-mon[32335]: Deploying daemon osd.2 on smithi203 2024-06-06T08:31:12.373 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:12 smithi203 ceph-mon[36579]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:12.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:12 smithi027 ceph-mon[32335]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:12.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:12 smithi176 ceph-mon[36786]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:14.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:14 smithi027 ceph-mon[32335]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:14.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:14.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:14.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:14.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:14 smithi203 ceph-mon[36579]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:14.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:14 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:14.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:14 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:14.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:14 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:14 smithi176 ceph-mon[36786]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:14 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:14 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:14 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:16 smithi027 ceph-mon[32335]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:16 smithi203 ceph-mon[36579]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:16 smithi176 ceph-mon[36786]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:17.654 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.655+0000 7f1c7d7fa700 1 -- 172.21.15.203:0/285840507 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1c800628d0 con 0x7f1c6c0a2b90 2024-06-06T08:31:17.655 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 2 on host 'smithi203' 2024-06-06T08:31:17.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c6c0a2b90 msgr2=0x7f1c6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:17.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c6c0a2b90 0x7f1c6c0a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c80103550 tx=0x7f1c6800a3b0 comp rx=0 tx=0).stop 2024-06-06T08:31:17.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c801036f0 msgr2=0x7f1c80198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:17.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c801036f0 0x7f1c80198a60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000c9e0 tx=0x7f1c7000ccf0 comp rx=0 tx=0).stop 2024-06-06T08:31:17.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 shutdown_connections 2024-06-06T08:31:17.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c6c0a2b90 0x7f1c6c0a5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:17.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c80104090 0x7f1c8019fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:17.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c801024f0 0x7f1c80198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:17.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 --2- 172.21.15.203:0/285840507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c801036f0 0x7f1c80198a60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:17.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 >> 172.21.15.203:0/285840507 conn(0x7f1c800fda90 msgr2=0x7f1c80106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:31:17.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 shutdown_connections 2024-06-06T08:31:17.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:31:17.658+0000 7f1c86659700 1 -- 172.21.15.203:0/285840507 wait complete. 2024-06-06T08:31:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:17 smithi203 ceph-mon[36579]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:17.959 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:17.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:17.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:17.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:17.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:17.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:17.960 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:18.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:17 smithi176 ceph-mon[36786]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:18.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:18.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:18.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:18.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:17 smithi027 ceph-mon[32335]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:18.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:18.285 DEBUG:teuthology.orchestra.run.smithi203:osd.2> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.2.service 2024-06-06T08:31:18.287 INFO:tasks.cephadm:Deploying osd.3 on smithi027 with /dev/vg_nvme/lv_3... 2024-06-06T08:31:18.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T08:31:18.546 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:31:18.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:18 smithi203 ceph-mon[36579]: from='osd.2 [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T08:31:18.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:18 smithi203 ceph-mon[36579]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T08:31:18.947 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:18 smithi027 ceph-mon[32335]: from='osd.2 [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T08:31:18.947 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:18 smithi027 ceph-mon[32335]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T08:31:19.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:18 smithi176 ceph-mon[36786]: from='osd.2 [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T08:31:19.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:18 smithi176 ceph-mon[36786]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T08:31:19.921 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:19 smithi027 ceph-mon[32335]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:19.921 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:19 smithi027 ceph-mon[32335]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T08:31:19.921 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:19 smithi027 ceph-mon[32335]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T08:31:19.921 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:19.921 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:19 smithi027 ceph-mon[32335]: from='osd.2 [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:31:19.921 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:19 smithi027 ceph-mon[32335]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:31:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:19 smithi203 ceph-mon[36579]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:19 smithi203 ceph-mon[36579]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T08:31:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:19 smithi203 ceph-mon[36579]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T08:31:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:19 smithi203 ceph-mon[36579]: from='osd.2 [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:31:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:19 smithi203 ceph-mon[36579]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:31:20.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:19 smithi176 ceph-mon[36786]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:20.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:19 smithi176 ceph-mon[36786]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T08:31:20.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:19 smithi176 ceph-mon[36786]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T08:31:20.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:20.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:19 smithi176 ceph-mon[36786]: from='osd.2 [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:31:20.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:19 smithi176 ceph-mon[36786]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:31:20.490 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2[41696]: 2024-06-06T08:31:20.190+0000 7f8368929700 -1 osd.2 0 waiting for initial osdmap 2024-06-06T08:31:20.490 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2[41696]: 2024-06-06T08:31:20.194+0000 7f8360714700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:31:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T08:31:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T08:31:20.520 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-06-06T08:31:20.520 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-06-06T08:31:20.521 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0140209 s, 748 MB/s 2024-06-06T08:31:20.521 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-06-06T08:31:20.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:20 smithi027 ceph-mon[32335]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:31:20.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:20 smithi027 ceph-mon[32335]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T08:31:20.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:20 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:20.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:20 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:20.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:20 smithi027 ceph-mon[32335]: from='osd.2 ' entity='osd.2' 2024-06-06T08:31:20.855 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:20 smithi203 ceph-mon[36579]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:31:20.856 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:20 smithi203 ceph-mon[36579]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T08:31:20.856 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:20 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:20.856 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:20 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:20.856 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:20 smithi203 ceph-mon[36579]: from='osd.2 ' entity='osd.2' 2024-06-06T08:31:21.116 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:20 smithi176 ceph-mon[36786]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:31:21.116 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:20 smithi176 ceph-mon[36786]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T08:31:21.116 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:20 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:21.116 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:20 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:21.116 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:20 smithi176 ceph-mon[36786]: from='osd.2 ' entity='osd.2' 2024-06-06T08:31:21.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2024-06-06T08:31:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:31:21.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:31:21.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:31:21.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:21.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:21.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: osd.2 [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] boot 2024-06-06T08:31:21.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T08:31:21.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:21.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:21.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:21.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:31:21.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:21.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:21.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:21.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: osd.2 [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] boot 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:31:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:31:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:31:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T08:31:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: osd.2 [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] boot 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:22.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:22 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi203 2024-06-06T08:31:22.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:22 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi203 to 17088M 2024-06-06T08:31:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.225+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/898751798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f68108200 msgr2=0x7f4f6810a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:23.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.225+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/898751798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f68108200 0x7f4f6810a670 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000b0a0 tx=0x7f4f6000b3b0 comp rx=0 tx=0).stop 2024-06-06T08:31:23.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.225+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/898751798 shutdown_connections 2024-06-06T08:31:23.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.225+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/898751798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f68108200 0x7f4f6810a670 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:23.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.225+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/898751798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f68102fa0 0x7f4f68105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:23.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.225+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/898751798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f681058d0 0x7f4f68107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.225+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/898751798 >> 172.21.15.27:0/898751798 conn(0x7f4f680fc9b0 msgr2=0x7f4f680fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/898751798 shutdown_connections 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/898751798 wait complete. 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 Processor -- start 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 -- start start 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f68102fa0 0x7f4f68198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f681058d0 0x7f4f68198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f68108200 0x7f4f6819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6810e0e0 con 0x7f4f681058d0 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6810df60 con 0x7f4f68108200 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6f9b8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6810e260 con 0x7f4f68102fa0 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.226+0000 7f4f6cf53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f681058d0 0x7f4f68198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:31:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6cf53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f681058d0 0x7f4f68198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39302/0 (socket says 172.21.15.27:39302) 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6cf53700 1 -- 172.21.15.27:0/2840435634 learned_addr learned my addr 172.21.15.27:0/2840435634 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6df55700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f68108200 0x7f4f6819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6d754700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f68102fa0 0x7f4f68198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6cf53700 1 -- 172.21.15.27:0/2840435634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f68102fa0 msgr2=0x7f4f68198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6cf53700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f68102fa0 0x7f4f68198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6cf53700 1 -- 172.21.15.27:0/2840435634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f68108200 msgr2=0x7f4f6819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6cf53700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f68108200 0x7f4f6819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6cf53700 1 -- 172.21.15.27:0/2840435634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f681a02f0 con 0x7f4f681058d0 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6df55700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f68108200 0x7f4f6819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6cf53700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f681058d0 0x7f4f68198bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00e9f0 tx=0x7f4f5c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f5a7fc700 1 -- 172.21.15.27:0/2840435634 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00cbd0 con 0x7f4f681058d0 2024-06-06T08:31:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f5a7fc700 1 -- 172.21.15.27:0/2840435634 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f5c00cd70 con 0x7f4f681058d0 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f681a05e0 con 0x7f4f681058d0 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f5a7fc700 1 -- 172.21.15.27:0/2840435634 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c010430 con 0x7f4f681058d0 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.227+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f681a0ad0 con 0x7f4f681058d0 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.228+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f68069180 con 0x7f4f681058d0 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.230+0000 7f4f5a7fc700 1 -- 172.21.15.27:0/2840435634 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f4f5c004750 con 0x7f4f681058d0 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.230+0000 7f4f5a7fc700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4f540a2b90 0x7f4f540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.230+0000 7f4f6d754700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4f540a2b90 0x7f4f540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.231+0000 7f4f5a7fc700 1 -- 172.21.15.27:0/2840435634 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f4f5c0ce1a0 con 0x7f4f681058d0 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.231+0000 7f4f6d754700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4f540a2b90 0x7f4f540a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f64005950 tx=0x7f4f640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:31:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.234+0000 7f4f5a7fc700 1 -- 172.21.15.27:0/2840435634 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f5c053850 con 0x7f4f681058d0 2024-06-06T08:31:23.260 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:22 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi203 2024-06-06T08:31:23.260 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:22 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi203 to 17088M 2024-06-06T08:31:23.260 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:22 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi203 2024-06-06T08:31:23.260 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:22 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi203 to 17088M 2024-06-06T08:31:23.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:23.451+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4f680628d0 con 0x7f4f540a2b90 2024-06-06T08:31:24.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:23 smithi027 ceph-mon[32335]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:24.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:31:24.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T08:31:24.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:23 smithi027 ceph-mon[32335]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T08:31:24.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:31:24.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:31:24.134 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:31:24.134 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:23 smithi203 ceph-mon[36579]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:23 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:31:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:23 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T08:31:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:23 smithi203 ceph-mon[36579]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T08:31:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:23 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:31:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:23 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:31:24.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:23 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:31:24.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:23 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:24.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:23 smithi176 ceph-mon[36786]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:24.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:23 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:31:24.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:23 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T08:31:24.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:23 smithi176 ceph-mon[36786]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T08:31:24.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:23 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:31:24.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:23 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:31:24.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:23 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:31:24.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:23 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:24.706 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:31:24 smithi027 sudo[51240]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T08:31:24.706 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:31:24 smithi027 sudo[51240]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:24.706 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:31:24 smithi027 sudo[51240]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:24.706 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:31:24 smithi027 sudo[51240]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:24.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:24.902+0000 7f4f5a7fc700 1 -- 172.21.15.27:0/2840435634 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4f5c098530 con 0x7f4f681058d0 2024-06-06T08:31:25.163 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:24 smithi176 ceph-mon[36786]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:31:25.163 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T08:31:25.163 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:24 smithi176 ceph-mon[36786]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T08:31:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:24 smithi027 ceph-mon[32335]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:31:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T08:31:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:24 smithi027 ceph-mon[32335]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T08:31:25.206 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:31:24 smithi027 sudo[51245]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T08:31:25.206 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:31:24 smithi027 sudo[51245]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:25.207 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:31:24 smithi027 sudo[51245]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:25.207 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:31:24 smithi027 sudo[51245]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:24 smithi203 ceph-mon[36579]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:31:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T08:31:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:24 smithi203 ceph-mon[36579]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T08:31:25.414 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:31:25 smithi176 sudo[45493]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T08:31:25.414 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:31:25 smithi176 sudo[45493]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:25.414 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:31:25 smithi176 sudo[45493]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:25.414 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:31:25 smithi176 sudo[45493]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:25.757 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:31:25 smithi176 sudo[45496]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T08:31:25.757 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:31:25 smithi176 sudo[45496]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:25.757 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:31:25 smithi176 sudo[45496]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:25.757 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:31:25 smithi176 sudo[45496]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:25.934 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:25 smithi203 sudo[45609]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T08:31:25.934 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:25 smithi203 sudo[45609]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:25.934 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:25 smithi203 sudo[45609]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:25.935 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:25 smithi203 sudo[45609]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:25.935 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:25 smithi203 ceph-mon[36579]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:25.935 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:25 smithi203 ceph-mon[36579]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T08:31:25.935 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:25 smithi203 ceph-mon[36579]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T08:31:25.935 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:25.935 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:25 smithi027 ceph-mon[32335]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:25 smithi027 ceph-mon[32335]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T08:31:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:25 smithi027 ceph-mon[32335]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T08:31:26.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:26.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:26.240 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:25 smithi203 sudo[45612]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T08:31:26.240 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:25 smithi203 sudo[45612]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:26.240 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:25 smithi203 sudo[45612]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:26.240 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:31:25 smithi203 sudo[45612]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:26.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:25 smithi176 ceph-mon[36786]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:26.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:25 smithi176 ceph-mon[36786]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T08:31:26.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:25 smithi176 ceph-mon[36786]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T08:31:26.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:26.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:26.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:26 smithi027 sudo[51310]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T08:31:26.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:26 smithi027 sudo[51310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:26.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:26 smithi027 sudo[51310]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:26.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:26 smithi027 sudo[51310]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:26 smithi027 sudo[51313]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T08:31:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:26 smithi027 sudo[51313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:26 smithi027 sudo[51313]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:26 smithi027 sudo[51313]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:27.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 sudo[45615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T08:31:27.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 sudo[45615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:27.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 sudo[45615]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:27.346 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 sudo[45615]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:27.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:27.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:27 smithi027 ceph-mon[32335]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T08:31:27.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:27 smithi027 ceph-mon[32335]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T08:31:27.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:31:27.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:31:27.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:31:27.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:31:27.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:31:27.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:31:27.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 sudo[45618]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T08:31:27.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 sudo[45618]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:27.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 sudo[45618]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:27.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 sudo[45618]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:27.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:27.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 ceph-mon[36579]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T08:31:27.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 ceph-mon[36579]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T08:31:27.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:31:27.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:31:27.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:31:27.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:31:27.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:31:27.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:31:27.847 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 sudo[45499]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T08:31:27.848 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 sudo[45499]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:27.848 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 sudo[45499]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:27.848 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 sudo[45499]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 sudo[45502]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 sudo[45502]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 sudo[45502]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 sudo[45502]: pam_unix(sudo:session): session closed for user root 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 ceph-mon[36786]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 ceph-mon[36786]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:31:28.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:31:28.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:31:28.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:31:28.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:28.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T08:31:28.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/82981986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa6a8c4e-0080-4151-aee8-052550a92b6c"}]: dispatch 2024-06-06T08:31:28.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T08:31:28.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T08:31:28.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:31:28.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:31:28.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:31:28.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/82981986' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa6a8c4e-0080-4151-aee8-052550a92b6c"}]': finished 2024-06-06T08:31:28.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: osdmap e22: 4 total, 3 up, 4 in 2024-06-06T08:31:28.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:28.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T08:31:28.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:28 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1454049075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:31:28.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:28.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T08:31:28.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/82981986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa6a8c4e-0080-4151-aee8-052550a92b6c"}]: dispatch 2024-06-06T08:31:28.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T08:31:28.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T08:31:28.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:31:28.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:31:28.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:31:28.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/82981986' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa6a8c4e-0080-4151-aee8-052550a92b6c"}]': finished 2024-06-06T08:31:28.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: osdmap e22: 4 total, 3 up, 4 in 2024-06-06T08:31:28.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:28.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T08:31:28.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:28 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1454049075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:31:28.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/82981986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa6a8c4e-0080-4151-aee8-052550a92b6c"}]: dispatch 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/82981986' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa6a8c4e-0080-4151-aee8-052550a92b6c"}]': finished 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: osdmap e22: 4 total, 3 up, 4 in 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:28.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T08:31:28.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:28 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1454049075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:31:29.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:29 smithi027 ceph-mon[32335]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:29.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:29 smithi203 ceph-mon[36579]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:29.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:29 smithi176 ceph-mon[36786]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:31.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:31 smithi027 ceph-mon[32335]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:31.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:31 smithi203 ceph-mon[36579]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:31.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:31 smithi176 ceph-mon[36786]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:33.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:33 smithi027 ceph-mon[32335]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:33 smithi203 ceph-mon[36579]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:33.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:33 smithi176 ceph-mon[36786]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:35.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:35 smithi027 ceph-mon[32335]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:35.696 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T08:31:35.696 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:35.696 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:35 smithi027 ceph-mon[32335]: Deploying daemon osd.3 on smithi027 2024-06-06T08:31:35.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:35 smithi203 ceph-mon[36579]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:35.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:35 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T08:31:35.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:35 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:35.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:35 smithi203 ceph-mon[36579]: Deploying daemon osd.3 on smithi027 2024-06-06T08:31:35.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:35 smithi176 ceph-mon[36786]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:35.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:35 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T08:31:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:35 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:35 smithi176 ceph-mon[36786]: Deploying daemon osd.3 on smithi027 2024-06-06T08:31:37.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:37 smithi027 ceph-mon[32335]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:37.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:37 smithi203 ceph-mon[36579]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:37.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:37 smithi176 ceph-mon[36786]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:39.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:39 smithi027 ceph-mon[32335]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:39.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:39 smithi203 ceph-mon[36579]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:39.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:39 smithi176 ceph-mon[36786]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:42.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:41 smithi027 ceph-mon[32335]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:42.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:41 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:42.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:41 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:42.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:41 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:42.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:41 smithi203 ceph-mon[36579]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:42.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:41 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:42.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:41 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:42.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:41 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:42.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:41 smithi176 ceph-mon[36786]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:42.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:41 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:42.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:41 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:42.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:41 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:44.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:43 smithi027 ceph-mon[32335]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:44.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:43 smithi203 ceph-mon[36579]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:44.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:43 smithi176 ceph-mon[36786]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:44.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.911+0000 7f4f5a7fc700 1 -- 172.21.15.27:0/2840435634 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4f680628d0 con 0x7f4f540a2b90 2024-06-06T08:31:44.912 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 3 on host 'smithi027' 2024-06-06T08:31:44.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4f540a2b90 msgr2=0x7f4f540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4f540a2b90 0x7f4f540a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f64005950 tx=0x7f4f640058e0 comp rx=0 tx=0).stop 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f681058d0 msgr2=0x7f4f68198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f681058d0 0x7f4f68198bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00e9f0 tx=0x7f4f5c00edb0 comp rx=0 tx=0).stop 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 shutdown_connections 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4f540a2b90 0x7f4f540a5050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f68108200 0x7f4f6819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f68102fa0 0x7f4f68198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 --2- 172.21.15.27:0/2840435634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f681058d0 0x7f4f68198bd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 >> 172.21.15.27:0/2840435634 conn(0x7f4f680fc9b0 msgr2=0x7f4f681038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 shutdown_connections 2024-06-06T08:31:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:31:44.914+0000 7f4f6f9b8700 1 -- 172.21.15.27:0/2840435634 wait complete. 2024-06-06T08:31:45.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:45.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:45.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:44 smithi027 ceph-mon[32335]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T08:31:45.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:44 smithi203 ceph-mon[36579]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T08:31:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:45.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:45.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:44 smithi176 ceph-mon[36786]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T08:31:45.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:45.598 DEBUG:teuthology.orchestra.run.smithi027:osd.3> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.3.service 2024-06-06T08:31:45.601 INFO:tasks.cephadm:Deploying osd.4 on smithi176 with /dev/vg_nvme/lv_3... 2024-06-06T08:31:45.601 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T08:31:45.846 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:31:46.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:45 smithi027 ceph-mon[32335]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:46.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:45 smithi027 ceph-mon[32335]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T08:31:46.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:45 smithi027 ceph-mon[32335]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T08:31:46.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:45 smithi027 ceph-mon[32335]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:31:46.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:46.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:46.207 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:31:45 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3[53334]: 2024-06-06T08:31:45.940+0000 7fc6e88ff700 -1 osd.3 0 waiting for initial osdmap 2024-06-06T08:31:46.208 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:31:45 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3[53334]: 2024-06-06T08:31:45.944+0000 7fc6e46f2700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:31:46.234 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:45 smithi176 ceph-mon[36786]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:46.234 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:45 smithi176 ceph-mon[36786]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T08:31:46.235 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:45 smithi176 ceph-mon[36786]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T08:31:46.235 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:45 smithi176 ceph-mon[36786]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:31:46.235 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:46.235 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:46.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:45 smithi203 ceph-mon[36579]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:46.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:45 smithi203 ceph-mon[36579]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T08:31:46.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:45 smithi203 ceph-mon[36579]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T08:31:46.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:45 smithi203 ceph-mon[36579]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:31:46.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:46.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:31:47.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:46 smithi027 ceph-mon[32335]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:31:47.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:46 smithi027 ceph-mon[32335]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T08:31:47.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:46 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:47.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:46 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:47.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:46 smithi203 ceph-mon[36579]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:31:47.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:46 smithi203 ceph-mon[36579]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T08:31:47.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:46 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:47.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:46 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:47.248 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:46 smithi176 ceph-mon[36786]: from='osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:31:47.248 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:46 smithi176 ceph-mon[36786]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T08:31:47.248 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:46 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:47.248 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:46 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:47.819 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T08:31:47.819 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T08:31:47.819 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-06-06T08:31:47.819 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-06-06T08:31:47.819 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123226 s, 851 MB/s 2024-06-06T08:31:47.819 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-06-06T08:31:48.186 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:47 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:31:48.186 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:47 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:31:48.187 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:47 smithi176 ceph-mon[36786]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:48.187 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:48.187 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:47 smithi176 ceph-mon[36786]: osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] boot 2024-06-06T08:31:48.187 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:47 smithi176 ceph-mon[36786]: osdmap e25: 4 total, 4 up, 4 in 2024-06-06T08:31:48.187 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:48.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:47 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:31:48.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:47 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:31:48.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:47 smithi027 ceph-mon[32335]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:48.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:48.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:47 smithi027 ceph-mon[32335]: osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] boot 2024-06-06T08:31:48.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:47 smithi027 ceph-mon[32335]: osdmap e25: 4 total, 4 up, 4 in 2024-06-06T08:31:48.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:48.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:47 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:31:48.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:47 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:31:48.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:47 smithi203 ceph-mon[36579]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-06T08:31:48.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:48.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:47 smithi203 ceph-mon[36579]: osd.3 [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] boot 2024-06-06T08:31:48.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:47 smithi203 ceph-mon[36579]: osdmap e25: 4 total, 4 up, 4 in 2024-06-06T08:31:48.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:31:48.497 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_3 2024-06-06T08:31:48.731 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:31:49.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:48 smithi027 ceph-mon[32335]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T08:31:49.131 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:48 smithi176 ceph-mon[36786]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T08:31:49.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:48 smithi203 ceph-mon[36579]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T08:31:50.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi027 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.222 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:31:50.222 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T08:31:50.222 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi027 2024-06-06T08:31:50.222 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.222 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.222 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:31:50.223 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:31:50.223 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.223 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:50.223 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:50.223 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:31:50.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T08:31:50.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi027 2024-06-06T08:31:50.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:31:50.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:31:50.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:50.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:31:50.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:31:50.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.329+0000 7fcb2232c700 1 -- 172.21.15.176:0/250421661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102fb0 msgr2=0x7fcb1c109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:50.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.329+0000 7fcb2232c700 1 --2- 172.21.15.176:0/250421661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102fb0 0x7fcb1c109860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400b0d0 tx=0x7fcb1400b3e0 comp rx=0 tx=0).stop 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.329+0000 7fcb2232c700 1 -- 172.21.15.176:0/250421661 shutdown_connections 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.329+0000 7fcb2232c700 1 --2- 172.21.15.176:0/250421661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102fb0 0x7fcb1c109860 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.329+0000 7fcb2232c700 1 --2- 172.21.15.176:0/250421661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb1c101410 0x7fcb1c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.329+0000 7fcb2232c700 1 --2- 172.21.15.176:0/250421661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb1c102610 0x7fcb1c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.329+0000 7fcb2232c700 1 -- 172.21.15.176:0/250421661 >> 172.21.15.176:0/250421661 conn(0x7fcb1c0fc9f0 msgr2=0x7fcb1c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.329+0000 7fcb2232c700 1 -- 172.21.15.176:0/250421661 shutdown_connections 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.329+0000 7fcb2232c700 1 -- 172.21.15.176:0/250421661 wait complete. 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb2232c700 1 Processor -- start 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb2232c700 1 -- start start 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb2232c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb1c101410 0x7fcb1c197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:31:50.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb2232c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102610 0x7fcb1c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb2232c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb1c102fb0 0x7fcb1c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb2232c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c10cfd0 con 0x7fcb1c102fb0 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb2232c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c10ce50 con 0x7fcb1c102610 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb2232c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c10d150 con 0x7fcb1c101410 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb1b7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102610 0x7fcb1c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb1b7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102610 0x7fcb1c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:56076/0 (socket says 172.21.15.176:56076) 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb1b7fe700 1 -- 172.21.15.176:0/619478520 learned_addr learned my addr 172.21.15.176:0/619478520 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.330+0000 7fcb208c9700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb1c102fb0 0x7fcb1c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb1bfff700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb1c101410 0x7fcb1c197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb1b7fe700 1 -- 172.21.15.176:0/619478520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb1c101410 msgr2=0x7fcb1c197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb1b7fe700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb1c101410 0x7fcb1c197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:50.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb1b7fe700 1 -- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb1c102fb0 msgr2=0x7fcb1c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:31:50.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb1b7fe700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb1c102fb0 0x7fcb1c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:31:50.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb1b7fe700 1 -- 172.21.15.176:0/619478520 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c19f1c0 con 0x7fcb1c102610 2024-06-06T08:31:50.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb1b7fe700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102610 0x7fcb1c197aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d950 tx=0x7fcb1000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:31:50.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb197fa700 1 -- 172.21.15.176:0/619478520 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10016070 con 0x7fcb1c102610 2024-06-06T08:31:50.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c06ac60 con 0x7fcb1c102610 2024-06-06T08:31:50.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb197fa700 1 -- 172.21.15.176:0/619478520 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1000fd70 con 0x7fcb1c102610 2024-06-06T08:31:50.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c06b1f0 con 0x7fcb1c102610 2024-06-06T08:31:50.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.331+0000 7fcb197fa700 1 -- 172.21.15.176:0/619478520 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10020060 con 0x7fcb1c102610 2024-06-06T08:31:50.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.332+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1c069180 con 0x7fcb1c102610 2024-06-06T08:31:50.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.333+0000 7fcb197fa700 1 -- 172.21.15.176:0/619478520 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcb10019070 con 0x7fcb1c102610 2024-06-06T08:31:50.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.334+0000 7fcb197fa700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcb040a2c20 0x7fcb040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:31:50.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.334+0000 7fcb197fa700 1 -- 172.21.15.176:0/619478520 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fcb100cdbf0 con 0x7fcb1c102610 2024-06-06T08:31:50.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.334+0000 7fcb1bfff700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcb040a2c20 0x7fcb040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:31:50.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.334+0000 7fcb1bfff700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcb040a2c20 0x7fcb040a50e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c102470 tx=0x7fcb0c00d620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:31:50.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.339+0000 7fcb197fa700 1 -- 172.21.15.176:0/619478520 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcb1009bd70 con 0x7fcb1c102610 2024-06-06T08:31:50.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:31:50.542+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcb1c06b450 con 0x7fcb040a2c20 2024-06-06T08:31:51.161 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:50 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi027 to 8544M 2024-06-06T08:31:51.161 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:50 smithi176 ceph-mon[36786]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T08:31:51.161 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:50 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:31:51.161 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:50 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:31:51.161 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:50 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:51.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:50 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi027 to 8544M 2024-06-06T08:31:51.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:50 smithi203 ceph-mon[36579]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T08:31:51.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:50 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:31:51.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:50 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:31:51.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:50 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:51.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:50 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi027 to 8544M 2024-06-06T08:31:51.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:50 smithi027 ceph-mon[32335]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T08:31:51.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:50 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:31:51.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:50 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:31:51.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:50 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:31:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:51 smithi203 ceph-mon[36579]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:31:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:51 smithi203 ceph-mon[36579]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:31:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:51 smithi203 ceph-mon[36579]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T08:31:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:51 smithi176 ceph-mon[36786]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:31:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:51 smithi176 ceph-mon[36786]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:31:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:51 smithi176 ceph-mon[36786]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T08:31:52.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:51 smithi027 ceph-mon[32335]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:31:52.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:51 smithi027 ceph-mon[32335]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:31:52.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:51 smithi027 ceph-mon[32335]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T08:31:54.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:53 smithi203 ceph-mon[36579]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:31:54.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:53 smithi203 ceph-mon[36579]: from='client.? 172.21.15.176:0/4167924783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b29f8a35-7dd1-4418-9aa3-dca189a351b6"}]: dispatch 2024-06-06T08:31:54.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:53 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b29f8a35-7dd1-4418-9aa3-dca189a351b6"}]: dispatch 2024-06-06T08:31:54.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:53 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b29f8a35-7dd1-4418-9aa3-dca189a351b6"}]': finished 2024-06-06T08:31:54.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:53 smithi203 ceph-mon[36579]: osdmap e30: 5 total, 4 up, 5 in 2024-06-06T08:31:54.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:31:54.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:53 smithi203 ceph-mon[36579]: from='client.? 172.21.15.176:0/899203263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:31:54.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:53 smithi176 ceph-mon[36786]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:31:54.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:53 smithi176 ceph-mon[36786]: from='client.? 172.21.15.176:0/4167924783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b29f8a35-7dd1-4418-9aa3-dca189a351b6"}]: dispatch 2024-06-06T08:31:54.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:53 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b29f8a35-7dd1-4418-9aa3-dca189a351b6"}]: dispatch 2024-06-06T08:31:54.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:53 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b29f8a35-7dd1-4418-9aa3-dca189a351b6"}]': finished 2024-06-06T08:31:54.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:53 smithi176 ceph-mon[36786]: osdmap e30: 5 total, 4 up, 5 in 2024-06-06T08:31:54.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:31:54.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:53 smithi176 ceph-mon[36786]: from='client.? 172.21.15.176:0/899203263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:31:54.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:53 smithi027 ceph-mon[32335]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:31:54.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:53 smithi027 ceph-mon[32335]: from='client.? 172.21.15.176:0/4167924783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b29f8a35-7dd1-4418-9aa3-dca189a351b6"}]: dispatch 2024-06-06T08:31:54.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:53 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b29f8a35-7dd1-4418-9aa3-dca189a351b6"}]: dispatch 2024-06-06T08:31:54.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:53 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b29f8a35-7dd1-4418-9aa3-dca189a351b6"}]': finished 2024-06-06T08:31:54.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:53 smithi027 ceph-mon[32335]: osdmap e30: 5 total, 4 up, 5 in 2024-06-06T08:31:54.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:31:54.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:53 smithi027 ceph-mon[32335]: from='client.? 172.21.15.176:0/899203263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:31:56.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:55 smithi203 ceph-mon[36579]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-06-06T08:31:56.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:55 smithi176 ceph-mon[36786]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-06-06T08:31:56.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:55 smithi027 ceph-mon[32335]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-06-06T08:31:58.220 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:57 smithi176 ceph-mon[36786]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-06T08:31:58.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:57 smithi203 ceph-mon[36579]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-06T08:31:58.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:57 smithi027 ceph-mon[32335]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-06T08:32:00.116 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:31:59 smithi176 ceph-mon[36786]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:32:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:31:59 smithi203 ceph-mon[36579]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:32:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:31:59 smithi027 ceph-mon[32335]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:32:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T08:32:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:01.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T08:32:01.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:01.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T08:32:01.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:02.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:01 smithi176 ceph-mon[36786]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-06T08:32:02.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:01 smithi176 ceph-mon[36786]: Deploying daemon osd.4 on smithi176 2024-06-06T08:32:02.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:01 smithi203 ceph-mon[36579]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-06T08:32:02.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:01 smithi203 ceph-mon[36579]: Deploying daemon osd.4 on smithi176 2024-06-06T08:32:02.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:01 smithi027 ceph-mon[32335]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-06T08:32:02.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:01 smithi027 ceph-mon[32335]: Deploying daemon osd.4 on smithi176 2024-06-06T08:32:04.226 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:03 smithi176 ceph-mon[36786]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:32:04.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:03 smithi203 ceph-mon[36579]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:32:04.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:03 smithi027 ceph-mon[32335]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:32:06.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:05 smithi203 ceph-mon[36579]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:06.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:05 smithi176 ceph-mon[36786]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:06.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:05 smithi027 ceph-mon[32335]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:07.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:07 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:07.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:07 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:07.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:07 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:07 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:07 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:07 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:07.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:07 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:07.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:07 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:07.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:07 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:08.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:08 smithi176 ceph-mon[36786]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:08.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:08 smithi027 ceph-mon[32335]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:08.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:08 smithi203 ceph-mon[36579]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:09.709 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 4 on host 'smithi176' 2024-06-06T08:32:09.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.709+0000 7fcb197fa700 1 -- 172.21.15.176:0/619478520 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcb1c06b450 con 0x7fcb040a2c20 2024-06-06T08:32:09.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcb040a2c20 msgr2=0x7fcb040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:09.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcb040a2c20 0x7fcb040a50e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c102470 tx=0x7fcb0c00d620 comp rx=0 tx=0).stop 2024-06-06T08:32:09.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102610 msgr2=0x7fcb1c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:09.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102610 0x7fcb1c197aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d950 tx=0x7fcb1000dc60 comp rx=0 tx=0).stop 2024-06-06T08:32:09.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 shutdown_connections 2024-06-06T08:32:09.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcb040a2c20 0x7fcb040a50e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:09.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb1c102fb0 0x7fcb1c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:09.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb1c101410 0x7fcb1c197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:09.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 --2- 172.21.15.176:0/619478520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcb1c102610 0x7fcb1c197aa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:09.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 >> 172.21.15.176:0/619478520 conn(0x7fcb1c0fc9f0 msgr2=0x7fcb1c105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:32:09.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 shutdown_connections 2024-06-06T08:32:09.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:32:09.712+0000 7fcb2232c700 1 -- 172.21.15.176:0/619478520 wait complete. 2024-06-06T08:32:09.893 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:09 smithi176 ceph-mon[36786]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:09.893 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:09.893 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:09.893 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:09.893 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:09.893 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:09.893 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:09.893 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:09.894 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:10.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:09 smithi027 ceph-mon[32335]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:10.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:10.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:10.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:09 smithi203 ceph-mon[36579]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:10.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:10.415 DEBUG:teuthology.orchestra.run.smithi176:osd.4> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.4.service 2024-06-06T08:32:10.417 INFO:tasks.cephadm:Deploying osd.5 on smithi203 with /dev/vg_nvme/lv_3... 2024-06-06T08:32:10.417 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T08:32:10.662 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:32:10.976 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:10 smithi176 ceph-mon[36786]: from='osd.4 [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T08:32:10.976 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:10 smithi176 ceph-mon[36786]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T08:32:11.064 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:10 smithi203 ceph-mon[36579]: from='osd.4 [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T08:32:11.064 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:10 smithi203 ceph-mon[36579]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T08:32:11.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:10 smithi027 ceph-mon[32335]: from='osd.4 [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T08:32:11.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:10 smithi027 ceph-mon[32335]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T08:32:11.902 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:11 smithi176 ceph-mon[36786]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:11.902 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:11 smithi176 ceph-mon[36786]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T08:32:11.902 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:11 smithi176 ceph-mon[36786]: from='osd.4 [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:32:11.902 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:11 smithi176 ceph-mon[36786]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T08:32:11.902 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:11.902 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:11 smithi176 ceph-mon[36786]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:32:11.902 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:32:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4[48286]: 2024-06-06T08:32:11.788+0000 7fbb64f3d700 -1 osd.4 0 waiting for initial osdmap 2024-06-06T08:32:11.903 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:32:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4[48286]: 2024-06-06T08:32:11.795+0000 7fbb60d30700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:32:11.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:11 smithi203 ceph-mon[36579]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:11.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:11 smithi203 ceph-mon[36579]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T08:32:11.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:11 smithi203 ceph-mon[36579]: from='osd.4 [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:32:11.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:11 smithi203 ceph-mon[36579]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T08:32:11.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:11.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:11 smithi203 ceph-mon[36579]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:32:12.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:11 smithi027 ceph-mon[32335]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:12.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:11 smithi027 ceph-mon[32335]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T08:32:12.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:11 smithi027 ceph-mon[32335]: from='osd.4 [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:32:12.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:11 smithi027 ceph-mon[32335]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T08:32:12.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:12.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:11 smithi027 ceph-mon[32335]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:32:12.574 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T08:32:12.575 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T08:32:12.575 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-06-06T08:32:12.575 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-06-06T08:32:12.575 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122983 s, 853 MB/s 2024-06-06T08:32:12.575 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-06-06T08:32:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:12 smithi176 ceph-mon[36786]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:32:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:12 smithi176 ceph-mon[36786]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T08:32:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:13.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:12 smithi203 ceph-mon[36579]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:32:13.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:12 smithi203 ceph-mon[36579]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T08:32:13.166 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:13.167 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:13.194 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_3 2024-06-06T08:32:13.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:12 smithi027 ceph-mon[32335]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:32:13.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:12 smithi027 ceph-mon[32335]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T08:32:13.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:13.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:13.432 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:32:13.853 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:32:13.853 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:32:13.853 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:13.853 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:13.853 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: osd.4 [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] boot 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: osdmap e33: 5 total, 5 up, 5 in 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:13.854 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:14.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: osd.4 [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] boot 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: osdmap e33: 5 total, 5 up, 5 in 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:14.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:14.208 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:14.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:32:14.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:32:14.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-06-06T08:32:14.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:14.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: osd.4 [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] boot 2024-06-06T08:32:14.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: osdmap e33: 5 total, 5 up, 5 in 2024-06-06T08:32:14.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:32:14.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:14.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:14.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:14.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:14.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:14.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:14.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:14.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:14.909 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:14 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi176 2024-06-06T08:32:14.909 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:14 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi176 to 8544M 2024-06-06T08:32:14.909 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:14 smithi203 ceph-mon[36579]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T08:32:14.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.913+0000 7f214b59e700 1 -- 172.21.15.203:0/2984157153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c102510 msgr2=0x7f214c102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:14.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.913+0000 7f214b59e700 1 --2- 172.21.15.203:0/2984157153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c102510 0x7f214c102970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f213c009990 tx=0x7f213c009ca0 comp rx=0 tx=0).stop 2024-06-06T08:32:14.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.913+0000 7f214b59e700 1 -- 172.21.15.203:0/2984157153 shutdown_connections 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.913+0000 7f214b59e700 1 --2- 172.21.15.203:0/2984157153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f214c102eb0 0x7f214c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.913+0000 7f214b59e700 1 --2- 172.21.15.203:0/2984157153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f214c101310 0x7f214c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.913+0000 7f214b59e700 1 --2- 172.21.15.203:0/2984157153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c102510 0x7f214c102970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.913+0000 7f214b59e700 1 -- 172.21.15.203:0/2984157153 >> 172.21.15.203:0/2984157153 conn(0x7f214c0fc8b0 msgr2=0x7f214c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.913+0000 7f214b59e700 1 -- 172.21.15.203:0/2984157153 shutdown_connections 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.913+0000 7f214b59e700 1 -- 172.21.15.203:0/2984157153 wait complete. 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214b59e700 1 Processor -- start 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214b59e700 1 -- start start 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214b59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c101310 0x7f214c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f214c102510 0x7f214c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214b59e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f214c102eb0 0x7f214c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:32:14.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214b59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f214c10cff0 con 0x7f214c102510 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214b59e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f214c10ce70 con 0x7f214c102eb0 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214b59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f214c10d170 con 0x7f214c101310 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c101310 0x7f214c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c101310 0x7f214c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:58738/0 (socket says 172.21.15.203:58738) 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f214a59c700 1 -- 172.21.15.203:0/2160184305 learned_addr learned my addr 172.21.15.203:0/2160184305 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.914+0000 7f2149d9b700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f214c102510 0x7f214c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f214ad9d700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f214c102eb0 0x7f214c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f214a59c700 1 -- 172.21.15.203:0/2160184305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f214c102eb0 msgr2=0x7f214c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f214a59c700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f214c102eb0 0x7f214c19fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f214a59c700 1 -- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f214c102510 msgr2=0x7f214c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f214a59c700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f214c102510 0x7f214c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f214a59c700 1 -- 172.21.15.203:0/2160184305 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c1a03d0 con 0x7f214c101310 2024-06-06T08:32:14.917 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f2149d9b700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f214c102510 0x7f214c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:32:14.918 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f214a59c700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c101310 0x7f214c198830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f214000b530 tx=0x7f214000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:32:14.918 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f213b7fe700 1 -- 172.21.15.203:0/2160184305 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140015020 con 0x7f214c101310 2024-06-06T08:32:14.918 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f213b7fe700 1 -- 172.21.15.203:0/2160184305 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2140011d10 con 0x7f214c101310 2024-06-06T08:32:14.918 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f213b7fe700 1 -- 172.21.15.203:0/2160184305 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214000f2e0 con 0x7f214c101310 2024-06-06T08:32:14.918 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c1a06c0 con 0x7f214c101310 2024-06-06T08:32:14.918 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.915+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c1a0cd0 con 0x7f214c101310 2024-06-06T08:32:14.919 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.916+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214c069180 con 0x7f214c101310 2024-06-06T08:32:14.919 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.917+0000 7f213b7fe700 1 -- 172.21.15.203:0/2160184305 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2140018020 con 0x7f214c101310 2024-06-06T08:32:14.919 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.918+0000 7f213b7fe700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21340a2ca0 0x7f21340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:32:14.919 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.918+0000 7f213b7fe700 1 -- 172.21.15.203:0/2160184305 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f21400cd710 con 0x7f214c101310 2024-06-06T08:32:14.919 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.918+0000 7f2149d9b700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21340a2ca0 0x7f21340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:32:14.919 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.919+0000 7f2149d9b700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21340a2ca0 0x7f21340a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f213c00b5c0 tx=0x7f213c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:32:14.922 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:14.923+0000 7f213b7fe700 1 -- 172.21.15.203:0/2160184305 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f214001f020 con 0x7f214c101310 2024-06-06T08:32:15.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:15.126+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f214c1a0f90 con 0x7f21340a2ca0 2024-06-06T08:32:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:14 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi176 2024-06-06T08:32:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:14 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi176 to 8544M 2024-06-06T08:32:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:14 smithi027 ceph-mon[32335]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T08:32:15.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:14 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi176 2024-06-06T08:32:15.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:14 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi176 to 8544M 2024-06-06T08:32:15.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:14 smithi176 ceph-mon[36786]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T08:32:15.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:15 smithi203 ceph-mon[36579]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:15.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:15 smithi203 ceph-mon[36579]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:32:15.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:32:15.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:32:15.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:16.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:15 smithi027 ceph-mon[32335]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:16.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:15 smithi027 ceph-mon[32335]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:32:16.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:32:16.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:32:16.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:16.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:15 smithi176 ceph-mon[36786]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:16.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:15 smithi176 ceph-mon[36786]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:32:16.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:32:16.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:32:16.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:18.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:17 smithi203 ceph-mon[36579]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:18.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:17 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/3158955286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a98cab95-b7f2-405a-ba80-8814c9a60ad9"}]: dispatch 2024-06-06T08:32:18.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:17 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a98cab95-b7f2-405a-ba80-8814c9a60ad9"}]: dispatch 2024-06-06T08:32:18.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:17 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a98cab95-b7f2-405a-ba80-8814c9a60ad9"}]': finished 2024-06-06T08:32:18.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:17 smithi203 ceph-mon[36579]: osdmap e35: 6 total, 5 up, 6 in 2024-06-06T08:32:18.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:17 smithi027 ceph-mon[32335]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:17 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/3158955286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a98cab95-b7f2-405a-ba80-8814c9a60ad9"}]: dispatch 2024-06-06T08:32:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:17 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a98cab95-b7f2-405a-ba80-8814c9a60ad9"}]: dispatch 2024-06-06T08:32:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:17 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a98cab95-b7f2-405a-ba80-8814c9a60ad9"}]': finished 2024-06-06T08:32:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:17 smithi027 ceph-mon[32335]: osdmap e35: 6 total, 5 up, 6 in 2024-06-06T08:32:18.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:17 smithi176 ceph-mon[36786]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:17 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/3158955286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a98cab95-b7f2-405a-ba80-8814c9a60ad9"}]: dispatch 2024-06-06T08:32:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:17 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a98cab95-b7f2-405a-ba80-8814c9a60ad9"}]: dispatch 2024-06-06T08:32:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:17 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a98cab95-b7f2-405a-ba80-8814c9a60ad9"}]': finished 2024-06-06T08:32:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:17 smithi176 ceph-mon[36786]: osdmap e35: 6 total, 5 up, 6 in 2024-06-06T08:32:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:19.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:18 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/100289656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:32:19.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:18 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/100289656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:32:19.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:18 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/100289656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:32:20.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:19 smithi176 ceph-mon[36786]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:20.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:19 smithi027 ceph-mon[32335]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:20.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:19 smithi203 ceph-mon[36579]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:22.169 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:21 smithi203 ceph-mon[36579]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:22.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:21 smithi027 ceph-mon[32335]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:22.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:21 smithi176 ceph-mon[36786]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:24.069 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:23 smithi203 ceph-mon[36579]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:24.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:23 smithi027 ceph-mon[32335]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:24.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:23 smithi176 ceph-mon[36786]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:25.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:25 smithi203 ceph-mon[36579]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:25.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T08:32:25.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:25.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:25 smithi203 ceph-mon[36579]: Deploying daemon osd.5 on smithi203 2024-06-06T08:32:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:25 smithi027 ceph-mon[32335]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T08:32:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:25 smithi027 ceph-mon[32335]: Deploying daemon osd.5 on smithi203 2024-06-06T08:32:26.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:25 smithi176 ceph-mon[36786]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:26.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T08:32:26.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:26.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:25 smithi176 ceph-mon[36786]: Deploying daemon osd.5 on smithi203 2024-06-06T08:32:28.122 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:27 smithi203 ceph-mon[36579]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:27 smithi027 ceph-mon[32335]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:28.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:27 smithi176 ceph-mon[36786]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:29.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:29 smithi203 ceph-mon[36579]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:30.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:29 smithi027 ceph-mon[32335]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:30.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:29 smithi176 ceph-mon[36786]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:31.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:31 smithi027 ceph-mon[32335]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:31.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:31.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:31.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:31.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:31 smithi203 ceph-mon[36579]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:31.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:31.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:31.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:32.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:31 smithi176 ceph-mon[36786]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:32.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:32.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:32.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:33.915 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:33 smithi203 ceph-mon[36579]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:33.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:33 smithi027 ceph-mon[32335]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:33.995 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 5 on host 'smithi203' 2024-06-06T08:32:33.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.995+0000 7f213b7fe700 1 -- 172.21.15.203:0/2160184305 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f214c1a0f90 con 0x7f21340a2ca0 2024-06-06T08:32:33.998 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21340a2ca0 msgr2=0x7f21340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:33.998 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21340a2ca0 0x7f21340a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f213c00b5c0 tx=0x7f213c005fb0 comp rx=0 tx=0).stop 2024-06-06T08:32:33.998 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c101310 msgr2=0x7f214c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:33.998 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c101310 0x7f214c198830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f214000b530 tx=0x7f214000b8f0 comp rx=0 tx=0).stop 2024-06-06T08:32:33.998 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 shutdown_connections 2024-06-06T08:32:33.998 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21340a2ca0 0x7f21340a5160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:33.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f214c102eb0 0x7f214c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:33.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f214c101310 0x7f214c198830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:33.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 --2- 172.21.15.203:0/2160184305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f214c102510 0x7f214c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:33.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 >> 172.21.15.203:0/2160184305 conn(0x7f214c0fc8b0 msgr2=0x7f214c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:32:33.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 shutdown_connections 2024-06-06T08:32:33.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:32:33.999+0000 7f214b59e700 1 -- 172.21.15.203:0/2160184305 wait complete. 2024-06-06T08:32:34.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:33 smithi176 ceph-mon[36786]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:34.645 DEBUG:teuthology.orchestra.run.smithi203:osd.5> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.5.service 2024-06-06T08:32:34.647 INFO:tasks.cephadm:Deploying osd.6 on smithi027 with /dev/vg_nvme/lv_2... 2024-06-06T08:32:34.647 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T08:32:34.905 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:32:35.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:34 smithi027 ceph-mon[32335]: from='osd.5 [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T08:32:35.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:34 smithi027 ceph-mon[32335]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T08:32:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:34 smithi203 ceph-mon[36579]: from='osd.5 [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T08:32:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:34 smithi203 ceph-mon[36579]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T08:32:35.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.259 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.259 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.259 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:35.259 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:34 smithi176 ceph-mon[36786]: from='osd.5 [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T08:32:35.259 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:34 smithi176 ceph-mon[36786]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T08:32:36.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:36 smithi027 ceph-mon[32335]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:36.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:36 smithi027 ceph-mon[32335]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T08:32:36.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:36 smithi027 ceph-mon[32335]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T08:32:36.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:36 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:36.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:36 smithi027 ceph-mon[32335]: from='osd.5 [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:32:36.326 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:36 smithi027 ceph-mon[32335]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:32:36.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:36 smithi203 ceph-mon[36579]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:36.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:36 smithi203 ceph-mon[36579]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T08:32:36.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:36 smithi203 ceph-mon[36579]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T08:32:36.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:36 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:36.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:36 smithi203 ceph-mon[36579]: from='osd.5 [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:32:36.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:36 smithi203 ceph-mon[36579]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:32:36.491 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:32:36 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:32:36.115+0000 7f1a63119700 -1 osd.5 0 waiting for initial osdmap 2024-06-06T08:32:36.491 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:32:36 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:32:36.126+0000 7f1a5ef0c700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:32:36.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:36 smithi176 ceph-mon[36786]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:36.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:36 smithi176 ceph-mon[36786]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T08:32:36.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:36 smithi176 ceph-mon[36786]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T08:32:36.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:36 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:36.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:36 smithi176 ceph-mon[36786]: from='osd.5 [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:32:36.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:36 smithi176 ceph-mon[36786]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:32:37.099 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T08:32:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T08:32:37.100 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-06-06T08:32:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-06-06T08:32:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0138764 s, 756 MB/s 2024-06-06T08:32:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-06-06T08:32:37.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:37 smithi203 ceph-mon[36579]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:32:37.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:37 smithi203 ceph-mon[36579]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T08:32:37.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:37 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:37.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:37 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:37.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:37 smithi203 ceph-mon[36579]: osd.5 [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] boot 2024-06-06T08:32:37.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:37 smithi203 ceph-mon[36579]: osdmap e38: 6 total, 6 up, 6 in 2024-06-06T08:32:37.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:37 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:37.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:37 smithi027 ceph-mon[32335]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:32:37.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:37 smithi027 ceph-mon[32335]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T08:32:37.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:37 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:37.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:37 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:37.414 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:37 smithi027 ceph-mon[32335]: osd.5 [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] boot 2024-06-06T08:32:37.415 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:37 smithi027 ceph-mon[32335]: osdmap e38: 6 total, 6 up, 6 in 2024-06-06T08:32:37.415 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:37 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:37.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:37 smithi176 ceph-mon[36786]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:32:37.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:37 smithi176 ceph-mon[36786]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T08:32:37.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:37 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:37.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:37 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:37.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:37 smithi176 ceph-mon[36786]: osd.5 [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] boot 2024-06-06T08:32:37.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:37 smithi176 ceph-mon[36786]: osdmap e38: 6 total, 6 up, 6 in 2024-06-06T08:32:37.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:37 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:32:37.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2024-06-06T08:32:38.085 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:32:38.344 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:32:38.344 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:32:38.344 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.345 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:38 smithi203 ceph-mon[36579]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T08:32:38.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:32:38.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:38 smithi027 ceph-mon[32335]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:38.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:38.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:38.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:32:38.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:38.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:38 smithi176 ceph-mon[36786]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T08:32:39.321 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:39 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi203 2024-06-06T08:32:39.322 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:39 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi203 to 8544M 2024-06-06T08:32:39.322 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:39 smithi027 ceph-mon[32335]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T08:32:39.408 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:39 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi203 2024-06-06T08:32:39.408 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:39 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi203 to 8544M 2024-06-06T08:32:39.408 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:39 smithi176 ceph-mon[36786]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T08:32:39.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:39 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi203 2024-06-06T08:32:39.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:39 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi203 to 8544M 2024-06-06T08:32:39.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:39 smithi203 ceph-mon[36579]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T08:32:39.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 -- 172.21.15.27:0/1904089204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 msgr2=0x7f275c109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:39.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 --2- 172.21.15.27:0/1904089204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 0x7f275c109860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f275400b0d0 tx=0x7f275400b3e0 comp rx=0 tx=0).stop 2024-06-06T08:32:39.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 -- 172.21.15.27:0/1904089204 shutdown_connections 2024-06-06T08:32:39.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 --2- 172.21.15.27:0/1904089204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 0x7f275c109860 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:39.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 --2- 172.21.15.27:0/1904089204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c101410 0x7f275c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:39.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 --2- 172.21.15.27:0/1904089204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f275c102610 0x7f275c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:39.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 -- 172.21.15.27:0/1904089204 >> 172.21.15.27:0/1904089204 conn(0x7f275c0fc9f0 msgr2=0x7f275c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:32:39.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 -- 172.21.15.27:0/1904089204 shutdown_connections 2024-06-06T08:32:39.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 -- 172.21.15.27:0/1904089204 wait complete. 2024-06-06T08:32:39.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 Processor -- start 2024-06-06T08:32:39.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.922+0000 7f276402a700 1 -- start start 2024-06-06T08:32:39.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f276402a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c101410 0x7f275c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:32:39.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f276402a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f275c102610 0x7f275c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:32:39.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f276402a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 0x7f275c19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f276402a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f275c10d050 con 0x7f275c102fb0 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f276402a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f275c10ced0 con 0x7f275c102610 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f276402a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f275c10d1d0 con 0x7f275c101410 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27625c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 0x7f275c19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27625c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 0x7f275c19fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44108/0 (socket says 172.21.15.27:44108) 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27625c7700 1 -- 172.21.15.27:0/4075988049 learned_addr learned my addr 172.21.15.27:0/4075988049 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f2761dc6700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c101410 0x7f275c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27615c5700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f275c102610 0x7f275c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27625c7700 1 -- 172.21.15.27:0/4075988049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c101410 msgr2=0x7f275c06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27625c7700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c101410 0x7f275c06b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27625c7700 1 -- 172.21.15.27:0/4075988049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f275c102610 msgr2=0x7f275c19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:32:39.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27625c7700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f275c102610 0x7f275c19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:32:39.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27625c7700 1 -- 172.21.15.27:0/4075988049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c1a0320 con 0x7f275c102fb0 2024-06-06T08:32:39.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f27625c7700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 0x7f275c19fa50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2754007fa0 tx=0x7f2754007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:32:39.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f2752ffd700 1 -- 172.21.15.27:0/4075988049 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275401f020 con 0x7f275c102fb0 2024-06-06T08:32:39.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275c1a05b0 con 0x7f275c102fb0 2024-06-06T08:32:39.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f2752ffd700 1 -- 172.21.15.27:0/4075988049 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2754003e10 con 0x7f275c102fb0 2024-06-06T08:32:39.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.923+0000 7f2752ffd700 1 -- 172.21.15.27:0/4075988049 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275401b6a0 con 0x7f275c102fb0 2024-06-06T08:32:39.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.924+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275c1a0a90 con 0x7f275c102fb0 2024-06-06T08:32:39.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.925+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f275c069180 con 0x7f275c102fb0 2024-06-06T08:32:39.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.926+0000 7f2752ffd700 1 -- 172.21.15.27:0/4075988049 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2754022020 con 0x7f275c102fb0 2024-06-06T08:32:39.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.926+0000 7f2752ffd700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f27480a2c20 0x7f27480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:32:39.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.927+0000 7f2761dc6700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f27480a2c20 0x7f27480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:32:39.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.927+0000 7f2752ffd700 1 -- 172.21.15.27:0/4075988049 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f275400f020 con 0x7f275c102fb0 2024-06-06T08:32:39.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.927+0000 7f2761dc6700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f27480a2c20 0x7f27480a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2758005950 tx=0x7f275800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:32:39.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:39.931+0000 7f2752ffd700 1 -- 172.21.15.27:0/4075988049 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f275409cec0 con 0x7f275c102fb0 2024-06-06T08:32:40.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:32:40.154+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f275c0628d0 con 0x7f27480a2c20 2024-06-06T08:32:40.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:40 smithi027 ceph-mon[32335]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:40.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:40 smithi203 ceph-mon[36579]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:40.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:40 smithi176 ceph-mon[36786]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:41.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:41 smithi027 ceph-mon[32335]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T08:32:41.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:41 smithi027 ceph-mon[32335]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:32:41.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:41 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:32:41.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:41 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:32:41.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:41 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:41.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:41 smithi203 ceph-mon[36579]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T08:32:41.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:41 smithi203 ceph-mon[36579]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:32:41.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:41 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:32:41.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:41 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:32:41.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:41 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:41.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:41 smithi176 ceph-mon[36786]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T08:32:41.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:41 smithi176 ceph-mon[36786]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:32:41.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:41 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:32:41.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:41 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:32:41.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:41 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:42 smithi027 ceph-mon[32335]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:42 smithi027 ceph-mon[32335]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T08:32:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:42 smithi203 ceph-mon[36579]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:42 smithi203 ceph-mon[36579]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T08:32:42.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:42 smithi176 ceph-mon[36786]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:42.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:42 smithi176 ceph-mon[36786]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T08:32:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:43 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1028313584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4261660-83ed-4dbc-9789-405ffbb4389a"}]: dispatch 2024-06-06T08:32:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:43 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1028313584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4261660-83ed-4dbc-9789-405ffbb4389a"}]': finished 2024-06-06T08:32:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:43 smithi027 ceph-mon[32335]: osdmap e43: 7 total, 6 up, 7 in 2024-06-06T08:32:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:43 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:32:43.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:43 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1028313584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4261660-83ed-4dbc-9789-405ffbb4389a"}]: dispatch 2024-06-06T08:32:43.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:43 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1028313584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4261660-83ed-4dbc-9789-405ffbb4389a"}]': finished 2024-06-06T08:32:43.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:43 smithi203 ceph-mon[36579]: osdmap e43: 7 total, 6 up, 7 in 2024-06-06T08:32:43.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:32:43.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:43 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1028313584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4261660-83ed-4dbc-9789-405ffbb4389a"}]: dispatch 2024-06-06T08:32:43.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:43 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1028313584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4261660-83ed-4dbc-9789-405ffbb4389a"}]': finished 2024-06-06T08:32:43.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:43 smithi176 ceph-mon[36786]: osdmap e43: 7 total, 6 up, 7 in 2024-06-06T08:32:43.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:43 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:32:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:44 smithi027 ceph-mon[32335]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:44 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/342977012' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:32:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:44 smithi203 ceph-mon[36579]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:44 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/342977012' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:32:44.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:44 smithi176 ceph-mon[36786]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:44.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:44 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/342977012' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:32:46.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:46 smithi027 ceph-mon[32335]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-06-06T08:32:46.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:46 smithi203 ceph-mon[36579]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-06-06T08:32:46.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:46 smithi176 ceph-mon[36786]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-06-06T08:32:48.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:48 smithi027 ceph-mon[32335]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-06-06T08:32:48.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:48 smithi203 ceph-mon[36579]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-06-06T08:32:48.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:48 smithi176 ceph-mon[36786]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-06-06T08:32:50.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:50 smithi027 ceph-mon[32335]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:32:50.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:50 smithi203 ceph-mon[36579]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:32:50.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:50 smithi176 ceph-mon[36786]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:32:51.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T08:32:51.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:51.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T08:32:51.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:51.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T08:32:51.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:32:52.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:52 smithi027 ceph-mon[32335]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-06T08:32:52.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:52 smithi027 ceph-mon[32335]: Deploying daemon osd.6 on smithi027 2024-06-06T08:32:52.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:52 smithi203 ceph-mon[36579]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-06T08:32:52.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:52 smithi203 ceph-mon[36579]: Deploying daemon osd.6 on smithi027 2024-06-06T08:32:52.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:52 smithi176 ceph-mon[36786]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-06T08:32:52.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:52 smithi176 ceph-mon[36786]: Deploying daemon osd.6 on smithi027 2024-06-06T08:32:53.258 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:53 smithi027 ceph-mon[32335]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:32:53.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:53 smithi203 ceph-mon[36579]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:32:53.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:53 smithi176 ceph-mon[36786]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:32:55.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:55 smithi027 ceph-mon[32335]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-06-06T08:32:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:55 smithi203 ceph-mon[36579]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-06-06T08:32:55.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:55 smithi176 ceph-mon[36786]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-06-06T08:32:58.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:57 smithi176 ceph-mon[36786]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:58.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:58.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:58.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:57 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:58.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:57 smithi027 ceph-mon[32335]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:58.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:58.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:58.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:57 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:32:58.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:57 smithi203 ceph-mon[36579]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:32:58.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:58.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:32:58.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:57 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:00.061 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:59 smithi027 ceph-mon[32335]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:33:00.061 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:00.061 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:00.061 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:00.061 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:00.061 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:32:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:59 smithi176 ceph-mon[36786]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:33:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:32:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:59 smithi203 ceph-mon[36579]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:33:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:32:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:01.125 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:01 smithi027 ceph-mon[32335]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T08:33:01.211 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 6 on host 'smithi027' 2024-06-06T08:33:01.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.211+0000 7f2752ffd700 1 -- 172.21.15.27:0/4075988049 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f275c0628d0 con 0x7f27480a2c20 2024-06-06T08:33:01.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f27480a2c20 msgr2=0x7f27480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f27480a2c20 0x7f27480a50e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2758005950 tx=0x7f275800a300 comp rx=0 tx=0).stop 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 msgr2=0x7f275c19fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 0x7f275c19fa50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2754007fa0 tx=0x7f2754007d10 comp rx=0 tx=0).stop 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 shutdown_connections 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f27480a2c20 0x7f27480a50e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f275c102fb0 0x7f275c19fa50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c101410 0x7f275c06b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 --2- 172.21.15.27:0/4075988049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f275c102610 0x7f275c19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 >> 172.21.15.27:0/4075988049 conn(0x7f275c0fc9f0 msgr2=0x7f275c105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 shutdown_connections 2024-06-06T08:33:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:01.214+0000 7f276402a700 1 -- 172.21.15.27:0/4075988049 wait complete. 2024-06-06T08:33:01.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:01 smithi176 ceph-mon[36786]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T08:33:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:01 smithi203 ceph-mon[36579]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T08:33:01.958 DEBUG:teuthology.orchestra.run.smithi027:osd.6> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.6.service 2024-06-06T08:33:01.960 INFO:tasks.cephadm:Deploying osd.7 on smithi176 with /dev/vg_nvme/lv_2... 2024-06-06T08:33:01.960 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T08:33:02.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-mon[32335]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:33:02.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-mon[32335]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T08:33:02.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-mon[32335]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T08:33:02.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-mon[32335]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:33:02.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:02.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:02.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:02.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:02.207 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:33:02.012+0000 7f5f149dd700 -1 osd.6 0 waiting for initial osdmap 2024-06-06T08:33:02.207 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:33:02 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:33:02.021+0000 7f5f107d0700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:33:02.207 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:33:02.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:02 smithi176 ceph-mon[36786]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:33:02.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:02 smithi176 ceph-mon[36786]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T08:33:02.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:02 smithi176 ceph-mon[36786]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T08:33:02.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:02 smithi176 ceph-mon[36786]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:33:02.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:02.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:02.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:02.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:02 smithi203 ceph-mon[36579]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:33:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:02 smithi203 ceph-mon[36579]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T08:33:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:02 smithi203 ceph-mon[36579]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T08:33:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:02 smithi203 ceph-mon[36579]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:33:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:02.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:02.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:03.290 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:03 smithi027 ceph-mon[32335]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:33:03.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:03 smithi027 ceph-mon[32335]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T08:33:03.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:03.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:03.414 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:03 smithi176 ceph-mon[36786]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:33:03.414 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:03 smithi176 ceph-mon[36786]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T08:33:03.414 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:03.414 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:03.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:03 smithi203 ceph-mon[36579]: from='osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:33:03.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:03 smithi203 ceph-mon[36579]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T08:33:03.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:03.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:04 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:33:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:04 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:33:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:04 smithi176 ceph-mon[36786]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:33:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:04 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:04 smithi176 ceph-mon[36786]: osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] boot 2024-06-06T08:33:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:04 smithi176 ceph-mon[36786]: osdmap e46: 7 total, 7 up, 7 in 2024-06-06T08:33:04.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:04 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:04.391 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T08:33:04.392 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T08:33:04.392 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-06-06T08:33:04.392 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-06-06T08:33:04.392 INFO:teuthology.orchestra.run.smithi176.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127255 s, 824 MB/s 2024-06-06T08:33:04.392 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-06-06T08:33:04.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:04 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:33:04.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:04 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:33:04.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:04 smithi027 ceph-mon[32335]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:33:04.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:04 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:04.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:04 smithi027 ceph-mon[32335]: osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] boot 2024-06-06T08:33:04.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:04 smithi027 ceph-mon[32335]: osdmap e46: 7 total, 7 up, 7 in 2024-06-06T08:33:04.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:04 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:04 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:33:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:04 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:33:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:04 smithi203 ceph-mon[36579]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-06-06T08:33:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:04 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:04 smithi203 ceph-mon[36579]: osd.6 [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] boot 2024-06-06T08:33:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:04 smithi203 ceph-mon[36579]: osdmap e46: 7 total, 7 up, 7 in 2024-06-06T08:33:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:04 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:33:05.055 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_2 2024-06-06T08:33:05.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:05 smithi027 ceph-mon[32335]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T08:33:05.295 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:33:05.417 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:05 smithi176 ceph-mon[36786]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T08:33:05.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:05 smithi203 ceph-mon[36579]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T08:33:06.432 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:06.432 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.432 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.433 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:06.433 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:06.433 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:06.433 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.433 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:06.433 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:06.433 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:06.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:06.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:06.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:06.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:06.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:06.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:06.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:06.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:06.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 -- 172.21.15.176:0/2005254141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 msgr2=0x7f9a20105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:06.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 --2- 172.21.15.176:0/2005254141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 0x7f9a20105360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b0a0 tx=0x7f9a1000b3b0 comp rx=0 tx=0).stop 2024-06-06T08:33:06.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 -- 172.21.15.176:0/2005254141 shutdown_connections 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 --2- 172.21.15.176:0/2005254141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a201081d0 0x7f9a2010a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 --2- 172.21.15.176:0/2005254141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 0x7f9a20105360 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 --2- 172.21.15.176:0/2005254141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a201058a0 0x7f9a20107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 -- 172.21.15.176:0/2005254141 >> 172.21.15.176:0/2005254141 conn(0x7f9a200fc980 msgr2=0x7f9a200fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 -- 172.21.15.176:0/2005254141 shutdown_connections 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 -- 172.21.15.176:0/2005254141 wait complete. 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.941+0000 7f9a253f8700 1 Processor -- start 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a253f8700 1 -- start start 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a253f8700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 0x7f9a201987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a253f8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a201058a0 0x7f9a20198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a253f8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a201081d0 0x7f9a2019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:06.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a253f8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2010e180 con 0x7f9a201081d0 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a253f8700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2010e000 con 0x7f9a20102f70 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a253f8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2010e300 con 0x7f9a201058a0 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 0x7f9a201987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 0x7f9a201987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:33396/0 (socket says 172.21.15.176:33396) 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1effd700 1 -- 172.21.15.176:0/1679372904 learned_addr learned my addr 172.21.15.176:0/1679372904 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1e7fc700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a201058a0 0x7f9a20198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1f7fe700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a201081d0 0x7f9a2019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1effd700 1 -- 172.21.15.176:0/1679372904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a201058a0 msgr2=0x7f9a20198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1effd700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a201058a0 0x7f9a20198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1effd700 1 -- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a201081d0 msgr2=0x7f9a2019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:06.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1effd700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a201081d0 0x7f9a2019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1effd700 1 -- 172.21.15.176:0/1679372904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a201a0400 con 0x7f9a20102f70 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1e7fc700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a201058a0 0x7f9a20198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1f7fe700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a201081d0 0x7f9a2019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.942+0000 7f9a1effd700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 0x7f9a201987a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a100062a0 tx=0x7f9a10004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.943+0000 7f9a17fff700 1 -- 172.21.15.176:0/1679372904 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10021070 con 0x7f9a20102f70 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.943+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a201a0690 con 0x7f9a20102f70 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.943+0000 7f9a17fff700 1 -- 172.21.15.176:0/1679372904 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1001d930 con 0x7f9a20102f70 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.943+0000 7f9a17fff700 1 -- 172.21.15.176:0/1679372904 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a100055c0 con 0x7f9a20102f70 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.943+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201a0b90 con 0x7f9a20102f70 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.944+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a20069180 con 0x7f9a20102f70 2024-06-06T08:33:06.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.945+0000 7f9a17fff700 1 -- 172.21.15.176:0/1679372904 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9a10024070 con 0x7f9a20102f70 2024-06-06T08:33:06.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.946+0000 7f9a17fff700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a0c0a2c20 0x7f9a0c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:06.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.946+0000 7f9a17fff700 1 -- 172.21.15.176:0/1679372904 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f9a1000f020 con 0x7f9a20102f70 2024-06-06T08:33:06.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.946+0000 7f9a1e7fc700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a0c0a2c20 0x7f9a0c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:06.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.947+0000 7f9a1e7fc700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a0c0a2c20 0x7f9a0c0a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a08007900 tx=0x7f9a08008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:33:06.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:06.950+0000 7f9a17fff700 1 -- 172.21.15.176:0/1679372904 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9a1002b020 con 0x7f9a20102f70 2024-06-06T08:33:07.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:07.155+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9a200628d0 con 0x7f9a0c0a2c20 2024-06-06T08:33:07.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:07 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi027 2024-06-06T08:33:07.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:07 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi027 to 5696M 2024-06-06T08:33:07.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:07 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi027 2024-06-06T08:33:07.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:07 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi027 to 5696M 2024-06-06T08:33:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:07 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi027 2024-06-06T08:33:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:07 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi027 to 5696M 2024-06-06T08:33:08.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:08 smithi027 ceph-mon[32335]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:08.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:08 smithi027 ceph-mon[32335]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:33:08.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:33:08.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:33:08.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:08.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:08 smithi203 ceph-mon[36579]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:08.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:08 smithi203 ceph-mon[36579]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:33:08.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:33:08.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:33:08.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:08.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:08 smithi176 ceph-mon[36786]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:08.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:08 smithi176 ceph-mon[36786]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:33:08.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:33:08.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:33:08.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:10.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:10 smithi027 ceph-mon[32335]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:10.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:10 smithi027 ceph-mon[32335]: from='client.? 172.21.15.176:0/1396009511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8f52b13-40bd-443c-a8e1-d3ce46bb4397"}]: dispatch 2024-06-06T08:33:10.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:10 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8f52b13-40bd-443c-a8e1-d3ce46bb4397"}]: dispatch 2024-06-06T08:33:10.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:10 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8f52b13-40bd-443c-a8e1-d3ce46bb4397"}]': finished 2024-06-06T08:33:10.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:10 smithi027 ceph-mon[32335]: osdmap e48: 8 total, 7 up, 8 in 2024-06-06T08:33:10.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:10 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:10.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:10 smithi203 ceph-mon[36579]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:10.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:10 smithi203 ceph-mon[36579]: from='client.? 172.21.15.176:0/1396009511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8f52b13-40bd-443c-a8e1-d3ce46bb4397"}]: dispatch 2024-06-06T08:33:10.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:10 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8f52b13-40bd-443c-a8e1-d3ce46bb4397"}]: dispatch 2024-06-06T08:33:10.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:10 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8f52b13-40bd-443c-a8e1-d3ce46bb4397"}]': finished 2024-06-06T08:33:10.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:10 smithi203 ceph-mon[36579]: osdmap e48: 8 total, 7 up, 8 in 2024-06-06T08:33:10.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:10 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:10.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:10 smithi176 ceph-mon[36786]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:10.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:10 smithi176 ceph-mon[36786]: from='client.? 172.21.15.176:0/1396009511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8f52b13-40bd-443c-a8e1-d3ce46bb4397"}]: dispatch 2024-06-06T08:33:10.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:10 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8f52b13-40bd-443c-a8e1-d3ce46bb4397"}]: dispatch 2024-06-06T08:33:10.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:10 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8f52b13-40bd-443c-a8e1-d3ce46bb4397"}]': finished 2024-06-06T08:33:10.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:10 smithi176 ceph-mon[36786]: osdmap e48: 8 total, 7 up, 8 in 2024-06-06T08:33:10.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:10 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:11.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:11 smithi027 ceph-mon[32335]: from='client.? 172.21.15.176:0/955414963' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:33:11.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:11 smithi203 ceph-mon[36579]: from='client.? 172.21.15.176:0/955414963' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:33:11.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:11 smithi176 ceph-mon[36786]: from='client.? 172.21.15.176:0/955414963' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:33:12.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:12 smithi027 ceph-mon[32335]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:12.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:12 smithi203 ceph-mon[36579]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:12.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:12 smithi176 ceph-mon[36786]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:14.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:14 smithi027 ceph-mon[32335]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:14 smithi203 ceph-mon[36579]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:14 smithi176 ceph-mon[36786]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:16 smithi027 ceph-mon[32335]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:16 smithi203 ceph-mon[36579]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:16 smithi176 ceph-mon[36786]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:18.120 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:18 smithi176 ceph-mon[36786]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:18.120 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:18 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T08:33:18.120 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:18 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:18.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:18 smithi027 ceph-mon[32335]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:18.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:18 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T08:33:18.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:18 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:18 smithi203 ceph-mon[36579]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:18 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T08:33:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:18 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:19.183 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:19 smithi176 ceph-mon[36786]: Deploying daemon osd.7 on smithi176 2024-06-06T08:33:19.357 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:19 smithi027 ceph-mon[32335]: Deploying daemon osd.7 on smithi176 2024-06-06T08:33:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:19 smithi203 ceph-mon[36579]: Deploying daemon osd.7 on smithi176 2024-06-06T08:33:20.165 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:20 smithi176 ceph-mon[36786]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:20.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:20 smithi027 ceph-mon[32335]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:20.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:20 smithi203 ceph-mon[36579]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:22.199 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:22 smithi176 ceph-mon[36786]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:22.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:22 smithi027 ceph-mon[32335]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:22.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:22 smithi203 ceph-mon[36579]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:24.150 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:23 smithi176 ceph-mon[36786]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:24.150 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:23 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:24.151 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:23 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:24.151 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:23 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:23 smithi027 ceph-mon[32335]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:23 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:23 smithi203 ceph-mon[36579]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:23 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:23 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:23 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:25 smithi027 ceph-mon[32335]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:26.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:26.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:25 smithi203 ceph-mon[36579]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:26.245 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:25 smithi176 ceph-mon[36786]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:26.245 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:26.245 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:26.245 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:26.245 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:26.245 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:26.624 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 7 on host 'smithi176' 2024-06-06T08:33:26.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.625+0000 7f9a17fff700 1 -- 172.21.15.176:0/1679372904 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9a200628d0 con 0x7f9a0c0a2c20 2024-06-06T08:33:26.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a0c0a2c20 msgr2=0x7f9a0c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:26.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a0c0a2c20 0x7f9a0c0a50e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a08007900 tx=0x7f9a08008040 comp rx=0 tx=0).stop 2024-06-06T08:33:26.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 msgr2=0x7f9a201987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:26.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 0x7f9a201987a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a100062a0 tx=0x7f9a10004ef0 comp rx=0 tx=0).stop 2024-06-06T08:33:26.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 shutdown_connections 2024-06-06T08:33:26.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a0c0a2c20 0x7f9a0c0a50e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:26.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a201081d0 0x7f9a2019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:26.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a20102f70 0x7f9a201987a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:26.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 --2- 172.21.15.176:0/1679372904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a201058a0 0x7f9a20198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:26.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 >> 172.21.15.176:0/1679372904 conn(0x7f9a200fc980 msgr2=0x7f9a201039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:33:26.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 shutdown_connections 2024-06-06T08:33:26.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:33:26.628+0000 7f9a253f8700 1 -- 172.21.15.176:0/1679372904 wait complete. 2024-06-06T08:33:27.314 DEBUG:teuthology.orchestra.run.smithi176:osd.7> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.7.service 2024-06-06T08:33:27.316 INFO:tasks.cephadm:Deploying osd.8 on smithi203 with /dev/vg_nvme/lv_2... 2024-06-06T08:33:27.316 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T08:33:27.563 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:33:27.684 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:27 smithi203 ceph-mon[36579]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:27.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:27.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:27.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:27 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:27.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:27 smithi203 ceph-mon[36579]: from='osd.7 [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T08:33:27.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:27 smithi203 ceph-mon[36579]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T08:33:27.895 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:27 smithi176 ceph-mon[36786]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:27.895 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:27.895 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:27.896 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:27 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:27.896 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:27 smithi176 ceph-mon[36786]: from='osd.7 [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T08:33:27.896 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:27 smithi176 ceph-mon[36786]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T08:33:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:27 smithi027 ceph-mon[32335]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:27 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:27 smithi027 ceph-mon[32335]: from='osd.7 [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T08:33:27.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:27 smithi027 ceph-mon[32335]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T08:33:28.859 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:33:28 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:33:28.594+0000 7f9893664700 -1 osd.7 0 waiting for initial osdmap 2024-06-06T08:33:28.860 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:33:28 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:33:28.603+0000 7f988f457700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:33:28.860 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:28 smithi176 ceph-mon[36786]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T08:33:28.860 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:28 smithi176 ceph-mon[36786]: from='osd.7 [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:33:28.860 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:28 smithi176 ceph-mon[36786]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T08:33:28.860 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:28.860 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:28 smithi176 ceph-mon[36786]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:33:28.904 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:28 smithi203 ceph-mon[36579]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T08:33:28.904 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:28 smithi203 ceph-mon[36579]: from='osd.7 [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:33:28.904 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:28 smithi203 ceph-mon[36579]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T08:33:28.904 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:28.905 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:28 smithi203 ceph-mon[36579]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:33:28.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:28 smithi027 ceph-mon[32335]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T08:33:28.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:28 smithi027 ceph-mon[32335]: from='osd.7 [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:33:28.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:28 smithi027 ceph-mon[32335]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T08:33:28.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:28.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:28 smithi027 ceph-mon[32335]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:33:29.775 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T08:33:29.775 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T08:33:29.775 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-06-06T08:33:29.775 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-06-06T08:33:29.775 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0182583 s, 574 MB/s 2024-06-06T08:33:29.775 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-06-06T08:33:29.776 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:29 smithi203 ceph-mon[36579]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:29.776 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:29 smithi203 ceph-mon[36579]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:33:29.776 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:29 smithi203 ceph-mon[36579]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T08:33:29.776 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:29 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:29.776 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:29 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:29.776 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:29 smithi203 ceph-mon[36579]: osd.7 [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] boot 2024-06-06T08:33:29.776 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:29 smithi203 ceph-mon[36579]: osdmap e51: 8 total, 8 up, 8 in 2024-06-06T08:33:29.776 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:29 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:29.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:29 smithi027 ceph-mon[32335]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:29.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:29 smithi027 ceph-mon[32335]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:33:29.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:29 smithi027 ceph-mon[32335]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T08:33:29.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:29 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:29.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:29 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:29.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:29 smithi027 ceph-mon[32335]: osd.7 [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] boot 2024-06-06T08:33:29.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:29 smithi027 ceph-mon[32335]: osdmap e51: 8 total, 8 up, 8 in 2024-06-06T08:33:29.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:29 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:29.973 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:29 smithi176 ceph-mon[36786]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-06-06T08:33:29.973 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:29 smithi176 ceph-mon[36786]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:33:29.973 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:29 smithi176 ceph-mon[36786]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T08:33:29.973 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:29 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:29.973 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:29 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:29.974 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:29 smithi176 ceph-mon[36786]: osd.7 [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] boot 2024-06-06T08:33:29.974 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:29 smithi176 ceph-mon[36786]: osdmap e51: 8 total, 8 up, 8 in 2024-06-06T08:33:29.974 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:29 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:33:30.385 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_2 2024-06-06T08:33:30.637 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:33:30.684 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:33:30.684 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:33:30.684 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T08:33:30.684 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:30.684 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:30.684 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:30.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:30.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:30.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:30.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:30.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:30.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:30 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:30.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:33:30.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:33:30.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T08:33:30.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:30.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:30.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:30.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:30.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:30.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:30.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:30.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:30.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:30 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:31.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:31.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:31.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:30 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:31.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:31 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi176 2024-06-06T08:33:31.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:31 smithi203 ceph-mon[36579]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:31.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:31 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi176 to 5696M 2024-06-06T08:33:31.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:31 smithi203 ceph-mon[36579]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T08:33:31.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:31 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi176 2024-06-06T08:33:31.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:31 smithi027 ceph-mon[32335]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:31.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:31 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi176 to 5696M 2024-06-06T08:33:31.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:31 smithi027 ceph-mon[32335]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T08:33:32.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:31 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi176 2024-06-06T08:33:32.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:31 smithi176 ceph-mon[36786]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:32.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:31 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi176 to 5696M 2024-06-06T08:33:32.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:31 smithi176 ceph-mon[36786]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T08:33:32.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.239+0000 7f4fc59e9700 1 -- 172.21.15.203:0/2273962440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc0102f90 msgr2=0x7f4fc0109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:32.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.239+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/2273962440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc0102f90 0x7f4fc0109840 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009990 tx=0x7f4fb0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:33:32.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.239+0000 7f4fc59e9700 1 -- 172.21.15.203:0/2273962440 shutdown_connections 2024-06-06T08:33:32.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.239+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/2273962440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc0102f90 0x7f4fc0109840 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.239+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/2273962440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc01013f0 0x7f4fc0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.239+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/2273962440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4fc01025f0 0x7f4fc0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.239+0000 7f4fc59e9700 1 -- 172.21.15.203:0/2273962440 >> 172.21.15.203:0/2273962440 conn(0x7f4fc00fc9b0 msgr2=0x7f4fc00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.239+0000 7f4fc59e9700 1 -- 172.21.15.203:0/2273962440 shutdown_connections 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.239+0000 7f4fc59e9700 1 -- 172.21.15.203:0/2273962440 wait complete. 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fc59e9700 1 Processor -- start 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fc59e9700 1 -- start start 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fc59e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc01013f0 0x7f4fc0197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fc59e9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc01025f0 0x7f4fc0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:32.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fc59e9700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4fc0102f90 0x7f4fc019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fc59e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc010cfb0 con 0x7f4fc01013f0 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fc59e9700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc010ce30 con 0x7f4fc0102f90 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fc59e9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc010d130 con 0x7f4fc01025f0 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fbe7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc01025f0 0x7f4fc0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fbe7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc01025f0 0x7f4fc0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:34790/0 (socket says 172.21.15.203:34790) 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fbe7fc700 1 -- 172.21.15.203:0/1340874344 learned_addr learned my addr 172.21.15.203:0/1340874344 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fbeffd700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc01013f0 0x7f4fc0197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fbf7fe700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4fc0102f90 0x7f4fc019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fbe7fc700 1 -- 172.21.15.203:0/1340874344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4fc0102f90 msgr2=0x7f4fc019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.240+0000 7f4fbe7fc700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4fc0102f90 0x7f4fc019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.241+0000 7f4fbe7fc700 1 -- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc01013f0 msgr2=0x7f4fc0197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.241+0000 7f4fbe7fc700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc01013f0 0x7f4fc0197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:32.242 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.241+0000 7f4fbe7fc700 1 -- 172.21.15.203:0/1340874344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc019f190 con 0x7f4fc01025f0 2024-06-06T08:33:32.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.241+0000 7f4fbe7fc700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc01025f0 0x7f4fc0197a70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400d980 tx=0x7f4fb400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:33:32.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.241+0000 7f4fc49e7700 1 -- 172.21.15.203:0/1340874344 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4016070 con 0x7f4fc01025f0 2024-06-06T08:33:32.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.241+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc019f480 con 0x7f4fc01025f0 2024-06-06T08:33:32.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.241+0000 7f4fc49e7700 1 -- 172.21.15.203:0/1340874344 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb4009db0 con 0x7f4fc01025f0 2024-06-06T08:33:32.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.241+0000 7f4fc49e7700 1 -- 172.21.15.203:0/1340874344 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400f3c0 con 0x7f4fc01025f0 2024-06-06T08:33:32.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.241+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc019fa10 con 0x7f4fc01025f0 2024-06-06T08:33:32.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.242+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc0069180 con 0x7f4fc01025f0 2024-06-06T08:33:32.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.243+0000 7f4fc49e7700 1 -- 172.21.15.203:0/1340874344 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4fb4019070 con 0x7f4fc01025f0 2024-06-06T08:33:32.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.243+0000 7f4fc49e7700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4fa80a2c20 0x7f4fa80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:32.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.244+0000 7f4fc49e7700 1 -- 172.21.15.203:0/1340874344 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f4fb40cdfe0 con 0x7f4fc01025f0 2024-06-06T08:33:32.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.244+0000 7f4fbeffd700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4fa80a2c20 0x7f4fa80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:32.244 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.244+0000 7f4fbeffd700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4fa80a2c20 0x7f4fa80a50e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009550 tx=0x7f4fb80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:33:32.247 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.248+0000 7f4fc49e7700 1 -- 172.21.15.203:0/1340874344 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4fb409ba80 con 0x7f4fc01025f0 2024-06-06T08:33:32.453 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:32.454+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4fc0062990 con 0x7f4fa80a2c20 2024-06-06T08:33:33.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:33 smithi027 ceph-mon[32335]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T08:33:33.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:33 smithi027 ceph-mon[32335]: pgmap v181: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:33.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:33 smithi027 ceph-mon[32335]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:33:33.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:33 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:33:33.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:33 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:33:33.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:33 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:33 smithi203 ceph-mon[36579]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T08:33:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:33 smithi203 ceph-mon[36579]: pgmap v181: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:33 smithi203 ceph-mon[36579]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:33:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:33 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:33:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:33 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:33:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:33 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:33.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:33 smithi176 ceph-mon[36786]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T08:33:33.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:33 smithi176 ceph-mon[36786]: pgmap v181: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:33.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:33 smithi176 ceph-mon[36786]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:33:33.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:33 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:33:33.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:33 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:33:33.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:33 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:34.642 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:34 smithi203 ceph-mon[36579]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T08:33:34.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:34 smithi027 ceph-mon[32335]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T08:33:34.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:34 smithi176 ceph-mon[36786]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T08:33:35.691 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:35 smithi203 ceph-mon[36579]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:35.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:35 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/2763375287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd6572ce-6f94-49c6-9602-07e0196dada4"}]: dispatch 2024-06-06T08:33:35.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:35 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd6572ce-6f94-49c6-9602-07e0196dada4"}]: dispatch 2024-06-06T08:33:35.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:35 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd6572ce-6f94-49c6-9602-07e0196dada4"}]': finished 2024-06-06T08:33:35.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:35 smithi203 ceph-mon[36579]: osdmap e56: 9 total, 8 up, 9 in 2024-06-06T08:33:35.692 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:35 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:35 smithi027 ceph-mon[32335]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:35 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/2763375287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd6572ce-6f94-49c6-9602-07e0196dada4"}]: dispatch 2024-06-06T08:33:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:35 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd6572ce-6f94-49c6-9602-07e0196dada4"}]: dispatch 2024-06-06T08:33:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:35 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd6572ce-6f94-49c6-9602-07e0196dada4"}]': finished 2024-06-06T08:33:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:35 smithi027 ceph-mon[32335]: osdmap e56: 9 total, 8 up, 9 in 2024-06-06T08:33:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:35.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:35 smithi176 ceph-mon[36786]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:35 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/2763375287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd6572ce-6f94-49c6-9602-07e0196dada4"}]: dispatch 2024-06-06T08:33:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:35 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd6572ce-6f94-49c6-9602-07e0196dada4"}]: dispatch 2024-06-06T08:33:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:35 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd6572ce-6f94-49c6-9602-07e0196dada4"}]': finished 2024-06-06T08:33:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:35 smithi176 ceph-mon[36786]: osdmap e56: 9 total, 8 up, 9 in 2024-06-06T08:33:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:35 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:36.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:36 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/404991604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:33:36.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:36 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/404991604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:33:36.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:36 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/404991604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:33:37.640 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:37 smithi203 ceph-mon[36579]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:37.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:37 smithi027 ceph-mon[32335]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:37.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:37 smithi176 ceph-mon[36786]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:39.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:39 smithi027 ceph-mon[32335]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:39.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:39 smithi203 ceph-mon[36579]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:39.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:39 smithi176 ceph-mon[36786]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:41.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:41 smithi027 ceph-mon[32335]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:41.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:41 smithi203 ceph-mon[36579]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:41.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:41 smithi176 ceph-mon[36786]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:43.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:43 smithi027 ceph-mon[32335]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:43.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:43 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T08:33:43.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:43 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:43.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:43 smithi027 ceph-mon[32335]: Deploying daemon osd.8 on smithi203 2024-06-06T08:33:43.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:43 smithi203 ceph-mon[36579]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:43.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T08:33:43.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:43 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:43.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:43 smithi203 ceph-mon[36579]: Deploying daemon osd.8 on smithi203 2024-06-06T08:33:43.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:43 smithi176 ceph-mon[36786]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:43.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:43 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T08:33:43.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:43 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:43.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:43 smithi176 ceph-mon[36786]: Deploying daemon osd.8 on smithi203 2024-06-06T08:33:45.584 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:45 smithi203 ceph-mon[36579]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:45.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:45 smithi176 ceph-mon[36786]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:45.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:45 smithi027 ceph-mon[32335]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:47.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:47 smithi203 ceph-mon[36579]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:47.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:47 smithi176 ceph-mon[36786]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:47.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:47 smithi027 ceph-mon[32335]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:49.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:49.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:49.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:49.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:48 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:49.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:48 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:49.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:48 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:49.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:49.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:49.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:50.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:50 smithi203 ceph-mon[36579]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:50.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:50 smithi176 ceph-mon[36786]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:50.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:50 smithi027 ceph-mon[32335]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:51.769 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.770+0000 7f4fc49e7700 1 -- 172.21.15.203:0/1340874344 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4fc0062990 con 0x7f4fa80a2c20 2024-06-06T08:33:51.770 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 8 on host 'smithi203' 2024-06-06T08:33:51.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4fa80a2c20 msgr2=0x7f4fa80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4fa80a2c20 0x7f4fa80a50e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009550 tx=0x7f4fb80091b0 comp rx=0 tx=0).stop 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc01025f0 msgr2=0x7f4fc0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc01025f0 0x7f4fc0197a70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400d980 tx=0x7f4fb400dc90 comp rx=0 tx=0).stop 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 shutdown_connections 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4fa80a2c20 0x7f4fa80a50e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4fc0102f90 0x7f4fc019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc01013f0 0x7f4fc0197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 --2- 172.21.15.203:0/1340874344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fc01025f0 0x7f4fc0197a70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 >> 172.21.15.203:0/1340874344 conn(0x7f4fc00fc9b0 msgr2=0x7f4fc0105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 shutdown_connections 2024-06-06T08:33:51.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:33:51.772+0000 7f4fc59e9700 1 -- 172.21.15.203:0/1340874344 wait complete. 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='osd.8 [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T08:33:51.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:51 smithi203 ceph-mon[36579]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T08:33:52.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:52.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:52.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:52.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:52.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='osd.8 [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T08:33:52.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:51 smithi027 ceph-mon[32335]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:33:52.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='osd.8 [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T08:33:52.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:51 smithi176 ceph-mon[36786]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T08:33:52.424 DEBUG:teuthology.orchestra.run.smithi203:osd.8> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.8.service 2024-06-06T08:33:52.427 INFO:tasks.cephadm:Deploying osd.9 on smithi027 with /dev/vg_nvme/lv_1... 2024-06-06T08:33:52.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T08:33:52.688 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:33:53.982 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:53 smithi027 ceph-mon[32335]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:53.982 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:53 smithi027 ceph-mon[32335]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T08:33:53.982 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:53 smithi027 ceph-mon[32335]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T08:33:53.982 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:53 smithi027 ceph-mon[32335]: from='osd.8 [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:33:53.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:53.983 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:53 smithi027 ceph-mon[32335]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:33:53.988 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:53 smithi203 ceph-mon[36579]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:53.988 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:53 smithi203 ceph-mon[36579]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T08:33:53.988 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:53 smithi203 ceph-mon[36579]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T08:33:53.988 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:53 smithi203 ceph-mon[36579]: from='osd.8 [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:33:53.988 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:53.988 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:53 smithi203 ceph-mon[36579]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:33:53.988 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:33:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:33:53.737+0000 7f37b795d700 -1 osd.8 0 waiting for initial osdmap 2024-06-06T08:33:53.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:33:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:33:53.751+0000 7f37b3750700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:33:54.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:53 smithi176 ceph-mon[36786]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:54.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:53 smithi176 ceph-mon[36786]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T08:33:54.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:53 smithi176 ceph-mon[36786]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T08:33:54.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:53 smithi176 ceph-mon[36786]: from='osd.8 [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:33:54.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:54.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:53 smithi176 ceph-mon[36786]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:33:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T08:33:54.908 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T08:33:54.908 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-06-06T08:33:54.908 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-06-06T08:33:54.908 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0145485 s, 721 MB/s 2024-06-06T08:33:54.908 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-06-06T08:33:54.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:54 smithi203 ceph-mon[36579]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:33:54.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:54 smithi203 ceph-mon[36579]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T08:33:54.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:54 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:54.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:54 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:54.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:54 smithi203 ceph-mon[36579]: osd.8 [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] boot 2024-06-06T08:33:54.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:54 smithi203 ceph-mon[36579]: osdmap e59: 9 total, 9 up, 9 in 2024-06-06T08:33:54.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:54 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:55.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:54 smithi176 ceph-mon[36786]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:33:55.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:54 smithi176 ceph-mon[36786]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T08:33:55.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:54 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:55.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:54 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:55.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:54 smithi176 ceph-mon[36786]: osd.8 [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] boot 2024-06-06T08:33:55.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:54 smithi176 ceph-mon[36786]: osdmap e59: 9 total, 9 up, 9 in 2024-06-06T08:33:55.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:54 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:55.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:54 smithi027 ceph-mon[32335]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:33:55.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:54 smithi027 ceph-mon[32335]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T08:33:55.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:54 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:55.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:54 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:55.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:54 smithi027 ceph-mon[32335]: osd.8 [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] boot 2024-06-06T08:33:55.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:54 smithi027 ceph-mon[32335]: osdmap e59: 9 total, 9 up, 9 in 2024-06-06T08:33:55.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:54 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:33:55.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_1 2024-06-06T08:33:55.893 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:55.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:55.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:55.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:55.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:55.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:33:55.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:33:55.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:55.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T08:33:55.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:55.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:55.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:55.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:55.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:55.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:55.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:55.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:55.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:56.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:33:56.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:33:56.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:56 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi203 2024-06-06T08:33:56.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:56 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi203 to 5696M 2024-06-06T08:33:56.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:56 smithi203 ceph-mon[36579]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T08:33:57.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:56 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi203 2024-06-06T08:33:57.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:56 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi203 to 5696M 2024-06-06T08:33:57.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:56 smithi176 ceph-mon[36786]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T08:33:57.057 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:56 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi203 2024-06-06T08:33:57.057 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:56 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi203 to 5696M 2024-06-06T08:33:57.057 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:56 smithi027 ceph-mon[32335]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T08:33:57.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 -- 172.21.15.27:0/1614925564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c102f90 msgr2=0x7ff27c109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:57.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 --2- 172.21.15.27:0/1614925564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c102f90 0x7ff27c109840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff27400b0d0 tx=0x7ff27400b3e0 comp rx=0 tx=0).stop 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 -- 172.21.15.27:0/1614925564 shutdown_connections 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 --2- 172.21.15.27:0/1614925564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c102f90 0x7ff27c109840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 --2- 172.21.15.27:0/1614925564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff27c1013f0 0x7ff27c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 --2- 172.21.15.27:0/1614925564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff27c1025f0 0x7ff27c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 -- 172.21.15.27:0/1614925564 >> 172.21.15.27:0/1614925564 conn(0x7ff27c0fc9b0 msgr2=0x7ff27c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 -- 172.21.15.27:0/1614925564 shutdown_connections 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 -- 172.21.15.27:0/1614925564 wait complete. 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 Processor -- start 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.621+0000 7ff282e8c700 1 -- start start 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff282e8c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c1013f0 0x7ff27c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff282e8c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff27c1025f0 0x7ff27c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:57.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff282e8c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff27c102f90 0x7ff27c1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff282e8c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c10cfd0 con 0x7ff27c1013f0 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff282e8c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c10ce50 con 0x7ff27c1025f0 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff282e8c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c10d150 con 0x7ff27c102f90 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff280c28700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c1013f0 0x7ff27c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff280c28700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c1013f0 0x7ff27c19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38584/0 (socket says 172.21.15.27:38584) 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff280c28700 1 -- 172.21.15.27:0/3357943178 learned_addr learned my addr 172.21.15.27:0/3357943178 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff27bfff700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff27c1025f0 0x7ff27c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff281429700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff27c102f90 0x7ff27c1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff280c28700 1 -- 172.21.15.27:0/3357943178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff27c102f90 msgr2=0x7ff27c1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff280c28700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff27c102f90 0x7ff27c1a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff280c28700 1 -- 172.21.15.27:0/3357943178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff27c1025f0 msgr2=0x7ff27c19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:33:57.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff280c28700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff27c1025f0 0x7ff27c19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:33:57.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff280c28700 1 -- 172.21.15.27:0/3357943178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c1a2510 con 0x7ff27c1013f0 2024-06-06T08:33:57.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff27bfff700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff27c1025f0 0x7ff27c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:33:57.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.622+0000 7ff281429700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff27c102f90 0x7ff27c1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:33:57.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.623+0000 7ff280c28700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c1013f0 0x7ff27c19a8b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff27000d9b0 tx=0x7ff27000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:33:57.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.623+0000 7ff279ffb700 1 -- 172.21.15.27:0/3357943178 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270017070 con 0x7ff27c1013f0 2024-06-06T08:33:57.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.623+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27c1a2800 con 0x7ff27c1013f0 2024-06-06T08:33:57.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.623+0000 7ff279ffb700 1 -- 172.21.15.27:0/3357943178 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff270009b40 con 0x7ff27c1013f0 2024-06-06T08:33:57.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.623+0000 7ff279ffb700 1 -- 172.21.15.27:0/3357943178 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27000f620 con 0x7ff27c1013f0 2024-06-06T08:33:57.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.623+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27c1a2d40 con 0x7ff27c1013f0 2024-06-06T08:33:57.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.624+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27c069180 con 0x7ff27c1013f0 2024-06-06T08:33:57.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.625+0000 7ff279ffb700 1 -- 172.21.15.27:0/3357943178 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff270009cf0 con 0x7ff27c1013f0 2024-06-06T08:33:57.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.626+0000 7ff279ffb700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff26c0a2c20 0x7ff26c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:33:57.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.626+0000 7ff279ffb700 1 -- 172.21.15.27:0/3357943178 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7ff2700ce320 con 0x7ff27c1013f0 2024-06-06T08:33:57.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.626+0000 7ff27bfff700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff26c0a2c20 0x7ff26c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:33:57.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.626+0000 7ff27bfff700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff26c0a2c20 0x7ff26c0a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff268005950 tx=0x7ff2680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:33:57.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.631+0000 7ff279ffb700 1 -- 172.21.15.27:0/3357943178 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff27009bc30 con 0x7ff27c1013f0 2024-06-06T08:33:57.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:33:57.847+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff27c0fead0 con 0x7ff26c0a2c20 2024-06-06T08:33:57.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:57 smithi027 ceph-mon[32335]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:33:57.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:57 smithi027 ceph-mon[32335]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T08:33:57.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:57 smithi203 ceph-mon[36579]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:33:57.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:57 smithi203 ceph-mon[36579]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T08:33:58.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:57 smithi176 ceph-mon[36786]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:33:58.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:57 smithi176 ceph-mon[36786]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T08:33:58.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:58 smithi203 ceph-mon[36579]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:33:58.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:33:58.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:33:58.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:58 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:58.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:58 smithi203 ceph-mon[36579]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T08:33:59.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:58 smithi176 ceph-mon[36786]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:33:59.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:33:59.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:33:59.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:58 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:59.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:58 smithi176 ceph-mon[36786]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T08:33:59.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:58 smithi027 ceph-mon[32335]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:33:59.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:33:59.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:33:59.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:58 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:33:59.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:58 smithi027 ceph-mon[32335]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T08:33:59.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:33:59 smithi027 ceph-mon[32335]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:00.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:33:59 smithi176 ceph-mon[36786]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:00.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:33:59 smithi203 ceph-mon[36579]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:02.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:01 smithi176 ceph-mon[36786]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-06-06T08:34:02.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:01 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4197684890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bece157-d4cb-4072-a56d-a3e3d3c09f85"}]: dispatch 2024-06-06T08:34:02.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:01 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4197684890' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bece157-d4cb-4072-a56d-a3e3d3c09f85"}]': finished 2024-06-06T08:34:02.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:01 smithi176 ceph-mon[36786]: osdmap e64: 10 total, 9 up, 10 in 2024-06-06T08:34:02.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:01 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:02.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:01 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1782370847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:34:02.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:01 smithi027 ceph-mon[32335]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-06-06T08:34:02.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:01 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4197684890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bece157-d4cb-4072-a56d-a3e3d3c09f85"}]: dispatch 2024-06-06T08:34:02.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:01 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4197684890' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bece157-d4cb-4072-a56d-a3e3d3c09f85"}]': finished 2024-06-06T08:34:02.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:01 smithi027 ceph-mon[32335]: osdmap e64: 10 total, 9 up, 10 in 2024-06-06T08:34:02.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:01 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:02.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:01 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1782370847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:34:02.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:01 smithi203 ceph-mon[36579]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-06-06T08:34:02.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:01 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4197684890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bece157-d4cb-4072-a56d-a3e3d3c09f85"}]: dispatch 2024-06-06T08:34:02.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:01 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4197684890' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bece157-d4cb-4072-a56d-a3e3d3c09f85"}]': finished 2024-06-06T08:34:02.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:01 smithi203 ceph-mon[36579]: osdmap e64: 10 total, 9 up, 10 in 2024-06-06T08:34:02.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:01 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:02.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:01 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1782370847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:34:04.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:03 smithi176 ceph-mon[36786]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T08:34:04.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:03 smithi027 ceph-mon[32335]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T08:34:04.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:03 smithi203 ceph-mon[36579]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T08:34:06.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:05 smithi176 ceph-mon[36786]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-06-06T08:34:06.107 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:05 smithi027 ceph-mon[32335]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-06-06T08:34:06.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:05 smithi203 ceph-mon[36579]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-06-06T08:34:08.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:07 smithi176 ceph-mon[36786]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:34:08.042 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:07 smithi203 ceph-mon[36579]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:34:08.133 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:07 smithi027 ceph-mon[32335]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:34:09.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:09 smithi027 ceph-mon[32335]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:34:09.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T08:34:09.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:09 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:09.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:09 smithi027 ceph-mon[32335]: Deploying daemon osd.9 on smithi027 2024-06-06T08:34:10.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:09 smithi176 ceph-mon[36786]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:34:10.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T08:34:10.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:09 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:10.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:09 smithi176 ceph-mon[36786]: Deploying daemon osd.9 on smithi027 2024-06-06T08:34:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:09 smithi203 ceph-mon[36579]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:34:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T08:34:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:09 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:09 smithi203 ceph-mon[36579]: Deploying daemon osd.9 on smithi027 2024-06-06T08:34:11.891 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:11 smithi027 ceph-mon[32335]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:12.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:11 smithi176 ceph-mon[36786]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:12.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:11 smithi203 ceph-mon[36579]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:14.101 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:13 smithi027 ceph-mon[32335]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:14.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:13 smithi203 ceph-mon[36579]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:14.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:13 smithi176 ceph-mon[36786]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:15.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:15 smithi027 ceph-mon[32335]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:15.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:15.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:15.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:16.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:15 smithi203 ceph-mon[36579]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:16.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:16.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:16.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:16.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:15 smithi176 ceph-mon[36786]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:16.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:16.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:16.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:18.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:17 smithi027 ceph-mon[32335]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:18.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:17 smithi203 ceph-mon[36579]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:18.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:17 smithi176 ceph-mon[36786]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.571+0000 7ff279ffb700 1 -- 172.21.15.27:0/3357943178 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff27c0fead0 con 0x7ff26c0a2c20 2024-06-06T08:34:19.571 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 9 on host 'smithi027' 2024-06-06T08:34:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff26c0a2c20 msgr2=0x7ff26c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff26c0a2c20 0x7ff26c0a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff268005950 tx=0x7ff2680058e0 comp rx=0 tx=0).stop 2024-06-06T08:34:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c1013f0 msgr2=0x7ff27c19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c1013f0 0x7ff27c19a8b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff27000d9b0 tx=0x7ff27000dd70 comp rx=0 tx=0).stop 2024-06-06T08:34:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 shutdown_connections 2024-06-06T08:34:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff26c0a2c20 0x7ff26c0a50e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:19.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff27c102f90 0x7ff27c1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:19.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff27c1013f0 0x7ff27c19a8b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:19.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 --2- 172.21.15.27:0/3357943178 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff27c1025f0 0x7ff27c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:19.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 >> 172.21.15.27:0/3357943178 conn(0x7ff27c0fc9b0 msgr2=0x7ff27c0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:34:19.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 shutdown_connections 2024-06-06T08:34:19.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:34:19.573+0000 7ff282e8c700 1 -- 172.21.15.27:0/3357943178 wait complete. 2024-06-06T08:34:19.925 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:19.925 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.925 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:19.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:19.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T08:34:19.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:19 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T08:34:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:19.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:19 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:20.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:20.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:20.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:20.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:20.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:20.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:20.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T08:34:20.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:20.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:20.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:19 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:20.394 DEBUG:teuthology.orchestra.run.smithi027:osd.9> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.9.service 2024-06-06T08:34:20.396 INFO:tasks.cephadm:Deploying osd.10 on smithi176 with /dev/vg_nvme/lv_1... 2024-06-06T08:34:20.396 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T08:34:20.641 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:34:20.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:20 smithi027 ceph-mon[32335]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T08:34:20.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:20 smithi027 ceph-mon[32335]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T08:34:20.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:20 smithi027 ceph-mon[32335]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:34:20.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:20 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:20.957 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:34:20 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:34:20.773+0000 7fe5d3eba700 -1 osd.9 0 waiting for initial osdmap 2024-06-06T08:34:20.957 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:34:20 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:34:20.786+0000 7fe5cfcad700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:34:21.107 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:20 smithi176 ceph-mon[36786]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T08:34:21.107 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:20 smithi176 ceph-mon[36786]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T08:34:21.107 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:20 smithi176 ceph-mon[36786]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:34:21.108 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:20 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:21.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:20 smithi203 ceph-mon[36579]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T08:34:21.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:20 smithi203 ceph-mon[36579]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T08:34:21.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:20 smithi203 ceph-mon[36579]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-06-06T08:34:21.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:20 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:21.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:21 smithi027 ceph-mon[32335]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:21.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:21 smithi027 ceph-mon[32335]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:34:21.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:21 smithi027 ceph-mon[32335]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T08:34:21.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:21.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:21 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:21 smithi176 ceph-mon[36786]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:21 smithi176 ceph-mon[36786]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:34:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:21 smithi176 ceph-mon[36786]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T08:34:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:21 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:21 smithi203 ceph-mon[36579]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:21 smithi203 ceph-mon[36579]: from='osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-06-06T08:34:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:21 smithi203 ceph-mon[36579]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T08:34:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:21 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:22.611 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T08:34:22.611 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T08:34:22.611 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-06-06T08:34:22.611 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-06-06T08:34:22.611 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124939 s, 839 MB/s 2024-06-06T08:34:22.612 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-06-06T08:34:23.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:22 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:34:23.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:22 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:34:23.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:22 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:23.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:22 smithi027 ceph-mon[32335]: osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] boot 2024-06-06T08:34:23.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:22 smithi027 ceph-mon[32335]: osdmap e67: 10 total, 10 up, 10 in 2024-06-06T08:34:23.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:22 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:22 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:34:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:22 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:34:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:22 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:22 smithi203 ceph-mon[36579]: osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] boot 2024-06-06T08:34:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:22 smithi203 ceph-mon[36579]: osdmap e67: 10 total, 10 up, 10 in 2024-06-06T08:34:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:22 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:23.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:22 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:34:23.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:22 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:34:23.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:22 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:23.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:22 smithi176 ceph-mon[36786]: osd.9 [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] boot 2024-06-06T08:34:23.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:22 smithi176 ceph-mon[36786]: osdmap e67: 10 total, 10 up, 10 in 2024-06-06T08:34:23.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:22 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:34:23.329 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi176:vg_nvme/lv_1 2024-06-06T08:34:23.571 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:34:23.862 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:23 smithi176 ceph-mon[36786]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:23.863 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:23 smithi176 ceph-mon[36786]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T08:34:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:23 smithi027 ceph-mon[32335]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:23 smithi027 ceph-mon[32335]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T08:34:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:23 smithi203 ceph-mon[36579]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-06-06T08:34:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:23 smithi203 ceph-mon[36579]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T08:34:25.123 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi027 2024-06-06T08:34:25.124 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.124 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.124 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.124 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.124 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.124 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.125 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi027 to 4272M 2024-06-06T08:34:25.125 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.125 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:25.125 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:25.125 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:24 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi027 2024-06-06T08:34:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi027 to 4272M 2024-06-06T08:34:25.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:25.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:25.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:24 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi027 2024-06-06T08:34:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi027 to 4272M 2024-06-06T08:34:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:25.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:25.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:24 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:25.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.351+0000 7ff82f690700 1 -- 172.21.15.176:0/2066682401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 msgr2=0x7ff828102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:25.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.351+0000 7ff82f690700 1 --2- 172.21.15.176:0/2066682401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 0x7ff828102a50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff8180099c0 tx=0x7ff818009cd0 comp rx=0 tx=0).stop 2024-06-06T08:34:25.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.351+0000 7ff82f690700 1 -- 172.21.15.176:0/2066682401 shutdown_connections 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.351+0000 7ff82f690700 1 --2- 172.21.15.176:0/2066682401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff828102f90 0x7ff828109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.351+0000 7ff82f690700 1 --2- 172.21.15.176:0/2066682401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8281013f0 0x7ff828101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.351+0000 7ff82f690700 1 --2- 172.21.15.176:0/2066682401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 0x7ff828102a50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.351+0000 7ff82f690700 1 -- 172.21.15.176:0/2066682401 >> 172.21.15.176:0/2066682401 conn(0x7ff8280fc990 msgr2=0x7ff8280fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.351+0000 7ff82f690700 1 -- 172.21.15.176:0/2066682401 shutdown_connections 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.351+0000 7ff82f690700 1 -- 172.21.15.176:0/2066682401 wait complete. 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82f690700 1 Processor -- start 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82f690700 1 -- start start 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82f690700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8281013f0 0x7ff828197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82f690700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 0x7ff828197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82f690700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff828102f90 0x7ff82819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:34:25.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82f690700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810d070 con 0x7ff8281013f0 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82f690700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810cef0 con 0x7ff8281025f0 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82f690700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810d1f0 con 0x7ff828102f90 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82cc2b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 0x7ff828197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82cc2b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 0x7ff828197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44582/0 (socket says 172.21.15.176:44582) 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82cc2b700 1 -- 172.21.15.176:0/318536321 learned_addr learned my addr 172.21.15.176:0/318536321 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82d42c700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8281013f0 0x7ff828197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82dc2d700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff828102f90 0x7ff82819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82cc2b700 1 -- 172.21.15.176:0/318536321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff828102f90 msgr2=0x7ff82819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82cc2b700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff828102f90 0x7ff82819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82cc2b700 1 -- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8281013f0 msgr2=0x7ff828197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.352+0000 7ff82cc2b700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8281013f0 0x7ff828197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.353+0000 7ff82cc2b700 1 -- 172.21.15.176:0/318536321 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82819f360 con 0x7ff8281025f0 2024-06-06T08:34:25.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.353+0000 7ff82dc2d700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff828102f90 0x7ff82819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:34:25.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.353+0000 7ff82cc2b700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 0x7ff828197c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff818006010 tx=0x7ff818004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:34:25.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.353+0000 7ff81e7fc700 1 -- 172.21.15.176:0/318536321 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818027070 con 0x7ff8281025f0 2024-06-06T08:34:25.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.353+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82819f5f0 con 0x7ff8281025f0 2024-06-06T08:34:25.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.353+0000 7ff81e7fc700 1 -- 172.21.15.176:0/318536321 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8180041d0 con 0x7ff8281025f0 2024-06-06T08:34:25.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.353+0000 7ff81e7fc700 1 -- 172.21.15.176:0/318536321 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81800f6c0 con 0x7ff8281025f0 2024-06-06T08:34:25.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.353+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82819fb20 con 0x7ff8281025f0 2024-06-06T08:34:25.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.354+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff828069180 con 0x7ff8281025f0 2024-06-06T08:34:25.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.355+0000 7ff81e7fc700 1 -- 172.21.15.176:0/318536321 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff81802a070 con 0x7ff8281025f0 2024-06-06T08:34:25.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.356+0000 7ff81e7fc700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff8140a2c20 0x7ff8140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:34:25.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.356+0000 7ff81e7fc700 1 -- 172.21.15.176:0/318536321 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7ff8180cfda0 con 0x7ff8281025f0 2024-06-06T08:34:25.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.356+0000 7ff82d42c700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff8140a2c20 0x7ff8140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:34:25.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.357+0000 7ff82d42c700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff8140a2c20 0x7ff8140a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff828102450 tx=0x7ff824009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:34:25.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.359+0000 7ff81e7fc700 1 -- 172.21.15.176:0/318536321 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff818031020 con 0x7ff8281025f0 2024-06-06T08:34:25.563 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:25.564+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff8280628d0 con 0x7ff8140a2c20 2024-06-06T08:34:26.199 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:25 smithi176 ceph-mon[36786]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:26.199 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:34:26.199 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:34:26.199 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:25 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:25 smithi027 ceph-mon[32335]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:34:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:34:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:25 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:25 smithi203 ceph-mon[36579]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:34:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:34:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:25 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:27.202 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:26 smithi176 ceph-mon[36786]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:34:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:26 smithi027 ceph-mon[32335]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:34:27.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:26 smithi203 ceph-mon[36579]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:34:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:27 smithi027 ceph-mon[32335]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:28.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:27 smithi203 ceph-mon[36579]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:28.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:27 smithi176 ceph-mon[36786]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:28 smithi027 ceph-mon[32335]: from='client.? 172.21.15.176:0/3240283650' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1044fbd0-7574-4faf-a3b1-da2a76a54e73"}]: dispatch 2024-06-06T08:34:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:28 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1044fbd0-7574-4faf-a3b1-da2a76a54e73"}]: dispatch 2024-06-06T08:34:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:28 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1044fbd0-7574-4faf-a3b1-da2a76a54e73"}]': finished 2024-06-06T08:34:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:28 smithi027 ceph-mon[32335]: osdmap e69: 11 total, 10 up, 11 in 2024-06-06T08:34:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:28 smithi203 ceph-mon[36579]: from='client.? 172.21.15.176:0/3240283650' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1044fbd0-7574-4faf-a3b1-da2a76a54e73"}]: dispatch 2024-06-06T08:34:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:28 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1044fbd0-7574-4faf-a3b1-da2a76a54e73"}]: dispatch 2024-06-06T08:34:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:28 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1044fbd0-7574-4faf-a3b1-da2a76a54e73"}]': finished 2024-06-06T08:34:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:28 smithi203 ceph-mon[36579]: osdmap e69: 11 total, 10 up, 11 in 2024-06-06T08:34:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:28 smithi176 ceph-mon[36786]: from='client.? 172.21.15.176:0/3240283650' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1044fbd0-7574-4faf-a3b1-da2a76a54e73"}]: dispatch 2024-06-06T08:34:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:28 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1044fbd0-7574-4faf-a3b1-da2a76a54e73"}]: dispatch 2024-06-06T08:34:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:28 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1044fbd0-7574-4faf-a3b1-da2a76a54e73"}]': finished 2024-06-06T08:34:29.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:28 smithi176 ceph-mon[36786]: osdmap e69: 11 total, 10 up, 11 in 2024-06-06T08:34:29.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:30.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:29 smithi027 ceph-mon[32335]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:30.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:29 smithi027 ceph-mon[32335]: from='client.? 172.21.15.176:0/3126505510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:34:30.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:29 smithi203 ceph-mon[36579]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:30.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:29 smithi203 ceph-mon[36579]: from='client.? 172.21.15.176:0/3126505510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:34:30.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:29 smithi176 ceph-mon[36786]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:30.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:29 smithi176 ceph-mon[36786]: from='client.? 172.21.15.176:0/3126505510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:34:32.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:31 smithi027 ceph-mon[32335]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:31 smithi203 ceph-mon[36579]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:31 smithi176 ceph-mon[36786]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:34.054 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:33 smithi176 ceph-mon[36786]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:33 smithi203 ceph-mon[36579]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:34.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:33 smithi027 ceph-mon[32335]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:36.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:35 smithi203 ceph-mon[36579]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:36.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:35 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T08:34:36.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:35 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:36.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:35 smithi176 ceph-mon[36786]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:36.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:35 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T08:34:36.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:35 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:36.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:35 smithi027 ceph-mon[32335]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:36.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T08:34:36.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:35 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:37.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:36 smithi176 ceph-mon[36786]: Deploying daemon osd.10 on smithi176 2024-06-06T08:34:37.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:36 smithi203 ceph-mon[36579]: Deploying daemon osd.10 on smithi176 2024-06-06T08:34:37.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:36 smithi027 ceph-mon[32335]: Deploying daemon osd.10 on smithi176 2024-06-06T08:34:38.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:37 smithi176 ceph-mon[36786]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:38.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:37 smithi203 ceph-mon[36579]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:38.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:37 smithi027 ceph-mon[32335]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:40.237 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:39 smithi176 ceph-mon[36786]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:40.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:39 smithi203 ceph-mon[36579]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:40.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:39 smithi027 ceph-mon[32335]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:42.104 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:41 smithi176 ceph-mon[36786]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:42.104 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:41 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:42.104 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:41 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:42.104 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:41 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:42.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:41 smithi203 ceph-mon[36579]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:42.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:41 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:42.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:41 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:42.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:41 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:41 smithi027 ceph-mon[32335]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:41 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:41 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:41 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:44.149 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:43 smithi176 ceph-mon[36786]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:44.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:43 smithi203 ceph-mon[36579]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:43 smithi027 ceph-mon[32335]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:45.383 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 10 on host 'smithi176' 2024-06-06T08:34:45.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.384+0000 7ff81e7fc700 1 -- 172.21.15.176:0/318536321 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff8280628d0 con 0x7ff8140a2c20 2024-06-06T08:34:45.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff8140a2c20 msgr2=0x7ff8140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff8140a2c20 0x7ff8140a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff828102450 tx=0x7ff824009190 comp rx=0 tx=0).stop 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 msgr2=0x7ff828197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 0x7ff828197c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff818006010 tx=0x7ff818004580 comp rx=0 tx=0).stop 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 shutdown_connections 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff8140a2c20 0x7ff8140a50e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff828102f90 0x7ff82819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8281013f0 0x7ff828197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 --2- 172.21.15.176:0/318536321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8281025f0 0x7ff828197c40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 >> 172.21.15.176:0/318536321 conn(0x7ff8280fc990 msgr2=0x7ff8281059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 shutdown_connections 2024-06-06T08:34:45.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-06-06T08:34:45.386+0000 7ff82f690700 1 -- 172.21.15.176:0/318536321 wait complete. 2024-06-06T08:34:45.702 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:45.702 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.702 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.702 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:45.702 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:45.702 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.702 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='osd.10 [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T08:34:45.702 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T08:34:45.702 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.703 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.703 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:45.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:45.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:45.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:45.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='osd.10 [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T08:34:45.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T08:34:45.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:45.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:45.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:45.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:45.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='osd.10 [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T08:34:45.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T08:34:45.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:45.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:34:46.036 DEBUG:teuthology.orchestra.run.smithi176:osd.10> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.10.service 2024-06-06T08:34:46.039 INFO:tasks.cephadm:Deploying osd.11 on smithi203 with /dev/vg_nvme/lv_1... 2024-06-06T08:34:46.039 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T08:34:46.285 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:34:46.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:46 smithi027 ceph-mon[32335]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T08:34:46.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:46 smithi027 ceph-mon[32335]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T08:34:46.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:46 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:46.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:46 smithi027 ceph-mon[32335]: from='osd.10 [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:34:46.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:46 smithi027 ceph-mon[32335]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:34:46.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:46 smithi203 ceph-mon[36579]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T08:34:46.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:46 smithi203 ceph-mon[36579]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T08:34:46.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:46 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:46.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:46 smithi203 ceph-mon[36579]: from='osd.10 [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:34:46.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:46 smithi203 ceph-mon[36579]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:34:46.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:46 smithi176 ceph-mon[36786]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T08:34:46.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:46 smithi176 ceph-mon[36786]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T08:34:46.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:46 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:46.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:46 smithi176 ceph-mon[36786]: from='osd.10 [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:34:46.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:46 smithi176 ceph-mon[36786]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-06-06T08:34:47.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:34:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:34:46.856+0000 7f7efad36700 -1 osd.10 0 waiting for initial osdmap 2024-06-06T08:34:47.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:34:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:34:46.869+0000 7f7ef2320700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:34:47.599 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:47 smithi176 ceph-mon[36786]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:47.599 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:47 smithi176 ceph-mon[36786]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:34:47.599 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:47 smithi176 ceph-mon[36786]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T08:34:47.599 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:47.599 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:47.599 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:47 smithi176 ceph-mon[36786]: from='osd.10 ' entity='osd.10' 2024-06-06T08:34:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:47 smithi027 ceph-mon[32335]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:47 smithi027 ceph-mon[32335]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:34:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:47 smithi027 ceph-mon[32335]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T08:34:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:47 smithi027 ceph-mon[32335]: from='osd.10 ' entity='osd.10' 2024-06-06T08:34:47.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:47 smithi203 ceph-mon[36579]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:47.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:47 smithi203 ceph-mon[36579]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-06-06T08:34:47.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:47 smithi203 ceph-mon[36579]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T08:34:47.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:47.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:47.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:47 smithi203 ceph-mon[36579]: from='osd.10 ' entity='osd.10' 2024-06-06T08:34:48.338 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T08:34:48.338 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T08:34:48.338 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-06-06T08:34:48.338 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-06-06T08:34:48.338 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125542 s, 835 MB/s 2024-06-06T08:34:48.338 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-06-06T08:34:48.686 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:48 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:34:48.686 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:48 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:34:48.686 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:48 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:48.686 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:48 smithi176 ceph-mon[36786]: osd.10 [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] boot 2024-06-06T08:34:48.686 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:48 smithi176 ceph-mon[36786]: osdmap e72: 11 total, 11 up, 11 in 2024-06-06T08:34:48.686 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:48 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:48 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:34:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:48 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:34:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:48 smithi027 ceph-mon[32335]: osd.10 [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] boot 2024-06-06T08:34:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:48 smithi027 ceph-mon[32335]: osdmap e72: 11 total, 11 up, 11 in 2024-06-06T08:34:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:48 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:48.720 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:48 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:34:48.720 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:48 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:34:48.720 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:48.720 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:48 smithi203 ceph-mon[36579]: osd.10 [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] boot 2024-06-06T08:34:48.720 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:48 smithi203 ceph-mon[36579]: osdmap e72: 11 total, 11 up, 11 in 2024-06-06T08:34:48.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:48 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:34:49.068 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_1 2024-06-06T08:34:49.308 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:34:49.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:49.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T08:34:49.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi176 2024-06-06T08:34:49.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi176 to 4272M 2024-06-06T08:34:49.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:49.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:49.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:49.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T08:34:49.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi176 2024-06-06T08:34:49.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.725 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.726 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi176 to 4272M 2024-06-06T08:34:49.726 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.726 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:49.726 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:49.726 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi176 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi176 to 4272M 2024-06-06T08:34:49.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:49.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:49.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:34:49.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:34:50.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:50 smithi027 ceph-mon[32335]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T08:34:50.713 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:50 smithi203 ceph-mon[36579]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T08:34:50.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:50 smithi176 ceph-mon[36786]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T08:34:51.066 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.066+0000 7fd01b227700 1 -- 172.21.15.203:0/1219150147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd014103810 msgr2=0x7fd014103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:51.066 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.066+0000 7fd01b227700 1 --2- 172.21.15.203:0/1219150147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd014103810 0x7fd014103c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd000009990 tx=0x7fd000009ca0 comp rx=0 tx=0).stop 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.066+0000 7fd01b227700 1 -- 172.21.15.203:0/1219150147 shutdown_connections 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.066+0000 7fd01b227700 1 --2- 172.21.15.203:0/1219150147 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd0141041b0 0x7fd01410aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.066+0000 7fd01b227700 1 --2- 172.21.15.203:0/1219150147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd014100790 0x7fd014100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.066+0000 7fd01b227700 1 --2- 172.21.15.203:0/1219150147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd014103810 0x7fd014103c70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.066+0000 7fd01b227700 1 -- 172.21.15.203:0/1219150147 >> 172.21.15.203:0/1219150147 conn(0x7fd0140fc0f0 msgr2=0x7fd0140fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.066+0000 7fd01b227700 1 -- 172.21.15.203:0/1219150147 shutdown_connections 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.066+0000 7fd01b227700 1 -- 172.21.15.203:0/1219150147 wait complete. 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd01b227700 1 Processor -- start 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd01b227700 1 -- start start 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd01b227700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd014100790 0x7fd0141986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd01b227700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd014103810 0x7fd014198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:34:51.067 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd01b227700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0141041b0 0x7fd01419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd01b227700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd01410e0f0 con 0x7fd014100790 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd01b227700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd01410df70 con 0x7fd014103810 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd01b227700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd01410e270 con 0x7fd0141041b0 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd0197c4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0141041b0 0x7fd01419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd0197c4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0141041b0 0x7fd01419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53326/0 (socket says 172.21.15.203:53326) 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd0197c4700 1 -- 172.21.15.203:0/202776561 learned_addr learned my addr 172.21.15.203:0/202776561 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd013fff700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd014103810 0x7fd014198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd018fc3700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd014100790 0x7fd0141986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd0197c4700 1 -- 172.21.15.203:0/202776561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd014103810 msgr2=0x7fd014198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd0197c4700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd014103810 0x7fd014198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd0197c4700 1 -- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd014100790 msgr2=0x7fd0141986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:34:51.068 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd0197c4700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd014100790 0x7fd0141986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:34:51.069 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd0197c4700 1 -- 172.21.15.203:0/202776561 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0141a0340 con 0x7fd0141041b0 2024-06-06T08:34:51.069 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.067+0000 7fd0197c4700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0141041b0 0x7fd01419fca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00b9a0 tx=0x7fd00c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:34:51.069 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.068+0000 7fd011ffb700 1 -- 172.21.15.203:0/202776561 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c019070 con 0x7fd0141041b0 2024-06-06T08:34:51.069 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.068+0000 7fd011ffb700 1 -- 172.21.15.203:0/202776561 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00c00ce10 con 0x7fd0141041b0 2024-06-06T08:34:51.069 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.068+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0141a0630 con 0x7fd0141041b0 2024-06-06T08:34:51.069 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.068+0000 7fd011ffb700 1 -- 172.21.15.203:0/202776561 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0073d0 con 0x7fd0141041b0 2024-06-06T08:34:51.069 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.068+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0141a0b70 con 0x7fd0141041b0 2024-06-06T08:34:51.070 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.069+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd014069180 con 0x7fd0141041b0 2024-06-06T08:34:51.070 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.070+0000 7fd011ffb700 1 -- 172.21.15.203:0/202776561 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd00c01c070 con 0x7fd0141041b0 2024-06-06T08:34:51.071 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.070+0000 7fd011ffb700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd0040a2c20 0x7fd0040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:34:51.071 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.070+0000 7fd011ffb700 1 -- 172.21.15.203:0/202776561 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7fd00c0cdd50 con 0x7fd0141041b0 2024-06-06T08:34:51.071 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.071+0000 7fd018fc3700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd0040a2c20 0x7fd0040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:34:51.071 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.071+0000 7fd018fc3700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd0040a2c20 0x7fd0040a50e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd014069eb0 tx=0x7fd00800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:34:51.073 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.075+0000 7fd011ffb700 1 -- 172.21.15.203:0/202776561 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd00c0d1030 con 0x7fd0141041b0 2024-06-06T08:34:51.275 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:34:51.277+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd0140628d0 con 0x7fd0040a2c20 2024-06-06T08:34:51.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:51 smithi203 ceph-mon[36579]: pgmap v240: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-06-06T08:34:51.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:51 smithi203 ceph-mon[36579]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-06-06T08:34:51.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:51 smithi203 ceph-mon[36579]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T08:34:51.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:34:51.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:34:51.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:51 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:51.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:51 smithi176 ceph-mon[36786]: pgmap v240: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-06-06T08:34:51.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:51 smithi176 ceph-mon[36786]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-06-06T08:34:51.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:51 smithi176 ceph-mon[36786]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T08:34:51.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:34:51.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:34:51.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:51 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:51 smithi027 ceph-mon[32335]: pgmap v240: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-06-06T08:34:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:51 smithi027 ceph-mon[32335]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-06-06T08:34:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:51 smithi027 ceph-mon[32335]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T08:34:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T08:34:51.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T08:34:51.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:51 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:34:52.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:52 smithi203 ceph-mon[36579]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:34:52.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:52 smithi203 ceph-mon[36579]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T08:34:52.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:52 smithi176 ceph-mon[36786]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:34:52.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:52 smithi176 ceph-mon[36786]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T08:34:52.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:52 smithi027 ceph-mon[32335]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:34:52.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:52 smithi027 ceph-mon[32335]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T08:34:53.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:53 smithi203 ceph-mon[36579]: pgmap v243: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-06-06T08:34:53.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:53 smithi176 ceph-mon[36786]: pgmap v243: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-06-06T08:34:53.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:53 smithi027 ceph-mon[32335]: pgmap v243: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-06-06T08:34:54.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:54 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/640009281' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5eef-bc5d-4858-abf0-42d889571c21"}]: dispatch 2024-06-06T08:34:54.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:54 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5eef-bc5d-4858-abf0-42d889571c21"}]: dispatch 2024-06-06T08:34:54.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:54 smithi203 ceph-mon[36579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c7f5eef-bc5d-4858-abf0-42d889571c21"}]': finished 2024-06-06T08:34:54.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:54 smithi203 ceph-mon[36579]: osdmap e77: 12 total, 11 up, 12 in 2024-06-06T08:34:54.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:54 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:34:54.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:54 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/640009281' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5eef-bc5d-4858-abf0-42d889571c21"}]: dispatch 2024-06-06T08:34:54.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:54 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5eef-bc5d-4858-abf0-42d889571c21"}]: dispatch 2024-06-06T08:34:54.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:54 smithi176 ceph-mon[36786]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c7f5eef-bc5d-4858-abf0-42d889571c21"}]': finished 2024-06-06T08:34:54.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:54 smithi176 ceph-mon[36786]: osdmap e77: 12 total, 11 up, 12 in 2024-06-06T08:34:54.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:54 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:34:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:54 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/640009281' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5eef-bc5d-4858-abf0-42d889571c21"}]: dispatch 2024-06-06T08:34:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:54 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c7f5eef-bc5d-4858-abf0-42d889571c21"}]: dispatch 2024-06-06T08:34:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:54 smithi027 ceph-mon[32335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c7f5eef-bc5d-4858-abf0-42d889571c21"}]': finished 2024-06-06T08:34:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:54 smithi027 ceph-mon[32335]: osdmap e77: 12 total, 11 up, 12 in 2024-06-06T08:34:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:54 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:34:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:55 smithi203 ceph-mon[36579]: pgmap v245: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-06-06T08:34:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:55 smithi203 ceph-mon[36579]: from='client.? 172.21.15.203:0/898661039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:34:55.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:55 smithi176 ceph-mon[36786]: pgmap v245: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-06-06T08:34:55.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:55 smithi176 ceph-mon[36786]: from='client.? 172.21.15.203:0/898661039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:34:55.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:55 smithi027 ceph-mon[32335]: pgmap v245: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-06-06T08:34:55.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:55 smithi027 ceph-mon[32335]: from='client.? 172.21.15.203:0/898661039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T08:34:56.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:56 smithi027 ceph-mon[32335]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-06-06T08:34:56.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:56 smithi027 ceph-mon[32335]: Cluster is now healthy 2024-06-06T08:34:56.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:56 smithi203 ceph-mon[36579]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-06-06T08:34:56.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:56 smithi203 ceph-mon[36579]: Cluster is now healthy 2024-06-06T08:34:57.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:56 smithi176 ceph-mon[36786]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-06-06T08:34:57.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:56 smithi176 ceph-mon[36786]: Cluster is now healthy 2024-06-06T08:34:57.779 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:57 smithi203 ceph-mon[36579]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T08:34:57.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:57 smithi027 ceph-mon[32335]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T08:34:58.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:57 smithi176 ceph-mon[36786]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T08:34:59.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:34:59 smithi027 ceph-mon[32335]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:34:59.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:34:59 smithi203 ceph-mon[36579]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:35:00.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:34:59 smithi176 ceph-mon[36786]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T08:35:01.655 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:01 smithi203 ceph-mon[36579]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T08:35:01.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:01 smithi027 ceph-mon[32335]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T08:35:02.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:01 smithi176 ceph-mon[36786]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T08:35:02.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T08:35:02.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:02.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:02 smithi203 ceph-mon[36579]: Deploying daemon osd.11 on smithi203 2024-06-06T08:35:02.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T08:35:02.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:02.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:02 smithi027 ceph-mon[32335]: Deploying daemon osd.11 on smithi203 2024-06-06T08:35:03.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T08:35:03.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:03.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:02 smithi176 ceph-mon[36786]: Deploying daemon osd.11 on smithi203 2024-06-06T08:35:03.591 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:03 smithi203 ceph-mon[36579]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:35:03.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:03 smithi027 ceph-mon[32335]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:35:04.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:03 smithi176 ceph-mon[36786]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T08:35:05.891 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:05 smithi203 ceph-mon[36579]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-06-06T08:35:05.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:05 smithi027 ceph-mon[32335]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-06-06T08:35:06.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:05 smithi176 ceph-mon[36786]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-06-06T08:35:07.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:07 smithi027 ceph-mon[32335]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T08:35:07.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:07 smithi203 ceph-mon[36579]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T08:35:08.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:07 smithi176 ceph-mon[36786]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T08:35:09.182 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:09.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:09.183 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:35:09.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:09.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:09.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:35:09.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:09.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:09.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:35:10.107 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:09 smithi203 ceph-mon[36579]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:10.157 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:09 smithi176 ceph-mon[36786]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:10.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:09 smithi027 ceph-mon[32335]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:11.397 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 11 on host 'smithi203' 2024-06-06T08:35:11.397 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.397+0000 7fd011ffb700 1 -- 172.21.15.203:0/202776561 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd0140628d0 con 0x7fd0040a2c20 2024-06-06T08:35:11.399 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd0040a2c20 msgr2=0x7fd0040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:11.399 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd0040a2c20 0x7fd0040a50e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd014069eb0 tx=0x7fd00800b5b0 comp rx=0 tx=0).stop 2024-06-06T08:35:11.399 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0141041b0 msgr2=0x7fd01419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0141041b0 0x7fd01419fca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00b9a0 tx=0x7fd00c00bcb0 comp rx=0 tx=0).stop 2024-06-06T08:35:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 shutdown_connections 2024-06-06T08:35:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd0040a2c20 0x7fd0040a50e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0141041b0 0x7fd01419fca0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd014100790 0x7fd0141986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 --2- 172.21.15.203:0/202776561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd014103810 0x7fd014198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 >> 172.21.15.203:0/202776561 conn(0x7fd0140fc0f0 msgr2=0x7fd014106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 shutdown_connections 2024-06-06T08:35:11.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:11.400+0000 7fd01b227700 1 -- 172.21.15.203:0/202776561 wait complete. 2024-06-06T08:35:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:11 smithi203 ceph-mon[36579]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:11 smithi203 ceph-mon[36579]: from='osd.11 [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T08:35:12.034 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:11 smithi203 ceph-mon[36579]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T08:35:12.061 DEBUG:teuthology.orchestra.run.smithi203:osd.11> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.11.service 2024-06-06T08:35:12.064 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-06T08:35:12.064 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-06T08:35:12.096 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:11 smithi027 ceph-mon[32335]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:12.096 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:12.096 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:12.096 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:11 smithi027 ceph-mon[32335]: from='osd.11 [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T08:35:12.096 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:11 smithi027 ceph-mon[32335]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T08:35:12.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:11 smithi176 ceph-mon[36786]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:12.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:12.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:12.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:11 smithi176 ceph-mon[36786]: from='osd.11 [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T08:35:12.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:11 smithi176 ceph-mon[36786]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T08:35:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:13 smithi027 ceph-mon[32335]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T08:35:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:13 smithi027 ceph-mon[32335]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T08:35:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:13 smithi027 ceph-mon[32335]: from='osd.11 [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:35:13.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:13 smithi027 ceph-mon[32335]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:35:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:13 smithi203 ceph-mon[36579]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T08:35:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:13 smithi203 ceph-mon[36579]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T08:35:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:13 smithi203 ceph-mon[36579]: from='osd.11 [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:35:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:13 smithi203 ceph-mon[36579]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:35:13.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:13.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:13.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:13 smithi176 ceph-mon[36786]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T08:35:13.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:13 smithi176 ceph-mon[36786]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T08:35:13.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:13.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:13 smithi176 ceph-mon[36786]: from='osd.11 [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:35:13.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:13 smithi176 ceph-mon[36786]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-06-06T08:35:14.131 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:35:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:35:13.791+0000 7fc7780dd700 -1 osd.11 0 waiting for initial osdmap 2024-06-06T08:35:14.131 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:35:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:35:13.803+0000 7fc76fec8700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T08:35:14.327 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:14 smithi027 ceph-mon[32335]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:14.328 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:14 smithi027 ceph-mon[32335]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:35:14.328 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:14 smithi027 ceph-mon[32335]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T08:35:14.328 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:14.328 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:14 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:14.328 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:14 smithi027 ceph-mon[32335]: from='osd.11 ' entity='osd.11' 2024-06-06T08:35:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 -- 172.21.15.27:0/2304436664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c00102f90 msgr2=0x7f2c00109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 --2- 172.21.15.27:0/2304436664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c00102f90 0x7f2c00109840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800b0d0 tx=0x7f2bf800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 -- 172.21.15.27:0/2304436664 shutdown_connections 2024-06-06T08:35:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 --2- 172.21.15.27:0/2304436664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c00102f90 0x7f2c00109840 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 --2- 172.21.15.27:0/2304436664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c001013f0 0x7f2c00101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 --2- 172.21.15.27:0/2304436664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c001025f0 0x7f2c00102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 -- 172.21.15.27:0/2304436664 >> 172.21.15.27:0/2304436664 conn(0x7f2c000fc990 msgr2=0x7f2c000fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 -- 172.21.15.27:0/2304436664 shutdown_connections 2024-06-06T08:35:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 -- 172.21.15.27:0/2304436664 wait complete. 2024-06-06T08:35:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.338+0000 7f2c0568d700 1 Processor -- start 2024-06-06T08:35:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2c0568d700 1 -- start start 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2c0568d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c001013f0 0x7f2c0006b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2c0568d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c001025f0 0x7f2c0019d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2c0568d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c00102f90 0x7f2c0019f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2c0568d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0010cfb0 con 0x7f2c001013f0 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2c0568d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0010ce30 con 0x7f2c001025f0 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2c0568d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0010d130 con 0x7f2c00102f90 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c001013f0 0x7f2c0006b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c001013f0 0x7f2c0006b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56092/0 (socket says 172.21.15.27:56092) 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfeffd700 1 -- 172.21.15.27:0/3582236766 learned_addr learned my addr 172.21.15.27:0/3582236766 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfe7fc700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c001025f0 0x7f2c0019d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:14.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfeffd700 1 -- 172.21.15.27:0/3582236766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c00102f90 msgr2=0x7f2c0019f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bff7fe700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c00102f90 0x7f2c0019f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfeffd700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c00102f90 0x7f2c0019f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfeffd700 1 -- 172.21.15.27:0/3582236766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c001025f0 msgr2=0x7f2c0019d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfeffd700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c001025f0 0x7f2c0019d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfeffd700 1 -- 172.21.15.27:0/3582236766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c001a0270 con 0x7f2c001013f0 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bff7fe700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c00102f90 0x7f2c0019f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.339+0000 7f2bfeffd700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c001013f0 0x7f2c0006b720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000d980 tx=0x7f2bf000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.340+0000 7f2be7fff700 1 -- 172.21.15.27:0/3582236766 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0017030 con 0x7f2c001013f0 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.340+0000 7f2be7fff700 1 -- 172.21.15.27:0/3582236766 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0004500 con 0x7f2c001013f0 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.340+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c001a0560 con 0x7f2c001013f0 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.340+0000 7f2be7fff700 1 -- 172.21.15.27:0/3582236766 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0003aa0 con 0x7f2c001013f0 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.340+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c001a0aa0 con 0x7f2c001013f0 2024-06-06T08:35:14.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.341+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c00069180 con 0x7f2c001013f0 2024-06-06T08:35:14.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.342+0000 7f2be7fff700 1 -- 172.21.15.27:0/3582236766 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2bf001a070 con 0x7f2c001013f0 2024-06-06T08:35:14.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.343+0000 7f2be7fff700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2be80a2c20 0x7f2be80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:14.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.343+0000 7f2bfe7fc700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2be80a2c20 0x7f2be80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:14.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.343+0000 7f2be7fff700 1 -- 172.21.15.27:0/3582236766 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f2bf00ce810 con 0x7f2c001013f0 2024-06-06T08:35:14.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.343+0000 7f2bfe7fc700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2be80a2c20 0x7f2be80a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c0006bca0 tx=0x7f2bf40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:14.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.347+0000 7f2be7fff700 1 -- 172.21.15.27:0/3582236766 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2bf009bc90 con 0x7f2c001013f0 2024-06-06T08:35:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:14 smithi203 ceph-mon[36579]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:14 smithi203 ceph-mon[36579]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:35:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:14 smithi203 ceph-mon[36579]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T08:35:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:14 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:14 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:14 smithi203 ceph-mon[36579]: from='osd.11 ' entity='osd.11' 2024-06-06T08:35:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:14 smithi176 ceph-mon[36786]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:14 smithi176 ceph-mon[36786]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-06-06T08:35:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:14 smithi176 ceph-mon[36786]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T08:35:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:14 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:14 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:14 smithi176 ceph-mon[36786]: from='osd.11 ' entity='osd.11' 2024-06-06T08:35:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.553+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2c0002dd20 con 0x7f2c001013f0 2024-06-06T08:35:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.554+0000 7f2be7fff700 1 -- 172.21.15.27:0/3582236766 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f2bf009b590 con 0x7f2c001013f0 2024-06-06T08:35:14.552 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:35:14.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.556+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2be80a2c20 msgr2=0x7f2be80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.556+0000 7f2c0568d700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2be80a2c20 0x7f2be80a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c0006bca0 tx=0x7f2bf40073d0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.556+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c001013f0 msgr2=0x7f2c0006b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.556+0000 7f2c0568d700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c001013f0 0x7f2c0006b720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000d980 tx=0x7f2bf000dc90 comp rx=0 tx=0).stop 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.557+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 shutdown_connections 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.557+0000 7f2c0568d700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2be80a2c20 0x7f2be80a50e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.557+0000 7f2c0568d700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c00102f90 0x7f2c0019f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.557+0000 7f2c0568d700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c001013f0 0x7f2c0006b720 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.557+0000 7f2c0568d700 1 --2- 172.21.15.27:0/3582236766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c001025f0 0x7f2c0019d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.557+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 >> 172.21.15.27:0/3582236766 conn(0x7f2c000fc990 msgr2=0x7f2c00105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.557+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 shutdown_connections 2024-06-06T08:35:14.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:14.557+0000 7f2c0568d700 1 -- 172.21.15.27:0/3582236766 wait complete. 2024-06-06T08:35:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:15 smithi027 ceph-mon[32335]: purged_snaps scrub starts 2024-06-06T08:35:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:15 smithi027 ceph-mon[32335]: purged_snaps scrub ok 2024-06-06T08:35:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:15 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3582236766' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T08:35:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:15 smithi027 ceph-mon[32335]: osd.11 [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] boot 2024-06-06T08:35:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:15 smithi027 ceph-mon[32335]: osdmap e80: 12 total, 12 up, 12 in 2024-06-06T08:35:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:15.252 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1717662887,"num_in_osds":12,"osd_in_since":1717662894,"num_remapped_pgs":0} 2024-06-06T08:35:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:15 smithi203 ceph-mon[36579]: purged_snaps scrub starts 2024-06-06T08:35:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:15 smithi203 ceph-mon[36579]: purged_snaps scrub ok 2024-06-06T08:35:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:15 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3582236766' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T08:35:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:15 smithi203 ceph-mon[36579]: osd.11 [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] boot 2024-06-06T08:35:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:15 smithi203 ceph-mon[36579]: osdmap e80: 12 total, 12 up, 12 in 2024-06-06T08:35:15.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:15.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:15 smithi176 ceph-mon[36786]: purged_snaps scrub starts 2024-06-06T08:35:15.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:15 smithi176 ceph-mon[36786]: purged_snaps scrub ok 2024-06-06T08:35:15.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:15.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:15 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3582236766' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T08:35:15.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:15 smithi176 ceph-mon[36786]: osd.11 [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] boot 2024-06-06T08:35:15.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:15 smithi176 ceph-mon[36786]: osdmap e80: 12 total, 12 up, 12 in 2024-06-06T08:35:15.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:35:16.254 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-06T08:35:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:16.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:35:16.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:16 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:35:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:16.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:35:16.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:16 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:35:16.504 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-06-06T08:35:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:16.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:16 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:35:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:17 smithi027 ceph-mon[32335]: Detected new or changed devices on smithi203 2024-06-06T08:35:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:17 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi203 to 4272M 2024-06-06T08:35:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:17 smithi027 ceph-mon[32335]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T08:35:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:35:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:17.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:17 smithi203 ceph-mon[36579]: Detected new or changed devices on smithi203 2024-06-06T08:35:17.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:17 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi203 to 4272M 2024-06-06T08:35:17.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:17 smithi203 ceph-mon[36579]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T08:35:17.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:17.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:35:17.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:17.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:17 smithi176 ceph-mon[36786]: Detected new or changed devices on smithi203 2024-06-06T08:35:17.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:17 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi203 to 4272M 2024-06-06T08:35:17.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:17 smithi176 ceph-mon[36786]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T08:35:17.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:17.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:35:17.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:18.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.348+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4133306964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c010a3f0 msgr2=0x7fc0c010c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:18.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.348+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4133306964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c010a3f0 0x7fc0c010c860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800b0a0 tx=0x7fc0b800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.348+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4133306964 shutdown_connections 2024-06-06T08:35:18.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.348+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4133306964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c010a3f0 0x7fc0c010c860 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.348+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4133306964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c0076440 0x7fc0c00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.348+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4133306964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0074e70 0x7fc0c00752f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.348+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4133306964 >> 172.21.15.27:0/4133306964 conn(0x7fc0c00fda90 msgr2=0x7fc0c00ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.348+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4133306964 shutdown_connections 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.348+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4133306964 wait complete. 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c7f55700 1 Processor -- start 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c7f55700 1 -- start start 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c7f55700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0074e70 0x7fc0c019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c7f55700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0076440 0x7fc0c019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c7f55700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c010a3f0 0x7fc0c01a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c7f55700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0110310 con 0x7fc0c0076440 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c7f55700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0110190 con 0x7fc0c0074e70 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c7f55700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0110490 con 0x7fc0c010a3f0 2024-06-06T08:35:18.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c54f0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0076440 0x7fc0c019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c54f0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0076440 0x7fc0c019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56106/0 (socket says 172.21.15.27:56106) 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.349+0000 7fc0c54f0700 1 -- 172.21.15.27:0/4289337969 learned_addr learned my addr 172.21.15.27:0/4289337969 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c5cf1700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0074e70 0x7fc0c019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c64f2700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c010a3f0 0x7fc0c01a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c54f0700 1 -- 172.21.15.27:0/4289337969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c010a3f0 msgr2=0x7fc0c01a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c54f0700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c010a3f0 0x7fc0c01a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c54f0700 1 -- 172.21.15.27:0/4289337969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0074e70 msgr2=0x7fc0c019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c54f0700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0074e70 0x7fc0c019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c54f0700 1 -- 172.21.15.27:0/4289337969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c01a24c0 con 0x7fc0c0076440 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c54f0700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0076440 0x7fc0c019ada0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c9e0 tx=0x7fc0b400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/4289337969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4007ac0 con 0x7fc0c0076440 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/4289337969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b4007c60 con 0x7fc0c0076440 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c01a27b0 con 0x7fc0c0076440 2024-06-06T08:35:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/4289337969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4018730 con 0x7fc0c0076440 2024-06-06T08:35:18.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.350+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c01a2cf0 con 0x7fc0c0076440 2024-06-06T08:35:18.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.351+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c0069180 con 0x7fc0c0076440 2024-06-06T08:35:18.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.352+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/4289337969 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc0b40188d0 con 0x7fc0c0076440 2024-06-06T08:35:18.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.353+0000 7fc0b2ffd700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2c20 0x7fc0ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:18.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.353+0000 7fc0c5cf1700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2c20 0x7fc0ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:18.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.353+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/4289337969 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc0b40cee00 con 0x7fc0c0076440 2024-06-06T08:35:18.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.354+0000 7fc0c5cf1700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2c20 0x7fc0ac0a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009a10 tx=0x7fc0bc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:18.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.357+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/4289337969 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0b4014030 con 0x7fc0c0076440 2024-06-06T08:35:18.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:18 smithi027 ceph-mon[32335]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:18 smithi203 ceph-mon[36579]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:18.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:18 smithi176 ceph-mon[36786]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:18.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.565+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc0c00630a0 con 0x7fc0c0076440 2024-06-06T08:35:18.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.565+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/4289337969 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fc0b409c290 con 0x7fc0c0076440 2024-06-06T08:35:18.564 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:35:18.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2c20 msgr2=0x7fc0ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:18.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2c20 0x7fc0ac0a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009a10 tx=0x7fc0bc009190 comp rx=0 tx=0).stop 2024-06-06T08:35:18.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0076440 msgr2=0x7fc0c019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:18.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0076440 0x7fc0c019ada0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c9e0 tx=0x7fc0b400ccf0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 shutdown_connections 2024-06-06T08:35:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2c20 0x7fc0ac0a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c010a3f0 0x7fc0c01a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0074e70 0x7fc0c019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 --2- 172.21.15.27:0/4289337969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0076440 0x7fc0c019ada0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.567+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 >> 172.21.15.27:0/4289337969 conn(0x7fc0c00fda90 msgr2=0x7fc0c0106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.568+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 shutdown_connections 2024-06-06T08:35:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:18.568+0000 7fc0c7f55700 1 -- 172.21.15.27:0/4289337969 wait complete. 2024-06-06T08:35:19.335 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1717662914,"num_in_osds":12,"osd_in_since":1717662894,"num_remapped_pgs":0} 2024-06-06T08:35:19.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:35:19.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:19 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4289337969' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T08:35:19.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:19 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4289337969' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T08:35:19.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:19 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4289337969' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T08:35:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:20.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:20 smithi027 ceph-mon[32335]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:20.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:20 smithi203 ceph-mon[36579]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:20.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:20 smithi176 ceph-mon[36786]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:21.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:21 smithi027 ceph-mon[32335]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:21.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.459+0000 7fd191dd6700 1 -- 172.21.15.27:0/1430760953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c101620 msgr2=0x7fd18c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:21.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.459+0000 7fd191dd6700 1 --2- 172.21.15.27:0/1430760953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c101620 0x7fd18c101a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0099c0 tx=0x7fd17c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.459+0000 7fd191dd6700 1 -- 172.21.15.27:0/1430760953 shutdown_connections 2024-06-06T08:35:21.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.459+0000 7fd191dd6700 1 --2- 172.21.15.27:0/1430760953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd18c1066b0 0x7fd18c10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.459+0000 7fd191dd6700 1 --2- 172.21.15.27:0/1430760953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c101620 0x7fd18c101a00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.459+0000 7fd191dd6700 1 --2- 172.21.15.27:0/1430760953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c101fd0 0x7fd18c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.459+0000 7fd191dd6700 1 -- 172.21.15.27:0/1430760953 >> 172.21.15.27:0/1430760953 conn(0x7fd18c078360 msgr2=0x7fd18c078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:21.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.459+0000 7fd191dd6700 1 -- 172.21.15.27:0/1430760953 shutdown_connections 2024-06-06T08:35:21.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.459+0000 7fd191dd6700 1 -- 172.21.15.27:0/1430760953 wait complete. 2024-06-06T08:35:21.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd191dd6700 1 Processor -- start 2024-06-06T08:35:21.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd191dd6700 1 -- start start 2024-06-06T08:35:21.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd191dd6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c101620 0x7fd18c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd191dd6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd18c101fd0 0x7fd18c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd191dd6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c1066b0 0x7fd18c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd191dd6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c111710 con 0x7fd18c1066b0 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd191dd6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c111590 con 0x7fd18c101620 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd191dd6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c111890 con 0x7fd18c101fd0 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd18bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c1066b0 0x7fd18c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd18bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c1066b0 0x7fd18c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40936/0 (socket says 172.21.15.27:40936) 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd18bfff700 1 -- 172.21.15.27:0/782601353 learned_addr learned my addr 172.21.15.27:0/782601353 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.460+0000 7fd18b7fe700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c101620 0x7fd18c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd18affd700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd18c101fd0 0x7fd18c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd18bfff700 1 -- 172.21.15.27:0/782601353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd18c101fd0 msgr2=0x7fd18c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd18bfff700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd18c101fd0 0x7fd18c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd18bfff700 1 -- 172.21.15.27:0/782601353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c101620 msgr2=0x7fd18c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd18bfff700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c101620 0x7fd18c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd18bfff700 1 -- 172.21.15.27:0/782601353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c199250 con 0x7fd18c1066b0 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd18affd700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd18c101fd0 0x7fd18c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd18bfff700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c1066b0 0x7fd18c198af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b970 tx=0x7fd18400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd188ff9700 1 -- 172.21.15.27:0/782601353 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184019070 con 0x7fd18c1066b0 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd188ff9700 1 -- 172.21.15.27:0/782601353 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18400ce10 con 0x7fd18c1066b0 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c199540 con 0x7fd18c1066b0 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd188ff9700 1 -- 172.21.15.27:0/782601353 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184007370 con 0x7fd18c1066b0 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.461+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c1a5800 con 0x7fd18c1066b0 2024-06-06T08:35:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.462+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18c069180 con 0x7fd18c1066b0 2024-06-06T08:35:21.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.463+0000 7fd188ff9700 1 -- 172.21.15.27:0/782601353 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd18401c070 con 0x7fd18c1066b0 2024-06-06T08:35:21.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.464+0000 7fd188ff9700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1780a2c20 0x7fd1780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:21.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.464+0000 7fd18b7fe700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1780a2c20 0x7fd1780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:21.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.464+0000 7fd188ff9700 1 -- 172.21.15.27:0/782601353 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd1840cdf10 con 0x7fd18c1066b0 2024-06-06T08:35:21.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.464+0000 7fd18b7fe700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1780a2c20 0x7fd1780a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd17c000c00 tx=0x7fd17c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:21.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.469+0000 7fd188ff9700 1 -- 172.21.15.27:0/782601353 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd18409b310 con 0x7fd18c1066b0 2024-06-06T08:35:21.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:21 smithi203 ceph-mon[36579]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:21.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:21 smithi176 ceph-mon[36786]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.676+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd18c02dca0 con 0x7fd18c1066b0 2024-06-06T08:35:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.677+0000 7fd188ff9700 1 -- 172.21.15.27:0/782601353 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18431 (secure 0 0 0) 0x7fd18409ac10 con 0x7fd18c1066b0 2024-06-06T08:35:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:35:21.676 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":81,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:35:16.179181+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:35:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1780a2c20 msgr2=0x7fd1780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1780a2c20 0x7fd1780a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd17c000c00 tx=0x7fd17c005fb0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c1066b0 msgr2=0x7fd18c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c1066b0 0x7fd18c198af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b970 tx=0x7fd18400bc80 comp rx=0 tx=0).stop 2024-06-06T08:35:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 shutdown_connections 2024-06-06T08:35:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1780a2c20 0x7fd1780a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd18c1066b0 0x7fd18c198af0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd18c101620 0x7fd18c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 --2- 172.21.15.27:0/782601353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd18c101fd0 0x7fd18c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 >> 172.21.15.27:0/782601353 conn(0x7fd18c078360 msgr2=0x7fd18c06af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 shutdown_connections 2024-06-06T08:35:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:21.680+0000 7fd191dd6700 1 -- 172.21.15.27:0/782601353 wait complete. 2024-06-06T08:35:22.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:22 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/782601353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:35:22.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:22 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/782601353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:35:22.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:22 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/782601353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:35:22.539 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T08:31:22.877337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-06T08:35:22.539 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-06T08:35:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:23.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:23 smithi203 ceph-mon[36579]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:23.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:23 smithi176 ceph-mon[36786]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:23.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:23 smithi027 ceph-mon[32335]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.630+0000 7f1f681cb700 1 -- 172.21.15.27:0/3937755913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 msgr2=0x7f1f6010f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 --2- 172.21.15.27:0/3937755913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 0x7f1f6010f3e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f580099c0 tx=0x7f1f58009cd0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 -- 172.21.15.27:0/3937755913 shutdown_connections 2024-06-06T08:35:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 --2- 172.21.15.27:0/3937755913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 0x7f1f6010f3e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 --2- 172.21.15.27:0/3937755913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f60102770 0x7f1f60102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 --2- 172.21.15.27:0/3937755913 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f60103120 0x7f1f60107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 -- 172.21.15.27:0/3937755913 >> 172.21.15.27:0/3937755913 conn(0x7f1f60079600 msgr2=0x7f1f60079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 -- 172.21.15.27:0/3937755913 shutdown_connections 2024-06-06T08:35:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 -- 172.21.15.27:0/3937755913 wait complete. 2024-06-06T08:35:24.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 Processor -- start 2024-06-06T08:35:24.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.631+0000 7f1f681cb700 1 -- start start 2024-06-06T08:35:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f681cb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f60102770 0x7f1f60074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f681cb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f60103120 0x7f1f60074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f681cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 0x7f1f60075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f681cb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f601127a0 con 0x7f1f60107860 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f681cb700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60112620 con 0x7f1f60102770 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f681cb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60112920 con 0x7f1f60103120 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f66768700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 0x7f1f60075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f66768700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 0x7f1f60075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40950/0 (socket says 172.21.15.27:40950) 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f66768700 1 -- 172.21.15.27:0/120481757 learned_addr learned my addr 172.21.15.27:0/120481757 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f65766700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f60103120 0x7f1f60074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f65f67700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f60102770 0x7f1f60074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f66768700 1 -- 172.21.15.27:0/120481757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f60103120 msgr2=0x7f1f60074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f66768700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f60103120 0x7f1f60074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f66768700 1 -- 172.21.15.27:0/120481757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f60102770 msgr2=0x7f1f60074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f66768700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f60102770 0x7f1f60074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f66768700 1 -- 172.21.15.27:0/120481757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f601a7040 con 0x7f1f60107860 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.632+0000 7f1f65f67700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f60102770 0x7f1f60074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.633+0000 7f1f66768700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 0x7f1f60075210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f58006010 tx=0x7f1f5800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.633+0000 7f1f5effd700 1 -- 172.21.15.27:0/120481757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58027070 con 0x7f1f60107860 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.633+0000 7f1f5effd700 1 -- 172.21.15.27:0/120481757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f58003680 con 0x7f1f60107860 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.633+0000 7f1f5effd700 1 -- 172.21.15.27:0/120481757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58017650 con 0x7f1f60107860 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.633+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f601a72d0 con 0x7f1f60107860 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.633+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f601a7780 con 0x7f1f60107860 2024-06-06T08:35:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.634+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6010aa30 con 0x7f1f60107860 2024-06-06T08:35:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.636+0000 7f1f5effd700 1 -- 172.21.15.27:0/120481757 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1f5802a070 con 0x7f1f60107860 2024-06-06T08:35:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.636+0000 7f1f5effd700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1f480a2ca0 0x7f1f480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.637+0000 7f1f65f67700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1f480a2ca0 0x7f1f480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.637+0000 7f1f5effd700 1 -- 172.21.15.27:0/120481757 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1f580cf970 con 0x7f1f60107860 2024-06-06T08:35:24.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.637+0000 7f1f65f67700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1f480a2ca0 0x7f1f480a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f500093a0 tx=0x7f1f5000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:24.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.640+0000 7f1f5effd700 1 -- 172.21.15.27:0/120481757 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1f58030020 con 0x7f1f60107860 2024-06-06T08:35:24.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.845+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f6010a410 con 0x7f1f60107860 2024-06-06T08:35:24.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.846+0000 7f1f5effd700 1 -- 172.21.15.27:0/120481757 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f1f58013070 con 0x7f1f60107860 2024-06-06T08:35:24.844 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1f480a2ca0 msgr2=0x7f1f480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1f480a2ca0 0x7f1f480a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f500093a0 tx=0x7f1f5000b040 comp rx=0 tx=0).stop 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 msgr2=0x7f1f60075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 0x7f1f60075210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f58006010 tx=0x7f1f5800fa00 comp rx=0 tx=0).stop 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 shutdown_connections 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1f480a2ca0 0x7f1f480a5160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f60107860 0x7f1f60075210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f60102770 0x7f1f60074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 --2- 172.21.15.27:0/120481757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f60103120 0x7f1f60074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:24.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 >> 172.21.15.27:0/120481757 conn(0x7f1f60079600 msgr2=0x7f1f60101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:24.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 shutdown_connections 2024-06-06T08:35:24.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:24.848+0000 7f1f681cb700 1 -- 172.21.15.27:0/120481757 wait complete. 2024-06-06T08:35:25.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:25 smithi027 ceph-mon[32335]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:25.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:25 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/120481757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:35:25.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:25 smithi203 ceph-mon[36579]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:25.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:25 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/120481757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:35:25.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:25 smithi176 ceph-mon[36786]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:25.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:25 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/120481757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:35:25.768 INFO:tasks.cephadm:Adding mds.a on smithi027 2024-06-06T08:35:25.768 INFO:tasks.cephadm:Adding mds.d on smithi027 2024-06-06T08:35:25.768 INFO:tasks.cephadm:Adding mds.g on smithi027 2024-06-06T08:35:25.768 INFO:tasks.cephadm:Adding mds.j on smithi027 2024-06-06T08:35:25.768 INFO:tasks.cephadm:Adding mds.b on smithi176 2024-06-06T08:35:25.769 INFO:tasks.cephadm:Adding mds.e on smithi176 2024-06-06T08:35:25.769 INFO:tasks.cephadm:Adding mds.h on smithi176 2024-06-06T08:35:25.769 INFO:tasks.cephadm:Adding mds.k on smithi176 2024-06-06T08:35:25.769 INFO:tasks.cephadm:Adding mds.c on smithi203 2024-06-06T08:35:25.769 INFO:tasks.cephadm:Adding mds.f on smithi203 2024-06-06T08:35:25.769 INFO:tasks.cephadm:Adding mds.i on smithi203 2024-06-06T08:35:25.769 INFO:tasks.cephadm:Adding mds.l on smithi203 2024-06-06T08:35:25.769 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l' 2024-06-06T08:35:26.017 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:35:27.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 -- 172.21.15.203:0/2029734628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801024f0 msgr2=0x7f5a80102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:27.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 --2- 172.21.15.203:0/2029734628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801024f0 0x7f5a80102910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009990 tx=0x7f5a70009ca0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 -- 172.21.15.203:0/2029734628 shutdown_connections 2024-06-06T08:35:27.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 --2- 172.21.15.203:0/2029734628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a80104090 0x7f5a8010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 --2- 172.21.15.203:0/2029734628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801024f0 0x7f5a80102910 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 --2- 172.21.15.203:0/2029734628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a801036f0 0x7f5a80103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 -- 172.21.15.203:0/2029734628 >> 172.21.15.203:0/2029734628 conn(0x7f5a800fda90 msgr2=0x7f5a800ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:27.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 -- 172.21.15.203:0/2029734628 shutdown_connections 2024-06-06T08:35:27.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 -- 172.21.15.203:0/2029734628 wait complete. 2024-06-06T08:35:27.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.695+0000 7f5a84853700 1 Processor -- start 2024-06-06T08:35:27.696 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a84853700 1 -- start start 2024-06-06T08:35:27.696 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a84853700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a801024f0 0x7f5a80198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:27.696 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a84853700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801036f0 0x7f5a80198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:27.696 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a84853700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a80104090 0x7f5a8019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a84853700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8010e0d0 con 0x7f5a801024f0 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a84853700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8010df50 con 0x7f5a80104090 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a84853700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8010e250 con 0x7f5a801036f0 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a75d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801036f0 0x7f5a80198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a75d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801036f0 0x7f5a80198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55260/0 (socket says 172.21.15.203:55260) 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a75d9b700 1 -- 172.21.15.203:0/2777855885 learned_addr learned my addr 172.21.15.203:0/2777855885 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a7e59c700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a801024f0 0x7f5a80198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a7ed9d700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a80104090 0x7f5a8019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a75d9b700 1 -- 172.21.15.203:0/2777855885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a80104090 msgr2=0x7f5a8019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a75d9b700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a80104090 0x7f5a8019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a75d9b700 1 -- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a801024f0 msgr2=0x7f5a80198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a75d9b700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a801024f0 0x7f5a80198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a75d9b700 1 -- 172.21.15.203:0/2777855885 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a801a03a0 con 0x7f5a801036f0 2024-06-06T08:35:27.697 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a7ed9d700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a80104090 0x7f5a8019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:35:27.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.696+0000 7f5a75d9b700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801036f0 0x7f5a80198c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c950 tx=0x7f5a6800cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:27.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.697+0000 7f5a77fff700 1 -- 172.21.15.203:0/2777855885 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68007ac0 con 0x7f5a801036f0 2024-06-06T08:35:27.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.697+0000 7f5a77fff700 1 -- 172.21.15.203:0/2777855885 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a68004d10 con 0x7f5a801036f0 2024-06-06T08:35:27.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.697+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a801a0690 con 0x7f5a801036f0 2024-06-06T08:35:27.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.697+0000 7f5a77fff700 1 -- 172.21.15.203:0/2777855885 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800f450 con 0x7f5a801036f0 2024-06-06T08:35:27.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.697+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a801a0bd0 con 0x7f5a801036f0 2024-06-06T08:35:27.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.698+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a80069180 con 0x7f5a801036f0 2024-06-06T08:35:27.699 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.699+0000 7f5a77fff700 1 -- 172.21.15.203:0/2777855885 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5a68004750 con 0x7f5a801036f0 2024-06-06T08:35:27.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.699+0000 7f5a77fff700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5a6c0a2c20 0x7f5a6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:27.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.699+0000 7f5a77fff700 1 -- 172.21.15.203:0/2777855885 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5a680cecf0 con 0x7f5a801036f0 2024-06-06T08:35:27.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.700+0000 7f5a7e59c700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5a6c0a2c20 0x7f5a6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:27.700 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.700+0000 7f5a7e59c700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5a6c0a2c20 0x7f5a6c0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5a700039c0 tx=0x7f5a70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:27.701 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.703+0000 7f5a77fff700 1 -- 172.21.15.203:0/2777855885 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a68014030 con 0x7f5a801036f0 2024-06-06T08:35:27.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:27 smithi203 ceph-mon[36579]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:27.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:27 smithi176 ceph-mon[36786]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:27.910 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.911+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}) -- 0x7f5a80062ad0 con 0x7f5a6c0a2c20 2024-06-06T08:35:27.918 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.919+0000 7f5a77fff700 1 -- 172.21.15.203:0/2777855885 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5a80062ad0 con 0x7f5a6c0a2c20 2024-06-06T08:35:27.918 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mds.all update... 2024-06-06T08:35:27.920 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.921+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5a6c0a2c20 msgr2=0x7f5a6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:27.920 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.921+0000 7f5a84853700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5a6c0a2c20 0x7f5a6c0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5a700039c0 tx=0x7f5a70005fb0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.920 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801036f0 msgr2=0x7f5a80198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:27.920 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801036f0 0x7f5a80198c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c950 tx=0x7f5a6800cd10 comp rx=0 tx=0).stop 2024-06-06T08:35:27.920 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 shutdown_connections 2024-06-06T08:35:27.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5a6c0a2c20 0x7f5a6c0a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5a80104090 0x7f5a8019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a801024f0 0x7f5a80198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 --2- 172.21.15.203:0/2777855885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a801036f0 0x7f5a80198c80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:27.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 >> 172.21.15.203:0/2777855885 conn(0x7f5a800fda90 msgr2=0x7f5a80106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:27.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 shutdown_connections 2024-06-06T08:35:27.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-06-06T08:35:27.922+0000 7f5a84853700 1 -- 172.21.15.203:0/2777855885 wait complete. 2024-06-06T08:35:27.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:27 smithi027 ceph-mon[32335]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:28.590 DEBUG:teuthology.orchestra.run.smithi027:mds.a> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.a.service 2024-06-06T08:35:28.594 DEBUG:teuthology.orchestra.run.smithi027:mds.d> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.d.service 2024-06-06T08:35:28.637 DEBUG:teuthology.orchestra.run.smithi027:mds.g> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.g.service 2024-06-06T08:35:28.680 DEBUG:teuthology.orchestra.run.smithi027:mds.j> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.j.service 2024-06-06T08:35:28.723 DEBUG:teuthology.orchestra.run.smithi176:mds.b> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.b.service 2024-06-06T08:35:28.726 DEBUG:teuthology.orchestra.run.smithi176:mds.e> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.e.service 2024-06-06T08:35:28.769 DEBUG:teuthology.orchestra.run.smithi176:mds.h> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.h.service 2024-06-06T08:35:28.812 DEBUG:teuthology.orchestra.run.smithi176:mds.k> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.k.service 2024-06-06T08:35:28.855 DEBUG:teuthology.orchestra.run.smithi203:mds.c> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.c.service 2024-06-06T08:35:28.858 DEBUG:teuthology.orchestra.run.smithi203:mds.f> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.f.service 2024-06-06T08:35:28.901 DEBUG:teuthology.orchestra.run.smithi203:mds.i> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.i.service 2024-06-06T08:35:28.944 DEBUG:teuthology.orchestra.run.smithi203:mds.l> sudo journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.l.service 2024-06-06T08:35:28.987 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-06T08:35:28.987 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:35:29.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:29.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:28 smithi176 ceph-mon[36786]: Deploying daemon mds.l on smithi203 2024-06-06T08:35:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:35:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-06-06T08:35:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:35:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:29.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:35:29.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:29.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:29.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:29.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:29.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:28 smithi027 ceph-mon[32335]: Deploying daemon mds.l on smithi203 2024-06-06T08:35:29.236 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:28 smithi203 ceph-mon[36579]: Deploying daemon mds.l on smithi203 2024-06-06T08:35:30.094 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:29 smithi203 ceph-mon[36579]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:30.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:29 smithi027 ceph-mon[32335]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:30.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:29 smithi176 ceph-mon[36786]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:30.490 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:35:30 smithi203 systemd[1]: Starting Ceph mds.l for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:35:30.749 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:35:30 smithi203 podman[66576]: 2024-06-06 08:35:30.395485121 +0000 UTC m=+0.017592706 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:35:30.749 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:35:30 smithi203 podman[66576]: 2024-06-06 08:35:30.506907529 +0000 UTC m=+0.129015096 container create ca8f63a505d194fde10e36358c7263241cab0dabb7469ccc50742cadb452e580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-06T08:35:30.750 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:35:30 smithi203 podman[66576]: 2024-06-06 08:35:30.751481331 +0000 UTC m=+0.373588912 container init ca8f63a505d194fde10e36358c7263241cab0dabb7469ccc50742cadb452e580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-06T08:35:31.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.215+0000 7ff127af2700 1 -- 172.21.15.27:0/2343382286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1201017f0 msgr2=0x7ff12010f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.215+0000 7ff127af2700 1 --2- 172.21.15.27:0/2343382286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1201017f0 0x7ff12010f260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff11800b0a0 tx=0x7ff11800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.215+0000 7ff127af2700 1 -- 172.21.15.27:0/2343382286 shutdown_connections 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.215+0000 7ff127af2700 1 --2- 172.21.15.27:0/2343382286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1201017f0 0x7ff12010f260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.215+0000 7ff127af2700 1 --2- 172.21.15.27:0/2343382286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff120107550 0x7ff120107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.215+0000 7ff127af2700 1 --2- 172.21.15.27:0/2343382286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff120100e30 0x7ff1201012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.215+0000 7ff127af2700 1 -- 172.21.15.27:0/2343382286 >> 172.21.15.27:0/2343382286 conn(0x7ff1200766b0 msgr2=0x7ff120076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.216+0000 7ff127af2700 1 -- 172.21.15.27:0/2343382286 shutdown_connections 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.216+0000 7ff127af2700 1 -- 172.21.15.27:0/2343382286 wait complete. 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.216+0000 7ff127af2700 1 Processor -- start 2024-06-06T08:35:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.216+0000 7ff127af2700 1 -- start start 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff127af2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120100e30 0x7ff12019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff127af2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1201017f0 0x7ff12019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff127af2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff120107550 0x7ff1201989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff127af2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1201127b0 con 0x7ff120100e30 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff127af2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff120112630 con 0x7ff120107550 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff127af2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff120112930 con 0x7ff1201017f0 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12588e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120100e30 0x7ff12019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12588e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120100e30 0x7ff12019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42412/0 (socket says 172.21.15.27:42412) 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12588e700 1 -- 172.21.15.27:0/1418818815 learned_addr learned my addr 172.21.15.27:0/1418818815 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12508d700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1201017f0 0x7ff12019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12608f700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff120107550 0x7ff1201989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12588e700 1 -- 172.21.15.27:0/1418818815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1201017f0 msgr2=0x7ff12019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12588e700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1201017f0 0x7ff12019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12588e700 1 -- 172.21.15.27:0/1418818815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff120107550 msgr2=0x7ff1201989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12588e700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff120107550 0x7ff1201989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12588e700 1 -- 172.21.15.27:0/1418818815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff120199150 con 0x7ff120100e30 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.217+0000 7ff12588e700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120100e30 0x7ff12019e7e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e950 tx=0x7ff11c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.218+0000 7ff112ffd700 1 -- 172.21.15.27:0/1418818815 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c004230 con 0x7ff120100e30 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.218+0000 7ff112ffd700 1 -- 172.21.15.27:0/1418818815 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff11c0043d0 con 0x7ff120100e30 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.218+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff120199440 con 0x7ff120100e30 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.218+0000 7ff112ffd700 1 -- 172.21.15.27:0/1418818815 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c003dd0 con 0x7ff120100e30 2024-06-06T08:35:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.218+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1201a56b0 con 0x7ff120100e30 2024-06-06T08:35:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.219+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff120069180 con 0x7ff120100e30 2024-06-06T08:35:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.220+0000 7ff112ffd700 1 -- 172.21.15.27:0/1418818815 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff11c00c7e0 con 0x7ff120100e30 2024-06-06T08:35:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.221+0000 7ff112ffd700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff10c0a2c20 0x7ff10c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:31.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.221+0000 7ff12508d700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff10c0a2c20 0x7ff10c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:31.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.221+0000 7ff112ffd700 1 -- 172.21.15.27:0/1418818815 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff11c0ceee0 con 0x7ff120100e30 2024-06-06T08:35:31.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.221+0000 7ff12508d700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff10c0a2c20 0x7ff10c0a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff120076cb0 tx=0x7ff114009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:31.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.226+0000 7ff112ffd700 1 -- 172.21.15.27:0/1418818815 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff11c09c2e0 con 0x7ff120100e30 2024-06-06T08:35:31.240 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:35:30 smithi203 podman[66576]: 2024-06-06 08:35:30.75407823 +0000 UTC m=+0.376185803 container start ca8f63a505d194fde10e36358c7263241cab0dabb7469ccc50742cadb452e580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-06T08:35:31.240 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:35:30 smithi203 ceph-mds[66596]: -- 172.21.15.203:0/1966006183 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf248e7a40 con 0x55cf248ba800 2024-06-06T08:35:31.240 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:35:30 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l[66592]: starting mds.l at 2024-06-06T08:35:31.240 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:35:30 smithi203 bash[66576]: ca8f63a505d194fde10e36358c7263241cab0dabb7469ccc50742cadb452e580 2024-06-06T08:35:31.240 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:35:30 smithi203 systemd[1]: Started Ceph mds.l for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.476+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff120199ed0 con 0x7ff120100e30 2024-06-06T08:35:31.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.477+0000 7ff112ffd700 1 -- 172.21.15.27:0/1418818815 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7ff11c09bbe0 con 0x7ff120100e30 2024-06-06T08:35:31.476 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:35:31.476 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-06T08:35:31.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff10c0a2c20 msgr2=0x7ff10c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff10c0a2c20 0x7ff10c0a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff120076cb0 tx=0x7ff114009260 comp rx=0 tx=0).stop 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120100e30 msgr2=0x7ff12019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120100e30 0x7ff12019e7e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e950 tx=0x7ff11c00ec60 comp rx=0 tx=0).stop 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 shutdown_connections 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff10c0a2c20 0x7ff10c0a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff120107550 0x7ff1201989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120100e30 0x7ff12019e7e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 --2- 172.21.15.27:0/1418818815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1201017f0 0x7ff12019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 >> 172.21.15.27:0/1418818815 conn(0x7ff1200766b0 msgr2=0x7ff1200ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 shutdown_connections 2024-06-06T08:35:31.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:31.479+0000 7ff127af2700 1 -- 172.21.15.27:0/1418818815 wait complete. 2024-06-06T08:35:31.479 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 2 2024-06-06T08:35:31.838 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6832/3789595568,v1:172.21.15.203:6833/3789595568] up:boot 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: fsmap 1 up:standby 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: Deploying daemon mds.k on smithi176 2024-06-06T08:35:31.839 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:31 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1418818815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:32.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:32.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6832/3789595568,v1:172.21.15.203:6833/3789595568] up:boot 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: fsmap 1 up:standby 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: Deploying daemon mds.k on smithi176 2024-06-06T08:35:32.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:31 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1418818815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6832/3789595568,v1:172.21.15.203:6833/3789595568] up:boot 2024-06-06T08:35:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: fsmap 1 up:standby 2024-06-06T08:35:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T08:35:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:32.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:32.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:32.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:32.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: Deploying daemon mds.k on smithi176 2024-06-06T08:35:32.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:31 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1418818815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:33.258 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:35:33 smithi176 systemd[1]: Starting Ceph mds.k for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:35:33.623 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:35:33 smithi176 podman[66380]: 2024-06-06 08:35:33.264284889 +0000 UTC m=+0.017421421 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:35:33.623 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:35:33 smithi176 podman[66380]: 2024-06-06 08:35:33.380951613 +0000 UTC m=+0.134088134 container create 2543aa97c256160cba8bcb3c053e8e076c1ae34fb63c552373a8e0eada355dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-06T08:35:33.624 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:35:33 smithi176 podman[66380]: 2024-06-06 08:35:33.625362214 +0000 UTC m=+0.378498729 container init 2543aa97c256160cba8bcb3c053e8e076c1ae34fb63c552373a8e0eada355dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-06T08:35:34.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:33 smithi176 ceph-mon[36786]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:34.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:33 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:34.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:33 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:34.007 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:35:33 smithi176 podman[66380]: 2024-06-06 08:35:33.628526362 +0000 UTC m=+0.381662886 container start 2543aa97c256160cba8bcb3c053e8e076c1ae34fb63c552373a8e0eada355dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:35:34.007 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:35:33 smithi176 ceph-mds[66399]: -- 172.21.15.176:0/1278553628 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558615051a40 con 0x558615024800 2024-06-06T08:35:34.007 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:35:33 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k[66395]: starting mds.k at 2024-06-06T08:35:34.007 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:35:33 smithi176 bash[66380]: 2543aa97c256160cba8bcb3c053e8e076c1ae34fb63c552373a8e0eada355dec 2024-06-06T08:35:34.008 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:35:33 smithi176 systemd[1]: Started Ceph mds.k for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:33 smithi027 ceph-mon[32335]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:33 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:33 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:33 smithi203 ceph-mon[36579]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:33 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:33 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:34.347 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:35:35.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:34 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6832/1587702098,v1:172.21.15.176:6833/1587702098] up:boot 2024-06-06T08:35:35.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:34 smithi027 ceph-mon[32335]: fsmap 2 up:standby 2024-06-06T08:35:35.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T08:35:35.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:35.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:35.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:35.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:34 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:35.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:34 smithi027 ceph-mon[32335]: Deploying daemon mds.j on smithi027 2024-06-06T08:35:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:34 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6832/1587702098,v1:172.21.15.176:6833/1587702098] up:boot 2024-06-06T08:35:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:34 smithi203 ceph-mon[36579]: fsmap 2 up:standby 2024-06-06T08:35:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T08:35:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:34 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:34 smithi203 ceph-mon[36579]: Deploying daemon mds.j on smithi027 2024-06-06T08:35:35.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:34 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6832/1587702098,v1:172.21.15.176:6833/1587702098] up:boot 2024-06-06T08:35:35.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:34 smithi176 ceph-mon[36786]: fsmap 2 up:standby 2024-06-06T08:35:35.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T08:35:35.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:35.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:35.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:35.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:34 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:35.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:34 smithi176 ceph-mon[36786]: Deploying daemon mds.j on smithi027 2024-06-06T08:35:35.713 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:35.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:35 smithi027 ceph-mon[32335]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:36.045 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:35 smithi176 ceph-mon[36786]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:36.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:35 smithi203 ceph-mon[36579]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:37.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:37 smithi027 ceph-mon[32335]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:37.956 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:35:37 smithi027 systemd[1]: Started Ceph mds.j for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:38.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.794+0000 7fa300653700 1 -- 172.21.15.27:0/1422979338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f8101610 msgr2=0x7fa2f81019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:38.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.794+0000 7fa300653700 1 --2- 172.21.15.27:0/1422979338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f8101610 0x7fa2f81019f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800b0d0 tx=0x7fa2e800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.794+0000 7fa300653700 1 -- 172.21.15.27:0/1422979338 shutdown_connections 2024-06-06T08:35:38.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.794+0000 7fa300653700 1 --2- 172.21.15.27:0/1422979338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f81066f0 0x7fa2f810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.794+0000 7fa300653700 1 --2- 172.21.15.27:0/1422979338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f8101610 0x7fa2f81019f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.794+0000 7fa300653700 1 --2- 172.21.15.27:0/1422979338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2f8101fc0 0x7fa2f8106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.794+0000 7fa300653700 1 -- 172.21.15.27:0/1422979338 >> 172.21.15.27:0/1422979338 conn(0x7fa2f8078430 msgr2=0x7fa2f8078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:38.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.794+0000 7fa300653700 1 -- 172.21.15.27:0/1422979338 shutdown_connections 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.794+0000 7fa300653700 1 -- 172.21.15.27:0/1422979338 wait complete. 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa300653700 1 Processor -- start 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa300653700 1 -- start start 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa300653700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2f8101610 0x7fa2f819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa300653700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8101fc0 0x7fa2f819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa300653700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f81066f0 0x7fa2f819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa300653700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8111770 con 0x7fa2f81066f0 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa300653700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81115f0 con 0x7fa2f8101610 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa300653700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81118f0 con 0x7fa2f8101fc0 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa2febf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f81066f0 0x7fa2f819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa2febf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f81066f0 0x7fa2f819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42434/0 (socket says 172.21.15.27:42434) 2024-06-06T08:35:38.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa2febf0700 1 -- 172.21.15.27:0/3554417110 learned_addr learned my addr 172.21.15.27:0/3554417110 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.795+0000 7fa2fdbee700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8101fc0 0x7fa2f819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2fe3ef700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2f8101610 0x7fa2f819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2febf0700 1 -- 172.21.15.27:0/3554417110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8101fc0 msgr2=0x7fa2f819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2febf0700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8101fc0 0x7fa2f819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2febf0700 1 -- 172.21.15.27:0/3554417110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2f8101610 msgr2=0x7fa2f819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2febf0700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2f8101610 0x7fa2f819aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2febf0700 1 -- 172.21.15.27:0/3554417110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f819feb0 con 0x7fa2f81066f0 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2febf0700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f81066f0 0x7fa2f819f770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d980 tx=0x7fa2f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2f77fe700 1 -- 172.21.15.27:0/3554417110 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0017070 con 0x7fa2f81066f0 2024-06-06T08:35:38.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2f77fe700 1 -- 172.21.15.27:0/3554417110 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f0005020 con 0x7fa2f81066f0 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa2f77fe700 1 -- 172.21.15.27:0/3554417110 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0009b50 con 0x7fa2f81066f0 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f81a01a0 con 0x7fa2f81066f0 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.796+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f81a7a20 con 0x7fa2f81066f0 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.797+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f8069180 con 0x7fa2f81066f0 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.798+0000 7fa2f77fe700 1 -- 172.21.15.27:0/3554417110 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa2f0009cf0 con 0x7fa2f81066f0 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.799+0000 7fa2f77fe700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa2e00a2ca0 0x7fa2e00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.799+0000 7fa2fe3ef700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa2e00a2ca0 0x7fa2e00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.799+0000 7fa2f77fe700 1 -- 172.21.15.27:0/3554417110 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa2f00ce990 con 0x7fa2f81066f0 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.799+0000 7fa2fe3ef700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa2e00a2ca0 0x7fa2e00a5160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800b820 tx=0x7fa2e800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:38.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:37.804+0000 7fa2f77fe700 1 -- 172.21.15.27:0/3554417110 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa2f0020020 con 0x7fa2f81066f0 2024-06-06T08:35:38.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.079+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2f81a7bf0 con 0x7fa2f81066f0 2024-06-06T08:35:38.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.079+0000 7fa2f77fe700 1 -- 172.21.15.27:0/3554417110 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fa2f009bd90 con 0x7fa2f81066f0 2024-06-06T08:35:38.079 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:35:38.079 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-06T08:35:38.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa2e00a2ca0 msgr2=0x7fa2e00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:38.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa2e00a2ca0 0x7fa2e00a5160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800b820 tx=0x7fa2e800b040 comp rx=0 tx=0).stop 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f81066f0 msgr2=0x7fa2f819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f81066f0 0x7fa2f819f770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d980 tx=0x7fa2f000dc90 comp rx=0 tx=0).stop 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 shutdown_connections 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa2e00a2ca0 0x7fa2e00a5160 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f81066f0 0x7fa2f819f770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2f8101610 0x7fa2f819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 --2- 172.21.15.27:0/3554417110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2f8101fc0 0x7fa2f819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 >> 172.21.15.27:0/3554417110 conn(0x7fa2f8078430 msgr2=0x7fa2f8107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 shutdown_connections 2024-06-06T08:35:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:38.083+0000 7fa300653700 1 -- 172.21.15.27:0/3554417110 wait complete. 2024-06-06T08:35:38.083 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-06-06T08:35:38.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:37 smithi203 ceph-mon[36579]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:38.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:37 smithi176 ceph-mon[36786]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:38.990 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: Deploying daemon mds.d on smithi027 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3554417110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.27:6834/1517816957,v1:172.21.15.27:6835/1517816957] up:boot 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: fsmap 3 up:standby 2024-06-06T08:35:38.991 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:38 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T08:35:39.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:39.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:39.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:39.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:39.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:39.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:39.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: Deploying daemon mds.d on smithi027 2024-06-06T08:35:39.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3554417110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:39.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.27:6834/1517816957,v1:172.21.15.27:6835/1517816957] up:boot 2024-06-06T08:35:39.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: fsmap 3 up:standby 2024-06-06T08:35:39.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:38 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T08:35:39.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: Deploying daemon mds.d on smithi027 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3554417110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.27:6834/1517816957,v1:172.21.15.27:6835/1517816957] up:boot 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: fsmap 3 up:standby 2024-06-06T08:35:39.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:38 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T08:35:40.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:39 smithi176 ceph-mon[36786]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:40.303 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:39 smithi027 ceph-mon[32335]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:40.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:39 smithi203 ceph-mon[36579]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:40.707 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:35:40 smithi027 systemd[1]: Starting Ceph mds.d for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:35:40.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:35:41.017 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:35:40 smithi027 podman[73071]: 2024-06-06 08:35:40.613386183 +0000 UTC m=+0.017955544 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:35:41.018 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:35:40 smithi027 podman[73071]: 2024-06-06 08:35:40.725061272 +0000 UTC m=+0.129630613 container create c86825e1d59d746a47ee936f69970038324c3ebdb1b2c1a40a1e7e137e4b85bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:35:41.018 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:35:41 smithi027 podman[73071]: 2024-06-06 08:35:41.019880106 +0000 UTC m=+0.424449449 container init c86825e1d59d746a47ee936f69970038324c3ebdb1b2c1a40a1e7e137e4b85bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T08:35:41.213 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:41.337 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:35:41 smithi027 podman[73071]: 2024-06-06 08:35:41.023228385 +0000 UTC m=+0.427797727 container start c86825e1d59d746a47ee936f69970038324c3ebdb1b2c1a40a1e7e137e4b85bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-06T08:35:41.337 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:35:41 smithi027 ceph-mds[73121]: -- 172.21.15.27:0/3165432411 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba86589a40 con 0x55ba8655d000 2024-06-06T08:35:41.337 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:35:41 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d[73104]: starting mds.d at 2024-06-06T08:35:41.337 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:35:41 smithi027 bash[73071]: c86825e1d59d746a47ee936f69970038324c3ebdb1b2c1a40a1e7e137e4b85bf 2024-06-06T08:35:41.337 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:35:41 smithi027 systemd[1]: Started Ceph mds.d for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:42.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:42.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:42.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:42.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.27:6836/1670386995,v1:172.21.15.27:6837/1670386995] up:boot 2024-06-06T08:35:42.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: fsmap 4 up:standby 2024-06-06T08:35:42.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:42 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.27:6836/1670386995,v1:172.21.15.27:6837/1670386995] up:boot 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: fsmap 4 up:standby 2024-06-06T08:35:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:42 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.27:6836/1670386995,v1:172.21.15.27:6837/1670386995] up:boot 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: fsmap 4 up:standby 2024-06-06T08:35:42.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:42 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T08:35:42.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.903+0000 7f1c7232e700 1 -- 172.21.15.27:0/2825647048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c6c101d50 msgr2=0x7f1c6c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.903+0000 7f1c7232e700 1 --2- 172.21.15.27:0/2825647048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c6c101d50 0x7f1c6c10e240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400b0a0 tx=0x7f1c6400b3b0 comp rx=0 tx=0).stop 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.904+0000 7f1c7232e700 1 -- 172.21.15.27:0/2825647048 shutdown_connections 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.904+0000 7f1c7232e700 1 --2- 172.21.15.27:0/2825647048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c6c101d50 0x7f1c6c10e240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.904+0000 7f1c7232e700 1 --2- 172.21.15.27:0/2825647048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c6c107400 0x7f1c6c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.904+0000 7f1c7232e700 1 --2- 172.21.15.27:0/2825647048 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c6c101390 0x7f1c6c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.904+0000 7f1c7232e700 1 -- 172.21.15.27:0/2825647048 >> 172.21.15.27:0/2825647048 conn(0x7f1c6c078340 msgr2=0x7f1c6c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.904+0000 7f1c7232e700 1 -- 172.21.15.27:0/2825647048 shutdown_connections 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.904+0000 7f1c7232e700 1 -- 172.21.15.27:0/2825647048 wait complete. 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.904+0000 7f1c7232e700 1 Processor -- start 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.904+0000 7f1c7232e700 1 -- start start 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c7232e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c6c101390 0x7f1c6c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:42.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c7232e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c6c101d50 0x7f1c6c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c7232e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c6c107400 0x7f1c6c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c7232e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c111670 con 0x7f1c6c101d50 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c7232e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1114f0 con 0x7f1c6c101390 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c7232e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1117f0 con 0x7f1c6c107400 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6bfff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c6c101390 0x7f1c6c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c6c101d50 0x7f1c6c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6bfff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c6c101390 0x7f1c6c19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:36134/0 (socket says 172.21.15.27:36134) 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6bfff700 1 -- 172.21.15.27:0/1770057530 learned_addr learned my addr 172.21.15.27:0/1770057530 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c708cb700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c6c107400 0x7f1c6c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6bfff700 1 -- 172.21.15.27:0/1770057530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c6c107400 msgr2=0x7f1c6c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6bfff700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c6c107400 0x7f1c6c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6bfff700 1 -- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c6c101d50 msgr2=0x7f1c6c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6bfff700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c6c101d50 0x7f1c6c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6bfff700 1 -- 172.21.15.27:0/1770057530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c199150 con 0x7f1c6c101390 2024-06-06T08:35:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6b7fe700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c6c101d50 0x7f1c6c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:35:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.905+0000 7f1c6bfff700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c6c101390 0x7f1c6c19e7e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00d950 tx=0x7f1c5c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.906+0000 7f1c697fa700 1 -- 172.21.15.27:0/1770057530 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c016070 con 0x7f1c6c101390 2024-06-06T08:35:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.906+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c199440 con 0x7f1c6c101390 2024-06-06T08:35:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.906+0000 7f1c697fa700 1 -- 172.21.15.27:0/1770057530 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c5c00fd70 con 0x7f1c6c101390 2024-06-06T08:35:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.906+0000 7f1c697fa700 1 -- 172.21.15.27:0/1770057530 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00e610 con 0x7f1c6c101390 2024-06-06T08:35:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.906+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a56b0 con 0x7f1c6c101390 2024-06-06T08:35:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.906+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6c069180 con 0x7f1c6c101390 2024-06-06T08:35:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.908+0000 7f1c697fa700 1 -- 172.21.15.27:0/1770057530 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1c5c019070 con 0x7f1c6c101390 2024-06-06T08:35:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.909+0000 7f1c697fa700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c540a2c20 0x7f1c540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.909+0000 7f1c6b7fe700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c540a2c20 0x7f1c540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.909+0000 7f1c697fa700 1 -- 172.21.15.27:0/1770057530 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1c5c0ce640 con 0x7f1c6c101390 2024-06-06T08:35:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.909+0000 7f1c6b7fe700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c540a2c20 0x7f1c540a50e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c078920 tx=0x7f1c6000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:42.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:42.912+0000 7f1c697fa700 1 -- 172.21.15.27:0/1770057530 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1c5c09b9a0 con 0x7f1c6c101390 2024-06-06T08:35:43.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.164+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c6c1a58e0 con 0x7f1c6c101390 2024-06-06T08:35:43.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.165+0000 7f1c697fa700 1 -- 172.21.15.27:0/1770057530 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f1c5c09b2a0 con 0x7f1c6c101390 2024-06-06T08:35:43.164 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:35:43.164 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-06T08:35:43.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c540a2c20 msgr2=0x7f1c540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:43.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c540a2c20 0x7f1c540a50e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c078920 tx=0x7f1c6000a300 comp rx=0 tx=0).stop 2024-06-06T08:35:43.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c6c101390 msgr2=0x7f1c6c19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:43.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c6c101390 0x7f1c6c19e7e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00d950 tx=0x7f1c5c00dc60 comp rx=0 tx=0).stop 2024-06-06T08:35:43.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 shutdown_connections 2024-06-06T08:35:43.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f1c540a2c20 0x7f1c540a50e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:43.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c6c107400 0x7f1c6c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:43.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c6c101390 0x7f1c6c19e7e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:43.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 --2- 172.21.15.27:0/1770057530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c6c101d50 0x7f1c6c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:43.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 >> 172.21.15.27:0/1770057530 conn(0x7f1c6c078340 msgr2=0x7f1c6c1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:43.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 shutdown_connections 2024-06-06T08:35:43.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:43.167+0000 7f1c7232e700 1 -- 172.21.15.27:0/1770057530 wait complete. 2024-06-06T08:35:43.169 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 5 2024-06-06T08:35:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:43 smithi027 ceph-mon[32335]: Deploying daemon mds.e on smithi176 2024-06-06T08:35:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:43 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1770057530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:43.459 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:43 smithi176 ceph-mon[36786]: Deploying daemon mds.e on smithi176 2024-06-06T08:35:43.459 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:43 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1770057530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:43.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:43 smithi203 ceph-mon[36579]: Deploying daemon mds.e on smithi176 2024-06-06T08:35:43.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:43 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1770057530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:43.757 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:35:43 smithi176 podman[66698]: 2024-06-06 08:35:43.58061897 +0000 UTC m=+0.017358147 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:35:44.193 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:35:43 smithi176 podman[66698]: 2024-06-06 08:35:43.922659352 +0000 UTC m=+0.359398510 container init 58df65fbc510362d6510c5ca146bdbbab100751a87cae778438e7f2949111561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:35:44.193 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:35:43 smithi176 podman[66698]: 2024-06-06 08:35:43.925303562 +0000 UTC m=+0.362042728 container start 58df65fbc510362d6510c5ca146bdbbab100751a87cae778438e7f2949111561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-06T08:35:44.193 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:35:43 smithi176 ceph-mds[66717]: -- 172.21.15.176:0/452105929 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5555b6687a40 con 0x5555b665b000 2024-06-06T08:35:44.194 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:35:43 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e[66713]: starting mds.e at 2024-06-06T08:35:44.194 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:35:44 smithi176 bash[66698]: 58df65fbc510362d6510c5ca146bdbbab100751a87cae778438e7f2949111561 2024-06-06T08:35:44.194 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:35:44 smithi176 systemd[1]: Started Ceph mds.e for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:44 smithi027 ceph-mon[32335]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:44 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:44 smithi203 ceph-mon[36579]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:44 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:44.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:44 smithi176 ceph-mon[36786]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:44.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:44.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:44.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:44.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:44.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:44.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:44 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:45 smithi027 ceph-mon[32335]: Deploying daemon mds.f on smithi203 2024-06-06T08:35:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:45 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6834/1368203014,v1:172.21.15.176:6835/1368203014] up:boot 2024-06-06T08:35:45.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:45 smithi027 ceph-mon[32335]: fsmap 5 up:standby 2024-06-06T08:35:45.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:45 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:35:45.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:45 smithi203 ceph-mon[36579]: Deploying daemon mds.f on smithi203 2024-06-06T08:35:45.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:45 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6834/1368203014,v1:172.21.15.176:6835/1368203014] up:boot 2024-06-06T08:35:45.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:45 smithi203 ceph-mon[36579]: fsmap 5 up:standby 2024-06-06T08:35:45.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:45 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:35:45.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:45 smithi176 ceph-mon[36786]: Deploying daemon mds.f on smithi203 2024-06-06T08:35:45.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:45 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6834/1368203014,v1:172.21.15.176:6835/1368203014] up:boot 2024-06-06T08:35:45.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:45 smithi176 ceph-mon[36786]: fsmap 5 up:standby 2024-06-06T08:35:45.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:45 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:35:45.865 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:35:46.130 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:46.200 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:35:46 smithi203 systemd[1]: Starting Ceph mds.f for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:35:46.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:46 smithi027 ceph-mon[32335]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:46.481 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:46 smithi176 ceph-mon[36786]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:46.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:46 smithi203 ceph-mon[36579]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:46.490 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:35:46 smithi203 podman[66880]: 2024-06-06 08:35:46.278388188 +0000 UTC m=+0.112849971 container create aaca192b1d3e53627969c00ca43a23f19b30e1b46f2f60127c7e3aa0d8976087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-06T08:35:46.490 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:35:46 smithi203 podman[66880]: 2024-06-06 08:35:46.182838683 +0000 UTC m=+0.017300486 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:35:46.990 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:35:46 smithi203 podman[66880]: 2024-06-06 08:35:46.523900793 +0000 UTC m=+0.358362588 container init aaca192b1d3e53627969c00ca43a23f19b30e1b46f2f60127c7e3aa0d8976087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-06T08:35:46.990 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:35:46 smithi203 podman[66880]: 2024-06-06 08:35:46.526359914 +0000 UTC m=+0.360821706 container start aaca192b1d3e53627969c00ca43a23f19b30e1b46f2f60127c7e3aa0d8976087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:35:46.990 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:35:46 smithi203 ceph-mds[66899]: -- 172.21.15.203:0/1352252656 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559bd2a5a40 con 0x5559bd278c00 2024-06-06T08:35:46.990 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:35:46 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f[66895]: starting mds.f at 2024-06-06T08:35:46.990 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:35:46 smithi203 bash[66880]: aaca192b1d3e53627969c00ca43a23f19b30e1b46f2f60127c7e3aa0d8976087 2024-06-06T08:35:46.990 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:35:46 smithi203 systemd[1]: Started Ceph mds.f for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.831+0000 7fdbd7933700 1 -- 172.21.15.27:0/3961766000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 msgr2=0x7fdbd010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.831+0000 7fdbd7933700 1 --2- 172.21.15.27:0/3961766000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 0x7fdbd010e240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009990 tx=0x7fdbc8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:35:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.831+0000 7fdbd7933700 1 -- 172.21.15.27:0/3961766000 shutdown_connections 2024-06-06T08:35:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.831+0000 7fdbd7933700 1 --2- 172.21.15.27:0/3961766000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 0x7fdbd010e240 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.831+0000 7fdbd7933700 1 --2- 172.21.15.27:0/3961766000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbd0107400 0x7fdbd01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.831+0000 7fdbd7933700 1 --2- 172.21.15.27:0/3961766000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbd0101390 0x7fdbd0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.831+0000 7fdbd7933700 1 -- 172.21.15.27:0/3961766000 >> 172.21.15.27:0/3961766000 conn(0x7fdbd0078340 msgr2=0x7fdbd0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.831+0000 7fdbd7933700 1 -- 172.21.15.27:0/3961766000 shutdown_connections 2024-06-06T08:35:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.831+0000 7fdbd7933700 1 -- 172.21.15.27:0/3961766000 wait complete. 2024-06-06T08:35:47.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:47.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:47.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:47.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:47.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:47.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:47.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:47.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: Deploying daemon mds.c on smithi203 2024-06-06T08:35:47.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6834/2823423002,v1:172.21.15.203:6835/2823423002] up:boot 2024-06-06T08:35:47.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: fsmap 6 up:standby 2024-06-06T08:35:47.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:47 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:35:47.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.832+0000 7fdbd7933700 1 Processor -- start 2024-06-06T08:35:47.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.832+0000 7fdbd7933700 1 -- start start 2024-06-06T08:35:47.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.832+0000 7fdbd7933700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbd0101390 0x7fdbd0103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:47.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.832+0000 7fdbd7933700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 0x7fdbd01035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:47.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.832+0000 7fdbd7933700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbd0107400 0x7fdbd0103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.832+0000 7fdbd7933700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0111710 con 0x7fdbd0101d50 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.832+0000 7fdbd7933700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0111590 con 0x7fdbd0101390 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.832+0000 7fdbd7933700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0111890 con 0x7fdbd0107400 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd4ece700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 0x7fdbd01035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd4ece700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 0x7fdbd01035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33982/0 (socket says 172.21.15.27:33982) 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd4ece700 1 -- 172.21.15.27:0/1503782214 learned_addr learned my addr 172.21.15.27:0/1503782214 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd56cf700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbd0101390 0x7fdbd0103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd5ed0700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbd0107400 0x7fdbd0103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd4ece700 1 -- 172.21.15.27:0/1503782214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbd0107400 msgr2=0x7fdbd0103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd4ece700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbd0107400 0x7fdbd0103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd4ece700 1 -- 172.21.15.27:0/1503782214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbd0101390 msgr2=0x7fdbd0103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd4ece700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbd0101390 0x7fdbd0103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd4ece700 1 -- 172.21.15.27:0/1503782214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd01a7160 con 0x7fdbd0101d50 2024-06-06T08:35:47.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd5ed0700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbd0107400 0x7fdbd0103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:35:47.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd4ece700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 0x7fdbd01035b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400b530 tx=0x7fdbc400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:47.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbce7fc700 1 -- 172.21.15.27:0/1503782214 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4015020 con 0x7fdbd0101d50 2024-06-06T08:35:47.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.833+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd01a7450 con 0x7fdbd0101d50 2024-06-06T08:35:47.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.834+0000 7fdbce7fc700 1 -- 172.21.15.27:0/1503782214 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc4011d10 con 0x7fdbd0101d50 2024-06-06T08:35:47.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.834+0000 7fdbce7fc700 1 -- 172.21.15.27:0/1503782214 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4010e50 con 0x7fdbd0101d50 2024-06-06T08:35:47.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.834+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd01a79e0 con 0x7fdbd0101d50 2024-06-06T08:35:47.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.835+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd0069180 con 0x7fdbd0101d50 2024-06-06T08:35:47.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.836+0000 7fdbce7fc700 1 -- 172.21.15.27:0/1503782214 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdbc4018020 con 0x7fdbd0101d50 2024-06-06T08:35:47.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.837+0000 7fdbce7fc700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdbb80a2dc0 0x7fdbb80a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:47.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.837+0000 7fdbd56cf700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdbb80a2dc0 0x7fdbb80a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:47.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.837+0000 7fdbce7fc700 1 -- 172.21.15.27:0/1503782214 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdbc40ce400 con 0x7fdbd0101d50 2024-06-06T08:35:47.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.837+0000 7fdbd56cf700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdbb80a2dc0 0x7fdbb80a5280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009d10 tx=0x7fdbc0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:47.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:47.841+0000 7fdbce7fc700 1 -- 172.21.15.27:0/1503782214 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdbc401f020 con 0x7fdbd0101d50 2024-06-06T08:35:47.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:47.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:47.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:47.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:47.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:47.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:47.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:47.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: Deploying daemon mds.c on smithi203 2024-06-06T08:35:47.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6834/2823423002,v1:172.21.15.203:6835/2823423002] up:boot 2024-06-06T08:35:47.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: fsmap 6 up:standby 2024-06-06T08:35:47.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:47 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:35:48.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:48.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:48.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:48.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:48.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:48.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:48.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:48.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: Deploying daemon mds.c on smithi203 2024-06-06T08:35:48.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6834/2823423002,v1:172.21.15.203:6835/2823423002] up:boot 2024-06-06T08:35:48.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: fsmap 6 up:standby 2024-06-06T08:35:48.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:47 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:35:48.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.095+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbd01a7c40 con 0x7fdbd0101d50 2024-06-06T08:35:48.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.095+0000 7fdbce7fc700 1 -- 172.21.15.27:0/1503782214 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fdbc409b800 con 0x7fdbd0101d50 2024-06-06T08:35:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:35:48.094 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdbb80a2dc0 msgr2=0x7fdbb80a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdbb80a2dc0 0x7fdbb80a5280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009d10 tx=0x7fdbc0009040 comp rx=0 tx=0).stop 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 msgr2=0x7fdbd01035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 0x7fdbd01035b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400b530 tx=0x7fdbc400b8f0 comp rx=0 tx=0).stop 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 shutdown_connections 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdbb80a2dc0 0x7fdbb80a5280 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbd0107400 0x7fdbd0103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbd0101390 0x7fdbd0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 --2- 172.21.15.27:0/1503782214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbd0101d50 0x7fdbd01035b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.097+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 >> 172.21.15.27:0/1503782214 conn(0x7fdbd0078340 msgr2=0x7fdbd01005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.098+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 shutdown_connections 2024-06-06T08:35:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:48.098+0000 7fdbd7933700 1 -- 172.21.15.27:0/1503782214 wait complete. 2024-06-06T08:35:48.098 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 7 2024-06-06T08:35:48.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:48 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1503782214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:48.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:48 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1503782214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:48.991 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:35:48 smithi203 systemd[1]: Starting Ceph mds.c for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:35:49.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:48 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1503782214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:49.342 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:35:49 smithi203 podman[67184]: 2024-06-06 08:35:48.946425522 +0000 UTC m=+0.016934508 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:35:49.342 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:35:49 smithi203 podman[67184]: 2024-06-06 08:35:49.073924713 +0000 UTC m=+0.144433686 container create 5c47540cd5cd4962bb2ad25d2202859c7502711d59863654cd93aaf93afbd017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-06T08:35:49.342 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:35:49 smithi203 podman[67184]: 2024-06-06 08:35:49.344097882 +0000 UTC m=+0.414606864 container init 5c47540cd5cd4962bb2ad25d2202859c7502711d59863654cd93aaf93afbd017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:35:49.668 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:35:49 smithi203 podman[67184]: 2024-06-06 08:35:49.346743181 +0000 UTC m=+0.417252164 container start 5c47540cd5cd4962bb2ad25d2202859c7502711d59863654cd93aaf93afbd017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:35:49.669 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:35:49 smithi203 ceph-mds[67203]: -- 172.21.15.203:0/1418222961 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc285b9a40 con 0x55cc2858c800 2024-06-06T08:35:49.669 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:35:49 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: starting mds.c at 2024-06-06T08:35:49.669 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:35:49 smithi203 bash[67184]: 5c47540cd5cd4962bb2ad25d2202859c7502711d59863654cd93aaf93afbd017 2024-06-06T08:35:49.669 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:35:49 smithi203 systemd[1]: Started Ceph mds.c for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:49.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:49 smithi027 ceph-mon[32335]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:49.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:49.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:49.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:49.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:49.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:49.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:49 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:49 smithi203 ceph-mon[36579]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:49.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:49 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:50.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:49 smithi176 ceph-mon[36786]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:50.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:50.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:50.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:50.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:50.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:50.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:49 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:50.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:50 smithi176 ceph-mon[36786]: Deploying daemon mds.h on smithi176 2024-06-06T08:35:50.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:50 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6836/266251643,v1:172.21.15.203:6837/266251643] up:boot 2024-06-06T08:35:50.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:50 smithi176 ceph-mon[36786]: fsmap 7 up:standby 2024-06-06T08:35:50.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:50 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:35:50.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:35:50.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:50 smithi027 ceph-mon[32335]: Deploying daemon mds.h on smithi176 2024-06-06T08:35:50.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:50 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6836/266251643,v1:172.21.15.203:6837/266251643] up:boot 2024-06-06T08:35:50.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:50 smithi027 ceph-mon[32335]: fsmap 7 up:standby 2024-06-06T08:35:50.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:50 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:35:50.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:50 smithi203 ceph-mon[36579]: Deploying daemon mds.h on smithi176 2024-06-06T08:35:50.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:50 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6836/266251643,v1:172.21.15.203:6837/266251643] up:boot 2024-06-06T08:35:50.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:50 smithi203 ceph-mon[36579]: fsmap 7 up:standby 2024-06-06T08:35:50.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:50 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:35:51.090 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:51.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:51 smithi027 ceph-mon[32335]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:51.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:51 smithi203 ceph-mon[36579]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:52.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:51 smithi176 ceph-mon[36786]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:52.007 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:35:51 smithi176 systemd[1]: Starting Ceph mds.h for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:35:52.326 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:35:52 smithi176 podman[67003]: 2024-06-06 08:35:51.912202894 +0000 UTC m=+0.017336474 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:35:52.326 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:35:52 smithi176 podman[67003]: 2024-06-06 08:35:52.042205447 +0000 UTC m=+0.147339014 container create b19b4f193920b50869d5e50b51cc84af4d835bad6072dc6535a128b42d596ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-06T08:35:52.756 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:35:52 smithi176 podman[67003]: 2024-06-06 08:35:52.328345612 +0000 UTC m=+0.433479172 container init b19b4f193920b50869d5e50b51cc84af4d835bad6072dc6535a128b42d596ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-06T08:35:52.757 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:35:52 smithi176 podman[67003]: 2024-06-06 08:35:52.331311725 +0000 UTC m=+0.436445302 container start b19b4f193920b50869d5e50b51cc84af4d835bad6072dc6535a128b42d596ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:35:52.757 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:35:52 smithi176 ceph-mds[67022]: -- 172.21.15.176:0/958974853 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ca88fda40 con 0x561ca88d1000 2024-06-06T08:35:52.757 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:35:52 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: starting mds.h at 2024-06-06T08:35:52.757 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:35:52 smithi176 bash[67003]: b19b4f193920b50869d5e50b51cc84af4d835bad6072dc6535a128b42d596ea7 2024-06-06T08:35:52.757 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:35:52 smithi176 systemd[1]: Started Ceph mds.h for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.807+0000 7ffa80030700 1 -- 172.21.15.27:0/2071126601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101390 msgr2=0x7ffa78101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.807+0000 7ffa80030700 1 --2- 172.21.15.27:0/2071126601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101390 0x7ffa78101810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009990 tx=0x7ffa6c009ca0 comp rx=0 tx=0).stop 2024-06-06T08:35:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.807+0000 7ffa80030700 1 -- 172.21.15.27:0/2071126601 shutdown_connections 2024-06-06T08:35:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.807+0000 7ffa80030700 1 --2- 172.21.15.27:0/2071126601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa78101d50 0x7ffa7810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.807+0000 7ffa80030700 1 --2- 172.21.15.27:0/2071126601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa78107400 0x7ffa781077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.807+0000 7ffa80030700 1 --2- 172.21.15.27:0/2071126601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101390 0x7ffa78101810 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.807+0000 7ffa80030700 1 -- 172.21.15.27:0/2071126601 >> 172.21.15.27:0/2071126601 conn(0x7ffa78078340 msgr2=0x7ffa78078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.807+0000 7ffa80030700 1 -- 172.21.15.27:0/2071126601 shutdown_connections 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.807+0000 7ffa80030700 1 -- 172.21.15.27:0/2071126601 wait complete. 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa80030700 1 Processor -- start 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa80030700 1 -- start start 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa80030700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa78101390 0x7ffa78073df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa80030700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101d50 0x7ffa7806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa80030700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa78107400 0x7ffa7806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa80030700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78111690 con 0x7ffa78101d50 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa80030700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78111510 con 0x7ffa78107400 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa80030700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78111810 con 0x7ffa78101390 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa7d5cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101d50 0x7ffa7806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa7d5cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101d50 0x7ffa7806eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56470/0 (socket says 172.21.15.27:56470) 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa7d5cb700 1 -- 172.21.15.27:0/689834447 learned_addr learned my addr 172.21.15.27:0/689834447 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.808+0000 7ffa7e5cd700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa78107400 0x7ffa7806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa7ddcc700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa78101390 0x7ffa78073df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa7e5cd700 1 -- 172.21.15.27:0/689834447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa78101390 msgr2=0x7ffa78073df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa7e5cd700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa78101390 0x7ffa78073df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa7e5cd700 1 -- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101d50 msgr2=0x7ffa7806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa7e5cd700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101d50 0x7ffa7806eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa7e5cd700 1 -- 172.21.15.27:0/689834447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7806f950 con 0x7ffa78107400 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa7d5cb700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101d50 0x7ffa7806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa7e5cd700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa78107400 0x7ffa7806f2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009d10 tx=0x7ffa7000c5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa76ffd700 1 -- 172.21.15.27:0/689834447 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70004830 con 0x7ffa78107400 2024-06-06T08:35:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7806cb90 con 0x7ffa78107400 2024-06-06T08:35:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa76ffd700 1 -- 172.21.15.27:0/689834447 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa70012050 con 0x7ffa78107400 2024-06-06T08:35:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa76ffd700 1 -- 172.21.15.27:0/689834447 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70005280 con 0x7ffa78107400 2024-06-06T08:35:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.809+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7806d120 con 0x7ffa78107400 2024-06-06T08:35:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.811+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78102ab0 con 0x7ffa78107400 2024-06-06T08:35:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.812+0000 7ffa76ffd700 1 -- 172.21.15.27:0/689834447 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffa70012200 con 0x7ffa78107400 2024-06-06T08:35:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.812+0000 7ffa76ffd700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ffa600a2ca0 0x7ffa600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.812+0000 7ffa7ddcc700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ffa600a2ca0 0x7ffa600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.812+0000 7ffa76ffd700 1 -- 172.21.15.27:0/689834447 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ffa700d2860 con 0x7ffa78107400 2024-06-06T08:35:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.813+0000 7ffa7ddcc700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ffa600a2ca0 0x7ffa600a5160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800f540 tx=0x7ffa6800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:52.817+0000 7ffa76ffd700 1 -- 172.21.15.27:0/689834447 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffa7009fbc0 con 0x7ffa78107400 2024-06-06T08:35:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.069+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa78069180 con 0x7ffa78107400 2024-06-06T08:35:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.069+0000 7ffa76ffd700 1 -- 172.21.15.27:0/689834447 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7ffa7009f4c0 con 0x7ffa78107400 2024-06-06T08:35:53.068 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:35:53.068 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-06-06T08:35:53.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ffa600a2ca0 msgr2=0x7ffa600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:53.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ffa600a2ca0 0x7ffa600a5160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800f540 tx=0x7ffa6800b040 comp rx=0 tx=0).stop 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa78107400 msgr2=0x7ffa7806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa78107400 0x7ffa7806f2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009d10 tx=0x7ffa7000c5b0 comp rx=0 tx=0).stop 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 shutdown_connections 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ffa600a2ca0 0x7ffa600a5160 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa78107400 0x7ffa7806f2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa78101390 0x7ffa78073df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 --2- 172.21.15.27:0/689834447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa78101d50 0x7ffa7806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 >> 172.21.15.27:0/689834447 conn(0x7ffa78078340 msgr2=0x7ffa780ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 shutdown_connections 2024-06-06T08:35:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:53.072+0000 7ffa80030700 1 -- 172.21.15.27:0/689834447 wait complete. 2024-06-06T08:35:53.072 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2024-06-06T08:35:53.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:53.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:53.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:53.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:53.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:53.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:53.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:53.704 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: Deploying daemon mds.i on smithi203 2024-06-06T08:35:53.705 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] up:boot 2024-06-06T08:35:53.705 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: fsmap 8 up:standby 2024-06-06T08:35:53.705 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:35:53.705 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:53 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/689834447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:53.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:53.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:53.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:53.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:53.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:53.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:53.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:53.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: Deploying daemon mds.i on smithi203 2024-06-06T08:35:53.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] up:boot 2024-06-06T08:35:53.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: fsmap 8 up:standby 2024-06-06T08:35:53.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:35:53.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:53 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/689834447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:53.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:53.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:53.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: Deploying daemon mds.i on smithi203 2024-06-06T08:35:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] up:boot 2024-06-06T08:35:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: fsmap 8 up:standby 2024-06-06T08:35:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:35:53.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:53 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/689834447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:54.740 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:35:54 smithi203 systemd[1]: Starting Ceph mds.i for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:35:55.135 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:35:54 smithi203 podman[67487]: 2024-06-06 08:35:54.784758415 +0000 UTC m=+0.017398394 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:35:55.135 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:35:54 smithi203 podman[67487]: 2024-06-06 08:35:54.908765722 +0000 UTC m=+0.141405691 container create 6ec074f172ee9fff2ab35fa00ff2544c98cdcf01e525428c10cc690e56fc2f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-06T08:35:55.135 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:35:55 smithi203 podman[67487]: 2024-06-06 08:35:55.136753884 +0000 UTC m=+0.369393852 container init 6ec074f172ee9fff2ab35fa00ff2544c98cdcf01e525428c10cc690e56fc2f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True) 2024-06-06T08:35:55.479 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:35:55 smithi203 podman[67487]: 2024-06-06 08:35:55.139456693 +0000 UTC m=+0.372096671 container start 6ec074f172ee9fff2ab35fa00ff2544c98cdcf01e525428c10cc690e56fc2f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:35:55.479 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:35:55 smithi203 ceph-mds[67506]: -- 172.21.15.203:0/623283014 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562790a3da40 con 0x562790a11000 2024-06-06T08:35:55.479 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:35:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: starting mds.i at 2024-06-06T08:35:55.479 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:35:55 smithi203 bash[67487]: 6ec074f172ee9fff2ab35fa00ff2544c98cdcf01e525428c10cc690e56fc2f77 2024-06-06T08:35:55.480 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:35:55 smithi203 systemd[1]: Started Ceph mds.i for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:55.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:35:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:55 smithi203 ceph-mon[36579]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:55 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:55.755 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:55 smithi027 ceph-mon[32335]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:55.755 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:55.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:55.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:55.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:55.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:55.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:55 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:55.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:55 smithi176 ceph-mon[36786]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:55.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:55.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:55.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:35:55.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:35:55.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:35:55.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:55 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:35:56.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:56 smithi027 ceph-mon[32335]: Deploying daemon mds.g on smithi027 2024-06-06T08:35:56.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:56 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] up:boot 2024-06-06T08:35:56.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:56 smithi027 ceph-mon[32335]: fsmap 9 up:standby 2024-06-06T08:35:56.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:56 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:35:56.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:56 smithi203 ceph-mon[36579]: Deploying daemon mds.g on smithi027 2024-06-06T08:35:56.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:56 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] up:boot 2024-06-06T08:35:56.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:56 smithi203 ceph-mon[36579]: fsmap 9 up:standby 2024-06-06T08:35:56.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:56 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:35:56.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:56 smithi176 ceph-mon[36786]: Deploying daemon mds.g on smithi027 2024-06-06T08:35:56.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:56 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] up:boot 2024-06-06T08:35:56.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:56 smithi176 ceph-mon[36786]: fsmap 9 up:standby 2024-06-06T08:35:56.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:56 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:35:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:35:57.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:57 smithi027 ceph-mon[32335]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:57.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:57 smithi176 ceph-mon[36786]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:57.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:57 smithi203 ceph-mon[36579]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:58.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.939+0000 7f9487fad700 1 -- 172.21.15.27:0/3437743926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 msgr2=0x7f948010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:58.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.939+0000 7f9487fad700 1 --2- 172.21.15.27:0/3437743926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 0x7f948010e240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f947800b0a0 tx=0x7f947800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:35:58.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.939+0000 7f9487fad700 1 -- 172.21.15.27:0/3437743926 shutdown_connections 2024-06-06T08:35:58.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.939+0000 7f9487fad700 1 --2- 172.21.15.27:0/3437743926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 0x7f948010e240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.939+0000 7f9487fad700 1 --2- 172.21.15.27:0/3437743926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9480107400 0x7f94801077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.939+0000 7f9487fad700 1 --2- 172.21.15.27:0/3437743926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9480101390 0x7f9480101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.939+0000 7f9487fad700 1 -- 172.21.15.27:0/3437743926 >> 172.21.15.27:0/3437743926 conn(0x7f9480078340 msgr2=0x7f9480078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.939+0000 7f9487fad700 1 -- 172.21.15.27:0/3437743926 shutdown_connections 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.939+0000 7f9487fad700 1 -- 172.21.15.27:0/3437743926 wait complete. 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.940+0000 7f9487fad700 1 Processor -- start 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.940+0000 7f9487fad700 1 -- start start 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.940+0000 7f9487fad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9480101390 0x7f948019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.940+0000 7f9487fad700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 0x7f948019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.940+0000 7f9487fad700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9480107400 0x7f9480198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.940+0000 7f9487fad700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94801116c0 con 0x7f9480101d50 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.940+0000 7f9487fad700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9480111540 con 0x7f9480107400 2024-06-06T08:35:58.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.940+0000 7f9487fad700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9480111840 con 0x7f9480101390 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.940+0000 7f9485548700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 0x7f948019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485548700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 0x7f948019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56490/0 (socket says 172.21.15.27:56490) 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485548700 1 -- 172.21.15.27:0/1111515892 learned_addr learned my addr 172.21.15.27:0/1111515892 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485d49700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9480101390 0x7f948019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f948654a700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9480107400 0x7f9480198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485548700 1 -- 172.21.15.27:0/1111515892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9480101390 msgr2=0x7f948019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485548700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9480101390 0x7f948019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485548700 1 -- 172.21.15.27:0/1111515892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9480107400 msgr2=0x7f9480198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485548700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9480107400 0x7f9480198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485548700 1 -- 172.21.15.27:0/1111515892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94801990b0 con 0x7f9480101d50 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f948654a700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9480107400 0x7f9480198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485d49700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9480101390 0x7f948019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:35:58.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9485548700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 0x7f948019ecd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f947000c9b0 tx=0x7f947000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:58.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9476ffd700 1 -- 172.21.15.27:0/1111515892 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470007b00 con 0x7f9480101d50 2024-06-06T08:35:58.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94801993a0 con 0x7f9480101d50 2024-06-06T08:35:58.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9476ffd700 1 -- 172.21.15.27:0/1111515892 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9470007ca0 con 0x7f9480101d50 2024-06-06T08:35:58.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9476ffd700 1 -- 172.21.15.27:0/1111515892 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470018780 con 0x7f9480101d50 2024-06-06T08:35:58.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.941+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801a5660 con 0x7f9480101d50 2024-06-06T08:35:58.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.943+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9480069180 con 0x7f9480101d50 2024-06-06T08:35:58.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.943+0000 7f9476ffd700 1 -- 172.21.15.27:0/1111515892 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f947000f450 con 0x7f9480101d50 2024-06-06T08:35:58.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.944+0000 7f9476ffd700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f946c0a2c20 0x7f946c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:35:58.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.944+0000 7f9476ffd700 1 -- 172.21.15.27:0/1111515892 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f94700cec10 con 0x7f9480101d50 2024-06-06T08:35:58.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.944+0000 7f9485d49700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f946c0a2c20 0x7f946c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:35:58.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.945+0000 7f9485d49700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f946c0a2c20 0x7f946c0a50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f947c005e20 tx=0x7f947c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:35:58.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:58.952+0000 7f9476ffd700 1 -- 172.21.15.27:0/1111515892 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9470014030 con 0x7f9480101d50 2024-06-06T08:35:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.205+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94801a5890 con 0x7f9480101d50 2024-06-06T08:35:59.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.205+0000 7f9476ffd700 1 -- 172.21.15.27:0/1111515892 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f947001f300 con 0x7f9480101d50 2024-06-06T08:35:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:35:59.204 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-06T08:35:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f946c0a2c20 msgr2=0x7f946c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:59.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f946c0a2c20 0x7f946c0a50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f947c005e20 tx=0x7f947c005db0 comp rx=0 tx=0).stop 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 msgr2=0x7f948019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 0x7f948019ecd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f947000c9b0 tx=0x7f947000cd70 comp rx=0 tx=0).stop 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 shutdown_connections 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f946c0a2c20 0x7f946c0a50e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9480107400 0x7f9480198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9480101390 0x7f948019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 --2- 172.21.15.27:0/1111515892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9480101d50 0x7f948019ecd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 >> 172.21.15.27:0/1111515892 conn(0x7f9480078340 msgr2=0x7f9480100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 shutdown_connections 2024-06-06T08:35:59.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:35:59.208+0000 7f9487fad700 1 -- 172.21.15.27:0/1111515892 wait complete. 2024-06-06T08:35:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 10 2024-06-06T08:35:59.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:59 smithi176 ceph-mon[36786]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:59.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:35:59 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1111515892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:59.956 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:35:59 smithi027 systemd[1]: Started Ceph mds.g for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:35:59.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:59 smithi027 ceph-mon[32335]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:59.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:35:59 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1111515892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:35:59.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:59 smithi203 ceph-mon[36579]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:35:59.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:35:59 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1111515892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:00.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.27:6838/2317631267,v1:172.21.15.27:6839/2317631267] up:boot 2024-06-06T08:36:00.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: fsmap 10 up:standby 2024-06-06T08:36:00.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T08:36:00.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:00.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:00.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:00.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:36:00.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:36:00.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:00.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:00 smithi027 ceph-mon[32335]: Deploying daemon mds.a on smithi027 2024-06-06T08:36:00.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.27:6838/2317631267,v1:172.21.15.27:6839/2317631267] up:boot 2024-06-06T08:36:00.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: fsmap 10 up:standby 2024-06-06T08:36:00.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T08:36:00.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:00.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:00.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:00.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:36:00.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:36:00.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:00.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:00 smithi176 ceph-mon[36786]: Deploying daemon mds.a on smithi027 2024-06-06T08:36:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.27:6838/2317631267,v1:172.21.15.27:6839/2317631267] up:boot 2024-06-06T08:36:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: fsmap 10 up:standby 2024-06-06T08:36:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T08:36:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:36:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:36:00.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:00.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:00 smithi203 ceph-mon[36579]: Deploying daemon mds.a on smithi027 2024-06-06T08:36:01.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:01 smithi027 ceph-mon[32335]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:01.956 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:36:01 smithi027 systemd[1]: Starting Ceph mds.a for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:36:02.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:01 smithi176 ceph-mon[36786]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:02.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:36:02.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:01 smithi203 ceph-mon[36579]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:02.335 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:36:02 smithi027 podman[74325]: 2024-06-06 08:36:01.965655583 +0000 UTC m=+0.018099268 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:36:02.336 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:36:02 smithi027 podman[74325]: 2024-06-06 08:36:02.109313786 +0000 UTC m=+0.161757457 container create bf14b0f8c55e9df0581459763680854d853dcd4b26c0c8cfa1ec8b8d3e99afd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:36:02.336 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:36:02 smithi027 podman[74325]: 2024-06-06 08:36:02.337422177 +0000 UTC m=+0.389865863 container init bf14b0f8c55e9df0581459763680854d853dcd4b26c0c8cfa1ec8b8d3e99afd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-06T08:36:02.512 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:02.636 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:36:02 smithi027 podman[74325]: 2024-06-06 08:36:02.340955379 +0000 UTC m=+0.393399053 container start bf14b0f8c55e9df0581459763680854d853dcd4b26c0c8cfa1ec8b8d3e99afd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-06T08:36:02.636 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:36:02 smithi027 ceph-mds[74364]: -- 172.21.15.27:0/903179222 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ccc261a40 con 0x558ccc234800 2024-06-06T08:36:02.637 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:36:02 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-a[74358]: starting mds.a at 2024-06-06T08:36:02.637 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:36:02 smithi027 bash[74325]: bf14b0f8c55e9df0581459763680854d853dcd4b26c0c8cfa1ec8b8d3e99afd9 2024-06-06T08:36:02.637 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:36:02 smithi027 systemd[1]: Started Ceph mds.a for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: Deploying daemon mds.b on smithi176 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.27:6840/1085633536,v1:172.21.15.27:6841/1085633536] up:boot 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: fsmap 11 up:standby 2024-06-06T08:36:03.612 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T08:36:03.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: Deploying daemon mds.b on smithi176 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.27:6840/1085633536,v1:172.21.15.27:6841/1085633536] up:boot 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: fsmap 11 up:standby 2024-06-06T08:36:03.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T08:36:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T08:36:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T08:36:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: Deploying daemon mds.b on smithi176 2024-06-06T08:36:03.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.27:6840/1085633536,v1:172.21.15.27:6841/1085633536] up:boot 2024-06-06T08:36:03.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: fsmap 11 up:standby 2024-06-06T08:36:03.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T08:36:04.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.190+0000 7f44c8858700 1 -- 172.21.15.27:0/3279987285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406aba0 msgr2=0x7f44c406af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:04.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.190+0000 7f44c8858700 1 --2- 172.21.15.27:0/3279987285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406aba0 0x7f44c406af80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009990 tx=0x7f44b4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.190+0000 7f44c8858700 1 -- 172.21.15.27:0/3279987285 shutdown_connections 2024-06-06T08:36:04.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.190+0000 7f44c8858700 1 --2- 172.21.15.27:0/3279987285 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44c410efd0 0x7f44c41113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.190+0000 7f44c8858700 1 --2- 172.21.15.27:0/3279987285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406aba0 0x7f44c406af80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.190+0000 7f44c8858700 1 --2- 172.21.15.27:0/3279987285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c406b4c0 0x7f44c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.190+0000 7f44c8858700 1 -- 172.21.15.27:0/3279987285 >> 172.21.15.27:0/3279987285 conn(0x7f44c40773e0 msgr2=0x7f44c40777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:04.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.190+0000 7f44c8858700 1 -- 172.21.15.27:0/3279987285 shutdown_connections 2024-06-06T08:36:04.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.190+0000 7f44c8858700 1 -- 172.21.15.27:0/3279987285 wait complete. 2024-06-06T08:36:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c8858700 1 Processor -- start 2024-06-06T08:36:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c8858700 1 -- start start 2024-06-06T08:36:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c8858700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44c406aba0 0x7f44c419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c8858700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406b4c0 0x7f44c419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c8858700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c410efd0 0x7f44c419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c8858700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4114a20 con 0x7f44c406b4c0 2024-06-06T08:36:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c8858700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41148a0 con 0x7f44c406aba0 2024-06-06T08:36:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c8858700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4114ba0 con 0x7f44c410efd0 2024-06-06T08:36:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c1d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406b4c0 0x7f44c419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c259c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44c406aba0 0x7f44c419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c1d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406b4c0 0x7f44c419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36878/0 (socket says 172.21.15.27:36878) 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c1d9b700 1 -- 172.21.15.27:0/500781805 learned_addr learned my addr 172.21.15.27:0/500781805 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.191+0000 7f44c2d9d700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c410efd0 0x7f44c419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44c259c700 1 -- 172.21.15.27:0/500781805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c410efd0 msgr2=0x7f44c419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44c259c700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c410efd0 0x7f44c419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44c259c700 1 -- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406b4c0 msgr2=0x7f44c419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44c259c700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406b4c0 0x7f44c419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44c259c700 1 -- 172.21.15.27:0/500781805 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c419fe00 con 0x7f44c406aba0 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44c1d9b700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406b4c0 0x7f44c419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44c259c700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44c406aba0 0x7f44c419adf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006010 tx=0x7f44b4003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44b37fe700 1 -- 172.21.15.27:0/500781805 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4027070 con 0x7f44c406aba0 2024-06-06T08:36:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c41a0090 con 0x7f44c406aba0 2024-06-06T08:36:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44b37fe700 1 -- 172.21.15.27:0/500781805 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b40056d0 con 0x7f44c406aba0 2024-06-06T08:36:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44b37fe700 1 -- 172.21.15.27:0/500781805 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4022bd0 con 0x7f44c406aba0 2024-06-06T08:36:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.192+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41a7970 con 0x7f44c406aba0 2024-06-06T08:36:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.193+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c4069180 con 0x7f44c406aba0 2024-06-06T08:36:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.194+0000 7f44b37fe700 1 -- 172.21.15.27:0/500781805 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f44b402a070 con 0x7f44c406aba0 2024-06-06T08:36:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.195+0000 7f44b37fe700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f44ac0a2c20 0x7f44ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.195+0000 7f44c1d9b700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f44ac0a2c20 0x7f44ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.195+0000 7f44b37fe700 1 -- 172.21.15.27:0/500781805 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f44b40d03b0 con 0x7f44c406aba0 2024-06-06T08:36:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.196+0000 7f44c1d9b700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f44ac0a2c20 0x7f44ac0a50e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44c4101030 tx=0x7f44b8007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.199+0000 7f44b37fe700 1 -- 172.21.15.27:0/500781805 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f44b4031020 con 0x7f44c406aba0 2024-06-06T08:36:04.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.452+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44c402dca0 con 0x7f44c406aba0 2024-06-06T08:36:04.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.453+0000 7f44b37fe700 1 -- 172.21.15.27:0/500781805 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f44b4013070 con 0x7f44c406aba0 2024-06-06T08:36:04.452 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:36:04.452 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-06T08:36:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.455+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f44ac0a2c20 msgr2=0x7f44ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f44ac0a2c20 0x7f44ac0a50e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44c4101030 tx=0x7f44b8007480 comp rx=0 tx=0).stop 2024-06-06T08:36:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44c406aba0 msgr2=0x7f44c419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44c406aba0 0x7f44c419adf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006010 tx=0x7f44b4003890 comp rx=0 tx=0).stop 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 shutdown_connections 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f44ac0a2c20 0x7f44ac0a50e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c410efd0 0x7f44c419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44c406aba0 0x7f44c419adf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 --2- 172.21.15.27:0/500781805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c406b4c0 0x7f44c419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 >> 172.21.15.27:0/500781805 conn(0x7f44c40773e0 msgr2=0x7f44c4110550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 shutdown_connections 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:04.456+0000 7f44c8858700 1 -- 172.21.15.27:0/500781805 wait complete. 2024-06-06T08:36:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 12 2024-06-06T08:36:04.981 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:04 smithi176 systemd[1]: Starting Ceph mds.b for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:36:05.256 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:04 smithi176 podman[67305]: 2024-06-06 08:36:04.885193379 +0000 UTC m=+0.017040850 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:36:05.256 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:05 smithi176 podman[67305]: 2024-06-06 08:36:05.006308899 +0000 UTC m=+0.138156380 container create 8210361d2b03276a5f83d713b7df95b16f1ae8ecca65b3dd4c41ded734a7287a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-06T08:36:05.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:05 smithi027 ceph-mon[32335]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:05.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:05 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/500781805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:05.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:05 smithi203 ceph-mon[36579]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:05.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:05 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/500781805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:05.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:05 smithi176 ceph-mon[36786]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:05.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:05 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/500781805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:05.756 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:05 smithi176 podman[67305]: 2024-06-06 08:36:05.267175063 +0000 UTC m=+0.399022536 container init 8210361d2b03276a5f83d713b7df95b16f1ae8ecca65b3dd4c41ded734a7287a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:36:05.757 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:05 smithi176 podman[67305]: 2024-06-06 08:36:05.269967923 +0000 UTC m=+0.401815382 container start 8210361d2b03276a5f83d713b7df95b16f1ae8ecca65b3dd4c41ded734a7287a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-06T08:36:05.757 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:05 smithi176 ceph-mds[67325]: -- 172.21.15.176:0/3392166042 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56463a60ba40 con 0x56463a5dec00 2024-06-06T08:36:05.757 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:05 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b[67321]: starting mds.b at 2024-06-06T08:36:05.757 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:05 smithi176 bash[67305]: 8210361d2b03276a5f83d713b7df95b16f1ae8ecca65b3dd4c41ded734a7287a 2024-06-06T08:36:05.757 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:05 smithi176 systemd[1]: Started Ceph mds.b for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:36:06.676 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:06 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] up:boot 2024-06-06T08:36:06.676 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:06 smithi203 ceph-mon[36579]: fsmap 12 up:standby 2024-06-06T08:36:06.677 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:36:06.677 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.677 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.677 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.677 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.677 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.677 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:06 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:36:06.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:06 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] up:boot 2024-06-06T08:36:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:06 smithi027 ceph-mon[32335]: fsmap 12 up:standby 2024-06-06T08:36:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:36:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:06 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:36:06.734 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:06 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] up:boot 2024-06-06T08:36:06.734 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:06 smithi176 ceph-mon[36786]: fsmap 12 up:standby 2024-06-06T08:36:06.734 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:36:06.734 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.734 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.734 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.735 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.735 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:06.735 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:06 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:36:07.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:36:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:07.703 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:07 smithi027 ceph-mon[32335]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:07.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:07 smithi203 ceph-mon[36579]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:07.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:07 smithi176 ceph-mon[36786]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:08.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:08.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:36:08.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:36:08.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:08 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:36:08.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:08 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:36:08.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:08 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:09.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.027+0000 7fc3d1354700 1 -- 172.21.15.27:0/3897156289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc1030f0 msgr2=0x7fc3cc107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:09.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.027+0000 7fc3d1354700 1 --2- 172.21.15.27:0/3897156289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc1030f0 0x7fc3cc107110 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40099c0 tx=0x7fc3b4009cd0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.027+0000 7fc3d1354700 1 -- 172.21.15.27:0/3897156289 shutdown_connections 2024-06-06T08:36:09.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.027+0000 7fc3d1354700 1 --2- 172.21.15.27:0/3897156289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc107840 0x7fc3cc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.027+0000 7fc3d1354700 1 --2- 172.21.15.27:0/3897156289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3cc102740 0x7fc3cc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.027+0000 7fc3d1354700 1 --2- 172.21.15.27:0/3897156289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc1030f0 0x7fc3cc107110 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.027+0000 7fc3d1354700 1 -- 172.21.15.27:0/3897156289 >> 172.21.15.27:0/3897156289 conn(0x7fc3cc0795f0 msgr2=0x7fc3cc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:09.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.027+0000 7fc3d1354700 1 -- 172.21.15.27:0/3897156289 shutdown_connections 2024-06-06T08:36:09.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.027+0000 7fc3d1354700 1 -- 172.21.15.27:0/3897156289 wait complete. 2024-06-06T08:36:09.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.028+0000 7fc3d1354700 1 Processor -- start 2024-06-06T08:36:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.028+0000 7fc3d1354700 1 -- start start 2024-06-06T08:36:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.028+0000 7fc3d1354700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3cc102740 0x7fc3cc19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.028+0000 7fc3d1354700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc1030f0 0x7fc3cc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.028+0000 7fc3d1354700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc107840 0x7fc3cc1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.028+0000 7fc3d1354700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc112770 con 0x7fc3cc107840 2024-06-06T08:36:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.028+0000 7fc3d1354700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1125f0 con 0x7fc3cc102740 2024-06-06T08:36:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.028+0000 7fc3d1354700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1128f0 con 0x7fc3cc1030f0 2024-06-06T08:36:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc107840 0x7fc3cc1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3caffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3cc102740 0x7fc3cc19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3ca7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc1030f0 0x7fc3cc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3caffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3cc102740 0x7fc3cc19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:33382/0 (socket says 172.21.15.27:33382) 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc107840 0x7fc3cc1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36892/0 (socket says 172.21.15.27:36892) 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3caffd700 1 -- 172.21.15.27:0/2276687417 learned_addr learned my addr 172.21.15.27:0/2276687417 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3cb7fe700 1 -- 172.21.15.27:0/2276687417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc1030f0 msgr2=0x7fc3cc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3cb7fe700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc1030f0 0x7fc3cc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3cb7fe700 1 -- 172.21.15.27:0/2276687417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3cc102740 msgr2=0x7fc3cc19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3cb7fe700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3cc102740 0x7fc3cc19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3cb7fe700 1 -- 172.21.15.27:0/2276687417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc199150 con 0x7fc3cc107840 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3caffd700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3cc102740 0x7fc3cc19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3cb7fe700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc107840 0x7fc3cc1989f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4007f10 tx=0x7fc3c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3c3fff700 1 -- 172.21.15.27:0/2276687417 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4004db0 con 0x7fc3cc107840 2024-06-06T08:36:09.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3c3fff700 1 -- 172.21.15.27:0/2276687417 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c4009e20 con 0x7fc3cc107840 2024-06-06T08:36:09.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc199440 con 0x7fc3cc107840 2024-06-06T08:36:09.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3c3fff700 1 -- 172.21.15.27:0/2276687417 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4015400 con 0x7fc3cc107840 2024-06-06T08:36:09.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.029+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc1a5700 con 0x7fc3cc107840 2024-06-06T08:36:09.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.030+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3cc069180 con 0x7fc3cc107840 2024-06-06T08:36:09.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.032+0000 7fc3c3fff700 1 -- 172.21.15.27:0/2276687417 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc3c400db20 con 0x7fc3cc107840 2024-06-06T08:36:09.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.033+0000 7fc3c3fff700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:09.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.033+0000 7fc3caffd700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:09.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.033+0000 7fc3c3fff700 1 -- 172.21.15.27:0/2276687417 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc3c40ced00 con 0x7fc3cc107840 2024-06-06T08:36:09.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.034+0000 7fc3caffd700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc199b30 tx=0x7fc3bc00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:09.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.037+0000 7fc3c3fff700 1 -- 172.21.15.27:0/2276687417 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc3c4010070 con 0x7fc3cc107840 2024-06-06T08:36:09.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.303+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3cc02dca0 con 0x7fc3cc107840 2024-06-06T08:36:09.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.304+0000 7fc3c3fff700 1 -- 172.21.15.27:0/2276687417 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7fc3c409c100 con 0x7fc3cc107840 2024-06-06T08:36:09.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:36:09.303 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-06T08:36:09.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc3b80a2c20 msgr2=0x7fc3b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:09.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc199b30 tx=0x7fc3bc00b500 comp rx=0 tx=0).stop 2024-06-06T08:36:09.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc107840 msgr2=0x7fc3cc1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:09.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc107840 0x7fc3cc1989f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4007f10 tx=0x7fc3c400d3b0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 shutdown_connections 2024-06-06T08:36:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc3b80a2c20 0x7fc3b80a50e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3cc107840 0x7fc3cc1989f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3cc102740 0x7fc3cc19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 --2- 172.21.15.27:0/2276687417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc1030f0 0x7fc3cc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.306+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 >> 172.21.15.27:0/2276687417 conn(0x7fc3cc0795f0 msgr2=0x7fc3cc100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.307+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 shutdown_connections 2024-06-06T08:36:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:09.307+0000 7fc3d1354700 1 -- 172.21.15.27:0/2276687417 wait complete. 2024-06-06T08:36:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2024-06-06T08:36:09.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:09 smithi027 ceph-mon[32335]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:09.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:09 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2276687417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:09.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:09 smithi203 ceph-mon[36579]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:09.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:09 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2276687417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:09.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:09 smithi176 ceph-mon[36786]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:09.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:09 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2276687417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:10.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-06T08:36:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:11.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.217 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:11 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:11 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:11.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:11 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:12.306 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:12 smithi027 ceph-mon[32335]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:12.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:12 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:12.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:12 smithi203 ceph-mon[36579]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:12.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:12 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:12.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:12 smithi176 ceph-mon[36786]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:12.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:12 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.560+0000 7f0db6e66700 1 -- 172.21.15.27:0/3812530113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 msgr2=0x7f0db010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.560+0000 7f0db6e66700 1 --2- 172.21.15.27:0/3812530113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 0x7f0db010f320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0da800b0a0 tx=0x7f0da800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.560+0000 7f0db6e66700 1 -- 172.21.15.27:0/3812530113 shutdown_connections 2024-06-06T08:36:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.560+0000 7f0db6e66700 1 --2- 172.21.15.27:0/3812530113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 0x7f0db010f320 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.560+0000 7f0db6e66700 1 --2- 172.21.15.27:0/3812530113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0db0108500 0x7f0db01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.560+0000 7f0db6e66700 1 --2- 172.21.15.27:0/3812530113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db006a170 0x7f0db006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.560+0000 7f0db6e66700 1 -- 172.21.15.27:0/3812530113 >> 172.21.15.27:0/3812530113 conn(0x7f0db0078340 msgr2=0x7f0db0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.560+0000 7f0db6e66700 1 -- 172.21.15.27:0/3812530113 shutdown_connections 2024-06-06T08:36:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.560+0000 7f0db6e66700 1 -- 172.21.15.27:0/3812530113 wait complete. 2024-06-06T08:36:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.561+0000 7f0db6e66700 1 Processor -- start 2024-06-06T08:36:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.561+0000 7f0db6e66700 1 -- start start 2024-06-06T08:36:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.561+0000 7f0db6e66700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0db006a170 0x7f0db0101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.561+0000 7f0db6e66700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 0x7f0db01019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0db6e66700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db0108500 0x7f0db0101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0db6e66700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0db0112760 con 0x7f0db006ab30 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0db6e66700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01125e0 con 0x7f0db006a170 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0db6e66700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01128e0 con 0x7f0db0108500 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0daffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 0x7f0db01019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0daffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 0x7f0db01019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60274/0 (socket says 172.21.15.27:60274) 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0daffff700 1 -- 172.21.15.27:0/214690159 learned_addr learned my addr 172.21.15.27:0/214690159 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0db4c02700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0db006a170 0x7f0db0101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0db5403700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db0108500 0x7f0db0101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0daffff700 1 -- 172.21.15.27:0/214690159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db0108500 msgr2=0x7f0db0101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0daffff700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db0108500 0x7f0db0101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0daffff700 1 -- 172.21.15.27:0/214690159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0db006a170 msgr2=0x7f0db0101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0daffff700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0db006a170 0x7f0db0101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0daffff700 1 -- 172.21.15.27:0/214690159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db01a7130 con 0x7f0db006ab30 2024-06-06T08:36:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0daffff700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 0x7f0db01019a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00e9a0 tx=0x7f0d9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0dadffb700 1 -- 172.21.15.27:0/214690159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00ca10 con 0x7f0db006ab30 2024-06-06T08:36:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db01a7420 con 0x7f0db006ab30 2024-06-06T08:36:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.562+0000 7f0dadffb700 1 -- 172.21.15.27:0/214690159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9c00cbb0 con 0x7f0db006ab30 2024-06-06T08:36:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.563+0000 7f0dadffb700 1 -- 172.21.15.27:0/214690159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c018680 con 0x7f0db006ab30 2024-06-06T08:36:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.563+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db01a79b0 con 0x7f0db006ab30 2024-06-06T08:36:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.564+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db0069180 con 0x7f0db006ab30 2024-06-06T08:36:12.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.565+0000 7f0dadffb700 1 -- 172.21.15.27:0/214690159 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0d9c010430 con 0x7f0db006ab30 2024-06-06T08:36:12.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.566+0000 7f0dadffb700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0da00a2c20 0x7f0da00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:12.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.566+0000 7f0db4c02700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0da00a2c20 0x7f0da00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:12.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.566+0000 7f0dadffb700 1 -- 172.21.15.27:0/214690159 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0d9c0cede0 con 0x7f0db006ab30 2024-06-06T08:36:12.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.566+0000 7f0db4c02700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0da00a2c20 0x7f0da00a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0da4007b60 tx=0x7f0da40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:12.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.570+0000 7f0dadffb700 1 -- 172.21.15.27:0/214690159 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0d9c09c1e0 con 0x7f0db006ab30 2024-06-06T08:36:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.779+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0db002dca0 con 0x7f0db006ab30 2024-06-06T08:36:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.779+0000 7f0dadffb700 1 -- 172.21.15.27:0/214690159 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f0d9c09bae0 con 0x7f0db006ab30 2024-06-06T08:36:12.778 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:36:12.778 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-06-06T08:36:12.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:36:12.779 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-06-06T08:36:12.779 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-06-06T08:36:12.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:36:12.779 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0da00a2c20 msgr2=0x7f0da00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0da00a2c20 0x7f0da00a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0da4007b60 tx=0x7f0da40073d0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 msgr2=0x7f0db01019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 0x7f0db01019a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00e9a0 tx=0x7f0d9c00ecb0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 shutdown_connections 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0da00a2c20 0x7f0da00a50e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db0108500 0x7f0db0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0db006a170 0x7f0db0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 --2- 172.21.15.27:0/214690159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0db006ab30 0x7f0db01019a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 >> 172.21.15.27:0/214690159 conn(0x7f0db0078340 msgr2=0x7f0db00fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 shutdown_connections 2024-06-06T08:36:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:12.782+0000 7f0db6e66700 1 -- 172.21.15.27:0/214690159 wait complete. 2024-06-06T08:36:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:36:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:13 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/214690159' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T08:36:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:36:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:13 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/214690159' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T08:36:13.494 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-06T08:36:13.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-06T08:36:13.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:13.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:36:13.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:36:13.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:36:13.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:13 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/214690159' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T08:36:13.745 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:14.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:14 smithi027 ceph-mon[32335]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:14 smithi203 ceph-mon[36579]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:14.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:14 smithi176 ceph-mon[36786]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 -- 172.21.15.27:0/430228390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64101390 msgr2=0x7faa64101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 --2- 172.21.15.27:0/430228390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64101390 0x7faa64101810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faa50009990 tx=0x7faa50009ca0 comp rx=0 tx=0).stop 2024-06-06T08:36:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 -- 172.21.15.27:0/430228390 shutdown_connections 2024-06-06T08:36:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 --2- 172.21.15.27:0/430228390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa64101d50 0x7faa6410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 --2- 172.21.15.27:0/430228390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa64107400 0x7faa641077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 --2- 172.21.15.27:0/430228390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64101390 0x7faa64101810 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 -- 172.21.15.27:0/430228390 >> 172.21.15.27:0/430228390 conn(0x7faa64078340 msgr2=0x7faa64078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 -- 172.21.15.27:0/430228390 shutdown_connections 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 -- 172.21.15.27:0/430228390 wait complete. 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 Processor -- start 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.440+0000 7faa6b047700 1 -- start start 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa6b047700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa64101390 0x7faa64103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa6b047700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa64101d50 0x7faa641035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa6b047700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64107400 0x7faa64103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa6b047700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faa641116f0 con 0x7faa64107400 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa6b047700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faa64111570 con 0x7faa64101d50 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa6b047700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa64111870 con 0x7faa64101390 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa695e4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64107400 0x7faa64103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa695e4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64107400 0x7faa64103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60298/0 (socket says 172.21.15.27:60298) 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa695e4700 1 -- 172.21.15.27:0/3623550502 learned_addr learned my addr 172.21.15.27:0/3623550502 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa68de3700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa64101390 0x7faa64103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa63fff700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa64101d50 0x7faa641035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa695e4700 1 -- 172.21.15.27:0/3623550502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa64101390 msgr2=0x7faa64103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa695e4700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa64101390 0x7faa64103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa695e4700 1 -- 172.21.15.27:0/3623550502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa64101d50 msgr2=0x7faa641035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa695e4700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa64101d50 0x7faa641035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa695e4700 1 -- 172.21.15.27:0/3623550502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa641a7190 con 0x7faa64107400 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa695e4700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64107400 0x7faa64103af0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faa5c007ee0 tx=0x7faa5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa61ffb700 1 -- 172.21.15.27:0/3623550502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c004bb0 con 0x7faa64107400 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa61ffb700 1 -- 172.21.15.27:0/3623550502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5c004d50 con 0x7faa64107400 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.441+0000 7faa61ffb700 1 -- 172.21.15.27:0/3623550502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c005960 con 0x7faa64107400 2024-06-06T08:36:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.442+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa641a7480 con 0x7faa64107400 2024-06-06T08:36:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.442+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa641a7a10 con 0x7faa64107400 2024-06-06T08:36:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.443+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa64069180 con 0x7faa64107400 2024-06-06T08:36:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.444+0000 7faa61ffb700 1 -- 172.21.15.27:0/3623550502 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faa5c01a020 con 0x7faa64107400 2024-06-06T08:36:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.444+0000 7faa61ffb700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7faa540a2ca0 0x7faa540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:15.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:15 smithi027 ceph-mon[32335]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.445+0000 7faa68de3700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7faa540a2ca0 0x7faa540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.445+0000 7faa61ffb700 1 -- 172.21.15.27:0/3623550502 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faa5c0ce220 con 0x7faa64107400 2024-06-06T08:36:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.445+0000 7faa68de3700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7faa540a2ca0 0x7faa540a5160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa58005950 tx=0x7faa5800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.450+0000 7faa61ffb700 1 -- 172.21.15.27:0/3623550502 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faa5c010070 con 0x7faa64107400 2024-06-06T08:36:15.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:15.659+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faa641a7c70 con 0x7faa64107400 2024-06-06T08:36:15.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:15 smithi203 ceph-mon[36579]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:15.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:15 smithi176 ceph-mon[36786]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:16.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.412+0000 7faa61ffb700 1 -- 172.21.15.27:0/3623550502 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7faa5c09b620 con 0x7faa64107400 2024-06-06T08:36:16.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7faa540a2ca0 msgr2=0x7faa540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:16.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7faa540a2ca0 0x7faa540a5160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa58005950 tx=0x7faa5800a300 comp rx=0 tx=0).stop 2024-06-06T08:36:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64107400 msgr2=0x7faa64103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64107400 0x7faa64103af0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faa5c007ee0 tx=0x7faa5c00d3b0 comp rx=0 tx=0).stop 2024-06-06T08:36:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 shutdown_connections 2024-06-06T08:36:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7faa540a2ca0 0x7faa540a5160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa64107400 0x7faa64103af0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa64101390 0x7faa64103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 --2- 172.21.15.27:0/3623550502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa64101d50 0x7faa641035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.415+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 >> 172.21.15.27:0/3623550502 conn(0x7faa64078340 msgr2=0x7faa64100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.416+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 shutdown_connections 2024-06-06T08:36:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:16.416+0000 7faa6b047700 1 -- 172.21.15.27:0/3623550502 wait complete. 2024-06-06T08:36:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-06-06T08:36:16.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:16 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3623550502' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T08:36:16.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:16 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3623550502' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T08:36:16.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:16 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3623550502' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T08:36:17.111 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-06T08:36:17.362 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:17.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:17 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3623550502' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T08:36:17.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:17 smithi027 ceph-mon[32335]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T08:36:17.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:17 smithi027 ceph-mon[32335]: pgmap v291: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:17.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:17 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3623550502' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T08:36:17.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:17 smithi203 ceph-mon[36579]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T08:36:17.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:17 smithi203 ceph-mon[36579]: pgmap v291: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:17.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:17 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3623550502' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T08:36:17.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:17 smithi176 ceph-mon[36786]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T08:36:17.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:17 smithi176 ceph-mon[36786]: pgmap v291: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:18.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:18 smithi027 ceph-mon[32335]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T08:36:18.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:18 smithi027 ceph-mon[32335]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T08:36:18.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:18 smithi203 ceph-mon[36579]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T08:36:18.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:18 smithi203 ceph-mon[36579]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T08:36:18.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:18 smithi176 ceph-mon[36786]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T08:36:18.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:18 smithi176 ceph-mon[36786]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T08:36:19.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.161+0000 7fc4cf88a700 1 -- 172.21.15.27:0/3195525741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 msgr2=0x7fc4c8107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:19.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.161+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/3195525741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 0x7fc4c8107830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800b0a0 tx=0x7fc4b800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.161+0000 7fc4cf88a700 1 -- 172.21.15.27:0/3195525741 shutdown_connections 2024-06-06T08:36:19.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.161+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/3195525741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4c8102ee0 0x7fc4c810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.161+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/3195525741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 0x7fc4c8107830 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.161+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/3195525741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4c8102540 0x7fc4c81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.161+0000 7fc4cf88a700 1 -- 172.21.15.27:0/3195525741 >> 172.21.15.27:0/3195525741 conn(0x7fc4c80795f0 msgr2=0x7fc4c8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:19.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.161+0000 7fc4cf88a700 1 -- 172.21.15.27:0/3195525741 shutdown_connections 2024-06-06T08:36:19.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.161+0000 7fc4cf88a700 1 -- 172.21.15.27:0/3195525741 wait complete. 2024-06-06T08:36:19.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.162+0000 7fc4cf88a700 1 Processor -- start 2024-06-06T08:36:19.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.162+0000 7fc4cf88a700 1 -- start start 2024-06-06T08:36:19.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.162+0000 7fc4cf88a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4c8102540 0x7fc4c8198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:19.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.162+0000 7fc4cf88a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4c8102ee0 0x7fc4c81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:19.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.162+0000 7fc4cf88a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 0x7fc4c819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:19.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.162+0000 7fc4cf88a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81128c0 con 0x7fc4c8107450 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.162+0000 7fc4cf88a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8112740 con 0x7fc4c8102540 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.162+0000 7fc4cf88a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8112a40 con 0x7fc4c8102ee0 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cde27700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 0x7fc4c819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cde27700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 0x7fc4c819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60322/0 (socket says 172.21.15.27:60322) 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cde27700 1 -- 172.21.15.27:0/235207800 learned_addr learned my addr 172.21.15.27:0/235207800 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cce25700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4c8102ee0 0x7fc4c81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cd626700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4c8102540 0x7fc4c8198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cde27700 1 -- 172.21.15.27:0/235207800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4c8102ee0 msgr2=0x7fc4c81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cde27700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4c8102ee0 0x7fc4c81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cde27700 1 -- 172.21.15.27:0/235207800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4c8102540 msgr2=0x7fc4c8198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:19.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cde27700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4c8102540 0x7fc4c8198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cde27700 1 -- 172.21.15.27:0/235207800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c819dd30 con 0x7fc4c8107450 2024-06-06T08:36:19.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cde27700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 0x7fc4c819d5f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000b410 tx=0x7fc4c000b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:19.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4c67fc700 1 -- 172.21.15.27:0/235207800 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0015020 con 0x7fc4c8107450 2024-06-06T08:36:19.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c819e020 con 0x7fc4c8107450 2024-06-06T08:36:19.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4c67fc700 1 -- 172.21.15.27:0/235207800 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c0011d10 con 0x7fc4c8107450 2024-06-06T08:36:19.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4c67fc700 1 -- 172.21.15.27:0/235207800 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0010dd0 con 0x7fc4c8107450 2024-06-06T08:36:19.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.163+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c81a58a0 con 0x7fc4c8107450 2024-06-06T08:36:19.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.164+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c8069180 con 0x7fc4c8107450 2024-06-06T08:36:19.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.166+0000 7fc4c67fc700 1 -- 172.21.15.27:0/235207800 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc4c0018020 con 0x7fc4c8107450 2024-06-06T08:36:19.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.166+0000 7fc4c67fc700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc4b00a2c20 0x7fc4b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:19.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.167+0000 7fc4c67fc700 1 -- 172.21.15.27:0/235207800 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7fc4c00ce8d0 con 0x7fc4c8107450 2024-06-06T08:36:19.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.167+0000 7fc4cd626700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc4b00a2c20 0x7fc4b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:19.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.167+0000 7fc4cd626700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc4b00a2c20 0x7fc4b00a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8006130 tx=0x7fc4b80060a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:19.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.170+0000 7fc4c67fc700 1 -- 172.21.15.27:0/235207800 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4c001f020 con 0x7fc4c8107450 2024-06-06T08:36:19.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.379+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fc4c802dca0 con 0x7fc4c8107450 2024-06-06T08:36:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.868+0000 7fc4c67fc700 1 -- 172.21.15.27:0/235207800 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v85) ==== 171+0+0 (secure 0 0 0) 0x7fc4c009bb60 con 0x7fc4c8107450 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc4b00a2c20 msgr2=0x7fc4b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc4b00a2c20 0x7fc4b00a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8006130 tx=0x7fc4b80060a0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 msgr2=0x7fc4c819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 0x7fc4c819d5f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000b410 tx=0x7fc4c000b720 comp rx=0 tx=0).stop 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 shutdown_connections 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc4b00a2c20 0x7fc4b00a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4c8107450 0x7fc4c819d5f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4c8102540 0x7fc4c8198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 --2- 172.21.15.27:0/235207800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4c8102ee0 0x7fc4c81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 >> 172.21.15.27:0/235207800 conn(0x7fc4c80795f0 msgr2=0x7fc4c81005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 shutdown_connections 2024-06-06T08:36:19.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:19.871+0000 7fc4cf88a700 1 -- 172.21.15.27:0/235207800 wait complete. 2024-06-06T08:36:19.872 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-06-06T08:36:20.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:19 smithi027 ceph-mon[32335]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T08:36:20.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:19 smithi027 ceph-mon[32335]: pgmap v294: 33 pgs: 8 creating+peering, 7 active+clean, 18 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:20.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:19 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/235207800' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T08:36:20.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:19 smithi203 ceph-mon[36579]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T08:36:20.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:19 smithi203 ceph-mon[36579]: pgmap v294: 33 pgs: 8 creating+peering, 7 active+clean, 18 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:20.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:19 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/235207800' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T08:36:20.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:19 smithi176 ceph-mon[36786]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T08:36:20.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:19 smithi176 ceph-mon[36786]: pgmap v294: 33 pgs: 8 creating+peering, 7 active+clean, 18 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:20.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:19 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/235207800' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T08:36:20.779 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-06T08:36:21.048 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:21.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:20 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/235207800' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T08:36:21.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:20 smithi027 ceph-mon[32335]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T08:36:21.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:20 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/235207800' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T08:36:21.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:20 smithi203 ceph-mon[36579]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T08:36:21.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:20 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/235207800' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T08:36:21.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:20 smithi176 ceph-mon[36786]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T08:36:22.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:21 smithi027 ceph-mon[32335]: pgmap v296: 97 pgs: 57 unknown, 15 creating+peering, 25 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:22.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:21 smithi027 ceph-mon[32335]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T08:36:22.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:21 smithi203 ceph-mon[36579]: pgmap v296: 97 pgs: 57 unknown, 15 creating+peering, 25 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:21 smithi203 ceph-mon[36579]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T08:36:22.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:21 smithi176 ceph-mon[36786]: pgmap v296: 97 pgs: 57 unknown, 15 creating+peering, 25 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:22.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:21 smithi176 ceph-mon[36786]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T08:36:22.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.926+0000 7f7023190700 1 -- 172.21.15.27:0/2254868310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c101390 msgr2=0x7f701c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:22.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.926+0000 7f7023190700 1 --2- 172.21.15.27:0/2254868310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c101390 0x7f701c101810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7004009990 tx=0x7f7004009ca0 comp rx=0 tx=0).stop 2024-06-06T08:36:22.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.926+0000 7f7023190700 1 -- 172.21.15.27:0/2254868310 shutdown_connections 2024-06-06T08:36:22.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.926+0000 7f7023190700 1 --2- 172.21.15.27:0/2254868310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f701c101d50 0x7f701c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:22.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.926+0000 7f7023190700 1 --2- 172.21.15.27:0/2254868310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f701c107400 0x7f701c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:22.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.926+0000 7f7023190700 1 --2- 172.21.15.27:0/2254868310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c101390 0x7f701c101810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:22.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.926+0000 7f7023190700 1 -- 172.21.15.27:0/2254868310 >> 172.21.15.27:0/2254868310 conn(0x7f701c078340 msgr2=0x7f701c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:22.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.926+0000 7f7023190700 1 -- 172.21.15.27:0/2254868310 shutdown_connections 2024-06-06T08:36:22.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.926+0000 7f7023190700 1 -- 172.21.15.27:0/2254868310 wait complete. 2024-06-06T08:36:22.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f7023190700 1 Processor -- start 2024-06-06T08:36:22.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f7023190700 1 -- start start 2024-06-06T08:36:22.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f7023190700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f701c101390 0x7f701c10ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:22.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f7023190700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f701c101d50 0x7f701c10f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:22.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f7023190700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c107400 0x7f701c10f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:22.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f7023190700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1127f0 con 0x7f701c107400 2024-06-06T08:36:22.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f7023190700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f701c112670 con 0x7f701c101d50 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f7023190700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f701c112970 con 0x7f701c101390 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f702172d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c107400 0x7f701c10f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f701bfff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f701c101d50 0x7f701c10f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f702172d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c107400 0x7f701c10f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36680/0 (socket says 172.21.15.27:36680) 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f702172d700 1 -- 172.21.15.27:0/1231069751 learned_addr learned my addr 172.21.15.27:0/1231069751 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.927+0000 7f7020f2c700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f701c101390 0x7f701c10ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f702172d700 1 -- 172.21.15.27:0/1231069751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f701c101390 msgr2=0x7f701c10ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f702172d700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f701c101390 0x7f701c10ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f702172d700 1 -- 172.21.15.27:0/1231069751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f701c101d50 msgr2=0x7f701c10f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f702172d700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f701c101d50 0x7f701c10f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f702172d700 1 -- 172.21.15.27:0/1231069751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c1a7190 con 0x7f701c107400 2024-06-06T08:36:22.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f701bfff700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f701c101d50 0x7f701c10f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:36:22.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f702172d700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c107400 0x7f701c10f750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f701400c900 tx=0x7f701400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:22.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f7019ffb700 1 -- 172.21.15.27:0/1231069751 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140078d0 con 0x7f701c107400 2024-06-06T08:36:22.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f7019ffb700 1 -- 172.21.15.27:0/1231069751 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701400f450 con 0x7f701c107400 2024-06-06T08:36:22.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c1a7480 con 0x7f701c107400 2024-06-06T08:36:22.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f7019ffb700 1 -- 172.21.15.27:0/1231069751 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014018660 con 0x7f701c107400 2024-06-06T08:36:22.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.928+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c1a7940 con 0x7f701c107400 2024-06-06T08:36:22.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.929+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c069180 con 0x7f701c107400 2024-06-06T08:36:22.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.931+0000 7f7019ffb700 1 -- 172.21.15.27:0/1231069751 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7014007a70 con 0x7f701c107400 2024-06-06T08:36:22.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.931+0000 7f7019ffb700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f70080a2c20 0x7f70080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:22.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.931+0000 7f7019ffb700 1 -- 172.21.15.27:0/1231069751 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7f70140ceed0 con 0x7f701c107400 2024-06-06T08:36:22.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.931+0000 7f7020f2c700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f70080a2c20 0x7f70080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:22.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.932+0000 7f7020f2c700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f70080a2c20 0x7f70080a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f700c003f10 tx=0x7f700c009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:22.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:22.935+0000 7f7019ffb700 1 -- 172.21.15.27:0/1231069751 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7014014030 con 0x7f701c107400 2024-06-06T08:36:23.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.188+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f701c02dca0 con 0x7f701c107400 2024-06-06T08:36:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:22 smithi027 ceph-mon[32335]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T08:36:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:22 smithi203 ceph-mon[36579]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T08:36:23.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:22 smithi176 ceph-mon[36786]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T08:36:23.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.896+0000 7f7019ffb700 1 -- 172.21.15.27:0/1231069751 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f701409bfe0 con 0x7f701c107400 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f70080a2c20 msgr2=0x7f70080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f70080a2c20 0x7f70080a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f700c003f10 tx=0x7f700c009180 comp rx=0 tx=0).stop 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c107400 msgr2=0x7f701c10f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c107400 0x7f701c10f750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f701400c900 tx=0x7f701400cc10 comp rx=0 tx=0).stop 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 shutdown_connections 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f70080a2c20 0x7f70080a50e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c107400 0x7f701c10f750 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f701c101390 0x7f701c10ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 --2- 172.21.15.27:0/1231069751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f701c101d50 0x7f701c10f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 >> 172.21.15.27:0/1231069751 conn(0x7f701c078340 msgr2=0x7f701c100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 shutdown_connections 2024-06-06T08:36:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:23.899+0000 7f7023190700 1 -- 172.21.15.27:0/1231069751 wait complete. 2024-06-06T08:36:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2024-06-06T08:36:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:23 smithi027 ceph-mon[32335]: pgmap v299: 97 pgs: 48 unknown, 7 creating+peering, 42 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:23 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1231069751' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T08:36:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:23 smithi027 ceph-mon[32335]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T08:36:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:23 smithi203 ceph-mon[36579]: pgmap v299: 97 pgs: 48 unknown, 7 creating+peering, 42 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:23 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1231069751' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T08:36:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:23 smithi203 ceph-mon[36579]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T08:36:24.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:23 smithi176 ceph-mon[36786]: pgmap v299: 97 pgs: 48 unknown, 7 creating+peering, 42 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:24.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:23 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1231069751' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T08:36:24.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:23 smithi176 ceph-mon[36786]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T08:36:24.687 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-06T08:36:24.688 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-06T08:36:24.948 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:24 smithi027 ceph-mon[32335]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T08:36:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:24 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1231069751' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T08:36:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:24 smithi027 ceph-mon[32335]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T08:36:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:24 smithi027 ceph-mon[32335]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T08:36:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:24 smithi027 ceph-mon[32335]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T08:36:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:24 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] up:active 2024-06-06T08:36:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:24 smithi027 ceph-mon[32335]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:36:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:24 smithi203 ceph-mon[36579]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T08:36:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:24 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1231069751' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T08:36:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:24 smithi203 ceph-mon[36579]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T08:36:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:24 smithi203 ceph-mon[36579]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T08:36:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:24 smithi203 ceph-mon[36579]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T08:36:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:24 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] up:active 2024-06-06T08:36:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:24 smithi203 ceph-mon[36579]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:36:25.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:24 smithi176 ceph-mon[36786]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T08:36:25.302 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:24 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1231069751' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T08:36:25.302 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:24 smithi176 ceph-mon[36786]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T08:36:25.302 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:24 smithi176 ceph-mon[36786]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T08:36:25.302 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:24 smithi176 ceph-mon[36786]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T08:36:25.302 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:24 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] up:active 2024-06-06T08:36:25.303 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:24 smithi176 ceph-mon[36786]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:36:25.757 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:36:25 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b[67321]: 2024-06-06T08:36:25.406+0000 7fa70c9b1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-06T08:36:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:25 smithi027 ceph-mon[32335]: pgmap v301: 97 pgs: 20 unknown, 7 creating+peering, 70 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-06T08:36:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:25 smithi027 ceph-mon[32335]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T08:36:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:25 smithi027 ceph-mon[32335]: Cluster is now healthy 2024-06-06T08:36:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:25 smithi203 ceph-mon[36579]: pgmap v301: 97 pgs: 20 unknown, 7 creating+peering, 70 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-06T08:36:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:25 smithi203 ceph-mon[36579]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T08:36:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:25 smithi203 ceph-mon[36579]: Cluster is now healthy 2024-06-06T08:36:26.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:25 smithi176 ceph-mon[36786]: pgmap v301: 97 pgs: 20 unknown, 7 creating+peering, 70 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-06T08:36:26.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:25 smithi176 ceph-mon[36786]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T08:36:26.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:25 smithi176 ceph-mon[36786]: Cluster is now healthy 2024-06-06T08:36:26.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.660+0000 7f11b24b5700 1 -- 172.21.15.27:0/3674994107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 msgr2=0x7f11ac102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.660+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3674994107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 0x7f11ac102b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f119c0099c0 tx=0x7f119c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.661+0000 7f11b24b5700 1 -- 172.21.15.27:0/3674994107 shutdown_connections 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.661+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3674994107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11ac107840 0x7f11ac10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.661+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3674994107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 0x7f11ac102b20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.661+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3674994107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11ac1030f0 0x7f11ac107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.661+0000 7f11b24b5700 1 -- 172.21.15.27:0/3674994107 >> 172.21.15.27:0/3674994107 conn(0x7f11ac0795f0 msgr2=0x7f11ac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.661+0000 7f11b24b5700 1 -- 172.21.15.27:0/3674994107 shutdown_connections 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.661+0000 7f11b24b5700 1 -- 172.21.15.27:0/3674994107 wait complete. 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.661+0000 7f11b24b5700 1 Processor -- start 2024-06-06T08:36:26.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.661+0000 7f11b24b5700 1 -- start start 2024-06-06T08:36:26.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11b24b5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 0x7f11ac19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:26.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11b24b5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11ac1030f0 0x7f11ac19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:26.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11b24b5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11ac107840 0x7f11ac198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:26.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11b24b5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac112760 con 0x7f11ac102740 2024-06-06T08:36:26.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11b24b5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1125e0 con 0x7f11ac107840 2024-06-06T08:36:26.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11b24b5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1128e0 con 0x7f11ac1030f0 2024-06-06T08:36:26.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11abfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 0x7f11ac19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11abfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 0x7f11ac19e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36700/0 (socket says 172.21.15.27:36700) 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11abfff700 1 -- 172.21.15.27:0/3663840977 learned_addr learned my addr 172.21.15.27:0/3663840977 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11b0a52700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11ac107840 0x7f11ac198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11ab7fe700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11ac1030f0 0x7f11ac19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11abfff700 1 -- 172.21.15.27:0/3663840977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11ac1030f0 msgr2=0x7f11ac19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11abfff700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11ac1030f0 0x7f11ac19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11abfff700 1 -- 172.21.15.27:0/3663840977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11ac107840 msgr2=0x7f11ac198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11abfff700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11ac107840 0x7f11ac198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11abfff700 1 -- 172.21.15.27:0/3663840977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac1990e0 con 0x7f11ac102740 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.662+0000 7f11abfff700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 0x7f11ac19e770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f119c006010 tx=0x7f119c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.663+0000 7f11a97fa700 1 -- 172.21.15.27:0/3663840977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c027070 con 0x7f11ac102740 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.663+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac199370 con 0x7f11ac102740 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.663+0000 7f11a97fa700 1 -- 172.21.15.27:0/3663840977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119c004050 con 0x7f11ac102740 2024-06-06T08:36:26.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.663+0000 7f11a97fa700 1 -- 172.21.15.27:0/3663840977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c022e40 con 0x7f11ac102740 2024-06-06T08:36:26.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.663+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac1a5640 con 0x7f11ac102740 2024-06-06T08:36:26.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.664+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ac069180 con 0x7f11ac102740 2024-06-06T08:36:26.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.665+0000 7f11a97fa700 1 -- 172.21.15.27:0/3663840977 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f119c02a070 con 0x7f11ac102740 2024-06-06T08:36:26.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.666+0000 7f11a97fa700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f11940a2c20 0x7f11940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.666+0000 7f11ab7fe700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f11940a2c20 0x7f11940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.666+0000 7f11a97fa700 1 -- 172.21.15.27:0/3663840977 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f119c0d0250 con 0x7f11ac102740 2024-06-06T08:36:26.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.666+0000 7f11ab7fe700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f11940a2c20 0x7f11940a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005ea0 tx=0x7f11a0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:26.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.669+0000 7f11a97fa700 1 -- 172.21.15.27:0/3663840977 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f119c013070 con 0x7f11ac102740 2024-06-06T08:36:26.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.886+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f11ac02dca0 con 0x7f11ac102740 2024-06-06T08:36:26.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.897+0000 7f11a97fa700 1 -- 172.21.15.27:0/3663840977 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v89) ==== 157+0+0 (secure 0 0 0) 0x7f119c09d390 con 0x7f11ac102740 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f11940a2c20 msgr2=0x7f11940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f11940a2c20 0x7f11940a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005ea0 tx=0x7f11a0005e30 comp rx=0 tx=0).stop 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 msgr2=0x7f11ac19e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 0x7f11ac19e770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f119c006010 tx=0x7f119c0038b0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 shutdown_connections 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f11940a2c20 0x7f11940a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f11ac107840 0x7f11ac198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11ac102740 0x7f11ac19e770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 --2- 172.21.15.27:0/3663840977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11ac1030f0 0x7f11ac19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 >> 172.21.15.27:0/3663840977 conn(0x7f11ac0795f0 msgr2=0x7f11ac100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 shutdown_connections 2024-06-06T08:36:26.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:26.900+0000 7f11b24b5700 1 -- 172.21.15.27:0/3663840977 wait complete. 2024-06-06T08:36:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:26 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3663840977' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T08:36:27.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:26 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3663840977' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T08:36:27.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:26 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3663840977' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T08:36:27.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-06T08:36:27.872 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:27 smithi027 ceph-mon[32335]: pgmap v302: 97 pgs: 97 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-06-06T08:36:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:27 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3663840977' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T08:36:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:27 smithi027 ceph-mon[32335]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T08:36:28.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:27 smithi203 ceph-mon[36579]: pgmap v302: 97 pgs: 97 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-06-06T08:36:28.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:27 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3663840977' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T08:36:28.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:27 smithi203 ceph-mon[36579]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T08:36:28.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:27 smithi176 ceph-mon[36786]: pgmap v302: 97 pgs: 97 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-06-06T08:36:28.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:27 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3663840977' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T08:36:28.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:27 smithi176 ceph-mon[36786]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T08:36:29.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.558+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3158348748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741066d0 msgr2=0x7f5c7410e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:29.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.558+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3158348748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741066d0 0x7f5c7410e2a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b0d0 tx=0x7f5c6c00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.558+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3158348748 shutdown_connections 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.558+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3158348748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741066d0 0x7f5c7410e2a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.558+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3158348748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c741015f0 0x7f5c741019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.558+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3158348748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74101fa0 0x7f5c74105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.558+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3158348748 >> 172.21.15.27:0/3158348748 conn(0x7f5c74078460 msgr2=0x7f5c74078870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.558+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3158348748 shutdown_connections 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.558+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3158348748 wait complete. 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7c3d7700 1 Processor -- start 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7c3d7700 1 -- start start 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7c3d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741015f0 0x7f5c74103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7c3d7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c74101fa0 0x7f5c741035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7c3d7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c741066d0 0x7f5c74103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:29.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7c3d7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c741116e0 con 0x7f5c741015f0 2024-06-06T08:36:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7c3d7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74111560 con 0x7f5c74101fa0 2024-06-06T08:36:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7c3d7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74111860 con 0x7f5c741066d0 2024-06-06T08:36:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7a173700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741015f0 0x7f5c74103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7a173700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741015f0 0x7f5c74103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36714/0 (socket says 172.21.15.27:36714) 2024-06-06T08:36:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.559+0000 7f5c7a173700 1 -- 172.21.15.27:0/3560187966 learned_addr learned my addr 172.21.15.27:0/3560187966 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7a974700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c741066d0 0x7f5c74103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c79972700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c74101fa0 0x7f5c741035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:29.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7a173700 1 -- 172.21.15.27:0/3560187966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c741066d0 msgr2=0x7f5c74103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7a173700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c741066d0 0x7f5c74103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7a173700 1 -- 172.21.15.27:0/3560187966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c74101fa0 msgr2=0x7f5c741035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7a173700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c74101fa0 0x7f5c741035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7a173700 1 -- 172.21.15.27:0/3560187966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c741a7050 con 0x7f5c741015f0 2024-06-06T08:36:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7a974700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c741066d0 0x7f5c74103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:36:29.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7a173700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741015f0 0x7f5c74103090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c920 tx=0x7f5c7000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c6b7fe700 1 -- 172.21.15.27:0/3560187966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70007b00 con 0x7f5c741015f0 2024-06-06T08:36:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c741a7340 con 0x7f5c741015f0 2024-06-06T08:36:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c6b7fe700 1 -- 172.21.15.27:0/3560187966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c70004d10 con 0x7f5c741015f0 2024-06-06T08:36:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c6b7fe700 1 -- 172.21.15.27:0/3560187966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c700056e0 con 0x7f5c741015f0 2024-06-06T08:36:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.560+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c741a78d0 con 0x7f5c741015f0 2024-06-06T08:36:29.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.561+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c74050a00 con 0x7f5c741015f0 2024-06-06T08:36:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.563+0000 7f5c6b7fe700 1 -- 172.21.15.27:0/3560187966 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5c70007ca0 con 0x7f5c741015f0 2024-06-06T08:36:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.563+0000 7f5c6b7fe700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c600a2cc0 0x7f5c600a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.563+0000 7f5c79972700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c600a2cc0 0x7f5c600a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.563+0000 7f5c6b7fe700 1 -- 172.21.15.27:0/3560187966 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f5c700cefe0 con 0x7f5c741015f0 2024-06-06T08:36:29.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.564+0000 7f5c79972700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c600a2cc0 0x7f5c600a5180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5c741079c0 tx=0x7f5c64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:29.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.567+0000 7f5c6b7fe700 1 -- 172.21.15.27:0/3560187966 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c70014030 con 0x7f5c741015f0 2024-06-06T08:36:29.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:29.775+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5c741a7b90 con 0x7f5c741015f0 2024-06-06T08:36:30.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:30 smithi027 ceph-mon[32335]: pgmap v304: 97 pgs: 97 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-06T08:36:30.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:30 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3560187966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T08:36:30.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:30 smithi203 ceph-mon[36579]: pgmap v304: 97 pgs: 97 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-06T08:36:30.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:30 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3560187966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T08:36:30.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:30 smithi176 ceph-mon[36786]: pgmap v304: 97 pgs: 97 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-06T08:36:30.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:30 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3560187966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T08:36:30.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.904+0000 7f5c6b7fe700 1 -- 172.21.15.27:0/3560187966 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v91) ==== 228+0+0 (secure 0 0 0) 0x7f5c7009bfb0 con 0x7f5c741015f0 2024-06-06T08:36:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c600a2cc0 msgr2=0x7f5c600a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c600a2cc0 0x7f5c600a5180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5c741079c0 tx=0x7f5c64009260 comp rx=0 tx=0).stop 2024-06-06T08:36:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741015f0 msgr2=0x7f5c74103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741015f0 0x7f5c74103090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000c920 tx=0x7f5c7000cce0 comp rx=0 tx=0).stop 2024-06-06T08:36:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 shutdown_connections 2024-06-06T08:36:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c600a2cc0 0x7f5c600a5180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c741066d0 0x7f5c74103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:30.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c741015f0 0x7f5c74103090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:30.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 --2- 172.21.15.27:0/3560187966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c74101fa0 0x7f5c741035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:30.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 >> 172.21.15.27:0/3560187966 conn(0x7f5c74078460 msgr2=0x7f5c7406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:30.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 shutdown_connections 2024-06-06T08:36:30.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:30.907+0000 7f5c7c3d7700 1 -- 172.21.15.27:0/3560187966 wait complete. 2024-06-06T08:36:30.913 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' created 2024-06-06T08:36:31.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:31 smithi027 ceph-mon[32335]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T08:36:31.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:31 smithi027 ceph-mon[32335]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T08:36:31.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:31 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3560187966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T08:36:31.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:31 smithi027 ceph-mon[32335]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T08:36:31.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:31 smithi203 ceph-mon[36579]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T08:36:31.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:31 smithi203 ceph-mon[36579]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T08:36:31.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:31 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3560187966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T08:36:31.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:31 smithi203 ceph-mon[36579]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T08:36:31.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-06T08:36:31.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:31 smithi176 ceph-mon[36786]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T08:36:31.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:31 smithi176 ceph-mon[36786]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T08:36:31.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:31 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3560187966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T08:36:31.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:31 smithi176 ceph-mon[36786]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T08:36:32.010 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:33.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:32 smithi027 ceph-mon[32335]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T08:36:33.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:32 smithi027 ceph-mon[32335]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T08:36:33.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:32 smithi203 ceph-mon[36579]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T08:36:33.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:32 smithi203 ceph-mon[36579]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T08:36:33.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:32 smithi176 ceph-mon[36786]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T08:36:33.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:32 smithi176 ceph-mon[36786]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T08:36:33.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.859+0000 7fb412e2d700 1 -- 172.21.15.27:0/2719809087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 msgr2=0x7fb40c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.859+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2719809087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 0x7fb40c1088e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb400009990 tx=0x7fb400009ca0 comp rx=0 tx=0).stop 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.860+0000 7fb412e2d700 1 -- 172.21.15.27:0/2719809087 shutdown_connections 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.860+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2719809087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb40c06ab30 0x7fb40c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.860+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2719809087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 0x7fb40c1088e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.860+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2719809087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb40c06a170 0x7fb40c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.860+0000 7fb412e2d700 1 -- 172.21.15.27:0/2719809087 >> 172.21.15.27:0/2719809087 conn(0x7fb40c078340 msgr2=0x7fb40c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.860+0000 7fb412e2d700 1 -- 172.21.15.27:0/2719809087 shutdown_connections 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.860+0000 7fb412e2d700 1 -- 172.21.15.27:0/2719809087 wait complete. 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.860+0000 7fb412e2d700 1 Processor -- start 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.860+0000 7fb412e2d700 1 -- start start 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb412e2d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb40c06a170 0x7fb40c101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb412e2d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb40c06ab30 0x7fb40c1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb412e2d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 0x7fb40c101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb412e2d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1127e0 con 0x7fb40c108500 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb412e2d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c112660 con 0x7fb40c06ab30 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb412e2d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c112960 con 0x7fb40c06a170 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb4113ca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 0x7fb40c101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb4113ca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 0x7fb40c101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47634/0 (socket says 172.21.15.27:47634) 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb4113ca700 1 -- 172.21.15.27:0/2085641497 learned_addr learned my addr 172.21.15.27:0/2085641497 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb410bc9700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb40c06a170 0x7fb40c101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb40bfff700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb40c06ab30 0x7fb40c1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb4113ca700 1 -- 172.21.15.27:0/2085641497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb40c06a170 msgr2=0x7fb40c101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb4113ca700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb40c06a170 0x7fb40c101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb4113ca700 1 -- 172.21.15.27:0/2085641497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb40c06ab30 msgr2=0x7fb40c1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb4113ca700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb40c06ab30 0x7fb40c1019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb4113ca700 1 -- 172.21.15.27:0/2085641497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1a7100 con 0x7fb40c108500 2024-06-06T08:36:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb4113ca700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 0x7fb40c101ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb40400b970 tx=0x7fb40400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.861+0000 7fb409ffb700 1 -- 172.21.15.27:0/2085641497 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404019070 con 0x7fb40c108500 2024-06-06T08:36:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.862+0000 7fb409ffb700 1 -- 172.21.15.27:0/2085641497 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40400ce10 con 0x7fb40c108500 2024-06-06T08:36:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.862+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c1a73f0 con 0x7fb40c108500 2024-06-06T08:36:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.862+0000 7fb409ffb700 1 -- 172.21.15.27:0/2085641497 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404007370 con 0x7fb40c108500 2024-06-06T08:36:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.862+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1a7930 con 0x7fb40c108500 2024-06-06T08:36:33.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.863+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c10a9e0 con 0x7fb40c108500 2024-06-06T08:36:33.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.864+0000 7fb409ffb700 1 -- 172.21.15.27:0/2085641497 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb40401c070 con 0x7fb40c108500 2024-06-06T08:36:33.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.865+0000 7fb409ffb700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb3fc0a2ca0 0x7fb3fc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:33.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.865+0000 7fb410bc9700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb3fc0a2ca0 0x7fb3fc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:33.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.865+0000 7fb409ffb700 1 -- 172.21.15.27:0/2085641497 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8770+0+0 (secure 0 0 0) 0x7fb4040cea40 con 0x7fb40c108500 2024-06-06T08:36:33.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.865+0000 7fb410bc9700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb3fc0a2ca0 0x7fb3fc0a5160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb40000b5c0 tx=0x7fb400005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:33.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:33.868+0000 7fb409ffb700 1 -- 172.21.15.27:0/2085641497 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb40409b780 con 0x7fb40c108500 2024-06-06T08:36:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.078+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fb40c10a3c0 con 0x7fb40c108500 2024-06-06T08:36:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:33 smithi027 ceph-mon[32335]: pgmap v309: 129 pgs: 4 creating+peering, 24 unknown, 101 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-06T08:36:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:33 smithi027 ceph-mon[32335]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T08:36:34.242 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:33 smithi203 ceph-mon[36579]: pgmap v309: 129 pgs: 4 creating+peering, 24 unknown, 101 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-06T08:36:34.242 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:33 smithi203 ceph-mon[36579]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T08:36:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:33 smithi176 ceph-mon[36786]: pgmap v309: 129 pgs: 4 creating+peering, 24 unknown, 101 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-06T08:36:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:33 smithi176 ceph-mon[36786]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T08:36:34.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.918+0000 7fb409ffb700 1 -- 172.21.15.27:0/2085641497 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v94) ==== 169+0+0 (secure 0 0 0) 0x7fb40409b080 con 0x7fb40c108500 2024-06-06T08:36:34.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.921+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb3fc0a2ca0 msgr2=0x7fb3fc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:34.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.921+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb3fc0a2ca0 0x7fb3fc0a5160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb40000b5c0 tx=0x7fb400005960 comp rx=0 tx=0).stop 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.921+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 msgr2=0x7fb40c101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.921+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 0x7fb40c101ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb40400b970 tx=0x7fb40400bc80 comp rx=0 tx=0).stop 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.921+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 shutdown_connections 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.921+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb3fc0a2ca0 0x7fb3fc0a5160 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.922+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb40c108500 0x7fb40c101ee0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.922+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb40c06a170 0x7fb40c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.922+0000 7fb412e2d700 1 --2- 172.21.15.27:0/2085641497 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb40c06ab30 0x7fb40c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.922+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 >> 172.21.15.27:0/2085641497 conn(0x7fb40c078340 msgr2=0x7fb40c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.922+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 shutdown_connections 2024-06-06T08:36:34.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:34.922+0000 7fb412e2d700 1 -- 172.21.15.27:0/2085641497 wait complete. 2024-06-06T08:36:34.922 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 4 allow_ec_overwrites to true 2024-06-06T08:36:35.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:34 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2085641497' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T08:36:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:34 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2085641497' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T08:36:35.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:34 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2085641497' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T08:36:35.703 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-06T08:36:35.963 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:36.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:35 smithi027 ceph-mon[32335]: pgmap v311: 129 pgs: 4 creating+peering, 7 unknown, 118 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:36.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:35 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2085641497' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T08:36:36.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:35 smithi027 ceph-mon[32335]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T08:36:36.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:35 smithi203 ceph-mon[36579]: pgmap v311: 129 pgs: 4 creating+peering, 7 unknown, 118 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:36.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:35 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2085641497' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T08:36:36.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:35 smithi203 ceph-mon[36579]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T08:36:36.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:35 smithi176 ceph-mon[36786]: pgmap v311: 129 pgs: 4 creating+peering, 7 unknown, 118 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:36.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:35 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2085641497' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T08:36:36.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:35 smithi176 ceph-mon[36786]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T08:36:37.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.765+0000 7ff5bda26700 1 -- 172.21.15.27:0/2783096454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b81084f0 msgr2=0x7ff5b81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.765+0000 7ff5bda26700 1 --2- 172.21.15.27:0/2783096454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b81084f0 0x7ff5b81088d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009990 tx=0x7ff5a8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.765+0000 7ff5bda26700 1 -- 172.21.15.27:0/2783096454 shutdown_connections 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.765+0000 7ff5bda26700 1 --2- 172.21.15.27:0/2783096454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5b806b3c0 0x7ff5b810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.765+0000 7ff5bda26700 1 --2- 172.21.15.27:0/2783096454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b81084f0 0x7ff5b81088d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.765+0000 7ff5bda26700 1 --2- 172.21.15.27:0/2783096454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5b806aa00 0x7ff5b806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.765+0000 7ff5bda26700 1 -- 172.21.15.27:0/2783096454 >> 172.21.15.27:0/2783096454 conn(0x7ff5b8078300 msgr2=0x7ff5b8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.765+0000 7ff5bda26700 1 -- 172.21.15.27:0/2783096454 shutdown_connections 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.765+0000 7ff5bda26700 1 -- 172.21.15.27:0/2783096454 wait complete. 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.766+0000 7ff5bda26700 1 Processor -- start 2024-06-06T08:36:37.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.766+0000 7ff5bda26700 1 -- start start 2024-06-06T08:36:37.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5bda26700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5b806aa00 0x7ff5b8198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:37.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5bda26700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b806b3c0 0x7ff5b81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:37.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5bda26700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5b81084f0 0x7ff5b819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:37.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5bda26700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8112850 con 0x7ff5b806b3c0 2024-06-06T08:36:37.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5bda26700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81126d0 con 0x7ff5b806aa00 2024-06-06T08:36:37.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5bda26700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81129d0 con 0x7ff5b81084f0 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5ae7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b806b3c0 0x7ff5b81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5ae7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b806b3c0 0x7ff5b81992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47662/0 (socket says 172.21.15.27:47662) 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5ae7fc700 1 -- 172.21.15.27:0/1053221841 learned_addr learned my addr 172.21.15.27:0/1053221841 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5b6ffd700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5b806aa00 0x7ff5b8198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5b77fe700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5b81084f0 0x7ff5b819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5ae7fc700 1 -- 172.21.15.27:0/1053221841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5b81084f0 msgr2=0x7ff5b819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5ae7fc700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5b81084f0 0x7ff5b819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5ae7fc700 1 -- 172.21.15.27:0/1053221841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5b806aa00 msgr2=0x7ff5b8198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5ae7fc700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5b806aa00 0x7ff5b8198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5ae7fc700 1 -- 172.21.15.27:0/1053221841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b819dd20 con 0x7ff5b806b3c0 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5b77fe700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5b81084f0 0x7ff5b819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5ae7fc700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b806b3c0 0x7ff5b81992a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000b530 tx=0x7ff5a000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:37.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5bca24700 1 -- 172.21.15.27:0/1053221841 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0015020 con 0x7ff5b806b3c0 2024-06-06T08:36:37.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5bca24700 1 -- 172.21.15.27:0/1053221841 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a0011d10 con 0x7ff5b806b3c0 2024-06-06T08:36:37.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.767+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b819e010 con 0x7ff5b806b3c0 2024-06-06T08:36:37.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.768+0000 7ff5bca24700 1 -- 172.21.15.27:0/1053221841 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000f2e0 con 0x7ff5b806b3c0 2024-06-06T08:36:37.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.768+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b81a5890 con 0x7ff5b806b3c0 2024-06-06T08:36:37.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.769+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b810a9d0 con 0x7ff5b806b3c0 2024-06-06T08:36:37.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.770+0000 7ff5bca24700 1 -- 172.21.15.27:0/1053221841 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff5a0018020 con 0x7ff5b806b3c0 2024-06-06T08:36:37.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.770+0000 7ff5bca24700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5a40a2c20 0x7ff5a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:37.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.771+0000 7ff5b6ffd700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5a40a2c20 0x7ff5a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:37.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.771+0000 7ff5bca24700 1 -- 172.21.15.27:0/1053221841 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7ff5a00ceb20 con 0x7ff5b806b3c0 2024-06-06T08:36:37.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.771+0000 7ff5b6ffd700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5a40a2c20 0x7ff5a40a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8006010 tx=0x7ff5a800b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:37.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:37.775+0000 7ff5bca24700 1 -- 172.21.15.27:0/1053221841 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff5a001f020 con 0x7ff5b806b3c0 2024-06-06T08:36:38.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.034+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7ff5b819a070 con 0x7ff5b806b3c0 2024-06-06T08:36:38.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:37 smithi027 ceph-mon[32335]: pgmap v313: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:38.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:37 smithi203 ceph-mon[36579]: pgmap v313: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:38.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:37 smithi176 ceph-mon[36786]: pgmap v313: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:38.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.930+0000 7ff5bca24700 1 -- 172.21.15.27:0/1053221841 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7ff5a009b860 con 0x7ff5b806b3c0 2024-06-06T08:36:38.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.933+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5a40a2c20 msgr2=0x7ff5a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:38.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.933+0000 7ff5bda26700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5a40a2c20 0x7ff5a40a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8006010 tx=0x7ff5a800b560 comp rx=0 tx=0).stop 2024-06-06T08:36:38.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.933+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b806b3c0 msgr2=0x7ff5b81992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:38.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.933+0000 7ff5bda26700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b806b3c0 0x7ff5b81992a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000b530 tx=0x7ff5a000b840 comp rx=0 tx=0).stop 2024-06-06T08:36:38.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.933+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 shutdown_connections 2024-06-06T08:36:38.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.933+0000 7ff5bda26700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5a40a2c20 0x7ff5a40a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:38.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.933+0000 7ff5bda26700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5b81084f0 0x7ff5b819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:38.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.933+0000 7ff5bda26700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5b806aa00 0x7ff5b8198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:38.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.933+0000 7ff5bda26700 1 --2- 172.21.15.27:0/1053221841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b806b3c0 0x7ff5b81992a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:38.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.934+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 >> 172.21.15.27:0/1053221841 conn(0x7ff5b8078300 msgr2=0x7ff5b8100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:38.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.934+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 shutdown_connections 2024-06-06T08:36:38.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:38.934+0000 7ff5bda26700 1 -- 172.21.15.27:0/1053221841 wait complete. 2024-06-06T08:36:38.933 INFO:teuthology.orchestra.run.smithi027.stderr:added data pool 4 to fsmap 2024-06-06T08:36:39.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:38 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1053221841' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T08:36:39.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:38 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1053221841' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T08:36:39.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:38 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1053221841' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T08:36:39.720 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:36:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:39 smithi027 ceph-mon[32335]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:39 smithi027 ceph-mon[32335]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T08:36:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:39 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1053221841' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T08:36:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:39 smithi027 ceph-mon[32335]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:36:39.981 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:40.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:39 smithi203 ceph-mon[36579]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:40.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:39 smithi203 ceph-mon[36579]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T08:36:40.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:39 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1053221841' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T08:36:40.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:39 smithi203 ceph-mon[36579]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:36:40.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:39 smithi176 ceph-mon[36786]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:40.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:39 smithi176 ceph-mon[36786]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T08:36:40.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:39 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1053221841' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T08:36:40.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:39 smithi176 ceph-mon[36786]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:36:41.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:40 smithi027 ceph-mon[32335]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T08:36:41.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:40 smithi027 ceph-mon[32335]: Cluster is now healthy 2024-06-06T08:36:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:40 smithi203 ceph-mon[36579]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T08:36:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:40 smithi203 ceph-mon[36579]: Cluster is now healthy 2024-06-06T08:36:41.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:40 smithi176 ceph-mon[36786]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T08:36:41.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:40 smithi176 ceph-mon[36786]: Cluster is now healthy 2024-06-06T08:36:41.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.664+0000 7f87ba575700 1 -- 172.21.15.27:0/3992392946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 msgr2=0x7f87b410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.664+0000 7f87ba575700 1 --2- 172.21.15.27:0/3992392946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 0x7f87b410e1d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00b0a0 tx=0x7f87ac00b3b0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.664+0000 7f87ba575700 1 -- 172.21.15.27:0/3992392946 shutdown_connections 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.664+0000 7f87ba575700 1 --2- 172.21.15.27:0/3992392946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 0x7f87b410e1d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.664+0000 7f87ba575700 1 --2- 172.21.15.27:0/3992392946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87b4107420 0x7f87b4107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.664+0000 7f87ba575700 1 --2- 172.21.15.27:0/3992392946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b4101370 0x7f87b41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.664+0000 7f87ba575700 1 -- 172.21.15.27:0/3992392946 >> 172.21.15.27:0/3992392946 conn(0x7f87b4078340 msgr2=0x7f87b4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.665+0000 7f87ba575700 1 -- 172.21.15.27:0/3992392946 shutdown_connections 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.665+0000 7f87ba575700 1 -- 172.21.15.27:0/3992392946 wait complete. 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.665+0000 7f87ba575700 1 Processor -- start 2024-06-06T08:36:41.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.665+0000 7f87ba575700 1 -- start start 2024-06-06T08:36:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87ba575700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87b4101370 0x7f87b419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87ba575700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 0x7f87b419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87ba575700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b4107420 0x7f87b4198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87ba575700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4111730 con 0x7f87b4101d30 2024-06-06T08:36:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87ba575700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41115b0 con 0x7f87b4101370 2024-06-06T08:36:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87ba575700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41118b0 con 0x7f87b4107420 2024-06-06T08:36:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 0x7f87b419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 0x7f87b419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35998/0 (socket says 172.21.15.27:35998) 2024-06-06T08:36:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b37fe700 1 -- 172.21.15.27:0/2511004899 learned_addr learned my addr 172.21.15.27:0/2511004899 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b8b12700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b4107420 0x7f87b4198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b3fff700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87b4101370 0x7f87b419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b37fe700 1 -- 172.21.15.27:0/2511004899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b4107420 msgr2=0x7f87b4198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b37fe700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b4107420 0x7f87b4198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b37fe700 1 -- 172.21.15.27:0/2511004899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87b4101370 msgr2=0x7f87b419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b37fe700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87b4101370 0x7f87b419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b37fe700 1 -- 172.21.15.27:0/2511004899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b41991e0 con 0x7f87b4101d30 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.666+0000 7f87b37fe700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 0x7f87b419ee00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87a800d980 tx=0x7f87a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.667+0000 7f87b17fa700 1 -- 172.21.15.27:0/2511004899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8017070 con 0x7f87b4101d30 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.667+0000 7f87b17fa700 1 -- 172.21.15.27:0/2511004899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a8005020 con 0x7f87b4101d30 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.667+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b41994d0 con 0x7f87b4101d30 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.667+0000 7f87b17fa700 1 -- 172.21.15.27:0/2511004899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8009bc0 con 0x7f87b4101d30 2024-06-06T08:36:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.667+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b41a5790 con 0x7f87b4101d30 2024-06-06T08:36:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.668+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b4069180 con 0x7f87b4101d30 2024-06-06T08:36:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.669+0000 7f87b17fa700 1 -- 172.21.15.27:0/2511004899 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f87a80051d0 con 0x7f87b4101d30 2024-06-06T08:36:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.670+0000 7f87b17fa700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f879c0a2ca0 0x7f879c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.670+0000 7f87b3fff700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f879c0a2ca0 0x7f879c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.670+0000 7f87b17fa700 1 -- 172.21.15.27:0/2511004899 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f87a80cf140 con 0x7f87b4101d30 2024-06-06T08:36:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.670+0000 7f87b3fff700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f879c0a2ca0 0x7f879c0a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87a4006fd0 tx=0x7f87a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.674+0000 7f87b17fa700 1 -- 172.21.15.27:0/2511004899 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f87a8020020 con 0x7f87b4101d30 2024-06-06T08:36:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.924+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87b41a5a80 con 0x7f87b4101d30 2024-06-06T08:36:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.925+0000 7f87b17fa700 1 -- 172.21.15.27:0/2511004899 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11809 (secure 0 0 0) 0x7f87a809be60 con 0x7f87b4101d30 2024-06-06T08:36:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:36:41.924 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:36:38.035723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-06-06T08:36:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f879c0a2ca0 msgr2=0x7f879c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f879c0a2ca0 0x7f879c0a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87a4006fd0 tx=0x7f87a4008040 comp rx=0 tx=0).stop 2024-06-06T08:36:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 msgr2=0x7f87b419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 0x7f87b419ee00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87a800d980 tx=0x7f87a800dc90 comp rx=0 tx=0).stop 2024-06-06T08:36:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 shutdown_connections 2024-06-06T08:36:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f879c0a2ca0 0x7f879c0a5160 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87b4107420 0x7f87b4198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f87b4101370 0x7f87b419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 --2- 172.21.15.27:0/2511004899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4101d30 0x7f87b419ee00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 >> 172.21.15.27:0/2511004899 conn(0x7f87b4078340 msgr2=0x7f87b4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 shutdown_connections 2024-06-06T08:36:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:41.928+0000 7f87ba575700 1 -- 172.21.15.27:0/2511004899 wait complete. 2024-06-06T08:36:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2024-06-06T08:36:42.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:41 smithi027 ceph-mon[32335]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:42.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:41 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2511004899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:42.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:41 smithi176 ceph-mon[36786]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:42.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:41 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2511004899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:42.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:41 smithi203 ceph-mon[36579]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:41 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2511004899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:42.740 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:36:42.993 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:44.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:43 smithi027 ceph-mon[32335]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:44.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:43 smithi176 ceph-mon[36786]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:43 smithi203 ceph-mon[36579]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.697+0000 7feaaa0ae700 1 -- 172.21.15.27:0/3604561412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa4108580 msgr2=0x7feaa4108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.697+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/3604561412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa4108580 0x7feaa4108960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fea940099c0 tx=0x7fea94009cd0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.697+0000 7feaaa0ae700 1 -- 172.21.15.27:0/3604561412 shutdown_connections 2024-06-06T08:36:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.697+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/3604561412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feaa406b620 0x7feaa410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.697+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/3604561412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa4108580 0x7feaa4108960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.697+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/3604561412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa406ac60 0x7feaa406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.697+0000 7feaaa0ae700 1 -- 172.21.15.27:0/3604561412 >> 172.21.15.27:0/3604561412 conn(0x7feaa4078430 msgr2=0x7feaa4078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:44.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.697+0000 7feaaa0ae700 1 -- 172.21.15.27:0/3604561412 shutdown_connections 2024-06-06T08:36:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.697+0000 7feaaa0ae700 1 -- 172.21.15.27:0/3604561412 wait complete. 2024-06-06T08:36:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaaa0ae700 1 Processor -- start 2024-06-06T08:36:44.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaaa0ae700 1 -- start start 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaaa0ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa406ac60 0x7feaa410e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaaa0ae700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feaa406b620 0x7feaa410eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaaa0ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4108580 0x7feaa41a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaaa0ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41128b0 con 0x7feaa406ac60 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaaa0ae700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4112730 con 0x7feaa406b620 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaaa0ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4112a30 con 0x7feaa4108580 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaa37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa406ac60 0x7feaa410e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaa37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa406ac60 0x7feaa410e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36020/0 (socket says 172.21.15.27:36020) 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.698+0000 7feaa37fe700 1 -- 172.21.15.27:0/40329844 learned_addr learned my addr 172.21.15.27:0/40329844 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7fea9affd700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feaa406b620 0x7feaa410eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa3fff700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4108580 0x7feaa41a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:44.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa37fe700 1 -- 172.21.15.27:0/40329844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4108580 msgr2=0x7feaa41a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa37fe700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4108580 0x7feaa41a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa37fe700 1 -- 172.21.15.27:0/40329844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feaa406b620 msgr2=0x7feaa410eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa37fe700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feaa406b620 0x7feaa410eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa37fe700 1 -- 172.21.15.27:0/40329844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa41a4350 con 0x7feaa406ac60 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7fea9affd700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feaa406b620 0x7feaa410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa3fff700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4108580 0x7feaa41a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa37fe700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa406ac60 0x7feaa410e640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea94009fd0 tx=0x7fea94003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa17fa700 1 -- 172.21.15.27:0/40329844 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94027070 con 0x7feaa406ac60 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa17fa700 1 -- 172.21.15.27:0/40329844 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea94004150 con 0x7feaa406ac60 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaa17fa700 1 -- 172.21.15.27:0/40329844 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9400f460 con 0x7feaa406ac60 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa41a45e0 con 0x7feaa406ac60 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.699+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa41abec0 con 0x7feaa406ac60 2024-06-06T08:36:44.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.700+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa4069180 con 0x7feaa406ac60 2024-06-06T08:36:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.701+0000 7feaa17fa700 1 -- 172.21.15.27:0/40329844 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fea9402a070 con 0x7feaa406ac60 2024-06-06T08:36:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.702+0000 7feaa17fa700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea900a2c20 0x7fea900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.702+0000 7fea9affd700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea900a2c20 0x7fea900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:44.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.703+0000 7feaa17fa700 1 -- 172.21.15.27:0/40329844 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fea940d0910 con 0x7feaa406ac60 2024-06-06T08:36:44.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.703+0000 7fea9affd700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea900a2c20 0x7fea900a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feaa4103420 tx=0x7fea8c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:44.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.706+0000 7feaa17fa700 1 -- 172.21.15.27:0/40329844 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea94031070 con 0x7feaa406ac60 2024-06-06T08:36:44.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.916+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaa41ac090 con 0x7feaa406ac60 2024-06-06T08:36:44.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.918+0000 7feaa17fa700 1 -- 172.21.15.27:0/40329844 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24103 (secure 0 0 0) 0x7fea94013070 con 0x7feaa406ac60 2024-06-06T08:36:44.917 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:36:44.917 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":95,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:36:38.035534+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:36:44.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea900a2c20 msgr2=0x7fea900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:44.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea900a2c20 0x7fea900a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feaa4103420 tx=0x7fea8c008040 comp rx=0 tx=0).stop 2024-06-06T08:36:44.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa406ac60 msgr2=0x7feaa410e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:44.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa406ac60 0x7feaa410e640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea94009fd0 tx=0x7fea94003730 comp rx=0 tx=0).stop 2024-06-06T08:36:44.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 shutdown_connections 2024-06-06T08:36:44.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fea900a2c20 0x7fea900a50e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaa4108580 0x7feaa41a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa406ac60 0x7feaa410e640 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 --2- 172.21.15.27:0/40329844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feaa406b620 0x7feaa410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:44.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.920+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 >> 172.21.15.27:0/40329844 conn(0x7feaa4078430 msgr2=0x7feaa40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:44.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.921+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 shutdown_connections 2024-06-06T08:36:44.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:44.921+0000 7feaaa0ae700 1 -- 172.21.15.27:0/40329844 wait complete. 2024-06-06T08:36:45.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:45 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/40329844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:36:45.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:45 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/40329844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:36:45.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:45 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/40329844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:36:45.757 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:36:46.023 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:46.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:46 smithi176 ceph-mon[36786]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:46.290 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:46 smithi027 ceph-mon[32335]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:46.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:46 smithi203 ceph-mon[36579]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:47.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.826+0000 7f5864ca1700 1 -- 172.21.15.27:0/2850037006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860107400 msgr2=0x7f58601077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:47.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.826+0000 7f5864ca1700 1 --2- 172.21.15.27:0/2850037006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860107400 0x7f58601077e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f585000b0a0 tx=0x7f585000b3b0 comp rx=0 tx=0).stop 2024-06-06T08:36:47.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.826+0000 7f5864ca1700 1 -- 172.21.15.27:0/2850037006 shutdown_connections 2024-06-06T08:36:47.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.826+0000 7f5864ca1700 1 --2- 172.21.15.27:0/2850037006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5860101d50 0x7f586010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:47.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.826+0000 7f5864ca1700 1 --2- 172.21.15.27:0/2850037006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860107400 0x7f58601077e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:47.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.826+0000 7f5864ca1700 1 --2- 172.21.15.27:0/2850037006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5860101390 0x7f5860101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:47.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.826+0000 7f5864ca1700 1 -- 172.21.15.27:0/2850037006 >> 172.21.15.27:0/2850037006 conn(0x7f5860078340 msgr2=0x7f5860078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.826+0000 7f5864ca1700 1 -- 172.21.15.27:0/2850037006 shutdown_connections 2024-06-06T08:36:47.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.826+0000 7f5864ca1700 1 -- 172.21.15.27:0/2850037006 wait complete. 2024-06-06T08:36:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f5864ca1700 1 Processor -- start 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f5864ca1700 1 -- start start 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f5864ca1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860101390 0x7f586019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f5864ca1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5860101d50 0x7f586019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f5864ca1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5860107400 0x7f58601989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f5864ca1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58601116a0 con 0x7f5860101390 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f5864ca1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5860111520 con 0x7f5860101d50 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f5864ca1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5860111820 con 0x7f5860107400 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f585e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860101390 0x7f586019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f585e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860101390 0x7f586019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36038/0 (socket says 172.21.15.27:36038) 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.827+0000 7f585e59c700 1 -- 172.21.15.27:0/1353654779 learned_addr learned my addr 172.21.15.27:0/1353654779 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f585dd9b700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5860101d50 0x7f586019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f585ed9d700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5860107400 0x7f58601989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f585e59c700 1 -- 172.21.15.27:0/1353654779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5860107400 msgr2=0x7f58601989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f585e59c700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5860107400 0x7f58601989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f585e59c700 1 -- 172.21.15.27:0/1353654779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5860101d50 msgr2=0x7f586019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f585e59c700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5860101d50 0x7f586019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f585e59c700 1 -- 172.21.15.27:0/1353654779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5860199140 con 0x7f5860101390 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f585e59c700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860101390 0x7f586019e820 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f585000b070 tx=0x7f5850004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f58577fe700 1 -- 172.21.15.27:0/1353654779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850021070 con 0x7f5860101390 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f58577fe700 1 -- 172.21.15.27:0/1353654779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5850007b80 con 0x7f5860101390 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f58577fe700 1 -- 172.21.15.27:0/1353654779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500052a0 con 0x7f5860101390 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58601993d0 con 0x7f5860101390 2024-06-06T08:36:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.828+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58601a56f0 con 0x7f5860101390 2024-06-06T08:36:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.830+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5860069180 con 0x7f5860101390 2024-06-06T08:36:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.831+0000 7f58577fe700 1 -- 172.21.15.27:0/1353654779 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5850024070 con 0x7f5860101390 2024-06-06T08:36:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.832+0000 7f58577fe700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f584c0a2c20 0x7f584c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.832+0000 7f585dd9b700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f584c0a2c20 0x7f584c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.832+0000 7f58577fe700 1 -- 172.21.15.27:0/1353654779 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f585000f020 con 0x7f5860101390 2024-06-06T08:36:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.832+0000 7f585dd9b700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f584c0a2c20 0x7f584c0a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f58600ffb70 tx=0x7f584800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:47.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:47.836+0000 7f58577fe700 1 -- 172.21.15.27:0/1353654779 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f585002b020 con 0x7f5860101390 2024-06-06T08:36:48.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.047+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58601a58c0 con 0x7f5860101390 2024-06-06T08:36:48.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.049+0000 7f58577fe700 1 -- 172.21.15.27:0/1353654779 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24103 (secure 0 0 0) 0x7f585009d6c0 con 0x7f5860101390 2024-06-06T08:36:48.048 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:36:48.048 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":95,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:36:38.035534+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:36:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f584c0a2c20 msgr2=0x7f584c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f584c0a2c20 0x7f584c0a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f58600ffb70 tx=0x7f584800b410 comp rx=0 tx=0).stop 2024-06-06T08:36:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860101390 msgr2=0x7f586019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860101390 0x7f586019e820 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f585000b070 tx=0x7f5850004e40 comp rx=0 tx=0).stop 2024-06-06T08:36:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 shutdown_connections 2024-06-06T08:36:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f584c0a2c20 0x7f584c0a50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5860107400 0x7f58601989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5860101390 0x7f586019e820 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 --2- 172.21.15.27:0/1353654779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5860101d50 0x7f586019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 >> 172.21.15.27:0/1353654779 conn(0x7f5860078340 msgr2=0x7f58601005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 shutdown_connections 2024-06-06T08:36:48.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:48.051+0000 7f5864ca1700 1 -- 172.21.15.27:0/1353654779 wait complete. 2024-06-06T08:36:48.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:48 smithi027 ceph-mon[32335]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:48.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:48 smithi176 ceph-mon[36786]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:48.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:48 smithi203 ceph-mon[36579]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:48.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:36:49.152 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:49.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:49 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1353654779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:36:49.310 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:49 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1353654779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:36:49.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:49 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1353654779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:36:50.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:50 smithi027 ceph-mon[32335]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:50.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:50 smithi203 ceph-mon[36579]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:50.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:50 smithi176 ceph-mon[36786]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:50.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.890+0000 7f4817a49700 1 -- 172.21.15.27:0/1225564823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4810101380 msgr2=0x7f4810101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:50.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.890+0000 7f4817a49700 1 --2- 172.21.15.27:0/1225564823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4810101380 0x7f4810101800 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4800009990 tx=0x7f4800009ca0 comp rx=0 tx=0).stop 2024-06-06T08:36:50.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.890+0000 7f4817a49700 1 -- 172.21.15.27:0/1225564823 shutdown_connections 2024-06-06T08:36:50.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.890+0000 7f4817a49700 1 --2- 172.21.15.27:0/1225564823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4810101d40 0x7f481010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:50.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.890+0000 7f4817a49700 1 --2- 172.21.15.27:0/1225564823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48101073f0 0x7f48101077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.890+0000 7f4817a49700 1 --2- 172.21.15.27:0/1225564823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4810101380 0x7f4810101800 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.890+0000 7f4817a49700 1 -- 172.21.15.27:0/1225564823 >> 172.21.15.27:0/1225564823 conn(0x7f4810078300 msgr2=0x7f4810078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.890+0000 7f4817a49700 1 -- 172.21.15.27:0/1225564823 shutdown_connections 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.890+0000 7f4817a49700 1 -- 172.21.15.27:0/1225564823 wait complete. 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4817a49700 1 Processor -- start 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4817a49700 1 -- start start 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4817a49700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4810101380 0x7f481010ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4817a49700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4810101d40 0x7f4810110490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4817a49700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48101073f0 0x7f48101109d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4817a49700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4810112760 con 0x7f48101073f0 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4817a49700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f48101125e0 con 0x7f4810101380 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4817a49700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f48101128e0 con 0x7f4810101d40 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4815fe6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48101073f0 0x7f48101109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4815fe6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48101073f0 0x7f48101109d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56736/0 (socket says 172.21.15.27:56736) 2024-06-06T08:36:50.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4815fe6700 1 -- 172.21.15.27:0/902333837 learned_addr learned my addr 172.21.15.27:0/902333837 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:50.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f48157e5700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4810101380 0x7f481010ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.891+0000 7f4814fe4700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4810101d40 0x7f4810110490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f4815fe6700 1 -- 172.21.15.27:0/902333837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4810101d40 msgr2=0x7f4810110490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f4815fe6700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4810101d40 0x7f4810110490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f4815fe6700 1 -- 172.21.15.27:0/902333837 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4810101380 msgr2=0x7f481010ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f4815fe6700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4810101380 0x7f481010ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f4815fe6700 1 -- 172.21.15.27:0/902333837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48101a7080 con 0x7f48101073f0 2024-06-06T08:36:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f4814fe4700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4810101d40 0x7f4810110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:36:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f4815fe6700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48101073f0 0x7f48101109d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4808007ee0 tx=0x7f480800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:50.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f48067fc700 1 -- 172.21.15.27:0/902333837 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480800dbd0 con 0x7f48101073f0 2024-06-06T08:36:50.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48101a7370 con 0x7f48101073f0 2024-06-06T08:36:50.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f48067fc700 1 -- 172.21.15.27:0/902333837 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f480800dd70 con 0x7f48101073f0 2024-06-06T08:36:50.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f48067fc700 1 -- 172.21.15.27:0/902333837 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808005a60 con 0x7f48101073f0 2024-06-06T08:36:50.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.892+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48101a7860 con 0x7f48101073f0 2024-06-06T08:36:50.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.893+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4810069180 con 0x7f48101073f0 2024-06-06T08:36:50.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.895+0000 7f48067fc700 1 -- 172.21.15.27:0/902333837 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f480801a020 con 0x7f48101073f0 2024-06-06T08:36:50.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.896+0000 7f48067fc700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47fc0a2c20 0x7f47fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:50.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.896+0000 7f48157e5700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47fc0a2c20 0x7f47fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:50.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.896+0000 7f48067fc700 1 -- 172.21.15.27:0/902333837 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f48080ce9b0 con 0x7f48101073f0 2024-06-06T08:36:50.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.896+0000 7f48157e5700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47fc0a2c20 0x7f47fc0a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f480c00abd0 tx=0x7f480c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:50.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:50.901+0000 7f48067fc700 1 -- 172.21.15.27:0/902333837 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4808010070 con 0x7f48101073f0 2024-06-06T08:36:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.174+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48101a7b60 con 0x7f48101073f0 2024-06-06T08:36:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.175+0000 7f48067fc700 1 -- 172.21.15.27:0/902333837 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11809 (secure 0 0 0) 0x7f480809b6d0 con 0x7f48101073f0 2024-06-06T08:36:51.175 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:36:51.175 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:36:38.035723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-06-06T08:36:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47fc0a2c20 msgr2=0x7f47fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47fc0a2c20 0x7f47fc0a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f480c00abd0 tx=0x7f480c00a380 comp rx=0 tx=0).stop 2024-06-06T08:36:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48101073f0 msgr2=0x7f48101109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48101073f0 0x7f48101109d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4808007ee0 tx=0x7f480800d3b0 comp rx=0 tx=0).stop 2024-06-06T08:36:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 shutdown_connections 2024-06-06T08:36:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47fc0a2c20 0x7f47fc0a50e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48101073f0 0x7f48101109d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4810101380 0x7f481010ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 --2- 172.21.15.27:0/902333837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4810101d40 0x7f4810110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.177+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 >> 172.21.15.27:0/902333837 conn(0x7f4810078300 msgr2=0x7f48100ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.178+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 shutdown_connections 2024-06-06T08:36:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:51.178+0000 7f4817a49700 1 -- 172.21.15.27:0/902333837 wait complete. 2024-06-06T08:36:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2024-06-06T08:36:51.814 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:36:52.066 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:52.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:52 smithi027 ceph-mon[32335]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:52.189 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:52 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/902333837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:52.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:52 smithi203 ceph-mon[36579]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:52.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:52 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/902333837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:52.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:52 smithi176 ceph-mon[36786]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:52.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:52 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/902333837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:36:53.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 -- 172.21.15.27:0/1753185352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806ac60 msgr2=0x7f745806b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:53.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 --2- 172.21.15.27:0/1753185352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806ac60 0x7f745806b0e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f744c0099c0 tx=0x7f744c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:36:53.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 -- 172.21.15.27:0/1753185352 shutdown_connections 2024-06-06T08:36:53.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 --2- 172.21.15.27:0/1753185352 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f745806b620 0x7f745810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:53.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 --2- 172.21.15.27:0/1753185352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74581085b0 0x7f7458108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 --2- 172.21.15.27:0/1753185352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806ac60 0x7f745806b0e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 -- 172.21.15.27:0/1753185352 >> 172.21.15.27:0/1753185352 conn(0x7f74580772a0 msgr2=0x7f74580776b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 -- 172.21.15.27:0/1753185352 shutdown_connections 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 -- 172.21.15.27:0/1753185352 wait complete. 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.800+0000 7f745e248700 1 Processor -- start 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f745e248700 1 -- start start 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f745e248700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f745806ac60 0x7f745819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f745e248700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806b620 0x7f745819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f745e248700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74581085b0 0x7f7458198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f745e248700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7458112860 con 0x7f745806b620 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f745e248700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f74581126e0 con 0x7f745806ac60 2024-06-06T08:36:53.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f745e248700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f74581129e0 con 0x7f74581085b0 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f7456ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806b620 0x7f745819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f7456ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806b620 0x7f745819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56746/0 (socket says 172.21.15.27:56746) 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f7456ffd700 1 -- 172.21.15.27:0/2597423390 learned_addr learned my addr 172.21.15.27:0/2597423390 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f74577fe700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f745806ac60 0x7f745819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f7456ffd700 1 -- 172.21.15.27:0/2597423390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74581085b0 msgr2=0x7f7458198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f7456ffd700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74581085b0 0x7f7458198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f7456ffd700 1 -- 172.21.15.27:0/2597423390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f745806ac60 msgr2=0x7f745819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f7456ffd700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f745806ac60 0x7f745819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f7456ffd700 1 -- 172.21.15.27:0/2597423390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7458199200 con 0x7f745806b620 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.801+0000 7f74577fe700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f745806ac60 0x7f745819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.802+0000 7f7456ffd700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806b620 0x7f745819ee20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f744c009fd0 tx=0x7f744c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:53.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.802+0000 7f7454ff9700 1 -- 172.21.15.27:0/2597423390 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c027070 con 0x7f745806b620 2024-06-06T08:36:53.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.802+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7458199490 con 0x7f745806b620 2024-06-06T08:36:53.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.802+0000 7f7454ff9700 1 -- 172.21.15.27:0/2597423390 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c004150 con 0x7f745806b620 2024-06-06T08:36:53.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.802+0000 7f7454ff9700 1 -- 172.21.15.27:0/2597423390 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c00f460 con 0x7f745806b620 2024-06-06T08:36:53.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.802+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74581a57b0 con 0x7f745806b620 2024-06-06T08:36:53.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.803+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7458103c20 con 0x7f745806b620 2024-06-06T08:36:53.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.804+0000 7f7454ff9700 1 -- 172.21.15.27:0/2597423390 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f744c02a070 con 0x7f745806b620 2024-06-06T08:36:53.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.805+0000 7f7454ff9700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f74400a2c20 0x7f74400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:53.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.805+0000 7f74577fe700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f74400a2c20 0x7f74400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:53.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.805+0000 7f7454ff9700 1 -- 172.21.15.27:0/2597423390 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f744c0d0250 con 0x7f745806b620 2024-06-06T08:36:53.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.805+0000 7f74577fe700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f74400a2c20 0x7f74400a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7448005950 tx=0x7f7448005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:53.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:53.810+0000 7f7454ff9700 1 -- 172.21.15.27:0/2597423390 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f744c05e850 con 0x7f745806b620 2024-06-06T08:36:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.023+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f745802dca0 con 0x7f745806b620 2024-06-06T08:36:54.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.025+0000 7f7454ff9700 1 -- 172.21.15.27:0/2597423390 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24103 (secure 0 0 0) 0x7f744c013070 con 0x7f745806b620 2024-06-06T08:36:54.024 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:36:54.024 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":95,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:36:38.035534+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:36:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f74400a2c20 msgr2=0x7f74400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f74400a2c20 0x7f74400a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7448005950 tx=0x7f7448005f50 comp rx=0 tx=0).stop 2024-06-06T08:36:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806b620 msgr2=0x7f745819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806b620 0x7f745819ee20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f744c009fd0 tx=0x7f744c003730 comp rx=0 tx=0).stop 2024-06-06T08:36:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 shutdown_connections 2024-06-06T08:36:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f74400a2c20 0x7f74400a50e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74581085b0 0x7f7458198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f745806ac60 0x7f745819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 --2- 172.21.15.27:0/2597423390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f745806b620 0x7f745819ee20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:54.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 >> 172.21.15.27:0/2597423390 conn(0x7f74580772a0 msgr2=0x7f74580fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:54.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 shutdown_connections 2024-06-06T08:36:54.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:54.027+0000 7f745e248700 1 -- 172.21.15.27:0/2597423390 wait complete. 2024-06-06T08:36:54.403 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:54 smithi027 ceph-mon[32335]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:54.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:54 smithi203 ceph-mon[36579]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:54.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:54 smithi176 ceph-mon[36786]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:54.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-06T08:36:23.190518+0000', 'modified': '2024-06-06T08:36:38.035723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.176:6839/1702529484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 1702529484}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 1702529484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24482, 'qdb_cluster': [24482]} 2024-06-06T08:36:54.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-06T08:36:54.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T08:36:54.978 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:55.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:55 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2597423390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:36:55.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:55 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2597423390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:36:55.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:55 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2597423390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:36:56.378 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:56 smithi027 ceph-mon[32335]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:56.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:56 smithi203 ceph-mon[36579]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:56.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:56 smithi176 ceph-mon[36786]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.634+0000 7f212ab16700 1 -- 172.21.15.27:0/3338712719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 msgr2=0x7f2124101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.634+0000 7f212ab16700 1 --2- 172.21.15.27:0/3338712719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 0x7f2124101a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f21180099c0 tx=0x7f2118009cd0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.635+0000 7f212ab16700 1 -- 172.21.15.27:0/3338712719 shutdown_connections 2024-06-06T08:36:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.635+0000 7f212ab16700 1 --2- 172.21.15.27:0/3338712719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21241066b0 0x7f212410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.635+0000 7f212ab16700 1 --2- 172.21.15.27:0/3338712719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 0x7f2124101a00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.635+0000 7f212ab16700 1 --2- 172.21.15.27:0/3338712719 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2124101fd0 0x7f2124105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.635+0000 7f212ab16700 1 -- 172.21.15.27:0/3338712719 >> 172.21.15.27:0/3338712719 conn(0x7f2124078340 msgr2=0x7f2124078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.635+0000 7f212ab16700 1 -- 172.21.15.27:0/3338712719 shutdown_connections 2024-06-06T08:36:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.635+0000 7f212ab16700 1 -- 172.21.15.27:0/3338712719 wait complete. 2024-06-06T08:36:56.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f212ab16700 1 Processor -- start 2024-06-06T08:36:56.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f212ab16700 1 -- start start 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f212ab16700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 0x7f2124199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f212ab16700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2124101fd0 0x7f212419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f212ab16700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21241066b0 0x7f21241a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f212ab16700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2124111710 con 0x7f2124101620 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f212ab16700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2124111590 con 0x7f2124101fd0 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f212ab16700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2124111890 con 0x7f21241066b0 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f21288b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 0x7f2124199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f21288b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 0x7f2124199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56766/0 (socket says 172.21.15.27:56766) 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f21288b2700 1 -- 172.21.15.27:0/1480446384 learned_addr learned my addr 172.21.15.27:0/1480446384 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f2123fff700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2124101fd0 0x7f212419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f21290b3700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21241066b0 0x7f21241a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.636+0000 7f21288b2700 1 -- 172.21.15.27:0/1480446384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21241066b0 msgr2=0x7f21241a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f21288b2700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21241066b0 0x7f21241a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f21288b2700 1 -- 172.21.15.27:0/1480446384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2124101fd0 msgr2=0x7f212419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f21288b2700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2124101fd0 0x7f212419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f21288b2700 1 -- 172.21.15.27:0/1480446384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21241a50d0 con 0x7f2124101620 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f2123fff700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2124101fd0 0x7f212419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f21290b3700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21241066b0 0x7f21241a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f21288b2700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 0x7f2124199b10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2118006010 tx=0x7f2118009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f2121ffb700 1 -- 172.21.15.27:0/1480446384 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118027070 con 0x7f2124101620 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f212ab16700 1 -- 172.21.15.27:0/1480446384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21241a52a0 con 0x7f2124101620 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f2121ffb700 1 -- 172.21.15.27:0/1480446384 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2118004050 con 0x7f2124101620 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f2121ffb700 1 -- 172.21.15.27:0/1480446384 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118022e10 con 0x7f2124101620 2024-06-06T08:36:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.637+0000 7f212ab16700 1 -- 172.21.15.27:0/1480446384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241a5750 con 0x7f2124101620 2024-06-06T08:36:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.639+0000 7f2121ffb700 1 -- 172.21.15.27:0/1480446384 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f211802a070 con 0x7f2124101620 2024-06-06T08:36:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.640+0000 7f2121ffb700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21140a2ca0 0x7f21140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.641+0000 7f2123fff700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21140a2ca0 0x7f21140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.641+0000 7f2121ffb700 1 -- 172.21.15.27:0/1480446384 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f21180d0010 con 0x7f2124101620 2024-06-06T08:36:56.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.641+0000 7f2123fff700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21140a2ca0 0x7f21140a5160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2110005950 tx=0x7f21100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.641+0000 7f2135399b80 1 Processor -- start 2024-06-06T08:36:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.641+0000 7f2135399b80 1 -- start start 2024-06-06T08:36:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2135399b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2135399b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2135399b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 0x55c2458e2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2135399b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c2457d4f10 con 0x55c2458e8830 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2135399b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c245833800 con 0x55c2458e2750 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2135399b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c24579b410 con 0x55c2458e7f10 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2123fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2123fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458e2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56768/0 (socket says 172.21.15.27:56768) 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2123fff700 1 -- 172.21.15.27:0/53005982 learned_addr learned my addr 172.21.15.27:0/53005982 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f21288b2700 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f21290b3700 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 0x55c2458e2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2123fff700 1 -- 172.21.15.27:0/53005982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 msgr2=0x55c2458e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2123fff700 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458e82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2123fff700 1 -- 172.21.15.27:0/53005982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 msgr2=0x55c2458e2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:36:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2123fff700 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 0x55c2458e2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2123fff700 1 -- 172.21.15.27:0/53005982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c245782880 con 0x55c2458e8830 2024-06-06T08:36:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f21290b3700 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 0x55c2458e2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:36:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.642+0000 7f2123fff700 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458e2210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f211000fd70 tx=0x7f2110012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb14f1244b8ab31 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f210effd700 1 -- 172.21.15.27:0/53005982 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110003880 con 0x55c2458e8830 2024-06-06T08:36:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f210effd700 1 -- 172.21.15.27:0/53005982 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2110024070 con 0x55c2458e8830 2024-06-06T08:36:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f210effd700 1 -- 172.21.15.27:0/53005982 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211001fdc0 con 0x55c2458e8830 2024-06-06T08:36:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f2135399b80 1 -- 172.21.15.27:0/53005982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 msgr2=0x55c2458e2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f2135399b80 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458e2210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f211000fd70 tx=0x7f2110012000 comp rx=0 tx=0).stop 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f2135399b80 1 -- 172.21.15.27:0/53005982 shutdown_connections 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f2135399b80 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 0x55c2458e2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f2135399b80 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f2135399b80 1 --2- 172.21.15.27:0/53005982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458e2210 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f2135399b80 1 -- 172.21.15.27:0/53005982 >> 172.21.15.27:0/53005982 conn(0x55c24580f030 msgr2=0x55c2458dd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f2135399b80 1 -- 172.21.15.27:0/53005982 shutdown_connections 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.643+0000 7f2135399b80 1 -- 172.21.15.27:0/53005982 wait complete. 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f2135399b80 1 -- start start 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f2135399b80 1 Processor -- start 2024-06-06T08:36:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f2135399b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 0x55c2458ee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f2135399b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458eb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f2135399b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458ebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f2135399b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c2457d3f70 con 0x55c2458e8830 2024-06-06T08:36:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f2135399b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c24582f7c0 con 0x55c2458e2750 2024-06-06T08:36:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f2135399b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c2458ef210 con 0x55c2458e7f10 2024-06-06T08:36:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f2135399b80 10 client.0 _ll_register_callbacks cb 0x55c2457c9560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f21290b3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458ebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f21290b3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458ebdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56770/0 (socket says 172.21.15.27:56770) 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.644+0000 7f21290b3700 1 -- 172.21.15.27:0/755417768 learned_addr learned my addr 172.21.15.27:0/755417768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f21288b2700 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 0x55c2458ee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2123fff700 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458eb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f21290b3700 1 -- 172.21.15.27:0/755417768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 msgr2=0x55c2458eb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f21290b3700 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458eb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f21290b3700 1 -- 172.21.15.27:0/755417768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 msgr2=0x55c2458ee570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f21290b3700 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 0x55c2458ee570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f21290b3700 1 -- 172.21.15.27:0/755417768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2457f9700 con 0x55c2458e8830 2024-06-06T08:36:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2123fff700 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458eb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:36:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f21290b3700 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458ebdb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f211c00da40 tx=0x7f211c00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a30e1d475e4592 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2103fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:36:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2103fff700 1 -- 172.21.15.27:0/755417768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c00f8e0 con 0x55c2458e8830 2024-06-06T08:36:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2103fff700 1 -- 172.21.15.27:0/755417768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f211c00fa80 con 0x55c2458e8830 2024-06-06T08:36:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2135399b80 1 -- 172.21.15.27:0/755417768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2110024070 con 0x55c2458e8830 2024-06-06T08:36:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2103fff700 1 -- 172.21.15.27:0/755417768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f211c022050 con 0x55c2458e8830 2024-06-06T08:36:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2135399b80 10 client.14673 fetch_fsmap learned FSMap version 16 2024-06-06T08:36:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2135399b80 1 -- 172.21.15.27:0/755417768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55c2458fdd50 con 0x55c2458e8830 2024-06-06T08:36:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.645+0000 7f2103fff700 1 -- 172.21.15.27:0/755417768 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2110024070 con 0x55c2458e8830 2024-06-06T08:36:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f2103fff700 1 -- 172.21.15.27:0/755417768 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f211c0163d0 con 0x55c2458e8830 2024-06-06T08:36:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f2135399b80 10 client.14673 fetch_fsmap finished waiting for FSMap version 16 2024-06-06T08:36:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f2135399b80 10 client.14673 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T08:36:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f2135399b80 20 client.14673 populate_metadata read hostname 'smithi027' 2024-06-06T08:36:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f2135399b80 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c245900400 0x55c245901870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f2135399b80 4 client.14673 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f2135399b80 1 -- 172.21.15.27:0/755417768 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c245840d00 con 0x55c245900400 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f21288b2700 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c245900400 0x55c245901870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f2103fff700 10 client.14673 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.646+0000 7f21288b2700 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c245900400 0x55c245901870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce74b83b682cca5 server_cookie=bc6700272adf5f4 in_seq=0 out_seq=0 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2103fff700 1 -- 172.21.15.27:0/755417768 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c245840d00 con 0x55c245900400 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2103fff700 10 client.14673 handle_command_reply: tid=0 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2135399b80 1 client.14673 shutdown 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2135399b80 1 -- 172.21.15.27:0/755417768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 msgr2=0x55c2458ebdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2135399b80 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458ebdb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f211c00da40 tx=0x7f211c00db20 comp rx=0 tx=0).stop 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2135399b80 1 -- 172.21.15.27:0/755417768 shutdown_connections 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2135399b80 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c245900400 0x55c245901870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2135399b80 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e8830 0x55c2458ebdb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2135399b80 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e2750 0x55c2458ee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2103fff700 0 client.14673 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:36:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.647+0000 7f2135399b80 1 --2- 172.21.15.27:0/755417768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458eb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.648+0000 7f2135399b80 1 -- 172.21.15.27:0/755417768 >> 172.21.15.27:0/755417768 conn(0x55c24580f030 msgr2=0x55c2458dc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.648+0000 7f2135399b80 1 -- 172.21.15.27:0/755417768 shutdown_connections 2024-06-06T08:36:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.648+0000 7f2135399b80 1 -- 172.21.15.27:0/755417768 wait complete. 2024-06-06T08:36:56.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.648+0000 7f2135399b80 20 client.14673 trim_cache size 0 max 16384 2024-06-06T08:36:56.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.669+0000 7f2135399b80 1 Processor -- start 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f2135399b80 1 -- start start 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f2135399b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e2750 0x55c2458ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f2135399b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f2135399b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e8830 0x55c2458dce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f2135399b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c2458ffbb0 con 0x55c2458e2750 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f2135399b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c245900280 con 0x55c2458e8830 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f2135399b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c245900950 con 0x55c2458e7f10 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f21288b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e2750 0x55c2458ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f21288b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e2750 0x55c2458ebe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56786/0 (socket says 172.21.15.27:56786) 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f21288b2700 1 -- 172.21.15.27:0/3157407019 learned_addr learned my addr 172.21.15.27:0/3157407019 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:56.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f2123fff700 1 --2- 172.21.15.27:0/3157407019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f21288b2700 1 -- 172.21.15.27:0/3157407019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 msgr2=0x55c2458e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f21288b2700 1 --2- 172.21.15.27:0/3157407019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f21288b2700 1 -- 172.21.15.27:0/3157407019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e8830 msgr2=0x55c2458dce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f21288b2700 1 --2- 172.21.15.27:0/3157407019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e8830 0x55c2458dce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f21288b2700 1 -- 172.21.15.27:0/3157407019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2458ef210 con 0x55c2458e2750 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f21288b2700 1 --2- 172.21.15.27:0/3157407019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e2750 0x55c2458ebe00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2118099090 tx=0x7f21180a3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c646827f68461 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.670+0000 7f210e7fc700 1 -- 172.21.15.27:0/3157407019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180a52a0 con 0x55c2458e2750 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f210e7fc700 1 -- 172.21.15.27:0/3157407019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21180870c0 con 0x55c2458e2750 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f210e7fc700 1 -- 172.21.15.27:0/3157407019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180a4e10 con 0x55c2458e2750 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f2135399b80 1 -- 172.21.15.27:0/3157407019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e2750 msgr2=0x55c2458ebe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:56.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f2135399b80 1 --2- 172.21.15.27:0/3157407019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e2750 0x55c2458ebe00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2118099090 tx=0x7f21180a3040 comp rx=0 tx=0).stop 2024-06-06T08:36:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f2135399b80 1 -- 172.21.15.27:0/3157407019 shutdown_connections 2024-06-06T08:36:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f2135399b80 1 --2- 172.21.15.27:0/3157407019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e8830 0x55c2458dce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f2135399b80 1 --2- 172.21.15.27:0/3157407019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e2750 0x55c2458ebe00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f2135399b80 1 --2- 172.21.15.27:0/3157407019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e7f10 0x55c2458e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f2135399b80 1 -- 172.21.15.27:0/3157407019 >> 172.21.15.27:0/3157407019 conn(0x55c24580f030 msgr2=0x55c2458e8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f2135399b80 1 -- 172.21.15.27:0/3157407019 shutdown_connections 2024-06-06T08:36:56.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.671+0000 7f2135399b80 1 -- 172.21.15.27:0/3157407019 wait complete. 2024-06-06T08:36:56.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2135399b80 1 -- start start 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2135399b80 1 Processor -- start 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2135399b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e2750 0x55c24590aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2135399b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e7f10 0x55c24590b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2135399b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e8830 0x55c2458dc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2135399b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c2459016f0 con 0x55c2458e7f10 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2135399b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c245901020 con 0x55c2458e8830 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2135399b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c245901dc0 con 0x55c2458e2750 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2135399b80 10 client.0 _ll_register_callbacks cb 0x55c2457c9560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2123fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e7f10 0x55c24590b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2123fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e7f10 0x55c24590b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56798/0 (socket says 172.21.15.27:56798) 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2123fff700 1 -- 172.21.15.27:0/818499988 learned_addr learned my addr 172.21.15.27:0/818499988 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f21290b3700 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e8830 0x55c2458dc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f21288b2700 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e2750 0x55c24590aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.672+0000 7f2123fff700 1 -- 172.21.15.27:0/818499988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e2750 msgr2=0x55c24590aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2123fff700 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e2750 0x55c24590aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2123fff700 1 -- 172.21.15.27:0/818499988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e8830 msgr2=0x55c2458dc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2123fff700 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e8830 0x55c2458dc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2123fff700 1 -- 172.21.15.27:0/818499988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2458dcb20 con 0x55c2458e7f10 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f21288b2700 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e2750 0x55c24590aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2123fff700 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e7f10 0x55c24590b530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f211000fd70 tx=0x7f2110003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a424696414775877 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2103fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2103fff700 1 -- 172.21.15.27:0/818499988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110003d90 con 0x55c2458e7f10 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2135399b80 1 -- 172.21.15.27:0/818499988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21180870c0 con 0x55c2458e7f10 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2103fff700 1 -- 172.21.15.27:0/818499988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f211001f460 con 0x55c2458e7f10 2024-06-06T08:36:56.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2103fff700 1 -- 172.21.15.27:0/818499988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f211001f660 con 0x55c2458e7f10 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2135399b80 10 client.14679 fetch_fsmap learned FSMap version 16 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2135399b80 1 -- 172.21.15.27:0/818499988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55c24590a9f0 con 0x55c2458e7f10 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2103fff700 1 -- 172.21.15.27:0/818499988 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21180870c0 con 0x55c2458e7f10 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2103fff700 1 -- 172.21.15.27:0/818499988 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f2110011780 con 0x55c2458e7f10 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.673+0000 7f2135399b80 10 client.14679 fetch_fsmap finished waiting for FSMap version 16 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f2135399b80 10 client.14679 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f2135399b80 20 client.14679 populate_metadata read hostname 'smithi027' 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f2135399b80 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c245970450 0x55c24596f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f2135399b80 4 client.14679 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f2135399b80 1 -- 172.21.15.27:0/818499988 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "status"}) -- 0x55c24583b280 con 0x55c245970450 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f21288b2700 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c245970450 0x55c24596f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f21288b2700 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c245970450 0x55c24596f160 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689aa4bb5f01443e server_cookie=8981b0239ca16d5a in_seq=0 out_seq=0 2024-06-06T08:36:56.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f2103fff700 10 client.14679 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f2103fff700 1 -- 172.21.15.27:0/818499988 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c24583b280 con 0x55c245970450 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f2103fff700 10 client.14679 handle_command_reply: tid=0 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.674+0000 7f2135399b80 1 client.14679 shutdown 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 -- 172.21.15.27:0/818499988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e7f10 msgr2=0x55c24590b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e7f10 0x55c24590b530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f211000fd70 tx=0x7f2110003960 comp rx=0 tx=0).stop 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 -- 172.21.15.27:0/818499988 shutdown_connections 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c2458e8830 0x55c2458dc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2458e2750 0x55c24590aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c245970450 0x55c24596f160 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 --2- 172.21.15.27:0/818499988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c2458e7f10 0x55c24590b530 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 -- 172.21.15.27:0/818499988 >> 172.21.15.27:0/818499988 conn(0x55c24580f030 msgr2=0x55c2458f74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2103fff700 0 client.14679 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 -- 172.21.15.27:0/818499988 shutdown_connections 2024-06-06T08:36:56.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 1 -- 172.21.15.27:0/818499988 wait complete. 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.675+0000 7f2135399b80 20 client.14679 trim_cache size 0 max 16384 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24482, 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 32.775678810000002, 2024-06-06T08:36:56.680 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 16, 2024-06-06T08:36:56.681 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 95, 2024-06-06T08:36:56.681 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 95, 2024-06-06T08:36:56.681 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 51.378602839999999 2024-06-06T08:36:56.681 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:36:56.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 -- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21140a2ca0 msgr2=0x7f21140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:56.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21140a2ca0 0x7f21140a5160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2110005950 tx=0x7f21100058e0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 -- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 msgr2=0x7f2124199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 0x7f2124199b10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2118006010 tx=0x7f2118009f80 comp rx=0 tx=0).stop 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 -- 172.21.15.27:0/1480446384 shutdown_connections 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f21140a2ca0 0x7f21140a5160 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21241066b0 0x7f21241a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2124101620 0x7f2124199b10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 --2- 172.21.15.27:0/1480446384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2124101fd0 0x7f212419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 -- 172.21.15.27:0/1480446384 >> 172.21.15.27:0/1480446384 conn(0x7f2124078340 msgr2=0x7f2124100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 -- 172.21.15.27:0/1480446384 shutdown_connections 2024-06-06T08:36:56.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:56.676+0000 7f212ab16700 1 -- 172.21.15.27:0/1480446384 wait complete. 2024-06-06T08:36:57.442 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-06T08:36:57.442 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-06T08:36:57.443 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-06T08:36:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:36:58.077 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:58 smithi027 ceph-mon[32335]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:58.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:58 smithi203 ceph-mon[36579]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:58.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:58 smithi176 ceph-mon[36786]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:36:59.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]: dispatch 2024-06-06T08:36:59.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-06T08:36:59.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-06-06T08:36:59.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-06-06T08:36:59.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-06-06T08:36:59.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-06-06T08:36:59.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-06-06T08:36:59.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-06-06T08:36:59.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:36:59 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-06-06T08:36:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.400+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/4052613294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 msgr2=0x7f6bbc10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.400+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/4052613294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 0x7f6bbc10e240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400b0a0 tx=0x7f6bb400b3b0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.400+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/4052613294 shutdown_connections 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.400+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/4052613294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 0x7f6bbc10e240 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.400+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/4052613294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bbc107400 0x7f6bbc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.400+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/4052613294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bbc101390 0x7f6bbc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.400+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/4052613294 >> 172.21.15.27:0/4052613294 conn(0x7f6bbc078340 msgr2=0x7f6bbc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.400+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/4052613294 shutdown_connections 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.400+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/4052613294 wait complete. 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc3b7f700 1 Processor -- start 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc3b7f700 1 -- start start 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc3b7f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bbc101390 0x7f6bbc199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc3b7f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 0x7f6bbc19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc3b7f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bbc107400 0x7f6bbc1a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc3b7f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc111740 con 0x7f6bbc101d50 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc3b7f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc1115c0 con 0x7f6bbc107400 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc3b7f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc1118c0 con 0x7f6bbc101390 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc111a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 0x7f6bbc19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc111a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 0x7f6bbc19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56814/0 (socket says 172.21.15.27:56814) 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc111a700 1 -- 172.21.15.27:0/3698661831 learned_addr learned my addr 172.21.15.27:0/3698661831 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc191b700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bbc101390 0x7f6bbc199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.401+0000 7f6bc211c700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bbc107400 0x7f6bbc1a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bc111a700 1 -- 172.21.15.27:0/3698661831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bbc101390 msgr2=0x7f6bbc199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bc111a700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bbc101390 0x7f6bbc199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bc111a700 1 -- 172.21.15.27:0/3698661831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bbc107400 msgr2=0x7f6bbc1a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bc111a700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bbc107400 0x7f6bbc1a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bc111a700 1 -- 172.21.15.27:0/3698661831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc1a5080 con 0x7f6bbc101d50 2024-06-06T08:36:59.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bc191b700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bbc101390 0x7f6bbc199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:36:59.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bc111a700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 0x7f6bbc19a000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00b530 tx=0x7f6bac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:59.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bb2ffd700 1 -- 172.21.15.27:0/3698661831 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac015020 con 0x7f6bbc101d50 2024-06-06T08:36:59.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc1a52b0 con 0x7f6bbc101d50 2024-06-06T08:36:59.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bb2ffd700 1 -- 172.21.15.27:0/3698661831 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bac011d10 con 0x7f6bbc101d50 2024-06-06T08:36:59.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bb2ffd700 1 -- 172.21.15.27:0/3698661831 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac00f2e0 con 0x7f6bbc101d50 2024-06-06T08:36:59.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.402+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc1a57c0 con 0x7f6bbc101d50 2024-06-06T08:36:59.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.403+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bbc102ab0 con 0x7f6bbc101d50 2024-06-06T08:36:59.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.404+0000 7f6bb2ffd700 1 -- 172.21.15.27:0/3698661831 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6bac018020 con 0x7f6bbc101d50 2024-06-06T08:36:59.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.405+0000 7f6bb2ffd700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ba80a2c20 0x7f6ba80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:36:59.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.405+0000 7f6bc191b700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ba80a2c20 0x7f6ba80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:36:59.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.405+0000 7f6bb2ffd700 1 -- 172.21.15.27:0/3698661831 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9071+0+0 (secure 0 0 0) 0x7f6bac0cee00 con 0x7f6bbc101d50 2024-06-06T08:36:59.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.406+0000 7f6bc191b700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ba80a2c20 0x7f6ba80a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8005950 tx=0x7f6bb8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:36:59.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.409+0000 7f6bb2ffd700 1 -- 172.21.15.27:0/3698661831 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6bac01f020 con 0x7f6bbc101d50 2024-06-06T08:36:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]: dispatch 2024-06-06T08:36:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-06T08:36:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-06-06T08:36:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-06-06T08:36:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-06-06T08:36:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-06-06T08:36:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-06-06T08:36:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-06-06T08:36:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:36:59 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-06-06T08:36:59.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]: dispatch 2024-06-06T08:36:59.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-06T08:36:59.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-06-06T08:36:59.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-06-06T08:36:59.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-06-06T08:36:59.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-06-06T08:36:59.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-06-06T08:36:59.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-06-06T08:36:59.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:36:59 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-06-06T08:36:59.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.666+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6bbc02dca0 con 0x7f6bbc101d50 2024-06-06T08:36:59.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.667+0000 7f6bb2ffd700 1 -- 172.21.15.27:0/3698661831 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f6bac09ba20 con 0x7f6bbc101d50 2024-06-06T08:36:59.666 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ba80a2c20 msgr2=0x7f6ba80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ba80a2c20 0x7f6ba80a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8005950 tx=0x7f6bb8005f50 comp rx=0 tx=0).stop 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 msgr2=0x7f6bbc19a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 0x7f6bbc19a000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00b530 tx=0x7f6bac00b840 comp rx=0 tx=0).stop 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 shutdown_connections 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6ba80a2c20 0x7f6ba80a50e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bbc107400 0x7f6bbc1a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bbc101390 0x7f6bbc199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 --2- 172.21.15.27:0/3698661831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc101d50 0x7f6bbc19a000 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:36:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 >> 172.21.15.27:0/3698661831 conn(0x7f6bbc078340 msgr2=0x7f6bbc0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:36:59.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 shutdown_connections 2024-06-06T08:36:59.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:36:59.669+0000 7f6bc3b7f700 1 -- 172.21.15.27:0/3698661831 wait complete. 2024-06-06T08:37:00.315 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]': finished 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T08:37:00.316 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:00 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3698661831' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T08:37:00.349 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-06T08:37:00.349 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-06T08:37:00.349 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-06-06T08:37:00.349 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-06T08:37:00.350 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-06T08:37:00.350 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-06T08:37:00.350 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-06T08:37:00.350 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-06T08:37:00.350 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-06T08:37:00.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:00.350 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-06-06T08:37:00.368 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42211sec preferred_lft 42211sec 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-06-06T08:37:00.369 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-06-06T08:37:00.370 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-06T08:37:00.370 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:37:00.370 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:37:00.370 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-06-06T08:37:00.370 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-06-06T08:37:00.370 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-06-06T08:37:00.371 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-06T08:37:00.371 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:37:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:37:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:37:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]': finished 2024-06-06T08:37:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-06T08:37:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-06-06T08:37:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-06-06T08:37:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-06-06T08:37:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-06-06T08:37:00.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-06-06T08:37:00.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-06-06T08:37:00.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-06-06T08:37:00.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T08:37:00.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:00 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3698661831' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T08:37:00.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:37:00.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4, 7, 11]}]': finished 2024-06-06T08:37:00.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-06T08:37:00.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-06-06T08:37:00.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-06-06T08:37:00.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-06-06T08:37:00.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-06-06T08:37:00.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-06-06T08:37:00.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-06-06T08:37:00.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-06-06T08:37:00.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T08:37:00.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:00 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3698661831' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T08:37:00.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:37:00.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:00.589 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-06T08:37:00.614 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-06-06T08:37:00.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:00.616 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-06-06T08:37:00.694 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-06-06T08:37:00.694 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-06T08:37:00.695 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-06T08:37:00.695 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-06T08:37:00.695 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-06T08:37:00.695 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:37:00.695 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:37:00.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-06T08:37:00.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-06T08:37:00.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-06T08:37:00.696 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:37:00.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:37:01.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:37:01.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:01.022 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-06-06T08:37:01.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:01.039 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-06-06T08:37:01.095 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:37:01.095 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:37:01.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T08:37:01.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-06T08:37:01.095 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:37:01.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:37:01.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:01 smithi027 ceph-mon[32335]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T08:37:01.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:37:01.219 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-06T08:37:01.219 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:37:01.220 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:37:01.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-06T08:37:01.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-06T08:37:01.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-06T08:37:01.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-06T08:37:01.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-06T08:37:01.220 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:37:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:37:01.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:37:01.355 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:37:01.355 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:37:01.355 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-06-06T08:37:01.355 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-06-06T08:37:01.355 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:37:01.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:37:01.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:37:01.437 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-06T08:37:01.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T08:37:01.437 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-06T08:37:01.459 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-06T08:37:01.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T08:37:01.460 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-06T08:37:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:01 smithi203 ceph-mon[36579]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T08:37:01.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:01 smithi176 ceph-mon[36786]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T08:37:01.518 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-06-06T08:37:01.585 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T08:37:01.642 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-06-06T08:37:01.642 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-06-06T08:37:01.642 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-06-06T08:37:01.642 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-06-06T08:37:01.642 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-06-06T08:37:01.643 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-06-06T08:37:01.643 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-06-06T08:37:01.643 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-06-06T08:37:01.643 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-06-06T08:37:01.643 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-06-06T08:37:01.643 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-06-06T08:37:01.643 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-06-06T08:37:01.643 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T08:37:01.644 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-06-06T08:37:01.645 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-06-06T08:37:01.645 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-06-06T08:37:01.645 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/3d530c915985816c5218f654a842e67ddc57e69181e490b3470ac9453969cd11/merged 2024-06-06T08:37:01.645 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1dcf77226f081bd5c95b5f84456fc9e675fb6e58ecf6b59c9dbd8330f6c22b2a/merged 2024-06-06T08:37:01.645 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-06-06T08:37:01.645 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T08:37:01.646 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/8b0bea1cabe90991826b3467c888c2f2a2599d0259a726751e09a2a06fae32b1/merged 2024-06-06T08:37:01.646 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/a68666cd512993969ebc496a2217df6aa3de7d658dfcb116d933cf3ecd1fd9fe/merged 2024-06-06T08:37:01.646 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/bafb4b412a701f7588c13275c8ac22218a95d4bcfefb9cd26945852d29e0c511/merged 2024-06-06T08:37:01.646 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/82d8db50a9e4c114c2e88c27794d1e28fa70da366193608a230b26d14f478401/merged 2024-06-06T08:37:01.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/26db7cf14f198310443e41c74970bfefc16299715f03f081a8f02818f023ffe6/merged 2024-06-06T08:37:01.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1d66b1c7d9519712072b919e68eb1504f8dc037496add7c0deaff0adca34f84b/merged 2024-06-06T08:37:01.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/a4f2c71ab1d7b0eafe3b4b01131de7649f801f6a52a06dbf54271ec6e1d9e9ba/merged 2024-06-06T08:37:01.647 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/3f5aa4526724d634d8f5887e5e30bac0e81a00a4076efeec9d6f2dafc6f5add3/merged 2024-06-06T08:37:01.647 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-06-06T08:37:01.647 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T08:37:01.647 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T08:37:01.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:01.648 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-06-06T08:37:01.698 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-06T08:37:01.698 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-06T08:37:01.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-06-06T08:37:01.767 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T08:37:01.825 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-06-06T08:37:01.826 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T08:37:01.827 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/3d530c915985816c5218f654a842e67ddc57e69181e490b3470ac9453969cd11/merged 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1dcf77226f081bd5c95b5f84456fc9e675fb6e58ecf6b59c9dbd8330f6c22b2a/merged 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/8b0bea1cabe90991826b3467c888c2f2a2599d0259a726751e09a2a06fae32b1/merged 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/a68666cd512993969ebc496a2217df6aa3de7d658dfcb116d933cf3ecd1fd9fe/merged 2024-06-06T08:37:01.828 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/bafb4b412a701f7588c13275c8ac22218a95d4bcfefb9cd26945852d29e0c511/merged 2024-06-06T08:37:01.829 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/82d8db50a9e4c114c2e88c27794d1e28fa70da366193608a230b26d14f478401/merged 2024-06-06T08:37:01.829 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/26db7cf14f198310443e41c74970bfefc16299715f03f081a8f02818f023ffe6/merged 2024-06-06T08:37:01.829 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/1d66b1c7d9519712072b919e68eb1504f8dc037496add7c0deaff0adca34f84b/merged 2024-06-06T08:37:01.829 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/a4f2c71ab1d7b0eafe3b4b01131de7649f801f6a52a06dbf54271ec6e1d9e9ba/merged 2024-06-06T08:37:01.829 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/3f5aa4526724d634d8f5887e5e30bac0e81a00a4076efeec9d6f2dafc6f5add3/merged 2024-06-06T08:37:01.829 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-06-06T08:37:01.829 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T08:37:01.829 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T08:37:01.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:01.829 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-06-06T08:37:01.865 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:2024-06-06T08:37:01.865+0000 7fcb06ce8480 -1 init, newargv = 0x55f964578840 newargc=15 2024-06-06T08:37:01.865 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[78352]: starting ceph client 2024-06-06T08:37:01.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[78352]: starting fuse 2024-06-06T08:37:01.881 INFO:teuthology.orchestra.run.smithi027.stdout:119 2024-06-06T08:37:01.881 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> import re 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> import os 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> 2024-06-06T08:37:01.882 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> return f 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.admin")) 2024-06-06T08:37:01.883 DEBUG:teuthology.orchestra.run.smithi027:> ' 2024-06-06T08:37:02.003 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.admin.78352.asok 2024-06-06T08:37:02.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:37:02.014 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78352.asok 2024-06-06T08:37:02.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:02.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78352.asok status 2024-06-06T08:37:02.153 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:37:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2024-06-06T08:37:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-06T08:37:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T08:37:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "admin", 2024-06-06T08:37:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2024-06-06T08:37:02.153 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "78352", 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 14691, 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 14691 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:37:02.154 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3958896977 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3958896977 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.14691 192.168.0.1:0/3958896977", 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3958896977", 2024-06-06T08:37:02.155 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2024-06-06T08:37:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 16, 2024-06-06T08:37:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 97, 2024-06-06T08:37:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2024-06-06T08:37:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2024-06-06T08:37:02.156 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2024-06-06T08:37:02.156 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:37:02.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:02 smithi027 ceph-mon[32335]: pgmap v328: 129 pgs: 2 activating, 127 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:37:02.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:02.162 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2024-06-06T08:37:02.162 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-06T08:37:02.178 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2024-06-06T08:37:02.178 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-06T08:37:02.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:02.179 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-06T08:37:02.246 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-06T08:37:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T08:37:02.362 INFO:teuthology.orchestra.run.smithi027.stdout:# file: . 2024-06-06T08:37:02.362 INFO:teuthology.orchestra.run.smithi027.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-06T08:37:02.362 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:37:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:37:02.368 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:37:02.368 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T08:37:02.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T08:37:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:02 smithi203 ceph-mon[36579]: pgmap v328: 129 pgs: 2 activating, 127 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:37:02.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:02 smithi176 ceph-mon[36786]: pgmap v328: 129 pgs: 2 activating, 127 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:37:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:04.161 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:04 smithi027 ceph-mon[32335]: pgmap v329: 129 pgs: 2 activating, 127 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:37:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:04 smithi203 ceph-mon[36579]: pgmap v329: 129 pgs: 2 activating, 127 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:37:04.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:04 smithi176 ceph-mon[36786]: pgmap v329: 129 pgs: 2 activating, 127 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:37:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.576+0000 7f6db8920700 1 -- 172.21.15.27:0/2688803789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41066b0 msgr2=0x7f6db410e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.576+0000 7f6db8920700 1 --2- 172.21.15.27:0/2688803789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41066b0 0x7f6db410e280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099c0 tx=0x7f6dac009cd0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.576+0000 7f6db8920700 1 -- 172.21.15.27:0/2688803789 shutdown_connections 2024-06-06T08:37:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.576+0000 7f6db8920700 1 --2- 172.21.15.27:0/2688803789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41066b0 0x7f6db410e280 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.577+0000 7f6db8920700 1 --2- 172.21.15.27:0/2688803789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6db41015d0 0x7f6db41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.577+0000 7f6db8920700 1 --2- 172.21.15.27:0/2688803789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6db4101f80 0x7f6db4105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.577+0000 7f6db8920700 1 -- 172.21.15.27:0/2688803789 >> 172.21.15.27:0/2688803789 conn(0x7f6db4078400 msgr2=0x7f6db4078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.577+0000 7f6db8920700 1 -- 172.21.15.27:0/2688803789 shutdown_connections 2024-06-06T08:37:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.577+0000 7f6db8920700 1 -- 172.21.15.27:0/2688803789 wait complete. 2024-06-06T08:37:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.577+0000 7f6db8920700 1 Processor -- start 2024-06-06T08:37:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.577+0000 7f6db8920700 1 -- start start 2024-06-06T08:37:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db8920700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41015d0 0x7f6db410ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db8920700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6db4101f80 0x7f6db410f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db8920700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6db41066b0 0x7f6db410f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db8920700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4112810 con 0x7f6db41015d0 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db8920700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4112690 con 0x7f6db41066b0 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db8920700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4112990 con 0x7f6db4101f80 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41015d0 0x7f6db410ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41015d0 0x7f6db410ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52588/0 (socket says 172.21.15.27:52588) 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db259c700 1 -- 172.21.15.27:0/85373586 learned_addr learned my addr 172.21.15.27:0/85373586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db2d9d700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6db41066b0 0x7f6db410f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db1d9b700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6db4101f80 0x7f6db410f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db259c700 1 -- 172.21.15.27:0/85373586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6db4101f80 msgr2=0x7f6db410f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db259c700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6db4101f80 0x7f6db410f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db259c700 1 -- 172.21.15.27:0/85373586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6db41066b0 msgr2=0x7f6db410f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db259c700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6db41066b0 0x7f6db410f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db259c700 1 -- 172.21.15.27:0/85373586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db41ab630 con 0x7f6db41015d0 2024-06-06T08:37:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db1d9b700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6db4101f80 0x7f6db410f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:37:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db2d9d700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6db41066b0 0x7f6db410f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.578+0000 7f6db259c700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41015d0 0x7f6db410ec60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6da400ba50 tx=0x7f6da400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.579+0000 7f6dab7fe700 1 -- 172.21.15.27:0/85373586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4019070 con 0x7f6db41015d0 2024-06-06T08:37:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.579+0000 7f6dab7fe700 1 -- 172.21.15.27:0/85373586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da4014440 con 0x7f6db41015d0 2024-06-06T08:37:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.579+0000 7f6dab7fe700 1 -- 172.21.15.27:0/85373586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da40075d0 con 0x7f6db41015d0 2024-06-06T08:37:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.579+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db41ab8c0 con 0x7f6db41015d0 2024-06-06T08:37:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.579+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db41abe50 con 0x7f6db41015d0 2024-06-06T08:37:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.580+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db4102ab0 con 0x7f6db41015d0 2024-06-06T08:37:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.581+0000 7f6dab7fe700 1 -- 172.21.15.27:0/85373586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6da401c070 con 0x7f6db41015d0 2024-06-06T08:37:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.582+0000 7f6dab7fe700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6da00a2ca0 0x7f6da00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.582+0000 7f6dab7fe700 1 -- 172.21.15.27:0/85373586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f6da40ceca0 con 0x7f6db41015d0 2024-06-06T08:37:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.582+0000 7f6db1d9b700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6da00a2ca0 0x7f6da00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.583+0000 7f6db1d9b700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6da00a2ca0 0x7f6da00a5160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c005fd0 tx=0x7f6d9c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.586+0000 7f6dab7fe700 1 -- 172.21.15.27:0/85373586 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6da409b8c0 con 0x7f6db41015d0 2024-06-06T08:37:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.806+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6db4110190 con 0x7f6db41015d0 2024-06-06T08:37:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.807+0000 7f6dab7fe700 1 -- 172.21.15.27:0/85373586 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+589 (secure 0 0 0) 0x7f6da409b1c0 con 0x7f6db41015d0 2024-06-06T08:37:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1905066984 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2182206949 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2182206949 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1183901040 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1624199721 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/897818490 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/897818490 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3611712452 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1620281536 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/752935431 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6da00a2ca0 msgr2=0x7f6da00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6da00a2ca0 0x7f6da00a5160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c005fd0 tx=0x7f6d9c005f00 comp rx=0 tx=0).stop 2024-06-06T08:37:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41015d0 msgr2=0x7f6db410ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41015d0 0x7f6db410ec60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6da400ba50 tx=0x7f6da400be10 comp rx=0 tx=0).stop 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 shutdown_connections 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6da00a2ca0 0x7f6da00a5160 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6db41066b0 0x7f6db410f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6db41015d0 0x7f6db410ec60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 --2- 172.21.15.27:0/85373586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6db4101f80 0x7f6db410f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 >> 172.21.15.27:0/85373586 conn(0x7f6db4078400 msgr2=0x7f6db40ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 shutdown_connections 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:04.809+0000 7f6db8920700 1 -- 172.21.15.27:0/85373586 wait complete. 2024-06-06T08:37:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:listed 10 entries 2024-06-06T08:37:05.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:05 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/85373586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:37:05.482 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:37:05.482 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T08:37:05.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:05 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/85373586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:37:05.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T08:37:05.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:05 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/85373586' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:37:05.791 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:06 smithi027 ceph-mon[32335]: pgmap v330: 129 pgs: 2 activating, 127 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-06-06T08:37:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:06 smithi203 ceph-mon[36579]: pgmap v330: 129 pgs: 2 activating, 127 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-06-06T08:37:06.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:06 smithi176 ceph-mon[36786]: pgmap v330: 129 pgs: 2 activating, 127 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-06-06T08:37:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.482+0000 7fc82a76a700 1 -- 172.21.15.27:0/1375738289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc824101d40 msgr2=0x7fc82410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.482+0000 7fc82a76a700 1 --2- 172.21.15.27:0/1375738289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc824101d40 0x7fc82410e230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00b0a0 tx=0x7fc81c00b3b0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.482+0000 7fc82a76a700 1 -- 172.21.15.27:0/1375738289 shutdown_connections 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.482+0000 7fc82a76a700 1 --2- 172.21.15.27:0/1375738289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc824101d40 0x7fc82410e230 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.482+0000 7fc82a76a700 1 --2- 172.21.15.27:0/1375738289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8241073f0 0x7fc8241077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.482+0000 7fc82a76a700 1 --2- 172.21.15.27:0/1375738289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc824101380 0x7fc824101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.482+0000 7fc82a76a700 1 -- 172.21.15.27:0/1375738289 >> 172.21.15.27:0/1375738289 conn(0x7fc824078300 msgr2=0x7fc824078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.482+0000 7fc82a76a700 1 -- 172.21.15.27:0/1375738289 shutdown_connections 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.482+0000 7fc82a76a700 1 -- 172.21.15.27:0/1375738289 wait complete. 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.483+0000 7fc82a76a700 1 Processor -- start 2024-06-06T08:37:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.483+0000 7fc82a76a700 1 -- start start 2024-06-06T08:37:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc82a76a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc824101380 0x7fc82419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc82a76a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc824101d40 0x7fc82419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc82a76a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8241073f0 0x7fc82419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc82a76a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc824111660 con 0x7fc8241073f0 2024-06-06T08:37:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc82a76a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241114e0 con 0x7fc824101380 2024-06-06T08:37:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc82a76a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241117e0 con 0x7fc824101d40 2024-06-06T08:37:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc823fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc824101380 0x7fc82419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc828d07700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8241073f0 0x7fc82419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc823fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc824101380 0x7fc82419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:34950/0 (socket says 172.21.15.27:34950) 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc823fff700 1 -- 172.21.15.27:0/725553403 learned_addr learned my addr 172.21.15.27:0/725553403 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc8237fe700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc824101d40 0x7fc82419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc828d07700 1 -- 172.21.15.27:0/725553403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc824101d40 msgr2=0x7fc82419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc828d07700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc824101d40 0x7fc82419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc828d07700 1 -- 172.21.15.27:0/725553403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc824101380 msgr2=0x7fc82419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc828d07700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc824101380 0x7fc82419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc828d07700 1 -- 172.21.15.27:0/725553403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82419fdc0 con 0x7fc8241073f0 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc828d07700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8241073f0 0x7fc82419f680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d010 tx=0x7fc81c004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.484+0000 7fc8217fa700 1 -- 172.21.15.27:0/725553403 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c021070 con 0x7fc8241073f0 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.485+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8241a0050 con 0x7fc8241073f0 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.485+0000 7fc8217fa700 1 -- 172.21.15.27:0/725553403 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc81c007b80 con 0x7fc8241073f0 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.485+0000 7fc8217fa700 1 -- 172.21.15.27:0/725553403 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0052a0 con 0x7fc8241073f0 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.485+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8241a7930 con 0x7fc8241073f0 2024-06-06T08:37:07.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.486+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc824069180 con 0x7fc8241073f0 2024-06-06T08:37:07.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.487+0000 7fc8217fa700 1 -- 172.21.15.27:0/725553403 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc81c024070 con 0x7fc8241073f0 2024-06-06T08:37:07.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.488+0000 7fc8217fa700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc8100a2c20 0x7fc8100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:07.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.488+0000 7fc823fff700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc8100a2c20 0x7fc8100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:07.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.488+0000 7fc8217fa700 1 -- 172.21.15.27:0/725553403 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7fc81c00f020 con 0x7fc8241073f0 2024-06-06T08:37:07.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.488+0000 7fc823fff700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc8100a2c20 0x7fc8100a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc814005950 tx=0x7fc81400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:07.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.491+0000 7fc8217fa700 1 -- 172.21.15.27:0/725553403 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc81c02b020 con 0x7fc8241073f0 2024-06-06T08:37:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.709+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc8241a7b00 con 0x7fc8241073f0 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.709+0000 7fc8217fa700 1 -- 172.21.15.27:0/725553403 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+589 (secure 0 0 0) 0x7fc81c09d650 con 0x7fc8241073f0 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1905066984 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2182206949 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2182206949 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1183901040 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1624199721 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/897818490 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/897818490 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3611712452 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1620281536 2024-06-07T08:28:31.314626+0000 2024-06-06T08:37:07.708 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/752935431 2024-06-07T08:28:58.346290+0000 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc8100a2c20 msgr2=0x7fc8100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc8100a2c20 0x7fc8100a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc814005950 tx=0x7fc81400b500 comp rx=0 tx=0).stop 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8241073f0 msgr2=0x7fc82419f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8241073f0 0x7fc82419f680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d010 tx=0x7fc81c004d50 comp rx=0 tx=0).stop 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 shutdown_connections 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc8100a2c20 0x7fc8100a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8241073f0 0x7fc82419f680 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc824101380 0x7fc82419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 --2- 172.21.15.27:0/725553403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc824101d40 0x7fc82419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 >> 172.21.15.27:0/725553403 conn(0x7fc824078300 msgr2=0x7fc8240ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 shutdown_connections 2024-06-06T08:37:07.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:07.711+0000 7fc82a76a700 1 -- 172.21.15.27:0/725553403 wait complete. 2024-06-06T08:37:07.711 INFO:teuthology.orchestra.run.smithi027.stderr:listed 10 entries 2024-06-06T08:37:08.329 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:08 smithi027 ceph-mon[32335]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-06-06T08:37:08.329 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:08 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/725553403' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:37:08.356 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2024-06-06T08:37:08.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:08.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-06T08:37:08.384 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[78352]: fuse finished with error 0 and tester_r 0 2024-06-06T08:37:08.386 INFO:teuthology.orchestra.run:waiting for 300 2024-06-06T08:37:08.439 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:37:08.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:37:08.440 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-06T08:37:08.463 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:37:08.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:08 smithi203 ceph-mon[36579]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-06-06T08:37:08.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:08 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/725553403' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:37:08.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:08 smithi176 ceph-mon[36786]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-06-06T08:37:08.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:08 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/725553403' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:37:08.750 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:10.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:10 smithi027 ceph-mon[32335]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 435 B/s wr, 0 op/s; 171 B/s, 0 objects/s recovering 2024-06-06T08:37:10.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.415+0000 7ff77738d700 1 -- 172.21.15.27:0/1034988539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7701030f0 msgr2=0x7ff770107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:10.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.415+0000 7ff77738d700 1 --2- 172.21.15.27:0/1034988539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7701030f0 0x7ff770107110 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff7600099c0 tx=0x7ff760009cd0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.416+0000 7ff77738d700 1 -- 172.21.15.27:0/1034988539 shutdown_connections 2024-06-06T08:37:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.416+0000 7ff77738d700 1 --2- 172.21.15.27:0/1034988539 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff770107840 0x7ff77010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.416+0000 7ff77738d700 1 --2- 172.21.15.27:0/1034988539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770102740 0x7ff770102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.416+0000 7ff77738d700 1 --2- 172.21.15.27:0/1034988539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7701030f0 0x7ff770107110 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.416+0000 7ff77738d700 1 -- 172.21.15.27:0/1034988539 >> 172.21.15.27:0/1034988539 conn(0x7ff7700795f0 msgr2=0x7ff770079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.416+0000 7ff77738d700 1 -- 172.21.15.27:0/1034988539 shutdown_connections 2024-06-06T08:37:10.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.416+0000 7ff77738d700 1 -- 172.21.15.27:0/1034988539 wait complete. 2024-06-06T08:37:10.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.416+0000 7ff77738d700 1 Processor -- start 2024-06-06T08:37:10.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.416+0000 7ff77738d700 1 -- start start 2024-06-06T08:37:10.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff77738d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff770102740 0x7ff77019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:10.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff77738d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7701030f0 0x7ff77019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:10.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff77738d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770107840 0x7ff770198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:10.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff77738d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701127d0 con 0x7ff770102740 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff77738d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff770112650 con 0x7ff7701030f0 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff77738d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff770112950 con 0x7ff770107840 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff775129700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff770102740 0x7ff77019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff775129700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff770102740 0x7ff77019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59084/0 (socket says 172.21.15.27:59084) 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff775129700 1 -- 172.21.15.27:0/2603707427 learned_addr learned my addr 172.21.15.27:0/2603707427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff774928700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7701030f0 0x7ff77019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff77592a700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770107840 0x7ff770198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff775129700 1 -- 172.21.15.27:0/2603707427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770107840 msgr2=0x7ff770198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff775129700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770107840 0x7ff770198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff775129700 1 -- 172.21.15.27:0/2603707427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7701030f0 msgr2=0x7ff77019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff775129700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7701030f0 0x7ff77019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff775129700 1 -- 172.21.15.27:0/2603707427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff770199250 con 0x7ff770102740 2024-06-06T08:37:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff77592a700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770107840 0x7ff770198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:37:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.417+0000 7ff775129700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff770102740 0x7ff77019e8e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b530 tx=0x7ff76c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.418+0000 7ff7667fc700 1 -- 172.21.15.27:0/2603707427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c015020 con 0x7ff770102740 2024-06-06T08:37:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.418+0000 7ff7667fc700 1 -- 172.21.15.27:0/2603707427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff76c011d10 con 0x7ff770102740 2024-06-06T08:37:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.418+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff770199540 con 0x7ff770102740 2024-06-06T08:37:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.418+0000 7ff7667fc700 1 -- 172.21.15.27:0/2603707427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c00f2e0 con 0x7ff770102740 2024-06-06T08:37:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.418+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7701a57b0 con 0x7ff770102740 2024-06-06T08:37:10.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.419+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77010aa80 con 0x7ff770102740 2024-06-06T08:37:10.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.420+0000 7ff7667fc700 1 -- 172.21.15.27:0/2603707427 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff76c018020 con 0x7ff770102740 2024-06-06T08:37:10.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.420+0000 7ff7667fc700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff75c0a2ca0 0x7ff75c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:10.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.421+0000 7ff774928700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff75c0a2ca0 0x7ff75c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:10.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.421+0000 7ff7667fc700 1 -- 172.21.15.27:0/2603707427 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7ff76c0cec40 con 0x7ff770102740 2024-06-06T08:37:10.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.421+0000 7ff774928700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff75c0a2ca0 0x7ff75c0a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff760000c00 tx=0x7ff760005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:10.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.425+0000 7ff7667fc700 1 -- 172.21.15.27:0/2603707427 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff76c01f020 con 0x7ff770102740 2024-06-06T08:37:10.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:10 smithi203 ceph-mon[36579]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 435 B/s wr, 0 op/s; 171 B/s, 0 objects/s recovering 2024-06-06T08:37:10.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:10 smithi176 ceph-mon[36786]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 435 B/s wr, 0 op/s; 171 B/s, 0 objects/s recovering 2024-06-06T08:37:10.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.633+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff770199fd0 con 0x7ff770102740 2024-06-06T08:37:10.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.634+0000 7ff7667fc700 1 -- 172.21.15.27:0/2603707427 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24602 (secure 0 0 0) 0x7ff76c09b800 con 0x7ff770102740 2024-06-06T08:37:10.633 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:37:10.633 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":97,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:37:00.035534+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:37:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff75c0a2ca0 msgr2=0x7ff75c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff75c0a2ca0 0x7ff75c0a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff760000c00 tx=0x7ff760005fb0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff770102740 msgr2=0x7ff77019e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff770102740 0x7ff77019e8e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b530 tx=0x7ff76c00b840 comp rx=0 tx=0).stop 2024-06-06T08:37:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 shutdown_connections 2024-06-06T08:37:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff75c0a2ca0 0x7ff75c0a5160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff770107840 0x7ff770198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff770102740 0x7ff77019e8e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 --2- 172.21.15.27:0/2603707427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7701030f0 0x7ff77019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:10.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.636+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 >> 172.21.15.27:0/2603707427 conn(0x7ff7700795f0 msgr2=0x7ff770108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:10.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.637+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 shutdown_connections 2024-06-06T08:37:10.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:10.637+0000 7ff77738d700 1 -- 172.21.15.27:0/2603707427 wait complete. 2024-06-06T08:37:11.289 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:11 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2603707427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:37:11.320 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:37:11.408 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:11 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2603707427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:37:11.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:11 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2603707427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:37:11.572 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:12.406 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:12 smithi027 ceph-mon[32335]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 492 B/s wr, 0 op/s; 155 B/s, 0 objects/s recovering 2024-06-06T08:37:12.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:12 smithi203 ceph-mon[36579]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 492 B/s wr, 0 op/s; 155 B/s, 0 objects/s recovering 2024-06-06T08:37:12.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:12 smithi176 ceph-mon[36786]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 492 B/s wr, 0 op/s; 155 B/s, 0 objects/s recovering 2024-06-06T08:37:13.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.285+0000 7f83ed0e8700 1 -- 172.21.15.27:0/4063449002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e8101380 msgr2=0x7f83e8101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:13.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.285+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/4063449002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e8101380 0x7f83e8101800 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009990 tx=0x7f83dc009ca0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.285+0000 7f83ed0e8700 1 -- 172.21.15.27:0/4063449002 shutdown_connections 2024-06-06T08:37:13.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.285+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/4063449002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83e8101d40 0x7f83e810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.285+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/4063449002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e81073f0 0x7f83e81077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.285+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/4063449002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e8101380 0x7f83e8101800 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.286+0000 7f83ed0e8700 1 -- 172.21.15.27:0/4063449002 >> 172.21.15.27:0/4063449002 conn(0x7f83e8078300 msgr2=0x7f83e8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:13.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.286+0000 7f83ed0e8700 1 -- 172.21.15.27:0/4063449002 shutdown_connections 2024-06-06T08:37:13.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.286+0000 7f83ed0e8700 1 -- 172.21.15.27:0/4063449002 wait complete. 2024-06-06T08:37:13.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.286+0000 7f83ed0e8700 1 Processor -- start 2024-06-06T08:37:13.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83ed0e8700 1 -- start start 2024-06-06T08:37:13.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83ed0e8700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83e8101380 0x7f83e819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:13.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83ed0e8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e8101d40 0x7f83e819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:13.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83ed0e8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e81073f0 0x7f83e81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:13.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83ed0e8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8111660 con 0x7f83e81073f0 2024-06-06T08:37:13.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83ed0e8700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81114e0 con 0x7f83e8101380 2024-06-06T08:37:13.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83ed0e8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81117e0 con 0x7f83e8101d40 2024-06-06T08:37:13.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e81073f0 0x7f83e81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:13.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e8101d40 0x7f83e819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:13.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e6d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83e8101380 0x7f83e819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:13.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e81073f0 0x7f83e81989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59096/0 (socket says 172.21.15.27:59096) 2024-06-06T08:37:13.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e8101d40 0x7f83e819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:37780/0 (socket says 172.21.15.27:37780) 2024-06-06T08:37:13.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e759e700 1 -- 172.21.15.27:0/2389033155 learned_addr learned my addr 172.21.15.27:0/2389033155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e6d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83e8101380 0x7f83e819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:37016/0 (socket says 172.21.15.27:37016) 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e6d9d700 1 -- 172.21.15.27:0/2389033155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e8101d40 msgr2=0x7f83e819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e6d9d700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e8101d40 0x7f83e819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e6d9d700 1 -- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e81073f0 msgr2=0x7f83e81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e6d9d700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e81073f0 0x7f83e81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e6d9d700 1 -- 172.21.15.27:0/2389033155 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e8199140 con 0x7f83e8101380 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.287+0000 7f83e759e700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e81073f0 0x7f83e81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.288+0000 7f83e6d9d700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83e8101380 0x7f83e819e820 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83d800d950 tx=0x7f83d800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.288+0000 7f83cffff700 1 -- 172.21.15.27:0/2389033155 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8016070 con 0x7f83e8101380 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.288+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e8199430 con 0x7f83e8101380 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.288+0000 7f83cffff700 1 -- 172.21.15.27:0/2389033155 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d800fd70 con 0x7f83e8101380 2024-06-06T08:37:13.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.288+0000 7f83cffff700 1 -- 172.21.15.27:0/2389033155 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d800e3b0 con 0x7f83e8101380 2024-06-06T08:37:13.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.288+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e81a56f0 con 0x7f83e8101380 2024-06-06T08:37:13.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.289+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e8069180 con 0x7f83e8101380 2024-06-06T08:37:13.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.290+0000 7f83cffff700 1 -- 172.21.15.27:0/2389033155 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f83d8019070 con 0x7f83e8101380 2024-06-06T08:37:13.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.291+0000 7f83cffff700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f83d00a2c20 0x7f83d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:13.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.291+0000 7f83e659c700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f83d00a2c20 0x7f83d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:13.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.291+0000 7f83cffff700 1 -- 172.21.15.27:0/2389033155 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f83d80ceea0 con 0x7f83e8101380 2024-06-06T08:37:13.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.291+0000 7f83e659c700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f83d00a2c20 0x7f83d00a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0098e0 tx=0x7f83dc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:13.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.295+0000 7f83cffff700 1 -- 172.21.15.27:0/2389033155 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f83d809ba60 con 0x7f83e8101380 2024-06-06T08:37:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:13 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:37:13.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:13 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:37:13.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:13 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:37:13.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.511+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83e8199f10 con 0x7f83e8101380 2024-06-06T08:37:13.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.513+0000 7f83cffff700 1 -- 172.21.15.27:0/2389033155 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24602 (secure 0 0 0) 0x7f83d809b360 con 0x7f83e8101380 2024-06-06T08:37:13.512 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:37:13.512 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":97,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:37:00.035534+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f83d00a2c20 msgr2=0x7f83d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f83d00a2c20 0x7f83d00a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0098e0 tx=0x7f83dc005960 comp rx=0 tx=0).stop 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83e8101380 msgr2=0x7f83e819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83e8101380 0x7f83e819e820 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83d800d950 tx=0x7f83d800dc60 comp rx=0 tx=0).stop 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 shutdown_connections 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f83d00a2c20 0x7f83d00a50e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83e81073f0 0x7f83e81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83e8101380 0x7f83e819e820 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 --2- 172.21.15.27:0/2389033155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e8101d40 0x7f83e819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 >> 172.21.15.27:0/2389033155 conn(0x7f83e8078300 msgr2=0x7f83e81005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 shutdown_connections 2024-06-06T08:37:13.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:13.516+0000 7f83ed0e8700 1 -- 172.21.15.27:0/2389033155 wait complete. 2024-06-06T08:37:14.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:14 smithi027 ceph-mon[32335]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-06-06T08:37:14.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:14 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2389033155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:37:14.184 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-06T08:37:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:14 smithi203 ceph-mon[36579]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-06-06T08:37:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:14 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2389033155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:37:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:14 smithi176 ceph-mon[36786]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-06-06T08:37:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:14 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2389033155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:37:14.922 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:15.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:15 smithi027 ceph-mon[32335]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-06-06T08:37:15.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:15.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:15.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:15.663 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:15 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:15.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:15 smithi203 ceph-mon[36579]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-06-06T08:37:15.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:15.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:15.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:15.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:15 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:16.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:15 smithi176 ceph-mon[36786]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-06-06T08:37:16.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:16.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:16.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:16.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:15 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.704+0000 7f00dbd13700 1 -- 172.21.15.27:0/388239652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 msgr2=0x7f00d41088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.704+0000 7f00dbd13700 1 --2- 172.21.15.27:0/388239652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 0x7f00d41088d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009990 tx=0x7f00d0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:37:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.704+0000 7f00dbd13700 1 -- 172.21.15.27:0/388239652 shutdown_connections 2024-06-06T08:37:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.704+0000 7f00dbd13700 1 --2- 172.21.15.27:0/388239652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d4102e80 0x7f00d410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.704+0000 7f00dbd13700 1 --2- 172.21.15.27:0/388239652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 0x7f00d41088d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.704+0000 7f00dbd13700 1 --2- 172.21.15.27:0/388239652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00d41024e0 0x7f00d4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.704+0000 7f00dbd13700 1 -- 172.21.15.27:0/388239652 >> 172.21.15.27:0/388239652 conn(0x7f00d4079530 msgr2=0x7f00d4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.704+0000 7f00dbd13700 1 -- 172.21.15.27:0/388239652 shutdown_connections 2024-06-06T08:37:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.704+0000 7f00dbd13700 1 -- 172.21.15.27:0/388239652 wait complete. 2024-06-06T08:37:16.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.705+0000 7f00dbd13700 1 Processor -- start 2024-06-06T08:37:16.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.705+0000 7f00dbd13700 1 -- start start 2024-06-06T08:37:16.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.705+0000 7f00dbd13700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41024e0 0x7f00d419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:16.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.705+0000 7f00dbd13700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00d4102e80 0x7f00d419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.705+0000 7f00dbd13700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 0x7f00d4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00dbd13700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00d41127b0 con 0x7f00d41084f0 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00dbd13700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f00d4112630 con 0x7f00d4102e80 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00dbd13700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00d4112930 con 0x7f00d41024e0 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00da2b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 0x7f00d4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00da2b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 0x7f00d4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59120/0 (socket says 172.21.15.27:59120) 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00da2b0700 1 -- 172.21.15.27:0/1251429467 learned_addr learned my addr 172.21.15.27:0/1251429467 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00d92ae700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00d4102e80 0x7f00d419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00d9aaf700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41024e0 0x7f00d419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00da2b0700 1 -- 172.21.15.27:0/1251429467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41024e0 msgr2=0x7f00d419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00da2b0700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41024e0 0x7f00d419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00da2b0700 1 -- 172.21.15.27:0/1251429467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00d4102e80 msgr2=0x7f00d419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00da2b0700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00d4102e80 0x7f00d419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:16.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00da2b0700 1 -- 172.21.15.27:0/1251429467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d4199250 con 0x7f00d41084f0 2024-06-06T08:37:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00d92ae700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00d4102e80 0x7f00d419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00da2b0700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 0x7f00d4198af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00ada0 tx=0x7f00cc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00caffd700 1 -- 172.21.15.27:0/1251429467 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc017020 con 0x7f00d41084f0 2024-06-06T08:37:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d4199540 con 0x7f00d41084f0 2024-06-06T08:37:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.706+0000 7f00caffd700 1 -- 172.21.15.27:0/1251429467 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00cc004bb0 con 0x7f00d41084f0 2024-06-06T08:37:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.707+0000 7f00caffd700 1 -- 172.21.15.27:0/1251429467 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc005520 con 0x7f00d41084f0 2024-06-06T08:37:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.707+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d41a57b0 con 0x7f00d41084f0 2024-06-06T08:37:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.708+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d4069180 con 0x7f00d41084f0 2024-06-06T08:37:16.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.709+0000 7f00caffd700 1 -- 172.21.15.27:0/1251429467 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f00cc0056c0 con 0x7f00d41084f0 2024-06-06T08:37:16.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.709+0000 7f00caffd700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00c00a2c20 0x7f00c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:16.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.710+0000 7f00d9aaf700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00c00a2c20 0x7f00c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:16.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.710+0000 7f00caffd700 1 -- 172.21.15.27:0/1251429467 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f00cc0ceb00 con 0x7f00d41084f0 2024-06-06T08:37:16.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.710+0000 7f00d9aaf700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00c00a2c20 0x7f00c00a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00d0006010 tx=0x7f00d000b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:16.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:16.714+0000 7f00caffd700 1 -- 172.21.15.27:0/1251429467 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f00cc021020 con 0x7f00d41084f0 2024-06-06T08:37:17.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.021+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f00d402dca0 con 0x7f00d41084f0 2024-06-06T08:37:17.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.876+0000 7f00caffd700 1 -- 172.21.15.27:0/1251429467 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f00cc09b7b0 con 0x7f00d41084f0 2024-06-06T08:37:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00c00a2c20 msgr2=0x7f00c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00c00a2c20 0x7f00c00a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00d0006010 tx=0x7f00d000b580 comp rx=0 tx=0).stop 2024-06-06T08:37:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 msgr2=0x7f00d4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 0x7f00d4198af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00ada0 tx=0x7f00cc00c330 comp rx=0 tx=0).stop 2024-06-06T08:37:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 shutdown_connections 2024-06-06T08:37:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f00c00a2c20 0x7f00c00a50e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d41084f0 0x7f00d4198af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:17.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d41024e0 0x7f00d419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:17.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 --2- 172.21.15.27:0/1251429467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00d4102e80 0x7f00d419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:17.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 >> 172.21.15.27:0/1251429467 conn(0x7f00d4079530 msgr2=0x7f00d41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:17.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 shutdown_connections 2024-06-06T08:37:17.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:17.878+0000 7f00dbd13700 1 -- 172.21.15.27:0/1251429467 wait complete. 2024-06-06T08:37:18.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:17 smithi027 ceph-mon[32335]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T08:37:18.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:18.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:18.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:17 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1251429467' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T08:37:18.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:37:18.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:37:18.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:17 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:18.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:17 smithi203 ceph-mon[36579]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T08:37:18.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:18.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:18.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:17 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1251429467' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T08:37:18.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:37:18.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:37:18.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:17 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:18.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:17 smithi176 ceph-mon[36786]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T08:37:18.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:18.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:17 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1251429467' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T08:37:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:37:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:37:18.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:17 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:37:18.535 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-06-06T08:37:18.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-06-06T08:37:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:19.130 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:18 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1251429467' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T08:37:19.131 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:18 smithi027 ceph-mon[32335]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:37:19.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:18 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1251429467' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T08:37:19.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:18 smithi203 ceph-mon[36579]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:37:19.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:18 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1251429467' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T08:37:19.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:18 smithi176 ceph-mon[36786]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:37:20.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:19 smithi027 ceph-mon[32335]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T08:37:20.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:19 smithi203 ceph-mon[36579]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T08:37:20.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:19 smithi176 ceph-mon[36786]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T08:37:20.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.473+0000 7f7980d53700 1 -- 172.21.15.27:0/3363766676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101d50 msgr2=0x7f797c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:20.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.473+0000 7f7980d53700 1 --2- 172.21.15.27:0/3363766676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101d50 0x7f797c10e240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f797400b0a0 tx=0x7f797400b3b0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.473+0000 7f7980d53700 1 -- 172.21.15.27:0/3363766676 shutdown_connections 2024-06-06T08:37:20.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.473+0000 7f7980d53700 1 --2- 172.21.15.27:0/3363766676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101d50 0x7f797c10e240 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.473+0000 7f7980d53700 1 --2- 172.21.15.27:0/3363766676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f797c107400 0x7f797c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.473+0000 7f7980d53700 1 --2- 172.21.15.27:0/3363766676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c101390 0x7f797c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.473+0000 7f7980d53700 1 -- 172.21.15.27:0/3363766676 >> 172.21.15.27:0/3363766676 conn(0x7f797c078340 msgr2=0x7f797c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.473+0000 7f7980d53700 1 -- 172.21.15.27:0/3363766676 shutdown_connections 2024-06-06T08:37:20.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.473+0000 7f7980d53700 1 -- 172.21.15.27:0/3363766676 wait complete. 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.474+0000 7f7980d53700 1 Processor -- start 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.474+0000 7f7980d53700 1 -- start start 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.474+0000 7f7980d53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101390 0x7f797c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f7980d53700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f797c101d50 0x7f797c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f7980d53700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c107400 0x7f797c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f7980d53700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1116a0 con 0x7f797c101390 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f7980d53700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f797c111520 con 0x7f797c101d50 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f7980d53700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f797c111820 con 0x7f797c107400 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101390 0x7f797c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101390 0x7f797c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43486/0 (socket says 172.21.15.27:43486) 2024-06-06T08:37:20.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797a59c700 1 -- 172.21.15.27:0/3594492726 learned_addr learned my addr 172.21.15.27:0/3594492726 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f7979d9b700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f797c101d50 0x7f797c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797ad9d700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c107400 0x7f797c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797a59c700 1 -- 172.21.15.27:0/3594492726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c107400 msgr2=0x7f797c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797a59c700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c107400 0x7f797c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797a59c700 1 -- 172.21.15.27:0/3594492726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f797c101d50 msgr2=0x7f797c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797a59c700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f797c101d50 0x7f797c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797a59c700 1 -- 172.21.15.27:0/3594492726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c199130 con 0x7f797c101390 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797ad9d700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c107400 0x7f797c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f797a59c700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101390 0x7f797c19e810 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c920 tx=0x7f796c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f796b7fe700 1 -- 172.21.15.27:0/3594492726 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c007b00 con 0x7f797c101390 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f796b7fe700 1 -- 172.21.15.27:0/3594492726 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f796c004d10 con 0x7f797c101390 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c199420 con 0x7f797c101390 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.475+0000 7f796b7fe700 1 -- 172.21.15.27:0/3594492726 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c005680 con 0x7f797c101390 2024-06-06T08:37:20.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.476+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c1a56e0 con 0x7f797c101390 2024-06-06T08:37:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.476+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797c069180 con 0x7f797c101390 2024-06-06T08:37:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.478+0000 7f796b7fe700 1 -- 172.21.15.27:0/3594492726 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f796c004750 con 0x7f797c101390 2024-06-06T08:37:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.478+0000 7f796b7fe700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f79640a2ca0 0x7f79640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.478+0000 7f7979d9b700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f79640a2ca0 0x7f79640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:20.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.478+0000 7f796b7fe700 1 -- 172.21.15.27:0/3594492726 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f796c0cee10 con 0x7f797c101390 2024-06-06T08:37:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.479+0000 7f7979d9b700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f79640a2ca0 0x7f79640a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f797c0788d0 tx=0x7f797000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.483+0000 7f796b7fe700 1 -- 172.21.15.27:0/3594492726 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f796c014030 con 0x7f797c101390 2024-06-06T08:37:20.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.742+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f797c1a59a0 con 0x7f797c101390 2024-06-06T08:37:20.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.892+0000 7f796b7fe700 1 -- 172.21.15.27:0/3594492726 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f796c09ba30 con 0x7f797c101390 2024-06-06T08:37:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f79640a2ca0 msgr2=0x7f79640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f79640a2ca0 0x7f79640a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f797c0788d0 tx=0x7f797000b3c0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101390 msgr2=0x7f797c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101390 0x7f797c19e810 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c920 tx=0x7f796c00cc30 comp rx=0 tx=0).stop 2024-06-06T08:37:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 shutdown_connections 2024-06-06T08:37:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f79640a2ca0 0x7f79640a5160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f797c107400 0x7f797c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797c101390 0x7f797c19e810 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 --2- 172.21.15.27:0/3594492726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f797c101d50 0x7f797c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 >> 172.21.15.27:0/3594492726 conn(0x7f797c078340 msgr2=0x7f797c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 shutdown_connections 2024-06-06T08:37:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:20.894+0000 7f7980d53700 1 -- 172.21.15.27:0/3594492726 wait complete. 2024-06-06T08:37:21.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:20 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3594492726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T08:37:21.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:20 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3594492726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T08:37:21.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:20 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3594492726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T08:37:21.689 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-06T08:37:21.952 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:21.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:21 smithi027 ceph-mon[32335]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T08:37:21.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:21 smithi027 ceph-mon[32335]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T08:37:21.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:21 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3594492726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-06T08:37:21.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:21 smithi027 ceph-mon[32335]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T08:37:21.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:21 smithi027 ceph-mon[32335]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:37:21.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:21 smithi027 ceph-mon[32335]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-06T08:37:21.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:21 smithi027 ceph-mon[32335]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-06T08:37:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:21 smithi203 ceph-mon[36579]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T08:37:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:21 smithi203 ceph-mon[36579]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T08:37:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:21 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3594492726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-06T08:37:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:21 smithi203 ceph-mon[36579]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T08:37:22.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:21 smithi203 ceph-mon[36579]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:37:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:21 smithi203 ceph-mon[36579]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-06T08:37:22.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:21 smithi203 ceph-mon[36579]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-06T08:37:22.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:21 smithi176 ceph-mon[36786]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T08:37:22.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:21 smithi176 ceph-mon[36786]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T08:37:22.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:21 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3594492726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-06T08:37:22.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:21 smithi176 ceph-mon[36786]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T08:37:22.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:21 smithi176 ceph-mon[36786]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T08:37:22.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:21 smithi176 ceph-mon[36786]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-06T08:37:22.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:21 smithi176 ceph-mon[36786]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-06T08:37:23.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:22 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] up:active 2024-06-06T08:37:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:22 smithi027 ceph-mon[32335]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T08:37:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:22 smithi027 ceph-mon[32335]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T08:37:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:22 smithi027 ceph-mon[32335]: Cluster is now healthy 2024-06-06T08:37:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:22 smithi027 ceph-mon[32335]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-06T08:37:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:22 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-06T08:37:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:22 smithi027 ceph-mon[32335]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-06T08:37:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:22 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] up:active 2024-06-06T08:37:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:22 smithi203 ceph-mon[36579]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T08:37:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:22 smithi203 ceph-mon[36579]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T08:37:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:22 smithi203 ceph-mon[36579]: Cluster is now healthy 2024-06-06T08:37:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:22 smithi203 ceph-mon[36579]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-06T08:37:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:22 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-06T08:37:23.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:22 smithi203 ceph-mon[36579]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-06T08:37:23.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:22 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] up:active 2024-06-06T08:37:23.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:22 smithi176 ceph-mon[36786]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T08:37:23.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:22 smithi176 ceph-mon[36786]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T08:37:23.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:22 smithi176 ceph-mon[36786]: Cluster is now healthy 2024-06-06T08:37:23.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:22 smithi176 ceph-mon[36786]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-06T08:37:23.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:22 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-06T08:37:23.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:22 smithi176 ceph-mon[36786]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-06T08:37:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.783+0000 7f9a99c6e700 1 -- 172.21.15.27:0/1577815396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 msgr2=0x7f9a9410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.783+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/1577815396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 0x7f9a9410e210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00b0a0 tx=0x7f9a8c00b3b0 comp rx=0 tx=0).stop 2024-06-06T08:37:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.783+0000 7f9a99c6e700 1 -- 172.21.15.27:0/1577815396 shutdown_connections 2024-06-06T08:37:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.783+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/1577815396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 0x7f9a9410e210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.783+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/1577815396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a941073d0 0x7f9a941077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.783+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/1577815396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94101390 0x7f9a941017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.783+0000 7f9a99c6e700 1 -- 172.21.15.27:0/1577815396 >> 172.21.15.27:0/1577815396 conn(0x7f9a940782c0 msgr2=0x7f9a940786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.784+0000 7f9a99c6e700 1 -- 172.21.15.27:0/1577815396 shutdown_connections 2024-06-06T08:37:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.784+0000 7f9a99c6e700 1 -- 172.21.15.27:0/1577815396 wait complete. 2024-06-06T08:37:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.784+0000 7f9a99c6e700 1 Processor -- start 2024-06-06T08:37:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a99c6e700 1 -- start start 2024-06-06T08:37:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a99c6e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a94101390 0x7f9a9419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a99c6e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 0x7f9a9419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a99c6e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a941073d0 0x7f9a94198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a99c6e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a94111650 con 0x7f9a94101d30 2024-06-06T08:37:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a99c6e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941114d0 con 0x7f9a94101390 2024-06-06T08:37:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a99c6e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941117d0 con 0x7f9a941073d0 2024-06-06T08:37:23.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a92ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 0x7f9a9419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a92ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 0x7f9a9419ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43498/0 (socket says 172.21.15.27:43498) 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a92ffd700 1 -- 172.21.15.27:0/4039067420 learned_addr learned my addr 172.21.15.27:0/4039067420 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a937fe700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a94101390 0x7f9a9419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a93fff700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a941073d0 0x7f9a94198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a92ffd700 1 -- 172.21.15.27:0/4039067420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a941073d0 msgr2=0x7f9a94198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a92ffd700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a941073d0 0x7f9a94198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a92ffd700 1 -- 172.21.15.27:0/4039067420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a94101390 msgr2=0x7f9a9419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.785+0000 7f9a92ffd700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a94101390 0x7f9a9419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.786+0000 7f9a92ffd700 1 -- 172.21.15.27:0/4039067420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a941990a0 con 0x7f9a94101d30 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.786+0000 7f9a93fff700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a941073d0 0x7f9a94198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.786+0000 7f9a92ffd700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 0x7f9a9419ecc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00d980 tx=0x7f9a7c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.786+0000 7f9a90ff9700 1 -- 172.21.15.27:0/4039067420 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c017070 con 0x7f9a94101d30 2024-06-06T08:37:23.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.786+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a94199390 con 0x7f9a94101d30 2024-06-06T08:37:23.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.786+0000 7f9a90ff9700 1 -- 172.21.15.27:0/4039067420 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a7c005020 con 0x7f9a94101d30 2024-06-06T08:37:23.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.786+0000 7f9a90ff9700 1 -- 172.21.15.27:0/4039067420 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c009bc0 con 0x7f9a94101d30 2024-06-06T08:37:23.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.786+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a941a5650 con 0x7f9a94101d30 2024-06-06T08:37:23.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.787+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a94069180 con 0x7f9a94101d30 2024-06-06T08:37:23.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.788+0000 7f9a90ff9700 1 -- 172.21.15.27:0/4039067420 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9a7c0051d0 con 0x7f9a94101d30 2024-06-06T08:37:23.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.789+0000 7f9a90ff9700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a800a2ca0 0x7f9a800a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:23.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.789+0000 7f9a937fe700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a800a2ca0 0x7f9a800a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:23.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.789+0000 7f9a90ff9700 1 -- 172.21.15.27:0/4039067420 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f9a7c0cf5f0 con 0x7f9a94101d30 2024-06-06T08:37:23.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.790+0000 7f9a937fe700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a800a2ca0 0x7f9a800a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9a94199a80 tx=0x7f9a8400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:23.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:23.793+0000 7f9a90ff9700 1 -- 172.21.15.27:0/4039067420 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9a7c020020 con 0x7f9a94101d30 2024-06-06T08:37:24.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.044+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f9a941a5880 con 0x7f9a94101d30 2024-06-06T08:37:24.043 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:23 smithi027 ceph-mon[32335]: pgmap v339: 129 pgs: 129 active+clean; 582 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-06T08:37:24.043 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:23 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] up:active 2024-06-06T08:37:24.043 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:23 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-06T08:37:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:23 smithi203 ceph-mon[36579]: pgmap v339: 129 pgs: 129 active+clean; 582 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-06T08:37:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:23 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] up:active 2024-06-06T08:37:24.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:23 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-06T08:37:24.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:23 smithi176 ceph-mon[36786]: pgmap v339: 129 pgs: 129 active+clean; 582 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-06T08:37:24.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:23 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] up:active 2024-06-06T08:37:24.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:23 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-06T08:37:24.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.834+0000 7f9a90ff9700 1 -- 172.21.15.27:0/4039067420 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f9a7c09c290 con 0x7f9a94101d30 2024-06-06T08:37:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a800a2ca0 msgr2=0x7f9a800a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a800a2ca0 0x7f9a800a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9a94199a80 tx=0x7f9a8400a380 comp rx=0 tx=0).stop 2024-06-06T08:37:24.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 msgr2=0x7f9a9419ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:24.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 0x7f9a9419ecc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00d980 tx=0x7f9a7c00dc90 comp rx=0 tx=0).stop 2024-06-06T08:37:24.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 shutdown_connections 2024-06-06T08:37:24.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f9a800a2ca0 0x7f9a800a5160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:24.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a941073d0 0x7f9a94198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:24.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a94101390 0x7f9a9419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:24.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 --2- 172.21.15.27:0/4039067420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a94101d30 0x7f9a9419ecc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:24.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 >> 172.21.15.27:0/4039067420 conn(0x7f9a940782c0 msgr2=0x7f9a94100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:24.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 shutdown_connections 2024-06-06T08:37:24.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:24.837+0000 7f9a99c6e700 1 -- 172.21.15.27:0/4039067420 wait complete. 2024-06-06T08:37:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:24 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4039067420' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T08:37:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:24 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4039067420' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T08:37:25.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:24 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:37:25.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:24 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4039067420' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T08:37:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:24 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4039067420' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T08:37:25.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:24 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:37:25.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:24 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4039067420' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T08:37:25.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:24 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4039067420' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T08:37:25.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:24 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:37:25.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-06T08:37:25.855 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:26.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:25 smithi027 ceph-mon[32335]: pgmap v340: 129 pgs: 129 active+clean; 583 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-06-06T08:37:26.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:25 smithi203 ceph-mon[36579]: pgmap v340: 129 pgs: 129 active+clean; 583 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-06-06T08:37:26.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:25 smithi176 ceph-mon[36786]: pgmap v340: 129 pgs: 129 active+clean; 583 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-06-06T08:37:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.559+0000 7f8f15097700 1 -- 172.21.15.27:0/2823293401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 msgr2=0x7f8f10101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.559+0000 7f8f15097700 1 --2- 172.21.15.27:0/2823293401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 0x7f8f10101810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009990 tx=0x7f8ef8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.559+0000 7f8f15097700 1 -- 172.21.15.27:0/2823293401 shutdown_connections 2024-06-06T08:37:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.559+0000 7f8f15097700 1 --2- 172.21.15.27:0/2823293401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f10101d50 0x7f8f1010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.559+0000 7f8f15097700 1 --2- 172.21.15.27:0/2823293401 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f10107400 0x7f8f101077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.559+0000 7f8f15097700 1 --2- 172.21.15.27:0/2823293401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 0x7f8f10101810 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.559+0000 7f8f15097700 1 -- 172.21.15.27:0/2823293401 >> 172.21.15.27:0/2823293401 conn(0x7f8f10078340 msgr2=0x7f8f10078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.559+0000 7f8f15097700 1 -- 172.21.15.27:0/2823293401 shutdown_connections 2024-06-06T08:37:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.559+0000 7f8f15097700 1 -- 172.21.15.27:0/2823293401 wait complete. 2024-06-06T08:37:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.560+0000 7f8f15097700 1 Processor -- start 2024-06-06T08:37:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.560+0000 7f8f15097700 1 -- start start 2024-06-06T08:37:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f15097700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 0x7f8f1019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f15097700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f10101d50 0x7f8f1019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f15097700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f10107400 0x7f8f10197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f15097700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10111670 con 0x7f8f10101390 2024-06-06T08:37:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f15097700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101114f0 con 0x7f8f10107400 2024-06-06T08:37:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f15097700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101117f0 con 0x7f8f10101d50 2024-06-06T08:37:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 0x7f8f1019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 0x7f8f1019d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43522/0 (socket says 172.21.15.27:43522) 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0ed9d700 1 -- 172.21.15.27:0/1244759509 learned_addr learned my addr 172.21.15.27:0/1244759509 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0e59c700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f10101d50 0x7f8f1019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0f59e700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f10107400 0x7f8f10197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0ed9d700 1 -- 172.21.15.27:0/1244759509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f10101d50 msgr2=0x7f8f1019dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0ed9d700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f10101d50 0x7f8f1019dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0ed9d700 1 -- 172.21.15.27:0/1244759509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f10107400 msgr2=0x7f8f10197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0ed9d700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f10107400 0x7f8f10197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0ed9d700 1 -- 172.21.15.27:0/1244759509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f101980b0 con 0x7f8f10101390 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0f59e700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f10107400 0x7f8f10197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.561+0000 7f8f0ed9d700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 0x7f8f1019d790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000c9b0 tx=0x7f8f0000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.562+0000 7f8f07fff700 1 -- 172.21.15.27:0/1244759509 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00007b00 con 0x7f8f10101390 2024-06-06T08:37:27.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.562+0000 7f8f07fff700 1 -- 172.21.15.27:0/1244759509 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f00007ca0 con 0x7f8f10101390 2024-06-06T08:37:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.562+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f101983a0 con 0x7f8f10101390 2024-06-06T08:37:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.562+0000 7f8f07fff700 1 -- 172.21.15.27:0/1244759509 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00018730 con 0x7f8f10101390 2024-06-06T08:37:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.562+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f101a4660 con 0x7f8f10101390 2024-06-06T08:37:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.563+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f10102ab0 con 0x7f8f10101390 2024-06-06T08:37:27.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.564+0000 7f8f07fff700 1 -- 172.21.15.27:0/1244759509 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8f000188d0 con 0x7f8f10101390 2024-06-06T08:37:27.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.565+0000 7f8f07fff700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8efc0a2c20 0x7f8efc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:27.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.565+0000 7f8f0e59c700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8efc0a2c20 0x7f8efc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:27.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.565+0000 7f8f07fff700 1 -- 172.21.15.27:0/1244759509 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f8f000cf580 con 0x7f8f10101390 2024-06-06T08:37:27.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.566+0000 7f8f0e59c700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8efc0a2c20 0x7f8efc0a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800b5c0 tx=0x7f8ef8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:27.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.568+0000 7f8f07fff700 1 -- 172.21.15.27:0/1244759509 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f00014030 con 0x7f8f10101390 2024-06-06T08:37:27.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.821+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f8f1002dca0 con 0x7f8f10101390 2024-06-06T08:37:27.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.920+0000 7f8f07fff700 1 -- 172.21.15.27:0/1244759509 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f8f0009c230 con 0x7f8f10101390 2024-06-06T08:37:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8efc0a2c20 msgr2=0x7f8efc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8efc0a2c20 0x7f8efc0a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800b5c0 tx=0x7f8ef8005960 comp rx=0 tx=0).stop 2024-06-06T08:37:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 msgr2=0x7f8f1019d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 0x7f8f1019d790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000c9b0 tx=0x7f8f0000ccc0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 shutdown_connections 2024-06-06T08:37:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f8efc0a2c20 0x7f8efc0a50e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f10107400 0x7f8f10197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f10101390 0x7f8f1019d790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 --2- 172.21.15.27:0/1244759509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f10101d50 0x7f8f1019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:27.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.922+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 >> 172.21.15.27:0/1244759509 conn(0x7f8f10078340 msgr2=0x7f8f100ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:27.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.923+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 shutdown_connections 2024-06-06T08:37:27.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:27.923+0000 7f8f15097700 1 -- 172.21.15.27:0/1244759509 wait complete. 2024-06-06T08:37:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:27 smithi027 ceph-mon[32335]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.9 KiB/s wr, 6 op/s 2024-06-06T08:37:28.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:27 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1244759509' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T08:37:28.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:27 smithi203 ceph-mon[36579]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.9 KiB/s wr, 6 op/s 2024-06-06T08:37:28.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:27 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1244759509' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T08:37:28.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:27 smithi176 ceph-mon[36786]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.9 KiB/s wr, 6 op/s 2024-06-06T08:37:28.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:27 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1244759509' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T08:37:28.582 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-06-06T08:37:28.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-06T08:37:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:29.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:28 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1244759509' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T08:37:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:28 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:37:29.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:28 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1244759509' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T08:37:29.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:28 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:37:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:28 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1244759509' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T08:37:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:28 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:37:30.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:29 smithi027 ceph-mon[32335]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-06T08:37:30.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:29 smithi203 ceph-mon[36579]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-06T08:37:30.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:29 smithi176 ceph-mon[36786]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-06T08:37:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 -- 172.21.15.27:0/2974353908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264102770 msgr2=0x7f4264102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 --2- 172.21.15.27:0/2974353908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264102770 0x7f4264102b50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f425400b0d0 tx=0x7f425400b3e0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 -- 172.21.15.27:0/2974353908 shutdown_connections 2024-06-06T08:37:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 --2- 172.21.15.27:0/2974353908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4264107860 0x7f426410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 --2- 172.21.15.27:0/2974353908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264102770 0x7f4264102b50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 --2- 172.21.15.27:0/2974353908 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4264103120 0x7f4264107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 -- 172.21.15.27:0/2974353908 >> 172.21.15.27:0/2974353908 conn(0x7f4264079600 msgr2=0x7f4264079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 -- 172.21.15.27:0/2974353908 shutdown_connections 2024-06-06T08:37:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 -- 172.21.15.27:0/2974353908 wait complete. 2024-06-06T08:37:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.592+0000 7f4269ef7700 1 Processor -- start 2024-06-06T08:37:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4269ef7700 1 -- start start 2024-06-06T08:37:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4269ef7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4264102770 0x7f426419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4269ef7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4264103120 0x7f426419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4269ef7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264107860 0x7f4264198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4269ef7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f42641127d0 con 0x7f4264107860 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4269ef7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4264112650 con 0x7f4264103120 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4269ef7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4264112950 con 0x7f4264102770 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4263fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264107860 0x7f4264198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4263fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264107860 0x7f4264198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43838/0 (socket says 172.21.15.27:43838) 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4263fff700 1 -- 172.21.15.27:0/854530360 learned_addr learned my addr 172.21.15.27:0/854530360 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f42637fe700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4264102770 0x7f426419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4262ffd700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4264103120 0x7f426419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4263fff700 1 -- 172.21.15.27:0/854530360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4264102770 msgr2=0x7f426419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4263fff700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4264102770 0x7f426419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4263fff700 1 -- 172.21.15.27:0/854530360 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4264103120 msgr2=0x7f426419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4263fff700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4264103120 0x7f426419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.593+0000 7f4263fff700 1 -- 172.21.15.27:0/854530360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4264199170 con 0x7f4264107860 2024-06-06T08:37:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.594+0000 7f4263fff700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264107860 0x7f4264198a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f425c00d980 tx=0x7f425c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.594+0000 7f4260ff9700 1 -- 172.21.15.27:0/854530360 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c017070 con 0x7f4264107860 2024-06-06T08:37:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.594+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4264199460 con 0x7f4264107860 2024-06-06T08:37:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.594+0000 7f4260ff9700 1 -- 172.21.15.27:0/854530360 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f425c005020 con 0x7f4264107860 2024-06-06T08:37:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.594+0000 7f4260ff9700 1 -- 172.21.15.27:0/854530360 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c009b50 con 0x7f4264107860 2024-06-06T08:37:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.594+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42641a5720 con 0x7f4264107860 2024-06-06T08:37:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.595+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426410aaa0 con 0x7f4264107860 2024-06-06T08:37:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.596+0000 7f4260ff9700 1 -- 172.21.15.27:0/854530360 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f425c009cf0 con 0x7f4264107860 2024-06-06T08:37:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.597+0000 7f4260ff9700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f42500a2c20 0x7f42500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.597+0000 7f42637fe700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f42500a2c20 0x7f42500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.597+0000 7f4260ff9700 1 -- 172.21.15.27:0/854530360 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f425c0cf520 con 0x7f4264107860 2024-06-06T08:37:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.597+0000 7f42637fe700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f42500a2c20 0x7f42500a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4254009fd0 tx=0x7f425400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:30.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.601+0000 7f4260ff9700 1 -- 172.21.15.27:0/854530360 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f425c020020 con 0x7f4264107860 2024-06-06T08:37:30.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.837+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f42640628d0 con 0x7f42500a2c20 2024-06-06T08:37:30.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.863+0000 7f4260ff9700 1 -- 172.21.15.27:0/854530360 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42640628d0 con 0x7f42500a2c20 2024-06-06T08:37:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f42500a2c20 msgr2=0x7f42500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f42500a2c20 0x7f42500a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4254009fd0 tx=0x7f425400b040 comp rx=0 tx=0).stop 2024-06-06T08:37:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264107860 msgr2=0x7f4264198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264107860 0x7f4264198a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f425c00d980 tx=0x7f425c00dc90 comp rx=0 tx=0).stop 2024-06-06T08:37:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 shutdown_connections 2024-06-06T08:37:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f42500a2c20 0x7f42500a50e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4264107860 0x7f4264198a10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4264102770 0x7f426419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 --2- 172.21.15.27:0/854530360 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4264103120 0x7f426419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 >> 172.21.15.27:0/854530360 conn(0x7f4264079600 msgr2=0x7f4264101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.865+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 shutdown_connections 2024-06-06T08:37:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:30.866+0000 7f4269ef7700 1 -- 172.21.15.27:0/854530360 wait complete. 2024-06-06T08:37:31.174 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:37:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:37:30.842+0000 7fc20469e700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:37:31.174 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:37:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:37:30.842+0000 7fc20469e700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:37:31.174 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:37:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:37:30.842+0000 7fc20469e700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:37:31.174 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:37:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:37:30.842+0000 7fc20469e700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:37:31.175 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:37:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:37:30.842+0000 7fc20469e700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:37:31.487 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-06T08:37:31.748 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:32.012 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:31 smithi027 ceph-mon[32335]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T08:37:32.012 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:31 smithi027 ceph-mon[32335]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:31 smithi203 ceph-mon[36579]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T08:37:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:31 smithi203 ceph-mon[36579]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:32.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:31 smithi176 ceph-mon[36786]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T08:37:32.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:31 smithi176 ceph-mon[36786]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:33.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.494+0000 7f01ababb700 1 -- 172.21.15.27:0/3694026426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 msgr2=0x7f01a4105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.494+0000 7f01ababb700 1 --2- 172.21.15.27:0/3694026426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 0x7f01a4105ff0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01980099c0 tx=0x7f0198009cd0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.495+0000 7f01ababb700 1 -- 172.21.15.27:0/3694026426 shutdown_connections 2024-06-06T08:37:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.495+0000 7f01ababb700 1 --2- 172.21.15.27:0/3694026426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a4106720 0x7f01a410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.495+0000 7f01ababb700 1 --2- 172.21.15.27:0/3694026426 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01a41019d0 0x7f01a4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.495+0000 7f01ababb700 1 --2- 172.21.15.27:0/3694026426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 0x7f01a4105ff0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.495+0000 7f01ababb700 1 -- 172.21.15.27:0/3694026426 >> 172.21.15.27:0/3694026426 conn(0x7f01a4078340 msgr2=0x7f01a4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.495+0000 7f01ababb700 1 -- 172.21.15.27:0/3694026426 shutdown_connections 2024-06-06T08:37:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.495+0000 7f01ababb700 1 -- 172.21.15.27:0/3694026426 wait complete. 2024-06-06T08:37:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.495+0000 7f01ababb700 1 Processor -- start 2024-06-06T08:37:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01ababb700 1 -- start start 2024-06-06T08:37:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01ababb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01a41019d0 0x7f01a4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01ababb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 0x7f01a4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01ababb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a4106720 0x7f01a419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01ababb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f01a4112830 con 0x7f01a4102380 2024-06-06T08:37:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01ababb700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f01a41126b0 con 0x7f01a41019d0 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01ababb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01a41129b0 con 0x7f01a4106720 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9056700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 0x7f01a4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9056700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 0x7f01a4199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43884/0 (socket says 172.21.15.27:43884) 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9056700 1 -- 172.21.15.27:0/1080267003 learned_addr learned my addr 172.21.15.27:0/1080267003 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9857700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01a41019d0 0x7f01a4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01aa058700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a4106720 0x7f01a419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9056700 1 -- 172.21.15.27:0/1080267003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a4106720 msgr2=0x7f01a419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9056700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a4106720 0x7f01a419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9056700 1 -- 172.21.15.27:0/1080267003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01a41019d0 msgr2=0x7f01a4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9056700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01a41019d0 0x7f01a4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9056700 1 -- 172.21.15.27:0/1080267003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a419dd50 con 0x7f01a4102380 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01a9857700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01a41019d0 0x7f01a4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.496+0000 7f01aa058700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a4106720 0x7f01a419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:33.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.497+0000 7f01a9056700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 0x7f01a4199280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0198009fd0 tx=0x7f0198005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:33.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.497+0000 7f0196ffd700 1 -- 172.21.15.27:0/1080267003 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198027070 con 0x7f01a4102380 2024-06-06T08:37:33.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.497+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a419dfe0 con 0x7f01a4102380 2024-06-06T08:37:33.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.497+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a41a58c0 con 0x7f01a4102380 2024-06-06T08:37:33.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.497+0000 7f0196ffd700 1 -- 172.21.15.27:0/1080267003 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0198004050 con 0x7f01a4102380 2024-06-06T08:37:33.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.497+0000 7f0196ffd700 1 -- 172.21.15.27:0/1080267003 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198022e00 con 0x7f01a4102380 2024-06-06T08:37:33.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.498+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0188005330 con 0x7f01a4102380 2024-06-06T08:37:33.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.500+0000 7f0196ffd700 1 -- 172.21.15.27:0/1080267003 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f019802a070 con 0x7f01a4102380 2024-06-06T08:37:33.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.500+0000 7f0196ffd700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f01900a2cb0 0x7f01900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:33.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.500+0000 7f01a9857700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f01900a2cb0 0x7f01900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:33.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.500+0000 7f0196ffd700 1 -- 172.21.15.27:0/1080267003 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f01980d0700 con 0x7f01a4102380 2024-06-06T08:37:33.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.501+0000 7f01a9857700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f01900a2cb0 0x7f01900a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005950 tx=0x7f01a0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:33.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.504+0000 7f0196ffd700 1 -- 172.21.15.27:0/1080267003 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0198013070 con 0x7f01a4102380 2024-06-06T08:37:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.743+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0188000bc0 con 0x7f01900a2cb0 2024-06-06T08:37:33.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.772+0000 7f0196ffd700 1 -- 172.21.15.27:0/1080267003 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0188000bc0 con 0x7f01900a2cb0 2024-06-06T08:37:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f01900a2cb0 msgr2=0x7f01900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f01900a2cb0 0x7f01900a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005950 tx=0x7f01a0005f50 comp rx=0 tx=0).stop 2024-06-06T08:37:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 msgr2=0x7f01a4199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 0x7f01a4199280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0198009fd0 tx=0x7f0198005f10 comp rx=0 tx=0).stop 2024-06-06T08:37:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 shutdown_connections 2024-06-06T08:37:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f01900a2cb0 0x7f01900a5170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01a4106720 0x7f01a419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f01a41019d0 0x7f01a4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 --2- 172.21.15.27:0/1080267003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a4102380 0x7f01a4199280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 >> 172.21.15.27:0/1080267003 conn(0x7f01a4078340 msgr2=0x7f01a40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.774+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 shutdown_connections 2024-06-06T08:37:33.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:33.775+0000 7f01ababb700 1 -- 172.21.15.27:0/1080267003 wait complete. 2024-06-06T08:37:34.135 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:33 smithi027 ceph-mon[32335]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T08:37:34.135 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:33 smithi027 ceph-mon[32335]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T08:37:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:33 smithi203 ceph-mon[36579]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T08:37:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:33 smithi203 ceph-mon[36579]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T08:37:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:33 smithi176 ceph-mon[36786]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-06T08:37:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:33 smithi176 ceph-mon[36786]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T08:37:34.434 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-06T08:37:34.690 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:35.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:34 smithi027 ceph-mon[32335]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:34 smithi203 ceph-mon[36579]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:35.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:34 smithi176 ceph-mon[36786]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:36.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:35 smithi027 ceph-mon[32335]: pgmap v345: 129 pgs: 129 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-06T08:37:36.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:35 smithi203 ceph-mon[36579]: pgmap v345: 129 pgs: 129 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-06T08:37:36.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:35 smithi176 ceph-mon[36786]: pgmap v345: 129 pgs: 129 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-06T08:37:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.458+0000 7fba918da700 1 -- 172.21.15.27:0/3066534742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c108510 msgr2=0x7fba8c1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.458+0000 7fba918da700 1 --2- 172.21.15.27:0/3066534742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c108510 0x7fba8c1088f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009990 tx=0x7fba7c009ca0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.458+0000 7fba918da700 1 -- 172.21.15.27:0/3066534742 shutdown_connections 2024-06-06T08:37:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.458+0000 7fba918da700 1 --2- 172.21.15.27:0/3066534742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba8c101440 0x7fba8c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.458+0000 7fba918da700 1 --2- 172.21.15.27:0/3066534742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c108510 0x7fba8c1088f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.458+0000 7fba918da700 1 --2- 172.21.15.27:0/3066534742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c100a80 0x7fba8c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.458+0000 7fba918da700 1 -- 172.21.15.27:0/3066534742 >> 172.21.15.27:0/3066534742 conn(0x7fba8c077a00 msgr2=0x7fba8c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:36.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.458+0000 7fba918da700 1 -- 172.21.15.27:0/3066534742 shutdown_connections 2024-06-06T08:37:36.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.458+0000 7fba918da700 1 -- 172.21.15.27:0/3066534742 wait complete. 2024-06-06T08:37:36.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba918da700 1 Processor -- start 2024-06-06T08:37:36.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba918da700 1 -- start start 2024-06-06T08:37:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba918da700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c100a80 0x7fba8c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba918da700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c101440 0x7fba8c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba918da700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba8c108510 0x7fba8c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba918da700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c112790 con 0x7fba8c100a80 2024-06-06T08:37:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba918da700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c112610 con 0x7fba8c108510 2024-06-06T08:37:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba918da700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c112910 con 0x7fba8c101440 2024-06-06T08:37:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba8affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c100a80 0x7fba8c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba8affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c100a80 0x7fba8c19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43896/0 (socket says 172.21.15.27:43896) 2024-06-06T08:37:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba8affd700 1 -- 172.21.15.27:0/1040149014 learned_addr learned my addr 172.21.15.27:0/1040149014 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.459+0000 7fba8b7fe700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba8c108510 0x7fba8c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba8a7fc700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c101440 0x7fba8c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba8affd700 1 -- 172.21.15.27:0/1040149014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c101440 msgr2=0x7fba8c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba8affd700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c101440 0x7fba8c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba8affd700 1 -- 172.21.15.27:0/1040149014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba8c108510 msgr2=0x7fba8c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba8affd700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba8c108510 0x7fba8c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba8affd700 1 -- 172.21.15.27:0/1040149014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c199110 con 0x7fba8c100a80 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba8a7fc700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c101440 0x7fba8c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba8b7fe700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba8c108510 0x7fba8c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba8affd700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c100a80 0x7fba8c19e7f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fba7c000c00 tx=0x7fba7c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba908d8700 1 -- 172.21.15.27:0/1040149014 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c027070 con 0x7fba8c100a80 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba908d8700 1 -- 172.21.15.27:0/1040149014 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba7c004050 con 0x7fba8c100a80 2024-06-06T08:37:36.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba908d8700 1 -- 172.21.15.27:0/1040149014 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00f460 con 0x7fba8c100a80 2024-06-06T08:37:36.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c1993a0 con 0x7fba8c100a80 2024-06-06T08:37:36.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.460+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c1a56c0 con 0x7fba8c100a80 2024-06-06T08:37:36.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.461+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8c10a9f0 con 0x7fba8c100a80 2024-06-06T08:37:36.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.462+0000 7fba908d8700 1 -- 172.21.15.27:0/1040149014 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fba7c02a070 con 0x7fba8c100a80 2024-06-06T08:37:36.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.463+0000 7fba908d8700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fba740a2d30 0x7fba740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:36.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.463+0000 7fba908d8700 1 -- 172.21.15.27:0/1040149014 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7fba7c0d0400 con 0x7fba8c100a80 2024-06-06T08:37:36.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.463+0000 7fba8a7fc700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fba740a2d30 0x7fba740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:36.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.464+0000 7fba8a7fc700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fba740a2d30 0x7fba740a51f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fba80005fd0 tx=0x7fba80005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:36.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.468+0000 7fba908d8700 1 -- 172.21.15.27:0/1040149014 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba7c05e850 con 0x7fba8c100a80 2024-06-06T08:37:36.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.708+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fba8c199ee0 con 0x7fba740a2d30 2024-06-06T08:37:36.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.713+0000 7fba908d8700 1 -- 172.21.15.27:0/1040149014 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fba8c199ee0 con 0x7fba740a2d30 2024-06-06T08:37:36.711 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_0/3cbf907b-ce75-4558-acca-a3d2903c7e5f 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fba740a2d30 msgr2=0x7fba740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fba740a2d30 0x7fba740a51f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fba80005fd0 tx=0x7fba80005ee0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c100a80 msgr2=0x7fba8c19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c100a80 0x7fba8c19e7f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fba7c000c00 tx=0x7fba7c0047b0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 shutdown_connections 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fba740a2d30 0x7fba740a51f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba8c108510 0x7fba8c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba8c100a80 0x7fba8c19e7f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 --2- 172.21.15.27:0/1040149014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba8c101440 0x7fba8c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 >> 172.21.15.27:0/1040149014 conn(0x7fba8c077a00 msgr2=0x7fba8c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 shutdown_connections 2024-06-06T08:37:36.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:36.715+0000 7fba918da700 1 -- 172.21.15.27:0/1040149014 wait complete. 2024-06-06T08:37:37.503 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-06T08:37:37.754 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:38.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:37 smithi027 ceph-mon[32335]: pgmap v346: 129 pgs: 129 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-06-06T08:37:38.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:37 smithi027 ceph-mon[32335]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:38.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:37 smithi203 ceph-mon[36579]: pgmap v346: 129 pgs: 129 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-06-06T08:37:38.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:37 smithi203 ceph-mon[36579]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:38.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:37 smithi176 ceph-mon[36786]: pgmap v346: 129 pgs: 129 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-06-06T08:37:38.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:37 smithi176 ceph-mon[36786]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 -- 172.21.15.27:0/1351917442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70107840 msgr2=0x7f6a7010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1351917442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70107840 0x7f6a7010f3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800b0d0 tx=0x7f6a6800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 -- 172.21.15.27:0/1351917442 shutdown_connections 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1351917442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70107840 0x7f6a7010f3c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1351917442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a70102740 0x7f6a70102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1351917442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a701030f0 0x7f6a70107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 -- 172.21.15.27:0/1351917442 >> 172.21.15.27:0/1351917442 conn(0x7f6a700795f0 msgr2=0x7f6a70079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 -- 172.21.15.27:0/1351917442 shutdown_connections 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 -- 172.21.15.27:0/1351917442 wait complete. 2024-06-06T08:37:39.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 Processor -- start 2024-06-06T08:37:39.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.524+0000 7f6a7869a700 1 -- start start 2024-06-06T08:37:39.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a7869a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70102740 0x7f6a7019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:39.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a7869a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a701030f0 0x7f6a7019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:39.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a7869a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70107840 0x7f6a70198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a7869a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701127d0 con 0x7f6a70102740 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a7869a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70112650 con 0x7f6a701030f0 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a7869a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a70112950 con 0x7f6a70107840 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76436700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70102740 0x7f6a7019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76436700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70102740 0x7f6a7019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43916/0 (socket says 172.21.15.27:43916) 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76436700 1 -- 172.21.15.27:0/1642869318 learned_addr learned my addr 172.21.15.27:0/1642869318 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a75c35700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a701030f0 0x7f6a7019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76c37700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70107840 0x7f6a70198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76436700 1 -- 172.21.15.27:0/1642869318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70107840 msgr2=0x7f6a70198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76436700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70107840 0x7f6a70198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76436700 1 -- 172.21.15.27:0/1642869318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a701030f0 msgr2=0x7f6a7019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76436700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a701030f0 0x7f6a7019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76436700 1 -- 172.21.15.27:0/1642869318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a701990d0 con 0x7f6a70102740 2024-06-06T08:37:39.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76c37700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70107840 0x7f6a70198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:39.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a75c35700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a701030f0 0x7f6a7019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:39.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.525+0000 7f6a76436700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70102740 0x7f6a7019e7b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c920 tx=0x7f6a6c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:39.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.526+0000 7f6a677fe700 1 -- 172.21.15.27:0/1642869318 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c007b00 con 0x7f6a70102740 2024-06-06T08:37:39.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.526+0000 7f6a677fe700 1 -- 172.21.15.27:0/1642869318 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a6c004d10 con 0x7f6a70102740 2024-06-06T08:37:39.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.526+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a701993c0 con 0x7f6a70102740 2024-06-06T08:37:39.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.526+0000 7f6a677fe700 1 -- 172.21.15.27:0/1642869318 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00f450 con 0x7f6a70102740 2024-06-06T08:37:39.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.526+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a701a5700 con 0x7f6a70102740 2024-06-06T08:37:39.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.527+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a70069180 con 0x7f6a70102740 2024-06-06T08:37:39.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.528+0000 7f6a677fe700 1 -- 172.21.15.27:0/1642869318 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6a6c004750 con 0x7f6a70102740 2024-06-06T08:37:39.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.529+0000 7f6a677fe700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6a5c0a2d30 0x7f6a5c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:39.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.529+0000 7f6a677fe700 1 -- 172.21.15.27:0/1642869318 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f6a6c0cf520 con 0x7f6a70102740 2024-06-06T08:37:39.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.529+0000 7f6a75c35700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6a5c0a2d30 0x7f6a5c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:39.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.529+0000 7f6a75c35700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6a5c0a2d30 0x7f6a5c0a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6a70108c90 tx=0x7f6a60008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.532+0000 7f6a677fe700 1 -- 172.21.15.27:0/1642869318 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a6c014030 con 0x7f6a70102740 2024-06-06T08:37:39.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.771+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6a70100b20 con 0x7f6a5c0a2d30 2024-06-06T08:37:39.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.798+0000 7f6a677fe700 1 -- 172.21.15.27:0/1642869318 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a70100b20 con 0x7f6a5c0a2d30 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6a5c0a2d30 msgr2=0x7f6a5c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6a5c0a2d30 0x7f6a5c0a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6a70108c90 tx=0x7f6a60008040 comp rx=0 tx=0).stop 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70102740 msgr2=0x7f6a7019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70102740 0x7f6a7019e7b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c920 tx=0x7f6a6c00cce0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 shutdown_connections 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6a5c0a2d30 0x7f6a5c0a51f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70107840 0x7f6a70198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a70102740 0x7f6a7019e7b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 --2- 172.21.15.27:0/1642869318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a701030f0 0x7f6a7019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 >> 172.21.15.27:0/1642869318 conn(0x7f6a700795f0 msgr2=0x7f6a70100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:39.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 shutdown_connections 2024-06-06T08:37:39.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:39.800+0000 7f6a7869a700 1 -- 172.21.15.27:0/1642869318 wait complete. 2024-06-06T08:37:40.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:39 smithi027 ceph-mon[32335]: pgmap v347: 129 pgs: 129 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-06T08:37:40.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:39 smithi203 ceph-mon[36579]: pgmap v347: 129 pgs: 129 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-06T08:37:40.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:39 smithi176 ceph-mon[36786]: pgmap v347: 129 pgs: 129 active+clean; 591 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-06T08:37:40.462 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-06T08:37:40.709 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:41.154 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:40 smithi027 ceph-mon[32335]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:40 smithi203 ceph-mon[36579]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:41.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:40 smithi176 ceph-mon[36786]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:42.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:41 smithi027 ceph-mon[32335]: pgmap v348: 129 pgs: 129 active+clean; 609 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-06-06T08:37:42.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:41 smithi203 ceph-mon[36579]: pgmap v348: 129 pgs: 129 active+clean; 609 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-06-06T08:37:42.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:41 smithi176 ceph-mon[36786]: pgmap v348: 129 pgs: 129 active+clean; 609 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2024-06-06T08:37:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.509+0000 7fe7f912e700 1 -- 172.21.15.27:0/2820606464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4106b80 msgr2=0x7fe7f4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.509+0000 7fe7f912e700 1 --2- 172.21.15.27:0/2820606464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4106b80 0x7fe7f4106f60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009990 tx=0x7fe7e4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.509+0000 7fe7f912e700 1 -- 172.21.15.27:0/2820606464 shutdown_connections 2024-06-06T08:37:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.509+0000 7fe7f912e700 1 --2- 172.21.15.27:0/2820606464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7f41014b0 0x7fe7f410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.509+0000 7fe7f912e700 1 --2- 172.21.15.27:0/2820606464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4106b80 0x7fe7f4106f60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.509+0000 7fe7f912e700 1 --2- 172.21.15.27:0/2820606464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7f4100af0 0x7fe7f4100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.509+0000 7fe7f912e700 1 -- 172.21.15.27:0/2820606464 >> 172.21.15.27:0/2820606464 conn(0x7fe7f4077960 msgr2=0x7fe7f40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.509+0000 7fe7f912e700 1 -- 172.21.15.27:0/2820606464 shutdown_connections 2024-06-06T08:37:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.509+0000 7fe7f912e700 1 -- 172.21.15.27:0/2820606464 wait complete. 2024-06-06T08:37:42.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f912e700 1 Processor -- start 2024-06-06T08:37:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f912e700 1 -- start start 2024-06-06T08:37:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f912e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4100af0 0x7fe7f419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f912e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7f41014b0 0x7fe7f419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f912e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7f4106b80 0x7fe7f4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f912e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4110590 con 0x7fe7f4100af0 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f912e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4110410 con 0x7fe7f4106b80 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f912e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4110710 con 0x7fe7f41014b0 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4100af0 0x7fe7f419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4100af0 0x7fe7f419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38572/0 (socket says 172.21.15.27:38572) 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f2d9d700 1 -- 172.21.15.27:0/4281421190 learned_addr learned my addr 172.21.15.27:0/4281421190 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f259c700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7f41014b0 0x7fe7f419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.510+0000 7fe7f359e700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7f4106b80 0x7fe7f4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7f2d9d700 1 -- 172.21.15.27:0/4281421190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7f41014b0 msgr2=0x7fe7f419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7f2d9d700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7f41014b0 0x7fe7f419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7f2d9d700 1 -- 172.21.15.27:0/4281421190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7f4106b80 msgr2=0x7fe7f4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7f2d9d700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7f4106b80 0x7fe7f4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7f2d9d700 1 -- 172.21.15.27:0/4281421190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f41990c0 con 0x7fe7f4100af0 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7f259c700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7f41014b0 0x7fe7f419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7f2d9d700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4100af0 0x7fe7f419e7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40038b0 tx=0x7fe7e4005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7dbfff700 1 -- 172.21.15.27:0/4281421190 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4027070 con 0x7fe7f4100af0 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7dbfff700 1 -- 172.21.15.27:0/4281421190 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e40040d0 con 0x7fe7f4100af0 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7dbfff700 1 -- 172.21.15.27:0/4281421190 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4022dd0 con 0x7fe7f4100af0 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f4199350 con 0x7fe7f4100af0 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.511+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f41a5670 con 0x7fe7f4100af0 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.512+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f4109080 con 0x7fe7f4100af0 2024-06-06T08:37:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.514+0000 7fe7dbfff700 1 -- 172.21.15.27:0/4281421190 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe7e402a070 con 0x7fe7f4100af0 2024-06-06T08:37:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.515+0000 7fe7dbfff700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe7dc0a2cb0 0x7fe7dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.515+0000 7fe7f259c700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe7dc0a2cb0 0x7fe7dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:42.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.515+0000 7fe7dbfff700 1 -- 172.21.15.27:0/4281421190 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7fe7e40d08c0 con 0x7fe7f4100af0 2024-06-06T08:37:42.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.515+0000 7fe7f259c700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe7dc0a2cb0 0x7fe7dc0a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005fd0 tx=0x7fe7e8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:42.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.518+0000 7fe7dbfff700 1 -- 172.21.15.27:0/4281421190 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe7e4031020 con 0x7fe7f4100af0 2024-06-06T08:37:42.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.762+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe7f40628d0 con 0x7fe7dc0a2cb0 2024-06-06T08:37:42.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.767+0000 7fe7dbfff700 1 -- 172.21.15.27:0/4281421190 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe7f40628d0 con 0x7fe7dc0a2cb0 2024-06-06T08:37:42.767 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86 2024-06-06T08:37:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe7dc0a2cb0 msgr2=0x7fe7dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe7dc0a2cb0 0x7fe7dc0a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005fd0 tx=0x7fe7e8005e30 comp rx=0 tx=0).stop 2024-06-06T08:37:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4100af0 msgr2=0x7fe7f419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4100af0 0x7fe7f419e7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40038b0 tx=0x7fe7e4005590 comp rx=0 tx=0).stop 2024-06-06T08:37:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 shutdown_connections 2024-06-06T08:37:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe7dc0a2cb0 0x7fe7dc0a5170 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7f4106b80 0x7fe7f4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7f4100af0 0x7fe7f419e7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 --2- 172.21.15.27:0/4281421190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7f41014b0 0x7fe7f419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.769+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 >> 172.21.15.27:0/4281421190 conn(0x7fe7f4077960 msgr2=0x7fe7f41039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.770+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 shutdown_connections 2024-06-06T08:37:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:42.770+0000 7fe7f912e700 1 -- 172.21.15.27:0/4281421190 wait complete. 2024-06-06T08:37:43.493 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:37:43.748 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:44.120 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:43 smithi027 ceph-mon[32335]: pgmap v349: 129 pgs: 129 active+clean; 609 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T08:37:44.120 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:43 smithi027 ceph-mon[32335]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:44.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:43 smithi203 ceph-mon[36579]: pgmap v349: 129 pgs: 129 active+clean; 609 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T08:37:44.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:43 smithi203 ceph-mon[36579]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:44.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:43 smithi176 ceph-mon[36786]: pgmap v349: 129 pgs: 129 active+clean; 609 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T08:37:44.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:43 smithi176 ceph-mon[36786]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:37:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.444+0000 7fac999c1700 1 -- 172.21.15.27:0/4253255657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 msgr2=0x7fac9410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.444+0000 7fac999c1700 1 --2- 172.21.15.27:0/4253255657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 0x7fac9410f310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0099c0 tx=0x7fac8c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.444+0000 7fac999c1700 1 -- 172.21.15.27:0/4253255657 shutdown_connections 2024-06-06T08:37:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.444+0000 7fac999c1700 1 --2- 172.21.15.27:0/4253255657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 0x7fac9410f310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.444+0000 7fac999c1700 1 --2- 172.21.15.27:0/4253255657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac94108540 0x7fac94108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.444+0000 7fac999c1700 1 --2- 172.21.15.27:0/4253255657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac94102490 0x7fac94102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.444+0000 7fac999c1700 1 -- 172.21.15.27:0/4253255657 >> 172.21.15.27:0/4253255657 conn(0x7fac940fdf40 msgr2=0x7fac94100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.444+0000 7fac999c1700 1 -- 172.21.15.27:0/4253255657 shutdown_connections 2024-06-06T08:37:45.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.444+0000 7fac999c1700 1 -- 172.21.15.27:0/4253255657 wait complete. 2024-06-06T08:37:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.445+0000 7fac999c1700 1 Processor -- start 2024-06-06T08:37:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.445+0000 7fac999c1700 1 -- start start 2024-06-06T08:37:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac999c1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac94102490 0x7fac9419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac999c1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 0x7fac9419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac999c1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac94108540 0x7fac94198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac999c1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fac941127a0 con 0x7fac94102e50 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac999c1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fac94112620 con 0x7fac94102490 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac999c1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fac94112920 con 0x7fac94108540 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac8a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 0x7fac9419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac8a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 0x7fac9419ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38586/0 (socket says 172.21.15.27:38586) 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac8a7fc700 1 -- 172.21.15.27:0/3857085666 learned_addr learned my addr 172.21.15.27:0/3857085666 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac92ffd700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac94102490 0x7fac9419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:45.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac937fe700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac94108540 0x7fac94198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac8a7fc700 1 -- 172.21.15.27:0/3857085666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac94108540 msgr2=0x7fac94198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac8a7fc700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac94108540 0x7fac94198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac8a7fc700 1 -- 172.21.15.27:0/3857085666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac94102490 msgr2=0x7fac9419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac8a7fc700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac94102490 0x7fac9419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac8a7fc700 1 -- 172.21.15.27:0/3857085666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac941990c0 con 0x7fac94102e50 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac92ffd700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac94102490 0x7fac9419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac937fe700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac94108540 0x7fac94198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac8a7fc700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 0x7fac9419ece0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00d980 tx=0x7fac7c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac989bf700 1 -- 172.21.15.27:0/3857085666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c017070 con 0x7fac94102e50 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac989bf700 1 -- 172.21.15.27:0/3857085666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac7c004d10 con 0x7fac94102e50 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.446+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac941993b0 con 0x7fac94102e50 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.447+0000 7fac989bf700 1 -- 172.21.15.27:0/3857085666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0055c0 con 0x7fac94102e50 2024-06-06T08:37:45.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.447+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac941a5670 con 0x7fac94102e50 2024-06-06T08:37:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.447+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac94069180 con 0x7fac94102e50 2024-06-06T08:37:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.449+0000 7fac989bf700 1 -- 172.21.15.27:0/3857085666 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac7c004750 con 0x7fac94102e50 2024-06-06T08:37:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.449+0000 7fac989bf700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fac800a2d30 0x7fac800a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.449+0000 7fac92ffd700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fac800a2d30 0x7fac800a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.449+0000 7fac989bf700 1 -- 172.21.15.27:0/3857085666 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7fac7c0cf820 con 0x7fac94102e50 2024-06-06T08:37:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.450+0000 7fac92ffd700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fac800a2d30 0x7fac800a51f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fac84007fd0 tx=0x7fac84007830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.453+0000 7fac989bf700 1 -- 172.21.15.27:0/3857085666 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fac7c020020 con 0x7fac94102e50 2024-06-06T08:37:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.701+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac941a58a0 con 0x7fac94102e50 2024-06-06T08:37:45.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.701+0000 7fac989bf700 1 -- 172.21.15.27:0/3857085666 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11882 (secure 0 0 0) 0x7fac7c09c360 con 0x7fac94102e50 2024-06-06T08:37:45.700 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:37:45.701 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:37:27.822986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24458,24460]},"id":1}]} 2024-06-06T08:37:45.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fac800a2d30 msgr2=0x7fac800a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:45.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fac800a2d30 0x7fac800a51f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fac84007fd0 tx=0x7fac84007830 comp rx=0 tx=0).stop 2024-06-06T08:37:45.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 msgr2=0x7fac9419ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:45.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 0x7fac9419ece0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00d980 tx=0x7fac7c00dd40 comp rx=0 tx=0).stop 2024-06-06T08:37:45.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 shutdown_connections 2024-06-06T08:37:45.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fac800a2d30 0x7fac800a51f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac94108540 0x7fac94198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac94102490 0x7fac9419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 --2- 172.21.15.27:0/3857085666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac94102e50 0x7fac9419ece0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:45.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 >> 172.21.15.27:0/3857085666 conn(0x7fac940fdf40 msgr2=0x7fac940ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:45.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 shutdown_connections 2024-06-06T08:37:45.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:45.704+0000 7fac999c1700 1 -- 172.21.15.27:0/3857085666 wait complete. 2024-06-06T08:37:45.704 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-06-06T08:37:46.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:45 smithi203 ceph-mon[36579]: pgmap v350: 129 pgs: 129 active+clean; 632 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-06-06T08:37:46.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:45 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3857085666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:37:46.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:45 smithi176 ceph-mon[36786]: pgmap v350: 129 pgs: 129 active+clean; 632 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-06-06T08:37:46.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:45 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3857085666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:37:46.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:45 smithi027 ceph-mon[32335]: pgmap v350: 129 pgs: 129 active+clean; 632 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-06-06T08:37:46.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:45 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3857085666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:37:46.528 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:37:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:47 smithi027 ceph-mon[32335]: pgmap v351: 129 pgs: 129 active+clean; 632 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-06T08:37:48.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:47 smithi203 ceph-mon[36579]: pgmap v351: 129 pgs: 129 active+clean; 632 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-06T08:37:48.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:47 smithi176 ceph-mon[36786]: pgmap v351: 129 pgs: 129 active+clean; 632 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-06T08:37:48.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.435+0000 7ff5ec47c700 1 -- 172.21.15.27:0/1484481175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101fd0 msgr2=0x7ff5e4105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.435+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/1484481175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101fd0 0x7ff5e4105fd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40099c0 tx=0x7ff5d4009cd0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.436+0000 7ff5ec47c700 1 -- 172.21.15.27:0/1484481175 shutdown_connections 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.436+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/1484481175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5e41066b0 0x7ff5e410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.436+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/1484481175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e4101620 0x7ff5e4101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.436+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/1484481175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101fd0 0x7ff5e4105fd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.436+0000 7ff5ec47c700 1 -- 172.21.15.27:0/1484481175 >> 172.21.15.27:0/1484481175 conn(0x7ff5e4078340 msgr2=0x7ff5e4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.436+0000 7ff5ec47c700 1 -- 172.21.15.27:0/1484481175 shutdown_connections 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.436+0000 7ff5ec47c700 1 -- 172.21.15.27:0/1484481175 wait complete. 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.436+0000 7ff5ec47c700 1 Processor -- start 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.436+0000 7ff5ec47c700 1 -- start start 2024-06-06T08:37:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ec47c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101620 0x7ff5e4198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ec47c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5e4101fd0 0x7ff5e4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ec47c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e41066b0 0x7ff5e419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ec47c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4111770 con 0x7ff5e4101620 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ec47c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41115f0 con 0x7ff5e4101fd0 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ec47c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41118f0 con 0x7ff5e41066b0 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ea218700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101620 0x7ff5e4198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ea218700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101620 0x7ff5e4198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38610/0 (socket says 172.21.15.27:38610) 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ea218700 1 -- 172.21.15.27:0/465312377 learned_addr learned my addr 172.21.15.27:0/465312377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5e9a17700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5e4101fd0 0x7ff5e4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5eaa19700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e41066b0 0x7ff5e419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ea218700 1 -- 172.21.15.27:0/465312377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e41066b0 msgr2=0x7ff5e419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ea218700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e41066b0 0x7ff5e419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ea218700 1 -- 172.21.15.27:0/465312377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5e4101fd0 msgr2=0x7ff5e4199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ea218700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5e4101fd0 0x7ff5e4199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5ea218700 1 -- 172.21.15.27:0/465312377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e419dd60 con 0x7ff5e4101620 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5eaa19700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e41066b0 0x7ff5e419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.437+0000 7ff5e9a17700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5e4101fd0 0x7ff5e4199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.438+0000 7ff5ea218700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101620 0x7ff5e4198d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000b530 tx=0x7ff5e000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.438+0000 7ff5db7fe700 1 -- 172.21.15.27:0/465312377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0015020 con 0x7ff5e4101620 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.438+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e419e050 con 0x7ff5e4101620 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.438+0000 7ff5db7fe700 1 -- 172.21.15.27:0/465312377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e0011d10 con 0x7ff5e4101620 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.438+0000 7ff5db7fe700 1 -- 172.21.15.27:0/465312377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0010e50 con 0x7ff5e4101620 2024-06-06T08:37:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.438+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e4102fa0 con 0x7ff5e4101620 2024-06-06T08:37:48.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.439+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e4102ab0 con 0x7ff5e4101620 2024-06-06T08:37:48.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.440+0000 7ff5db7fe700 1 -- 172.21.15.27:0/465312377 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff5e0018020 con 0x7ff5e4101620 2024-06-06T08:37:48.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.441+0000 7ff5db7fe700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5d00a2e50 0x7ff5d00a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:48.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.441+0000 7ff5e9a17700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5d00a2e50 0x7ff5d00a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:48.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.441+0000 7ff5db7fe700 1 -- 172.21.15.27:0/465312377 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7ff5e00cec60 con 0x7ff5e4101620 2024-06-06T08:37:48.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.441+0000 7ff5e9a17700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5d00a2e50 0x7ff5d00a5310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4000c00 tx=0x7ff5d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:48.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.445+0000 7ff5db7fe700 1 -- 172.21.15.27:0/465312377 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff5e001f020 con 0x7ff5e4101620 2024-06-06T08:37:48.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.653+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5e419a0b0 con 0x7ff5e4101620 2024-06-06T08:37:48.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.655+0000 7ff5db7fe700 1 -- 172.21.15.27:0/465312377 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24602 (secure 0 0 0) 0x7ff5e009b830 con 0x7ff5e4101620 2024-06-06T08:37:48.655 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:37:48.655 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":97,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:37:00.035534+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:37:48.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5d00a2e50 msgr2=0x7ff5d00a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:48.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5d00a2e50 0x7ff5d00a5310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4000c00 tx=0x7ff5d4005fd0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101620 msgr2=0x7ff5e4198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:48.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101620 0x7ff5e4198d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000b530 tx=0x7ff5e000b8f0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 shutdown_connections 2024-06-06T08:37:48.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff5d00a2e50 0x7ff5d00a5310 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e41066b0 0x7ff5e419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5e4101620 0x7ff5e4198d50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 --2- 172.21.15.27:0/465312377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff5e4101fd0 0x7ff5e4199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:48.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 >> 172.21.15.27:0/465312377 conn(0x7ff5e4078340 msgr2=0x7ff5e406af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:48.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 shutdown_connections 2024-06-06T08:37:48.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:48.657+0000 7ff5ec47c700 1 -- 172.21.15.27:0/465312377 wait complete. 2024-06-06T08:37:49.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:48 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/465312377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:37:49.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:48 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/465312377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:37:49.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:48 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/465312377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:37:49.507 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T08:37:49.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T08:37:49.760 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:50.163 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:49 smithi027 ceph-mon[32335]: pgmap v352: 129 pgs: 129 active+clean; 636 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-06-06T08:37:50.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:49 smithi203 ceph-mon[36579]: pgmap v352: 129 pgs: 129 active+clean; 636 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-06-06T08:37:50.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:49 smithi176 ceph-mon[36786]: pgmap v352: 129 pgs: 129 active+clean; 636 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-06-06T08:37:51.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.572+0000 7f58f757b700 1 -- 172.21.15.27:0/876707277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0102380 msgr2=0x7f58f0105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:51.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.572+0000 7f58f757b700 1 --2- 172.21.15.27:0/876707277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0102380 0x7f58f0105ff0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f58e00099c0 tx=0x7f58e0009cd0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.572+0000 7f58f757b700 1 -- 172.21.15.27:0/876707277 shutdown_connections 2024-06-06T08:37:51.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.572+0000 7f58f757b700 1 --2- 172.21.15.27:0/876707277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0106720 0x7f58f010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.572+0000 7f58f757b700 1 --2- 172.21.15.27:0/876707277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f01019d0 0x7f58f0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.572+0000 7f58f757b700 1 --2- 172.21.15.27:0/876707277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f0102380 0x7f58f0105ff0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.572+0000 7f58f757b700 1 -- 172.21.15.27:0/876707277 >> 172.21.15.27:0/876707277 conn(0x7f58f0078340 msgr2=0x7f58f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.572+0000 7f58f757b700 1 -- 172.21.15.27:0/876707277 shutdown_connections 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.572+0000 7f58f757b700 1 -- 172.21.15.27:0/876707277 wait complete. 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f757b700 1 Processor -- start 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f757b700 1 -- start start 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f757b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01019d0 0x7f58f0199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f757b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0102380 0x7f58f019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f757b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106720 0x7f58f01a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f757b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0111740 con 0x7f58f01019d0 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f757b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01115c0 con 0x7f58f0106720 2024-06-06T08:37:51.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f757b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01118c0 con 0x7f58f0102380 2024-06-06T08:37:51.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5317700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01019d0 0x7f58f0199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:51.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5317700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01019d0 0x7f58f0199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58706/0 (socket says 172.21.15.27:58706) 2024-06-06T08:37:51.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5317700 1 -- 172.21.15.27:0/4288749475 learned_addr learned my addr 172.21.15.27:0/4288749475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:51.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f4b16700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0102380 0x7f58f019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:51.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5b18700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106720 0x7f58f01a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:51.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5317700 1 -- 172.21.15.27:0/4288749475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0102380 msgr2=0x7f58f019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:51.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5317700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0102380 0x7f58f019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5317700 1 -- 172.21.15.27:0/4288749475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106720 msgr2=0x7f58f01a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:51.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5317700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106720 0x7f58f01a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5317700 1 -- 172.21.15.27:0/4288749475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f01a5100 con 0x7f58f01019d0 2024-06-06T08:37:51.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f4b16700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0102380 0x7f58f019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:51.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.573+0000 7f58f5b18700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106720 0x7f58f01a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:51.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.574+0000 7f58f5317700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01019d0 0x7f58f0199b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b530 tx=0x7f58ec00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:51.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.574+0000 7f58e67fc700 1 -- 172.21.15.27:0/4288749475 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec015020 con 0x7f58f01019d0 2024-06-06T08:37:51.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.574+0000 7f58e67fc700 1 -- 172.21.15.27:0/4288749475 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ec011d10 con 0x7f58f01019d0 2024-06-06T08:37:51.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.574+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f01a5390 con 0x7f58f01019d0 2024-06-06T08:37:51.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.574+0000 7f58e67fc700 1 -- 172.21.15.27:0/4288749475 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec010e50 con 0x7f58f01019d0 2024-06-06T08:37:51.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.574+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01a58a0 con 0x7f58f01019d0 2024-06-06T08:37:51.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.575+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f006a410 con 0x7f58f01019d0 2024-06-06T08:37:51.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.577+0000 7f58e67fc700 1 -- 172.21.15.27:0/4288749475 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f58ec018020 con 0x7f58f01019d0 2024-06-06T08:37:51.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.578+0000 7f58e67fc700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f58dc0a2e50 0x7f58dc0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:51.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.578+0000 7f58f4b16700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f58dc0a2e50 0x7f58dc0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:51.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.578+0000 7f58e67fc700 1 -- 172.21.15.27:0/4288749475 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f58ec0cf220 con 0x7f58f01019d0 2024-06-06T08:37:51.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.578+0000 7f58f4b16700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f58dc0a2e50 0x7f58dc0a5310 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009990 tx=0x7f58e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.581+0000 7f58e67fc700 1 -- 172.21.15.27:0/4288749475 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f58ec01f020 con 0x7f58f01019d0 2024-06-06T08:37:51.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.783+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58f00628d0 con 0x7f58dc0a2e50 2024-06-06T08:37:51.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.794+0000 7f58e67fc700 1 -- 172.21.15.27:0/4288749475 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340650 (secure 0 0 0) 0x7f58f00628d0 con 0x7f58dc0a2e50 2024-06-06T08:37:51.798 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:37:51.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f58dc0a2e50 msgr2=0x7f58dc0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:51.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f58dc0a2e50 0x7f58dc0a5310 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009990 tx=0x7f58e0005fb0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01019d0 msgr2=0x7f58f0199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01019d0 0x7f58f0199b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b530 tx=0x7f58ec00b8f0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 shutdown_connections 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f58dc0a2e50 0x7f58dc0a5310 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106720 0x7f58f01a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01019d0 0x7f58f0199b40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 --2- 172.21.15.27:0/4288749475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0102380 0x7f58f019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 >> 172.21.15.27:0/4288749475 conn(0x7f58f0078340 msgr2=0x7f58f0107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 shutdown_connections 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:51.797+0000 7f58f757b700 1 -- 172.21.15.27:0/4288749475 wait complete. 2024-06-06T08:37:51.802 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:37:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:51 smithi203 ceph-mon[36579]: pgmap v353: 129 pgs: 129 active+clean; 636 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-06-06T08:37:52.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:51 smithi176 ceph-mon[36786]: pgmap v353: 129 pgs: 129 active+clean; 636 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-06-06T08:37:52.403 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:51 smithi027 ceph-mon[32335]: pgmap v353: 129 pgs: 129 active+clean; 636 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-06-06T08:37:52.827 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-06-06T08:37:50.455837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651269,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":308,"num_read_kb":271,"num_write":347,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353836,"kb_used_data":13004,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587988,"statfs":{"total":1151940427776,"available":1151578099712,"internally_reserved":0,"allocated":13316096,"data_stored":8540777,"data_compressed":424455,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46344,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":90,"num_read_kb":92,"num_write":25,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002950"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050231+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:37:00.050231+0000","last_peered":"2024-06-06T08:37:00.050231+0000","last_clean":"2024-06-06T08:37:00.050231+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:37:00.050231+0000","last_undegraded":"2024-06-06T08:37:00.050231+0000","last_fullsized":"2024-06-06T08:37:00.050231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451282+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:37:00.451282+0000","last_peered":"2024-06-06T08:37:00.451282+0000","last_clean":"2024-06-06T08:37:00.451282+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:37:00.451282+0000","last_undegraded":"2024-06-06T08:37:00.451282+0000","last_fullsized":"2024-06-06T08:37:00.451282+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451041+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:37:00.451041+0000","last_peered":"2024-06-06T08:37:00.451041+0000","last_clean":"2024-06-06T08:37:00.451041+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:37:00.451041+0000","last_undegraded":"2024-06-06T08:37:00.451041+0000","last_fullsized":"2024-06-06T08:37:00.451041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050562+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:37:00.050562+0000","last_peered":"2024-06-06T08:37:00.050562+0000","last_clean":"2024-06-06T08:37:00.050562+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:37:00.050562+0000","last_undegraded":"2024-06-06T08:37:00.050562+0000","last_fullsized":"2024-06-06T08:37:00.050562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.330620+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:37:00.330620+0000","last_peered":"2024-06-06T08:37:00.330620+0000","last_clean":"2024-06-06T08:37:00.330620+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:37:00.330620+0000","last_undegraded":"2024-06-06T08:37:00.330620+0000","last_fullsized":"2024-06-06T08:37:00.330620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050290+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:37:00.050290+0000","last_peered":"2024-06-06T08:37:00.050290+0000","last_clean":"2024-06-06T08:37:00.050290+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:37:00.050290+0000","last_undegraded":"2024-06-06T08:37:00.050290+0000","last_fullsized":"2024-06-06T08:37:00.050290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681748+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:37:00.681748+0000","last_peered":"2024-06-06T08:37:00.681748+0000","last_clean":"2024-06-06T08:37:00.681748+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:37:00.681748+0000","last_undegraded":"2024-06-06T08:37:00.681748+0000","last_fullsized":"2024-06-06T08:37:00.681748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049965+0000","last_change":"2024-06-06T08:36:20.890919+0000","last_active":"2024-06-06T08:37:00.049965+0000","last_peered":"2024-06-06T08:37:00.049965+0000","last_clean":"2024-06-06T08:37:00.049965+0000","last_became_active":"2024-06-06T08:36:20.890501+0000","last_became_peered":"2024-06-06T08:36:20.890501+0000","last_unstale":"2024-06-06T08:37:00.049965+0000","last_undegraded":"2024-06-06T08:37:00.049965+0000","last_fullsized":"2024-06-06T08:37:00.049965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:51:24.461247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.054552+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:37:00.054552+0000","last_peered":"2024-06-06T08:37:00.054552+0000","last_clean":"2024-06-06T08:37:00.054552+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:37:00.054552+0000","last_undegraded":"2024-06-06T08:37:00.054552+0000","last_fullsized":"2024-06-06T08:37:00.054552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.052027+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:37:00.052027+0000","last_peered":"2024-06-06T08:37:00.052027+0000","last_clean":"2024-06-06T08:37:00.052027+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:37:00.052027+0000","last_undegraded":"2024-06-06T08:37:00.052027+0000","last_fullsized":"2024-06-06T08:37:00.052027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.682111+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:37:00.682111+0000","last_peered":"2024-06-06T08:37:00.682111+0000","last_clean":"2024-06-06T08:37:00.682111+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:37:00.682111+0000","last_undegraded":"2024-06-06T08:37:00.682111+0000","last_fullsized":"2024-06-06T08:37:00.682111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050692+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:37:00.050692+0000","last_peered":"2024-06-06T08:37:00.050692+0000","last_clean":"2024-06-06T08:37:00.050692+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:37:00.050692+0000","last_undegraded":"2024-06-06T08:37:00.050692+0000","last_fullsized":"2024-06-06T08:37:00.050692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049345+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:37:00.049345+0000","last_peered":"2024-06-06T08:37:00.049345+0000","last_clean":"2024-06-06T08:37:00.049345+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:37:00.049345+0000","last_undegraded":"2024-06-06T08:37:00.049345+0000","last_fullsized":"2024-06-06T08:37:00.049345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.450932+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:37:00.450932+0000","last_peered":"2024-06-06T08:37:00.450932+0000","last_clean":"2024-06-06T08:37:00.450932+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:37:00.450932+0000","last_undegraded":"2024-06-06T08:37:00.450932+0000","last_fullsized":"2024-06-06T08:37:00.450932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.450865+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:37:00.450865+0000","last_peered":"2024-06-06T08:37:00.450865+0000","last_clean":"2024-06-06T08:37:00.450865+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:37:00.450865+0000","last_undegraded":"2024-06-06T08:37:00.450865+0000","last_fullsized":"2024-06-06T08:37:00.450865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053308+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:37:00.053308+0000","last_peered":"2024-06-06T08:37:00.053308+0000","last_clean":"2024-06-06T08:37:00.053308+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:37:00.053308+0000","last_undegraded":"2024-06-06T08:37:00.053308+0000","last_fullsized":"2024-06-06T08:37:00.053308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049683+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:37:00.049683+0000","last_peered":"2024-06-06T08:37:00.049683+0000","last_clean":"2024-06-06T08:37:00.049683+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:37:00.049683+0000","last_undegraded":"2024-06-06T08:37:00.049683+0000","last_fullsized":"2024-06-06T08:37:00.049683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048456+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:37:00.048456+0000","last_peered":"2024-06-06T08:37:00.048456+0000","last_clean":"2024-06-06T08:37:00.048456+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:37:00.048456+0000","last_undegraded":"2024-06-06T08:37:00.048456+0000","last_fullsized":"2024-06-06T08:37:00.048456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046388+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:37:00.046388+0000","last_peered":"2024-06-06T08:37:00.046388+0000","last_clean":"2024-06-06T08:37:00.046388+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:37:00.046388+0000","last_undegraded":"2024-06-06T08:37:00.046388+0000","last_fullsized":"2024-06-06T08:37:00.046388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.045941+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:37:00.045941+0000","last_peered":"2024-06-06T08:37:00.045941+0000","last_clean":"2024-06-06T08:37:00.045941+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:37:00.045941+0000","last_undegraded":"2024-06-06T08:37:00.045941+0000","last_fullsized":"2024-06-06T08:37:00.045941+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.916397+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:37:21.916397+0000","last_peered":"2024-06-06T08:37:21.916397+0000","last_clean":"2024-06-06T08:37:21.916397+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:37:21.916397+0000","last_undegraded":"2024-06-06T08:37:21.916397+0000","last_fullsized":"2024-06-06T08:37:21.916397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050708+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:37:00.050708+0000","last_peered":"2024-06-06T08:37:00.050708+0000","last_clean":"2024-06-06T08:37:00.050708+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:37:00.050708+0000","last_undegraded":"2024-06-06T08:37:00.050708+0000","last_fullsized":"2024-06-06T08:37:00.050708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050402+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:37:00.050402+0000","last_peered":"2024-06-06T08:37:00.050402+0000","last_clean":"2024-06-06T08:37:00.050402+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:37:00.050402+0000","last_undegraded":"2024-06-06T08:37:00.050402+0000","last_fullsized":"2024-06-06T08:37:00.050402+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051034+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:37:00.051034+0000","last_peered":"2024-06-06T08:37:00.051034+0000","last_clean":"2024-06-06T08:37:00.051034+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:37:00.051034+0000","last_undegraded":"2024-06-06T08:37:00.051034+0000","last_fullsized":"2024-06-06T08:37:00.051034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451037+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:37:00.451037+0000","last_peered":"2024-06-06T08:37:00.451037+0000","last_clean":"2024-06-06T08:37:00.451037+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:37:00.451037+0000","last_undegraded":"2024-06-06T08:37:00.451037+0000","last_fullsized":"2024-06-06T08:37:00.451037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051195+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:37:00.051195+0000","last_peered":"2024-06-06T08:37:00.051195+0000","last_clean":"2024-06-06T08:37:00.051195+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:37:00.051195+0000","last_undegraded":"2024-06-06T08:37:00.051195+0000","last_fullsized":"2024-06-06T08:37:00.051195+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.914251+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:37:21.914251+0000","last_peered":"2024-06-06T08:37:21.914251+0000","last_clean":"2024-06-06T08:37:21.914251+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:37:21.914251+0000","last_undegraded":"2024-06-06T08:37:21.914251+0000","last_fullsized":"2024-06-06T08:37:21.914251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053743+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:37:00.053743+0000","last_peered":"2024-06-06T08:37:00.053743+0000","last_clean":"2024-06-06T08:37:00.053743+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:37:00.053743+0000","last_undegraded":"2024-06-06T08:37:00.053743+0000","last_fullsized":"2024-06-06T08:37:00.053743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050985+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:37:00.050985+0000","last_peered":"2024-06-06T08:37:00.050985+0000","last_clean":"2024-06-06T08:37:00.050985+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:37:00.050985+0000","last_undegraded":"2024-06-06T08:37:00.050985+0000","last_fullsized":"2024-06-06T08:37:00.050985+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.331133+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:37:00.331133+0000","last_peered":"2024-06-06T08:37:00.331133+0000","last_clean":"2024-06-06T08:37:00.331133+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:37:00.331133+0000","last_undegraded":"2024-06-06T08:37:00.331133+0000","last_fullsized":"2024-06-06T08:37:00.331133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185714+0000","last_change":"2024-06-06T08:36:20.902582+0000","last_active":"2024-06-06T08:37:00.185714+0000","last_peered":"2024-06-06T08:37:00.185714+0000","last_clean":"2024-06-06T08:37:00.185714+0000","last_became_active":"2024-06-06T08:36:20.902194+0000","last_became_peered":"2024-06-06T08:36:20.902194+0000","last_unstale":"2024-06-06T08:37:00.185714+0000","last_undegraded":"2024-06-06T08:37:00.185714+0000","last_fullsized":"2024-06-06T08:37:00.185714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:08:33.363612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046079+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:37:00.046079+0000","last_peered":"2024-06-06T08:37:00.046079+0000","last_clean":"2024-06-06T08:37:00.046079+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:37:00.046079+0000","last_undegraded":"2024-06-06T08:37:00.046079+0000","last_fullsized":"2024-06-06T08:37:00.046079+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":70,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:45.874308+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:37:45.874308+0000","last_peered":"2024-06-06T08:37:45.874308+0000","last_clean":"2024-06-06T08:37:45.874308+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:37:45.874308+0000","last_undegraded":"2024-06-06T08:37:45.874308+0000","last_fullsized":"2024-06-06T08:37:45.874308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.330869+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:37:00.330869+0000","last_peered":"2024-06-06T08:37:00.330869+0000","last_clean":"2024-06-06T08:37:00.330869+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:37:00.330869+0000","last_undegraded":"2024-06-06T08:37:00.330869+0000","last_fullsized":"2024-06-06T08:37:00.330869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046022+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:37:00.046022+0000","last_peered":"2024-06-06T08:37:00.046022+0000","last_clean":"2024-06-06T08:37:00.046022+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:37:00.046022+0000","last_undegraded":"2024-06-06T08:37:00.046022+0000","last_fullsized":"2024-06-06T08:37:00.046022+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.914900+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:37:21.914900+0000","last_peered":"2024-06-06T08:37:21.914900+0000","last_clean":"2024-06-06T08:37:21.914900+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:37:21.914900+0000","last_undegraded":"2024-06-06T08:37:21.914900+0000","last_fullsized":"2024-06-06T08:37:21.914900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185797+0000","last_change":"2024-06-06T08:36:20.899203+0000","last_active":"2024-06-06T08:37:00.185797+0000","last_peered":"2024-06-06T08:37:00.185797+0000","last_clean":"2024-06-06T08:37:00.185797+0000","last_became_active":"2024-06-06T08:36:20.898873+0000","last_became_peered":"2024-06-06T08:36:20.898873+0000","last_unstale":"2024-06-06T08:37:00.185797+0000","last_undegraded":"2024-06-06T08:37:00.185797+0000","last_fullsized":"2024-06-06T08:37:00.185797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:45.218350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049657+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:37:00.049657+0000","last_peered":"2024-06-06T08:37:00.049657+0000","last_clean":"2024-06-06T08:37:00.049657+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:37:00.049657+0000","last_undegraded":"2024-06-06T08:37:00.049657+0000","last_fullsized":"2024-06-06T08:37:00.049657+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.915421+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:37:21.915421+0000","last_peered":"2024-06-06T08:37:21.915421+0000","last_clean":"2024-06-06T08:37:21.915421+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:37:21.915421+0000","last_undegraded":"2024-06-06T08:37:21.915421+0000","last_fullsized":"2024-06-06T08:37:21.915421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050202+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:37:00.050202+0000","last_peered":"2024-06-06T08:37:00.050202+0000","last_clean":"2024-06-06T08:37:00.050202+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:37:00.050202+0000","last_undegraded":"2024-06-06T08:37:00.050202+0000","last_fullsized":"2024-06-06T08:37:00.050202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451228+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:37:00.451228+0000","last_peered":"2024-06-06T08:37:00.451228+0000","last_clean":"2024-06-06T08:37:00.451228+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:37:00.451228+0000","last_undegraded":"2024-06-06T08:37:00.451228+0000","last_fullsized":"2024-06-06T08:37:00.451228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.186017+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:37:00.186017+0000","last_peered":"2024-06-06T08:37:00.186017+0000","last_clean":"2024-06-06T08:37:00.186017+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:37:00.186017+0000","last_undegraded":"2024-06-06T08:37:00.186017+0000","last_fullsized":"2024-06-06T08:37:00.186017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.047726+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:37:00.047726+0000","last_peered":"2024-06-06T08:37:00.047726+0000","last_clean":"2024-06-06T08:37:00.047726+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:37:00.047726+0000","last_undegraded":"2024-06-06T08:37:00.047726+0000","last_fullsized":"2024-06-06T08:37:00.047726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.683085+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:37:00.683085+0000","last_peered":"2024-06-06T08:37:00.683085+0000","last_clean":"2024-06-06T08:37:00.683085+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:37:00.683085+0000","last_undegraded":"2024-06-06T08:37:00.683085+0000","last_fullsized":"2024-06-06T08:37:00.683085+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":67,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:46.875040+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:37:46.875040+0000","last_peered":"2024-06-06T08:37:46.875040+0000","last_clean":"2024-06-06T08:37:46.875040+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:37:46.875040+0000","last_undegraded":"2024-06-06T08:37:46.875040+0000","last_fullsized":"2024-06-06T08:37:46.875040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.054714+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:37:00.054714+0000","last_peered":"2024-06-06T08:37:00.054714+0000","last_clean":"2024-06-06T08:37:00.054714+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:37:00.054714+0000","last_undegraded":"2024-06-06T08:37:00.054714+0000","last_fullsized":"2024-06-06T08:37:00.054714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.047696+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:37:00.047696+0000","last_peered":"2024-06-06T08:37:00.047696+0000","last_clean":"2024-06-06T08:37:00.047696+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:37:00.047696+0000","last_undegraded":"2024-06-06T08:37:00.047696+0000","last_fullsized":"2024-06-06T08:37:00.047696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.911320+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:37:20.911320+0000","last_peered":"2024-06-06T08:37:20.911320+0000","last_clean":"2024-06-06T08:37:20.911320+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:37:20.911320+0000","last_undegraded":"2024-06-06T08:37:20.911320+0000","last_fullsized":"2024-06-06T08:37:20.911320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451030+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:37:00.451030+0000","last_peered":"2024-06-06T08:37:00.451030+0000","last_clean":"2024-06-06T08:37:00.451030+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:37:00.451030+0000","last_undegraded":"2024-06-06T08:37:00.451030+0000","last_fullsized":"2024-06-06T08:37:00.451030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051837+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:37:00.051837+0000","last_peered":"2024-06-06T08:37:00.051837+0000","last_clean":"2024-06-06T08:37:00.051837+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:37:00.051837+0000","last_undegraded":"2024-06-06T08:37:00.051837+0000","last_fullsized":"2024-06-06T08:37:00.051837+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.911971+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:37:20.911971+0000","last_peered":"2024-06-06T08:37:20.911971+0000","last_clean":"2024-06-06T08:37:20.911971+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:37:20.911971+0000","last_undegraded":"2024-06-06T08:37:20.911971+0000","last_fullsized":"2024-06-06T08:37:20.911971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053906+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:37:00.053906+0000","last_peered":"2024-06-06T08:37:00.053906+0000","last_clean":"2024-06-06T08:37:00.053906+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:37:00.053906+0000","last_undegraded":"2024-06-06T08:37:00.053906+0000","last_fullsized":"2024-06-06T08:37:00.053906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.332441+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:37:00.332441+0000","last_peered":"2024-06-06T08:37:00.332441+0000","last_clean":"2024-06-06T08:37:00.332441+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:37:00.332441+0000","last_undegraded":"2024-06-06T08:37:00.332441+0000","last_fullsized":"2024-06-06T08:37:00.332441+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"97'30","reported_seq":143,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:47.877685+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:37:47.877685+0000","last_peered":"2024-06-06T08:37:47.877685+0000","last_clean":"2024-06-06T08:37:47.877685+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:37:47.877685+0000","last_undegraded":"2024-06-06T08:37:47.877685+0000","last_fullsized":"2024-06-06T08:37:47.877685+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56667,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":57,"num_read_kb":84,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.331183+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:37:00.331183+0000","last_peered":"2024-06-06T08:37:00.331183+0000","last_clean":"2024-06-06T08:37:00.331183+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:37:00.331183+0000","last_undegraded":"2024-06-06T08:37:00.331183+0000","last_fullsized":"2024-06-06T08:37:00.331183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.186056+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:37:00.186056+0000","last_peered":"2024-06-06T08:37:00.186056+0000","last_clean":"2024-06-06T08:37:00.186056+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:37:00.186056+0000","last_undegraded":"2024-06-06T08:37:00.186056+0000","last_fullsized":"2024-06-06T08:37:00.186056+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.914971+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:37:21.914971+0000","last_peered":"2024-06-06T08:37:21.914971+0000","last_clean":"2024-06-06T08:37:21.914971+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:37:21.914971+0000","last_undegraded":"2024-06-06T08:37:21.914971+0000","last_fullsized":"2024-06-06T08:37:21.914971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049465+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:37:00.049465+0000","last_peered":"2024-06-06T08:37:00.049465+0000","last_clean":"2024-06-06T08:37:00.049465+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:37:00.049465+0000","last_undegraded":"2024-06-06T08:37:00.049465+0000","last_fullsized":"2024-06-06T08:37:00.049465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.683154+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:37:00.683154+0000","last_peered":"2024-06-06T08:37:00.683154+0000","last_clean":"2024-06-06T08:37:00.683154+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:37:00.683154+0000","last_undegraded":"2024-06-06T08:37:00.683154+0000","last_fullsized":"2024-06-06T08:37:00.683154+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.912644+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:37:20.912644+0000","last_peered":"2024-06-06T08:37:20.912644+0000","last_clean":"2024-06-06T08:37:20.912644+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:37:20.912644+0000","last_undegraded":"2024-06-06T08:37:20.912644+0000","last_fullsized":"2024-06-06T08:37:20.912644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681586+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:37:00.681586+0000","last_peered":"2024-06-06T08:37:00.681586+0000","last_clean":"2024-06-06T08:37:00.681586+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:37:00.681586+0000","last_undegraded":"2024-06-06T08:37:00.681586+0000","last_fullsized":"2024-06-06T08:37:00.681586+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.047857+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:37:00.047857+0000","last_peered":"2024-06-06T08:37:00.047857+0000","last_clean":"2024-06-06T08:37:00.047857+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:37:00.047857+0000","last_undegraded":"2024-06-06T08:37:00.047857+0000","last_fullsized":"2024-06-06T08:37:00.047857+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'4","reported_seq":71,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:46.876531+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:37:46.876531+0000","last_peered":"2024-06-06T08:37:46.876531+0000","last_clean":"2024-06-06T08:37:46.876531+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:37:46.876531+0000","last_undegraded":"2024-06-06T08:37:46.876531+0000","last_fullsized":"2024-06-06T08:37:46.876531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681824+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:37:00.681824+0000","last_peered":"2024-06-06T08:37:00.681824+0000","last_clean":"2024-06-06T08:37:00.681824+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:37:00.681824+0000","last_undegraded":"2024-06-06T08:37:00.681824+0000","last_fullsized":"2024-06-06T08:37:00.681824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049554+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:37:00.049554+0000","last_peered":"2024-06-06T08:37:00.049554+0000","last_clean":"2024-06-06T08:37:00.049554+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:37:00.049554+0000","last_undegraded":"2024-06-06T08:37:00.049554+0000","last_fullsized":"2024-06-06T08:37:00.049554+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":56,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.916448+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:37:21.916448+0000","last_peered":"2024-06-06T08:37:21.916448+0000","last_clean":"2024-06-06T08:37:21.916448+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:37:21.916448+0000","last_undegraded":"2024-06-06T08:37:21.916448+0000","last_fullsized":"2024-06-06T08:37:21.916448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185382+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:37:00.185382+0000","last_peered":"2024-06-06T08:37:00.185382+0000","last_clean":"2024-06-06T08:37:00.185382+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:37:00.185382+0000","last_undegraded":"2024-06-06T08:37:00.185382+0000","last_fullsized":"2024-06-06T08:37:00.185382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185360+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:37:00.185360+0000","last_peered":"2024-06-06T08:37:00.185360+0000","last_clean":"2024-06-06T08:37:00.185360+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:37:00.185360+0000","last_undegraded":"2024-06-06T08:37:00.185360+0000","last_fullsized":"2024-06-06T08:37:00.185360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:23.911268+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:37:23.911268+0000","last_peered":"2024-06-06T08:37:23.911268+0000","last_clean":"2024-06-06T08:37:23.911268+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:37:23.911268+0000","last_undegraded":"2024-06-06T08:37:23.911268+0000","last_fullsized":"2024-06-06T08:37:23.911268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049705+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:37:00.049705+0000","last_peered":"2024-06-06T08:37:00.049705+0000","last_clean":"2024-06-06T08:37:00.049705+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:37:00.049705+0000","last_undegraded":"2024-06-06T08:37:00.049705+0000","last_fullsized":"2024-06-06T08:37:00.049705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.330788+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:37:00.330788+0000","last_peered":"2024-06-06T08:37:00.330788+0000","last_clean":"2024-06-06T08:37:00.330788+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:37:00.330788+0000","last_undegraded":"2024-06-06T08:37:00.330788+0000","last_fullsized":"2024-06-06T08:37:00.330788+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048121+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:37:00.048121+0000","last_peered":"2024-06-06T08:37:00.048121+0000","last_clean":"2024-06-06T08:37:00.048121+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:37:00.048121+0000","last_undegraded":"2024-06-06T08:37:00.048121+0000","last_fullsized":"2024-06-06T08:37:00.048121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049936+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:37:00.049936+0000","last_peered":"2024-06-06T08:37:00.049936+0000","last_clean":"2024-06-06T08:37:00.049936+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:37:00.049936+0000","last_undegraded":"2024-06-06T08:37:00.049936+0000","last_fullsized":"2024-06-06T08:37:00.049936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048353+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:37:00.048353+0000","last_peered":"2024-06-06T08:37:00.048353+0000","last_clean":"2024-06-06T08:37:00.048353+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:37:00.048353+0000","last_undegraded":"2024-06-06T08:37:00.048353+0000","last_fullsized":"2024-06-06T08:37:00.048353+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681770+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:37:00.681770+0000","last_peered":"2024-06-06T08:37:00.681770+0000","last_clean":"2024-06-06T08:37:00.681770+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:37:00.681770+0000","last_undegraded":"2024-06-06T08:37:00.681770+0000","last_fullsized":"2024-06-06T08:37:00.681770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050565+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:37:00.050565+0000","last_peered":"2024-06-06T08:37:00.050565+0000","last_clean":"2024-06-06T08:37:00.050565+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:37:00.050565+0000","last_undegraded":"2024-06-06T08:37:00.050565+0000","last_fullsized":"2024-06-06T08:37:00.050565+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048736+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:37:00.048736+0000","last_peered":"2024-06-06T08:37:00.048736+0000","last_clean":"2024-06-06T08:37:00.048736+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:37:00.048736+0000","last_undegraded":"2024-06-06T08:37:00.048736+0000","last_fullsized":"2024-06-06T08:37:00.048736+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.186195+0000","last_change":"2024-06-06T08:36:20.903434+0000","last_active":"2024-06-06T08:37:00.186195+0000","last_peered":"2024-06-06T08:37:00.186195+0000","last_clean":"2024-06-06T08:37:00.186195+0000","last_became_active":"2024-06-06T08:36:20.901825+0000","last_became_peered":"2024-06-06T08:36:20.901825+0000","last_unstale":"2024-06-06T08:37:00.186195+0000","last_undegraded":"2024-06-06T08:37:00.186195+0000","last_fullsized":"2024-06-06T08:37:00.186195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:06.213780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":69,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:47.877370+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:37:47.877370+0000","last_peered":"2024-06-06T08:37:47.877370+0000","last_clean":"2024-06-06T08:37:47.877370+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:37:47.877370+0000","last_undegraded":"2024-06-06T08:37:47.877370+0000","last_fullsized":"2024-06-06T08:37:47.877370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681851+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:37:00.681851+0000","last_peered":"2024-06-06T08:37:00.681851+0000","last_clean":"2024-06-06T08:37:00.681851+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:37:00.681851+0000","last_undegraded":"2024-06-06T08:37:00.681851+0000","last_fullsized":"2024-06-06T08:37:00.681851+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050037+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:37:00.050037+0000","last_peered":"2024-06-06T08:37:00.050037+0000","last_clean":"2024-06-06T08:37:00.050037+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:37:00.050037+0000","last_undegraded":"2024-06-06T08:37:00.050037+0000","last_fullsized":"2024-06-06T08:37:00.050037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:24.840340+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:37:24.840340+0000","last_peered":"2024-06-06T08:37:24.840340+0000","last_clean":"2024-06-06T08:37:24.840340+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:37:24.840340+0000","last_undegraded":"2024-06-06T08:37:24.840340+0000","last_fullsized":"2024-06-06T08:37:24.840340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":371,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681869+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:37:00.681869+0000","last_peered":"2024-06-06T08:37:00.681869+0000","last_clean":"2024-06-06T08:37:00.681869+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:37:00.681869+0000","last_undegraded":"2024-06-06T08:37:00.681869+0000","last_fullsized":"2024-06-06T08:37:00.681869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185710+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:37:00.185710+0000","last_peered":"2024-06-06T08:37:00.185710+0000","last_clean":"2024-06-06T08:37:00.185710+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:37:00.185710+0000","last_undegraded":"2024-06-06T08:37:00.185710+0000","last_fullsized":"2024-06-06T08:37:00.185710+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048042+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:37:00.048042+0000","last_peered":"2024-06-06T08:37:00.048042+0000","last_clean":"2024-06-06T08:37:00.048042+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:37:00.048042+0000","last_undegraded":"2024-06-06T08:37:00.048042+0000","last_fullsized":"2024-06-06T08:37:00.048042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.913546+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:37:21.913546+0000","last_peered":"2024-06-06T08:37:21.913546+0000","last_clean":"2024-06-06T08:37:21.913546+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:37:21.913546+0000","last_undegraded":"2024-06-06T08:37:21.913546+0000","last_fullsized":"2024-06-06T08:37:21.913546+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049515+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:37:00.049515+0000","last_peered":"2024-06-06T08:37:00.049515+0000","last_clean":"2024-06-06T08:37:00.049515+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:37:00.049515+0000","last_undegraded":"2024-06-06T08:37:00.049515+0000","last_fullsized":"2024-06-06T08:37:00.049515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.911145+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:37:20.911145+0000","last_peered":"2024-06-06T08:37:20.911145+0000","last_clean":"2024-06-06T08:37:20.911145+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:37:20.911145+0000","last_undegraded":"2024-06-06T08:37:20.911145+0000","last_fullsized":"2024-06-06T08:37:20.911145+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048549+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:37:00.048549+0000","last_peered":"2024-06-06T08:37:00.048549+0000","last_clean":"2024-06-06T08:37:00.048549+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:37:00.048549+0000","last_undegraded":"2024-06-06T08:37:00.048549+0000","last_fullsized":"2024-06-06T08:37:00.048549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050088+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:37:00.050088+0000","last_peered":"2024-06-06T08:37:00.050088+0000","last_clean":"2024-06-06T08:37:00.050088+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:37:00.050088+0000","last_undegraded":"2024-06-06T08:37:00.050088+0000","last_fullsized":"2024-06-06T08:37:00.050088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'11","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:41.915304+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:37:41.915304+0000","last_peered":"2024-06-06T08:37:41.915304+0000","last_clean":"2024-06-06T08:37:41.915304+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:37:41.915304+0000","last_undegraded":"2024-06-06T08:37:41.915304+0000","last_fullsized":"2024-06-06T08:37:41.915304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050933+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:37:00.050933+0000","last_peered":"2024-06-06T08:37:00.050933+0000","last_clean":"2024-06-06T08:37:00.050933+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:37:00.050933+0000","last_undegraded":"2024-06-06T08:37:00.050933+0000","last_fullsized":"2024-06-06T08:37:00.050933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.045748+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:37:00.045748+0000","last_peered":"2024-06-06T08:37:00.045748+0000","last_clean":"2024-06-06T08:37:00.045748+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:37:00.045748+0000","last_undegraded":"2024-06-06T08:37:00.045748+0000","last_fullsized":"2024-06-06T08:37:00.045748+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.909814+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:37:20.909814+0000","last_peered":"2024-06-06T08:37:20.909814+0000","last_clean":"2024-06-06T08:37:20.909814+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:37:20.909814+0000","last_undegraded":"2024-06-06T08:37:20.909814+0000","last_fullsized":"2024-06-06T08:37:20.909814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451339+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:37:00.451339+0000","last_peered":"2024-06-06T08:37:00.451339+0000","last_clean":"2024-06-06T08:37:00.451339+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:37:00.451339+0000","last_undegraded":"2024-06-06T08:37:00.451339+0000","last_fullsized":"2024-06-06T08:37:00.451339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.187379+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:37:00.187379+0000","last_peered":"2024-06-06T08:37:00.187379+0000","last_clean":"2024-06-06T08:37:00.187379+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:37:00.187379+0000","last_undegraded":"2024-06-06T08:37:00.187379+0000","last_fullsized":"2024-06-06T08:37:00.187379+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.912032+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:37:20.912032+0000","last_peered":"2024-06-06T08:37:20.912032+0000","last_clean":"2024-06-06T08:37:20.912032+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:37:20.912032+0000","last_undegraded":"2024-06-06T08:37:20.912032+0000","last_fullsized":"2024-06-06T08:37:20.912032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.330864+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:37:00.330864+0000","last_peered":"2024-06-06T08:37:00.330864+0000","last_clean":"2024-06-06T08:37:00.330864+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:37:00.330864+0000","last_undegraded":"2024-06-06T08:37:00.330864+0000","last_fullsized":"2024-06-06T08:37:00.330864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048566+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:37:00.048566+0000","last_peered":"2024-06-06T08:37:00.048566+0000","last_clean":"2024-06-06T08:37:00.048566+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:37:00.048566+0000","last_undegraded":"2024-06-06T08:37:00.048566+0000","last_fullsized":"2024-06-06T08:37:00.048566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.910702+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:37:20.910702+0000","last_peered":"2024-06-06T08:37:20.910702+0000","last_clean":"2024-06-06T08:37:20.910702+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:37:20.910702+0000","last_undegraded":"2024-06-06T08:37:20.910702+0000","last_fullsized":"2024-06-06T08:37:20.910702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049530+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:37:00.049530+0000","last_peered":"2024-06-06T08:37:00.049530+0000","last_clean":"2024-06-06T08:37:00.049530+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:37:00.049530+0000","last_undegraded":"2024-06-06T08:37:00.049530+0000","last_fullsized":"2024-06-06T08:37:00.049530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051431+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:37:00.051431+0000","last_peered":"2024-06-06T08:37:00.051431+0000","last_clean":"2024-06-06T08:37:00.051431+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:37:00.051431+0000","last_undegraded":"2024-06-06T08:37:00.051431+0000","last_fullsized":"2024-06-06T08:37:00.051431+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046229+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:37:00.046229+0000","last_peered":"2024-06-06T08:37:00.046229+0000","last_clean":"2024-06-06T08:37:00.046229+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:37:00.046229+0000","last_undegraded":"2024-06-06T08:37:00.046229+0000","last_fullsized":"2024-06-06T08:37:00.046229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050658+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:37:00.050658+0000","last_peered":"2024-06-06T08:37:00.050658+0000","last_clean":"2024-06-06T08:37:00.050658+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:37:00.050658+0000","last_undegraded":"2024-06-06T08:37:00.050658+0000","last_fullsized":"2024-06-06T08:37:00.050658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051097+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:37:00.051097+0000","last_peered":"2024-06-06T08:37:00.051097+0000","last_clean":"2024-06-06T08:37:00.051097+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:37:00.051097+0000","last_undegraded":"2024-06-06T08:37:00.051097+0000","last_fullsized":"2024-06-06T08:37:00.051097+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.913095+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:37:20.913095+0000","last_peered":"2024-06-06T08:37:20.913095+0000","last_clean":"2024-06-06T08:37:20.913095+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:37:20.913095+0000","last_undegraded":"2024-06-06T08:37:20.913095+0000","last_fullsized":"2024-06-06T08:37:20.913095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681731+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:37:00.681731+0000","last_peered":"2024-06-06T08:37:00.681731+0000","last_clean":"2024-06-06T08:37:00.681731+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:37:00.681731+0000","last_undegraded":"2024-06-06T08:37:00.681731+0000","last_fullsized":"2024-06-06T08:37:00.681731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053427+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:37:00.053427+0000","last_peered":"2024-06-06T08:37:00.053427+0000","last_clean":"2024-06-06T08:37:00.053427+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:37:00.053427+0000","last_undegraded":"2024-06-06T08:37:00.053427+0000","last_fullsized":"2024-06-06T08:37:00.053427+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.916046+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:37:21.916046+0000","last_peered":"2024-06-06T08:37:21.916046+0000","last_clean":"2024-06-06T08:37:21.916046+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:37:21.916046+0000","last_undegraded":"2024-06-06T08:37:21.916046+0000","last_fullsized":"2024-06-06T08:37:21.916046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.682272+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:37:00.682272+0000","last_peered":"2024-06-06T08:37:00.682272+0000","last_clean":"2024-06-06T08:37:00.682272+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:37:00.682272+0000","last_undegraded":"2024-06-06T08:37:00.682272+0000","last_fullsized":"2024-06-06T08:37:00.682272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.450894+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:37:00.450894+0000","last_peered":"2024-06-06T08:37:00.450894+0000","last_clean":"2024-06-06T08:37:00.450894+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:37:00.450894+0000","last_undegraded":"2024-06-06T08:37:00.450894+0000","last_fullsized":"2024-06-06T08:37:00.450894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185873+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:37:00.185873+0000","last_peered":"2024-06-06T08:37:00.185873+0000","last_clean":"2024-06-06T08:37:00.185873+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:37:00.185873+0000","last_undegraded":"2024-06-06T08:37:00.185873+0000","last_fullsized":"2024-06-06T08:37:00.185873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681596+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:37:00.681596+0000","last_peered":"2024-06-06T08:37:00.681596+0000","last_clean":"2024-06-06T08:37:00.681596+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:37:00.681596+0000","last_undegraded":"2024-06-06T08:37:00.681596+0000","last_fullsized":"2024-06-06T08:37:00.681596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.184981+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:37:00.184981+0000","last_peered":"2024-06-06T08:37:00.184981+0000","last_clean":"2024-06-06T08:37:00.184981+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:37:00.184981+0000","last_undegraded":"2024-06-06T08:37:00.184981+0000","last_fullsized":"2024-06-06T08:37:00.184981+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.066312+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:37:00.066312+0000","last_peered":"2024-06-06T08:37:00.066312+0000","last_clean":"2024-06-06T08:37:00.066312+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:37:00.066312+0000","last_undegraded":"2024-06-06T08:37:00.066312+0000","last_fullsized":"2024-06-06T08:37:00.066312+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.682066+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:37:00.682066+0000","last_peered":"2024-06-06T08:37:00.682066+0000","last_clean":"2024-06-06T08:37:00.682066+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:37:00.682066+0000","last_undegraded":"2024-06-06T08:37:00.682066+0000","last_fullsized":"2024-06-06T08:37:00.682066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049989+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:37:00.049989+0000","last_peered":"2024-06-06T08:37:00.049989+0000","last_clean":"2024-06-06T08:37:00.049989+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:37:00.049989+0000","last_undegraded":"2024-06-06T08:37:00.049989+0000","last_fullsized":"2024-06-06T08:37:00.049989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046105+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:37:00.046105+0000","last_peered":"2024-06-06T08:37:00.046105+0000","last_clean":"2024-06-06T08:37:00.046105+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:37:00.046105+0000","last_undegraded":"2024-06-06T08:37:00.046105+0000","last_fullsized":"2024-06-06T08:37:00.046105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046072+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:37:00.046072+0000","last_peered":"2024-06-06T08:37:00.046072+0000","last_clean":"2024-06-06T08:37:00.046072+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:37:00.046072+0000","last_undegraded":"2024-06-06T08:37:00.046072+0000","last_fullsized":"2024-06-06T08:37:00.046072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451236+0000","last_change":"2024-06-06T08:36:20.903112+0000","last_active":"2024-06-06T08:37:00.451236+0000","last_peered":"2024-06-06T08:37:00.451236+0000","last_clean":"2024-06-06T08:37:00.451236+0000","last_became_active":"2024-06-06T08:36:20.902657+0000","last_became_peered":"2024-06-06T08:36:20.902657+0000","last_unstale":"2024-06-06T08:37:00.451236+0000","last_undegraded":"2024-06-06T08:37:00.451236+0000","last_fullsized":"2024-06-06T08:37:00.451236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:11.218419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048584+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:37:00.048584+0000","last_peered":"2024-06-06T08:37:00.048584+0000","last_clean":"2024-06-06T08:37:00.048584+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:37:00.048584+0000","last_undegraded":"2024-06-06T08:37:00.048584+0000","last_fullsized":"2024-06-06T08:37:00.048584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049863+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:37:00.049863+0000","last_peered":"2024-06-06T08:37:00.049863+0000","last_clean":"2024-06-06T08:37:00.049863+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:37:00.049863+0000","last_undegraded":"2024-06-06T08:37:00.049863+0000","last_fullsized":"2024-06-06T08:37:00.049863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046559+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:37:00.046559+0000","last_peered":"2024-06-06T08:37:00.046559+0000","last_clean":"2024-06-06T08:37:00.046559+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:37:00.046559+0000","last_undegraded":"2024-06-06T08:37:00.046559+0000","last_fullsized":"2024-06-06T08:37:00.046559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.332996+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:37:00.332996+0000","last_peered":"2024-06-06T08:37:00.332996+0000","last_clean":"2024-06-06T08:37:00.332996+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:37:00.332996+0000","last_undegraded":"2024-06-06T08:37:00.332996+0000","last_fullsized":"2024-06-06T08:37:00.332996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185862+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:37:00.185862+0000","last_peered":"2024-06-06T08:37:00.185862+0000","last_clean":"2024-06-06T08:37:00.185862+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:37:00.185862+0000","last_undegraded":"2024-06-06T08:37:00.185862+0000","last_fullsized":"2024-06-06T08:37:00.185862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050434+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:37:00.050434+0000","last_peered":"2024-06-06T08:37:00.050434+0000","last_clean":"2024-06-06T08:37:00.050434+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:37:00.050434+0000","last_undegraded":"2024-06-06T08:37:00.050434+0000","last_fullsized":"2024-06-06T08:37:00.050434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053800+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:37:00.053800+0000","last_peered":"2024-06-06T08:37:00.053800+0000","last_clean":"2024-06-06T08:37:00.053800+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:37:00.053800+0000","last_undegraded":"2024-06-06T08:37:00.053800+0000","last_fullsized":"2024-06-06T08:37:00.053800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049160+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:37:00.049160+0000","last_peered":"2024-06-06T08:37:00.049160+0000","last_clean":"2024-06-06T08:37:00.049160+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:37:00.049160+0000","last_undegraded":"2024-06-06T08:37:00.049160+0000","last_fullsized":"2024-06-06T08:37:00.049160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050715+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:37:00.050715+0000","last_peered":"2024-06-06T08:37:00.050715+0000","last_clean":"2024-06-06T08:37:00.050715+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:37:00.050715+0000","last_undegraded":"2024-06-06T08:37:00.050715+0000","last_fullsized":"2024-06-06T08:37:00.050715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280848,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60667,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":172,"num_read_kb":152,"num_write":164,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":182001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":550942,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32356,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712796,"statfs":{"total":95995035648,"available":95961903104,"internally_reserved":0,"allocated":1208320,"data_stored":1195245,"data_compressed":62162,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":555252,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":551054,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1146984,"data_compressed":62162,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":546332,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":602909,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":547720,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":547592,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":546526,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":608039,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1142182,"data_compressed":62162,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":57701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:37:52.839 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T08:37:53.108 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:53.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:52 smithi027 ceph-mon[32335]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:37:53.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:52 smithi203 ceph-mon[36579]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:37:53.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:52 smithi176 ceph-mon[36786]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:37:54.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:53 smithi176 ceph-mon[36786]: pgmap v354: 129 pgs: 129 active+clean; 636 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T08:37:54.391 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:53 smithi027 ceph-mon[32335]: pgmap v354: 129 pgs: 129 active+clean; 636 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T08:37:54.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:53 smithi203 ceph-mon[36579]: pgmap v354: 129 pgs: 129 active+clean; 636 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.947+0000 7f8723a5d700 1 -- 172.21.15.27:0/853871780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c107840 msgr2=0x7f871c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.947+0000 7f8723a5d700 1 --2- 172.21.15.27:0/853871780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c107840 0x7f871c10f3c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f871400b0d0 tx=0x7f871400b3e0 comp rx=0 tx=0).stop 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.947+0000 7f8723a5d700 1 -- 172.21.15.27:0/853871780 shutdown_connections 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.947+0000 7f8723a5d700 1 --2- 172.21.15.27:0/853871780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c107840 0x7f871c10f3c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.947+0000 7f8723a5d700 1 --2- 172.21.15.27:0/853871780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f871c102740 0x7f871c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.947+0000 7f8723a5d700 1 --2- 172.21.15.27:0/853871780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c1030f0 0x7f871c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.947+0000 7f8723a5d700 1 -- 172.21.15.27:0/853871780 >> 172.21.15.27:0/853871780 conn(0x7f871c0795f0 msgr2=0x7f871c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.947+0000 7f8723a5d700 1 -- 172.21.15.27:0/853871780 shutdown_connections 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.947+0000 7f8723a5d700 1 -- 172.21.15.27:0/853871780 wait complete. 2024-06-06T08:37:54.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8723a5d700 1 Processor -- start 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8723a5d700 1 -- start start 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8723a5d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c102740 0x7f871c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8723a5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c1030f0 0x7f871c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8723a5d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f871c107840 0x7f871c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8723a5d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f871c112760 con 0x7f871c1030f0 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8723a5d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1125e0 con 0x7f871c107840 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8723a5d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f871c1128e0 con 0x7f871c102740 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8720ff8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c1030f0 0x7f871c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8720ff8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c1030f0 0x7f871c19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58728/0 (socket says 172.21.15.27:58728) 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.948+0000 7f8720ff8700 1 -- 172.21.15.27:0/3521225995 learned_addr learned my addr 172.21.15.27:0/3521225995 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f87217f9700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c102740 0x7f871c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:54.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f8721ffa700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f871c107840 0x7f871c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f8720ff8700 1 -- 172.21.15.27:0/3521225995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c102740 msgr2=0x7f871c19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f8720ff8700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c102740 0x7f871c19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f8720ff8700 1 -- 172.21.15.27:0/3521225995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f871c107840 msgr2=0x7f871c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f8720ff8700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f871c107840 0x7f871c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f8720ff8700 1 -- 172.21.15.27:0/3521225995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c1990e0 con 0x7f871c1030f0 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f87217f9700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c102740 0x7f871c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f8720ff8700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c1030f0 0x7f871c19ecb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c920 tx=0x7f870c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f87127fc700 1 -- 172.21.15.27:0/3521225995 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c007b00 con 0x7f871c1030f0 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f87127fc700 1 -- 172.21.15.27:0/3521225995 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f870c004d10 con 0x7f871c1030f0 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c1993d0 con 0x7f871c1030f0 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f87127fc700 1 -- 172.21.15.27:0/3521225995 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c005680 con 0x7f871c1030f0 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.949+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c1a5640 con 0x7f871c1030f0 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.950+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c069180 con 0x7f871c1030f0 2024-06-06T08:37:54.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.951+0000 7f87127fc700 1 -- 172.21.15.27:0/3521225995 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f870c004750 con 0x7f871c1030f0 2024-06-06T08:37:54.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.952+0000 7f87127fc700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f87080a2d30 0x7f87080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:54.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.952+0000 7f87127fc700 1 -- 172.21.15.27:0/3521225995 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f870c0cf2f0 con 0x7f871c1030f0 2024-06-06T08:37:54.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.952+0000 7f87217f9700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f87080a2d30 0x7f87080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:54.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.952+0000 7f87217f9700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f87080a2d30 0x7f87080a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8718005950 tx=0x7f87180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:54.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:54.956+0000 7f87127fc700 1 -- 172.21.15.27:0/3521225995 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f870c014030 con 0x7f871c1030f0 2024-06-06T08:37:55.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.159+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f871c0628d0 con 0x7f87080a2d30 2024-06-06T08:37:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.170+0000 7f87127fc700 1 -- 172.21.15.27:0/3521225995 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340651 (secure 0 0 0) 0x7f871c0628d0 con 0x7f87080a2d30 2024-06-06T08:37:55.173 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:37:55.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f87080a2d30 msgr2=0x7f87080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f87080a2d30 0x7f87080a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8718005950 tx=0x7f87180058e0 comp rx=0 tx=0).stop 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c1030f0 msgr2=0x7f871c19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c1030f0 0x7f871c19ecb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c920 tx=0x7f870c00cc30 comp rx=0 tx=0).stop 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 shutdown_connections 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f87080a2d30 0x7f87080a51f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f871c107840 0x7f871c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c102740 0x7f871c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 --2- 172.21.15.27:0/3521225995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c1030f0 0x7f871c19ecb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 >> 172.21.15.27:0/3521225995 conn(0x7f871c0795f0 msgr2=0x7f871c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 shutdown_connections 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:55.174+0000 7f8723a5d700 1 -- 172.21.15.27:0/3521225995 wait complete. 2024-06-06T08:37:55.176 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:37:55.954 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-06-06T08:37:54.456835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656672,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":339,"num_read_kb":292,"num_write":352,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353848,"kb_used_data":13016,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587976,"statfs":{"total":1151940427776,"available":1151578087424,"internally_reserved":0,"allocated":13328384,"data_stored":8556986,"data_compressed":424455,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32982,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":89,"num_read_kb":79,"num_write":25,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002858"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050231+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:37:00.050231+0000","last_peered":"2024-06-06T08:37:00.050231+0000","last_clean":"2024-06-06T08:37:00.050231+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:37:00.050231+0000","last_undegraded":"2024-06-06T08:37:00.050231+0000","last_fullsized":"2024-06-06T08:37:00.050231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451282+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:37:00.451282+0000","last_peered":"2024-06-06T08:37:00.451282+0000","last_clean":"2024-06-06T08:37:00.451282+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:37:00.451282+0000","last_undegraded":"2024-06-06T08:37:00.451282+0000","last_fullsized":"2024-06-06T08:37:00.451282+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451041+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:37:00.451041+0000","last_peered":"2024-06-06T08:37:00.451041+0000","last_clean":"2024-06-06T08:37:00.451041+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:37:00.451041+0000","last_undegraded":"2024-06-06T08:37:00.451041+0000","last_fullsized":"2024-06-06T08:37:00.451041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050562+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:37:00.050562+0000","last_peered":"2024-06-06T08:37:00.050562+0000","last_clean":"2024-06-06T08:37:00.050562+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:37:00.050562+0000","last_undegraded":"2024-06-06T08:37:00.050562+0000","last_fullsized":"2024-06-06T08:37:00.050562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.330620+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:37:00.330620+0000","last_peered":"2024-06-06T08:37:00.330620+0000","last_clean":"2024-06-06T08:37:00.330620+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:37:00.330620+0000","last_undegraded":"2024-06-06T08:37:00.330620+0000","last_fullsized":"2024-06-06T08:37:00.330620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050290+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:37:00.050290+0000","last_peered":"2024-06-06T08:37:00.050290+0000","last_clean":"2024-06-06T08:37:00.050290+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:37:00.050290+0000","last_undegraded":"2024-06-06T08:37:00.050290+0000","last_fullsized":"2024-06-06T08:37:00.050290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681748+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:37:00.681748+0000","last_peered":"2024-06-06T08:37:00.681748+0000","last_clean":"2024-06-06T08:37:00.681748+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:37:00.681748+0000","last_undegraded":"2024-06-06T08:37:00.681748+0000","last_fullsized":"2024-06-06T08:37:00.681748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049965+0000","last_change":"2024-06-06T08:36:20.890919+0000","last_active":"2024-06-06T08:37:00.049965+0000","last_peered":"2024-06-06T08:37:00.049965+0000","last_clean":"2024-06-06T08:37:00.049965+0000","last_became_active":"2024-06-06T08:36:20.890501+0000","last_became_peered":"2024-06-06T08:36:20.890501+0000","last_unstale":"2024-06-06T08:37:00.049965+0000","last_undegraded":"2024-06-06T08:37:00.049965+0000","last_fullsized":"2024-06-06T08:37:00.049965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:51:24.461247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.054552+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:37:00.054552+0000","last_peered":"2024-06-06T08:37:00.054552+0000","last_clean":"2024-06-06T08:37:00.054552+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:37:00.054552+0000","last_undegraded":"2024-06-06T08:37:00.054552+0000","last_fullsized":"2024-06-06T08:37:00.054552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.052027+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:37:00.052027+0000","last_peered":"2024-06-06T08:37:00.052027+0000","last_clean":"2024-06-06T08:37:00.052027+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:37:00.052027+0000","last_undegraded":"2024-06-06T08:37:00.052027+0000","last_fullsized":"2024-06-06T08:37:00.052027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.682111+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:37:00.682111+0000","last_peered":"2024-06-06T08:37:00.682111+0000","last_clean":"2024-06-06T08:37:00.682111+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:37:00.682111+0000","last_undegraded":"2024-06-06T08:37:00.682111+0000","last_fullsized":"2024-06-06T08:37:00.682111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050692+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:37:00.050692+0000","last_peered":"2024-06-06T08:37:00.050692+0000","last_clean":"2024-06-06T08:37:00.050692+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:37:00.050692+0000","last_undegraded":"2024-06-06T08:37:00.050692+0000","last_fullsized":"2024-06-06T08:37:00.050692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049345+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:37:00.049345+0000","last_peered":"2024-06-06T08:37:00.049345+0000","last_clean":"2024-06-06T08:37:00.049345+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:37:00.049345+0000","last_undegraded":"2024-06-06T08:37:00.049345+0000","last_fullsized":"2024-06-06T08:37:00.049345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.450932+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:37:00.450932+0000","last_peered":"2024-06-06T08:37:00.450932+0000","last_clean":"2024-06-06T08:37:00.450932+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:37:00.450932+0000","last_undegraded":"2024-06-06T08:37:00.450932+0000","last_fullsized":"2024-06-06T08:37:00.450932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.450865+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:37:00.450865+0000","last_peered":"2024-06-06T08:37:00.450865+0000","last_clean":"2024-06-06T08:37:00.450865+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:37:00.450865+0000","last_undegraded":"2024-06-06T08:37:00.450865+0000","last_fullsized":"2024-06-06T08:37:00.450865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053308+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:37:00.053308+0000","last_peered":"2024-06-06T08:37:00.053308+0000","last_clean":"2024-06-06T08:37:00.053308+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:37:00.053308+0000","last_undegraded":"2024-06-06T08:37:00.053308+0000","last_fullsized":"2024-06-06T08:37:00.053308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049683+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:37:00.049683+0000","last_peered":"2024-06-06T08:37:00.049683+0000","last_clean":"2024-06-06T08:37:00.049683+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:37:00.049683+0000","last_undegraded":"2024-06-06T08:37:00.049683+0000","last_fullsized":"2024-06-06T08:37:00.049683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048456+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:37:00.048456+0000","last_peered":"2024-06-06T08:37:00.048456+0000","last_clean":"2024-06-06T08:37:00.048456+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:37:00.048456+0000","last_undegraded":"2024-06-06T08:37:00.048456+0000","last_fullsized":"2024-06-06T08:37:00.048456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046388+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:37:00.046388+0000","last_peered":"2024-06-06T08:37:00.046388+0000","last_clean":"2024-06-06T08:37:00.046388+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:37:00.046388+0000","last_undegraded":"2024-06-06T08:37:00.046388+0000","last_fullsized":"2024-06-06T08:37:00.046388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.045941+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:37:00.045941+0000","last_peered":"2024-06-06T08:37:00.045941+0000","last_clean":"2024-06-06T08:37:00.045941+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:37:00.045941+0000","last_undegraded":"2024-06-06T08:37:00.045941+0000","last_fullsized":"2024-06-06T08:37:00.045941+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.916397+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:37:21.916397+0000","last_peered":"2024-06-06T08:37:21.916397+0000","last_clean":"2024-06-06T08:37:21.916397+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:37:21.916397+0000","last_undegraded":"2024-06-06T08:37:21.916397+0000","last_fullsized":"2024-06-06T08:37:21.916397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050708+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:37:00.050708+0000","last_peered":"2024-06-06T08:37:00.050708+0000","last_clean":"2024-06-06T08:37:00.050708+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:37:00.050708+0000","last_undegraded":"2024-06-06T08:37:00.050708+0000","last_fullsized":"2024-06-06T08:37:00.050708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050402+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:37:00.050402+0000","last_peered":"2024-06-06T08:37:00.050402+0000","last_clean":"2024-06-06T08:37:00.050402+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:37:00.050402+0000","last_undegraded":"2024-06-06T08:37:00.050402+0000","last_fullsized":"2024-06-06T08:37:00.050402+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051034+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:37:00.051034+0000","last_peered":"2024-06-06T08:37:00.051034+0000","last_clean":"2024-06-06T08:37:00.051034+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:37:00.051034+0000","last_undegraded":"2024-06-06T08:37:00.051034+0000","last_fullsized":"2024-06-06T08:37:00.051034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451037+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:37:00.451037+0000","last_peered":"2024-06-06T08:37:00.451037+0000","last_clean":"2024-06-06T08:37:00.451037+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:37:00.451037+0000","last_undegraded":"2024-06-06T08:37:00.451037+0000","last_fullsized":"2024-06-06T08:37:00.451037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051195+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:37:00.051195+0000","last_peered":"2024-06-06T08:37:00.051195+0000","last_clean":"2024-06-06T08:37:00.051195+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:37:00.051195+0000","last_undegraded":"2024-06-06T08:37:00.051195+0000","last_fullsized":"2024-06-06T08:37:00.051195+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.914251+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:37:21.914251+0000","last_peered":"2024-06-06T08:37:21.914251+0000","last_clean":"2024-06-06T08:37:21.914251+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:37:21.914251+0000","last_undegraded":"2024-06-06T08:37:21.914251+0000","last_fullsized":"2024-06-06T08:37:21.914251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053743+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:37:00.053743+0000","last_peered":"2024-06-06T08:37:00.053743+0000","last_clean":"2024-06-06T08:37:00.053743+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:37:00.053743+0000","last_undegraded":"2024-06-06T08:37:00.053743+0000","last_fullsized":"2024-06-06T08:37:00.053743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050985+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:37:00.050985+0000","last_peered":"2024-06-06T08:37:00.050985+0000","last_clean":"2024-06-06T08:37:00.050985+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:37:00.050985+0000","last_undegraded":"2024-06-06T08:37:00.050985+0000","last_fullsized":"2024-06-06T08:37:00.050985+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.331133+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:37:00.331133+0000","last_peered":"2024-06-06T08:37:00.331133+0000","last_clean":"2024-06-06T08:37:00.331133+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:37:00.331133+0000","last_undegraded":"2024-06-06T08:37:00.331133+0000","last_fullsized":"2024-06-06T08:37:00.331133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185714+0000","last_change":"2024-06-06T08:36:20.902582+0000","last_active":"2024-06-06T08:37:00.185714+0000","last_peered":"2024-06-06T08:37:00.185714+0000","last_clean":"2024-06-06T08:37:00.185714+0000","last_became_active":"2024-06-06T08:36:20.902194+0000","last_became_peered":"2024-06-06T08:36:20.902194+0000","last_unstale":"2024-06-06T08:37:00.185714+0000","last_undegraded":"2024-06-06T08:37:00.185714+0000","last_fullsized":"2024-06-06T08:37:00.185714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:08:33.363612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046079+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:37:00.046079+0000","last_peered":"2024-06-06T08:37:00.046079+0000","last_clean":"2024-06-06T08:37:00.046079+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:37:00.046079+0000","last_undegraded":"2024-06-06T08:37:00.046079+0000","last_fullsized":"2024-06-06T08:37:00.046079+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":75,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:50.881340+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:37:50.881340+0000","last_peered":"2024-06-06T08:37:50.881340+0000","last_clean":"2024-06-06T08:37:50.881340+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:37:50.881340+0000","last_undegraded":"2024-06-06T08:37:50.881340+0000","last_fullsized":"2024-06-06T08:37:50.881340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.330869+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:37:00.330869+0000","last_peered":"2024-06-06T08:37:00.330869+0000","last_clean":"2024-06-06T08:37:00.330869+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:37:00.330869+0000","last_undegraded":"2024-06-06T08:37:00.330869+0000","last_fullsized":"2024-06-06T08:37:00.330869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046022+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:37:00.046022+0000","last_peered":"2024-06-06T08:37:00.046022+0000","last_clean":"2024-06-06T08:37:00.046022+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:37:00.046022+0000","last_undegraded":"2024-06-06T08:37:00.046022+0000","last_fullsized":"2024-06-06T08:37:00.046022+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.914900+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:37:21.914900+0000","last_peered":"2024-06-06T08:37:21.914900+0000","last_clean":"2024-06-06T08:37:21.914900+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:37:21.914900+0000","last_undegraded":"2024-06-06T08:37:21.914900+0000","last_fullsized":"2024-06-06T08:37:21.914900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185797+0000","last_change":"2024-06-06T08:36:20.899203+0000","last_active":"2024-06-06T08:37:00.185797+0000","last_peered":"2024-06-06T08:37:00.185797+0000","last_clean":"2024-06-06T08:37:00.185797+0000","last_became_active":"2024-06-06T08:36:20.898873+0000","last_became_peered":"2024-06-06T08:36:20.898873+0000","last_unstale":"2024-06-06T08:37:00.185797+0000","last_undegraded":"2024-06-06T08:37:00.185797+0000","last_fullsized":"2024-06-06T08:37:00.185797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:15:45.218350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049657+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:37:00.049657+0000","last_peered":"2024-06-06T08:37:00.049657+0000","last_clean":"2024-06-06T08:37:00.049657+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:37:00.049657+0000","last_undegraded":"2024-06-06T08:37:00.049657+0000","last_fullsized":"2024-06-06T08:37:00.049657+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.915421+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:37:21.915421+0000","last_peered":"2024-06-06T08:37:21.915421+0000","last_clean":"2024-06-06T08:37:21.915421+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:37:21.915421+0000","last_undegraded":"2024-06-06T08:37:21.915421+0000","last_fullsized":"2024-06-06T08:37:21.915421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050202+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:37:00.050202+0000","last_peered":"2024-06-06T08:37:00.050202+0000","last_clean":"2024-06-06T08:37:00.050202+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:37:00.050202+0000","last_undegraded":"2024-06-06T08:37:00.050202+0000","last_fullsized":"2024-06-06T08:37:00.050202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451228+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:37:00.451228+0000","last_peered":"2024-06-06T08:37:00.451228+0000","last_clean":"2024-06-06T08:37:00.451228+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:37:00.451228+0000","last_undegraded":"2024-06-06T08:37:00.451228+0000","last_fullsized":"2024-06-06T08:37:00.451228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.186017+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:37:00.186017+0000","last_peered":"2024-06-06T08:37:00.186017+0000","last_clean":"2024-06-06T08:37:00.186017+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:37:00.186017+0000","last_undegraded":"2024-06-06T08:37:00.186017+0000","last_fullsized":"2024-06-06T08:37:00.186017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.047726+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:37:00.047726+0000","last_peered":"2024-06-06T08:37:00.047726+0000","last_clean":"2024-06-06T08:37:00.047726+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:37:00.047726+0000","last_undegraded":"2024-06-06T08:37:00.047726+0000","last_fullsized":"2024-06-06T08:37:00.047726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.683085+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:37:00.683085+0000","last_peered":"2024-06-06T08:37:00.683085+0000","last_clean":"2024-06-06T08:37:00.683085+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:37:00.683085+0000","last_undegraded":"2024-06-06T08:37:00.683085+0000","last_fullsized":"2024-06-06T08:37:00.683085+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":72,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:51.882203+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:37:51.882203+0000","last_peered":"2024-06-06T08:37:51.882203+0000","last_clean":"2024-06-06T08:37:51.882203+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:37:51.882203+0000","last_undegraded":"2024-06-06T08:37:51.882203+0000","last_fullsized":"2024-06-06T08:37:51.882203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.054714+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:37:00.054714+0000","last_peered":"2024-06-06T08:37:00.054714+0000","last_clean":"2024-06-06T08:37:00.054714+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:37:00.054714+0000","last_undegraded":"2024-06-06T08:37:00.054714+0000","last_fullsized":"2024-06-06T08:37:00.054714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.047696+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:37:00.047696+0000","last_peered":"2024-06-06T08:37:00.047696+0000","last_clean":"2024-06-06T08:37:00.047696+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:37:00.047696+0000","last_undegraded":"2024-06-06T08:37:00.047696+0000","last_fullsized":"2024-06-06T08:37:00.047696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.911320+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:37:20.911320+0000","last_peered":"2024-06-06T08:37:20.911320+0000","last_clean":"2024-06-06T08:37:20.911320+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:37:20.911320+0000","last_undegraded":"2024-06-06T08:37:20.911320+0000","last_fullsized":"2024-06-06T08:37:20.911320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451030+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:37:00.451030+0000","last_peered":"2024-06-06T08:37:00.451030+0000","last_clean":"2024-06-06T08:37:00.451030+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:37:00.451030+0000","last_undegraded":"2024-06-06T08:37:00.451030+0000","last_fullsized":"2024-06-06T08:37:00.451030+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051837+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:37:00.051837+0000","last_peered":"2024-06-06T08:37:00.051837+0000","last_clean":"2024-06-06T08:37:00.051837+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:37:00.051837+0000","last_undegraded":"2024-06-06T08:37:00.051837+0000","last_fullsized":"2024-06-06T08:37:00.051837+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.911971+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:37:20.911971+0000","last_peered":"2024-06-06T08:37:20.911971+0000","last_clean":"2024-06-06T08:37:20.911971+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:37:20.911971+0000","last_undegraded":"2024-06-06T08:37:20.911971+0000","last_fullsized":"2024-06-06T08:37:20.911971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053906+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:37:00.053906+0000","last_peered":"2024-06-06T08:37:00.053906+0000","last_clean":"2024-06-06T08:37:00.053906+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:37:00.053906+0000","last_undegraded":"2024-06-06T08:37:00.053906+0000","last_fullsized":"2024-06-06T08:37:00.053906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.332441+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:37:00.332441+0000","last_peered":"2024-06-06T08:37:00.332441+0000","last_clean":"2024-06-06T08:37:00.332441+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:37:00.332441+0000","last_undegraded":"2024-06-06T08:37:00.332441+0000","last_fullsized":"2024-06-06T08:37:00.332441+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"97'33","reported_seq":157,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:52.884108+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:37:52.884108+0000","last_peered":"2024-06-06T08:37:52.884108+0000","last_clean":"2024-06-06T08:37:52.884108+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:37:52.884108+0000","last_undegraded":"2024-06-06T08:37:52.884108+0000","last_fullsized":"2024-06-06T08:37:52.884108+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62070,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":68,"num_read_kb":95,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.331183+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:37:00.331183+0000","last_peered":"2024-06-06T08:37:00.331183+0000","last_clean":"2024-06-06T08:37:00.331183+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:37:00.331183+0000","last_undegraded":"2024-06-06T08:37:00.331183+0000","last_fullsized":"2024-06-06T08:37:00.331183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.186056+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:37:00.186056+0000","last_peered":"2024-06-06T08:37:00.186056+0000","last_clean":"2024-06-06T08:37:00.186056+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:37:00.186056+0000","last_undegraded":"2024-06-06T08:37:00.186056+0000","last_fullsized":"2024-06-06T08:37:00.186056+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.914971+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:37:21.914971+0000","last_peered":"2024-06-06T08:37:21.914971+0000","last_clean":"2024-06-06T08:37:21.914971+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:37:21.914971+0000","last_undegraded":"2024-06-06T08:37:21.914971+0000","last_fullsized":"2024-06-06T08:37:21.914971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049465+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:37:00.049465+0000","last_peered":"2024-06-06T08:37:00.049465+0000","last_clean":"2024-06-06T08:37:00.049465+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:37:00.049465+0000","last_undegraded":"2024-06-06T08:37:00.049465+0000","last_fullsized":"2024-06-06T08:37:00.049465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.683154+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:37:00.683154+0000","last_peered":"2024-06-06T08:37:00.683154+0000","last_clean":"2024-06-06T08:37:00.683154+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:37:00.683154+0000","last_undegraded":"2024-06-06T08:37:00.683154+0000","last_fullsized":"2024-06-06T08:37:00.683154+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.912644+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:37:20.912644+0000","last_peered":"2024-06-06T08:37:20.912644+0000","last_clean":"2024-06-06T08:37:20.912644+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:37:20.912644+0000","last_undegraded":"2024-06-06T08:37:20.912644+0000","last_fullsized":"2024-06-06T08:37:20.912644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681586+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:37:00.681586+0000","last_peered":"2024-06-06T08:37:00.681586+0000","last_clean":"2024-06-06T08:37:00.681586+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:37:00.681586+0000","last_undegraded":"2024-06-06T08:37:00.681586+0000","last_fullsized":"2024-06-06T08:37:00.681586+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.047857+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:37:00.047857+0000","last_peered":"2024-06-06T08:37:00.047857+0000","last_clean":"2024-06-06T08:37:00.047857+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:37:00.047857+0000","last_undegraded":"2024-06-06T08:37:00.047857+0000","last_fullsized":"2024-06-06T08:37:00.047857+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'4","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:51.883323+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:37:51.883323+0000","last_peered":"2024-06-06T08:37:51.883323+0000","last_clean":"2024-06-06T08:37:51.883323+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:37:51.883323+0000","last_undegraded":"2024-06-06T08:37:51.883323+0000","last_fullsized":"2024-06-06T08:37:51.883323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681824+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:37:00.681824+0000","last_peered":"2024-06-06T08:37:00.681824+0000","last_clean":"2024-06-06T08:37:00.681824+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:37:00.681824+0000","last_undegraded":"2024-06-06T08:37:00.681824+0000","last_fullsized":"2024-06-06T08:37:00.681824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049554+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:37:00.049554+0000","last_peered":"2024-06-06T08:37:00.049554+0000","last_clean":"2024-06-06T08:37:00.049554+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:37:00.049554+0000","last_undegraded":"2024-06-06T08:37:00.049554+0000","last_fullsized":"2024-06-06T08:37:00.049554+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":56,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.916448+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:37:21.916448+0000","last_peered":"2024-06-06T08:37:21.916448+0000","last_clean":"2024-06-06T08:37:21.916448+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:37:21.916448+0000","last_undegraded":"2024-06-06T08:37:21.916448+0000","last_fullsized":"2024-06-06T08:37:21.916448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185382+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:37:00.185382+0000","last_peered":"2024-06-06T08:37:00.185382+0000","last_clean":"2024-06-06T08:37:00.185382+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:37:00.185382+0000","last_undegraded":"2024-06-06T08:37:00.185382+0000","last_fullsized":"2024-06-06T08:37:00.185382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185360+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:37:00.185360+0000","last_peered":"2024-06-06T08:37:00.185360+0000","last_clean":"2024-06-06T08:37:00.185360+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:37:00.185360+0000","last_undegraded":"2024-06-06T08:37:00.185360+0000","last_fullsized":"2024-06-06T08:37:00.185360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:23.911268+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:37:23.911268+0000","last_peered":"2024-06-06T08:37:23.911268+0000","last_clean":"2024-06-06T08:37:23.911268+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:37:23.911268+0000","last_undegraded":"2024-06-06T08:37:23.911268+0000","last_fullsized":"2024-06-06T08:37:23.911268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049705+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:37:00.049705+0000","last_peered":"2024-06-06T08:37:00.049705+0000","last_clean":"2024-06-06T08:37:00.049705+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:37:00.049705+0000","last_undegraded":"2024-06-06T08:37:00.049705+0000","last_fullsized":"2024-06-06T08:37:00.049705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.330788+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:37:00.330788+0000","last_peered":"2024-06-06T08:37:00.330788+0000","last_clean":"2024-06-06T08:37:00.330788+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:37:00.330788+0000","last_undegraded":"2024-06-06T08:37:00.330788+0000","last_fullsized":"2024-06-06T08:37:00.330788+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048121+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:37:00.048121+0000","last_peered":"2024-06-06T08:37:00.048121+0000","last_clean":"2024-06-06T08:37:00.048121+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:37:00.048121+0000","last_undegraded":"2024-06-06T08:37:00.048121+0000","last_fullsized":"2024-06-06T08:37:00.048121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049936+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:37:00.049936+0000","last_peered":"2024-06-06T08:37:00.049936+0000","last_clean":"2024-06-06T08:37:00.049936+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:37:00.049936+0000","last_undegraded":"2024-06-06T08:37:00.049936+0000","last_fullsized":"2024-06-06T08:37:00.049936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048353+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:37:00.048353+0000","last_peered":"2024-06-06T08:37:00.048353+0000","last_clean":"2024-06-06T08:37:00.048353+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:37:00.048353+0000","last_undegraded":"2024-06-06T08:37:00.048353+0000","last_fullsized":"2024-06-06T08:37:00.048353+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681770+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:37:00.681770+0000","last_peered":"2024-06-06T08:37:00.681770+0000","last_clean":"2024-06-06T08:37:00.681770+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:37:00.681770+0000","last_undegraded":"2024-06-06T08:37:00.681770+0000","last_fullsized":"2024-06-06T08:37:00.681770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050565+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:37:00.050565+0000","last_peered":"2024-06-06T08:37:00.050565+0000","last_clean":"2024-06-06T08:37:00.050565+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:37:00.050565+0000","last_undegraded":"2024-06-06T08:37:00.050565+0000","last_fullsized":"2024-06-06T08:37:00.050565+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048736+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:37:00.048736+0000","last_peered":"2024-06-06T08:37:00.048736+0000","last_clean":"2024-06-06T08:37:00.048736+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:37:00.048736+0000","last_undegraded":"2024-06-06T08:37:00.048736+0000","last_fullsized":"2024-06-06T08:37:00.048736+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.186195+0000","last_change":"2024-06-06T08:36:20.903434+0000","last_active":"2024-06-06T08:37:00.186195+0000","last_peered":"2024-06-06T08:37:00.186195+0000","last_clean":"2024-06-06T08:37:00.186195+0000","last_became_active":"2024-06-06T08:36:20.901825+0000","last_became_peered":"2024-06-06T08:36:20.901825+0000","last_unstale":"2024-06-06T08:37:00.186195+0000","last_undegraded":"2024-06-06T08:37:00.186195+0000","last_fullsized":"2024-06-06T08:37:00.186195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:06.213780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'4","reported_seq":74,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:52.884140+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:37:52.884140+0000","last_peered":"2024-06-06T08:37:52.884140+0000","last_clean":"2024-06-06T08:37:52.884140+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:37:52.884140+0000","last_undegraded":"2024-06-06T08:37:52.884140+0000","last_fullsized":"2024-06-06T08:37:52.884140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681851+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:37:00.681851+0000","last_peered":"2024-06-06T08:37:00.681851+0000","last_clean":"2024-06-06T08:37:00.681851+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:37:00.681851+0000","last_undegraded":"2024-06-06T08:37:00.681851+0000","last_fullsized":"2024-06-06T08:37:00.681851+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050037+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:37:00.050037+0000","last_peered":"2024-06-06T08:37:00.050037+0000","last_clean":"2024-06-06T08:37:00.050037+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:37:00.050037+0000","last_undegraded":"2024-06-06T08:37:00.050037+0000","last_fullsized":"2024-06-06T08:37:00.050037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:24.840340+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:37:24.840340+0000","last_peered":"2024-06-06T08:37:24.840340+0000","last_clean":"2024-06-06T08:37:24.840340+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:37:24.840340+0000","last_undegraded":"2024-06-06T08:37:24.840340+0000","last_fullsized":"2024-06-06T08:37:24.840340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":371,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681869+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:37:00.681869+0000","last_peered":"2024-06-06T08:37:00.681869+0000","last_clean":"2024-06-06T08:37:00.681869+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:37:00.681869+0000","last_undegraded":"2024-06-06T08:37:00.681869+0000","last_fullsized":"2024-06-06T08:37:00.681869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185710+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:37:00.185710+0000","last_peered":"2024-06-06T08:37:00.185710+0000","last_clean":"2024-06-06T08:37:00.185710+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:37:00.185710+0000","last_undegraded":"2024-06-06T08:37:00.185710+0000","last_fullsized":"2024-06-06T08:37:00.185710+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048042+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:37:00.048042+0000","last_peered":"2024-06-06T08:37:00.048042+0000","last_clean":"2024-06-06T08:37:00.048042+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:37:00.048042+0000","last_undegraded":"2024-06-06T08:37:00.048042+0000","last_fullsized":"2024-06-06T08:37:00.048042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.913546+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:37:21.913546+0000","last_peered":"2024-06-06T08:37:21.913546+0000","last_clean":"2024-06-06T08:37:21.913546+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:37:21.913546+0000","last_undegraded":"2024-06-06T08:37:21.913546+0000","last_fullsized":"2024-06-06T08:37:21.913546+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049515+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:37:00.049515+0000","last_peered":"2024-06-06T08:37:00.049515+0000","last_clean":"2024-06-06T08:37:00.049515+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:37:00.049515+0000","last_undegraded":"2024-06-06T08:37:00.049515+0000","last_fullsized":"2024-06-06T08:37:00.049515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.911145+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:37:20.911145+0000","last_peered":"2024-06-06T08:37:20.911145+0000","last_clean":"2024-06-06T08:37:20.911145+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:37:20.911145+0000","last_undegraded":"2024-06-06T08:37:20.911145+0000","last_fullsized":"2024-06-06T08:37:20.911145+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048549+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:37:00.048549+0000","last_peered":"2024-06-06T08:37:00.048549+0000","last_clean":"2024-06-06T08:37:00.048549+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:37:00.048549+0000","last_undegraded":"2024-06-06T08:37:00.048549+0000","last_fullsized":"2024-06-06T08:37:00.048549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050088+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:37:00.050088+0000","last_peered":"2024-06-06T08:37:00.050088+0000","last_clean":"2024-06-06T08:37:00.050088+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:37:00.050088+0000","last_undegraded":"2024-06-06T08:37:00.050088+0000","last_fullsized":"2024-06-06T08:37:00.050088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'12","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:44.915734+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:37:44.915734+0000","last_peered":"2024-06-06T08:37:44.915734+0000","last_clean":"2024-06-06T08:37:44.915734+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:37:44.915734+0000","last_undegraded":"2024-06-06T08:37:44.915734+0000","last_fullsized":"2024-06-06T08:37:44.915734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050933+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:37:00.050933+0000","last_peered":"2024-06-06T08:37:00.050933+0000","last_clean":"2024-06-06T08:37:00.050933+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:37:00.050933+0000","last_undegraded":"2024-06-06T08:37:00.050933+0000","last_fullsized":"2024-06-06T08:37:00.050933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.045748+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:37:00.045748+0000","last_peered":"2024-06-06T08:37:00.045748+0000","last_clean":"2024-06-06T08:37:00.045748+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:37:00.045748+0000","last_undegraded":"2024-06-06T08:37:00.045748+0000","last_fullsized":"2024-06-06T08:37:00.045748+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.909814+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:37:20.909814+0000","last_peered":"2024-06-06T08:37:20.909814+0000","last_clean":"2024-06-06T08:37:20.909814+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:37:20.909814+0000","last_undegraded":"2024-06-06T08:37:20.909814+0000","last_fullsized":"2024-06-06T08:37:20.909814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451339+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:37:00.451339+0000","last_peered":"2024-06-06T08:37:00.451339+0000","last_clean":"2024-06-06T08:37:00.451339+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:37:00.451339+0000","last_undegraded":"2024-06-06T08:37:00.451339+0000","last_fullsized":"2024-06-06T08:37:00.451339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.187379+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:37:00.187379+0000","last_peered":"2024-06-06T08:37:00.187379+0000","last_clean":"2024-06-06T08:37:00.187379+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:37:00.187379+0000","last_undegraded":"2024-06-06T08:37:00.187379+0000","last_fullsized":"2024-06-06T08:37:00.187379+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.912032+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:37:20.912032+0000","last_peered":"2024-06-06T08:37:20.912032+0000","last_clean":"2024-06-06T08:37:20.912032+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:37:20.912032+0000","last_undegraded":"2024-06-06T08:37:20.912032+0000","last_fullsized":"2024-06-06T08:37:20.912032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.330864+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:37:00.330864+0000","last_peered":"2024-06-06T08:37:00.330864+0000","last_clean":"2024-06-06T08:37:00.330864+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:37:00.330864+0000","last_undegraded":"2024-06-06T08:37:00.330864+0000","last_fullsized":"2024-06-06T08:37:00.330864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048566+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:37:00.048566+0000","last_peered":"2024-06-06T08:37:00.048566+0000","last_clean":"2024-06-06T08:37:00.048566+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:37:00.048566+0000","last_undegraded":"2024-06-06T08:37:00.048566+0000","last_fullsized":"2024-06-06T08:37:00.048566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.910702+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:37:20.910702+0000","last_peered":"2024-06-06T08:37:20.910702+0000","last_clean":"2024-06-06T08:37:20.910702+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:37:20.910702+0000","last_undegraded":"2024-06-06T08:37:20.910702+0000","last_fullsized":"2024-06-06T08:37:20.910702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049530+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:37:00.049530+0000","last_peered":"2024-06-06T08:37:00.049530+0000","last_clean":"2024-06-06T08:37:00.049530+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:37:00.049530+0000","last_undegraded":"2024-06-06T08:37:00.049530+0000","last_fullsized":"2024-06-06T08:37:00.049530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051431+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:37:00.051431+0000","last_peered":"2024-06-06T08:37:00.051431+0000","last_clean":"2024-06-06T08:37:00.051431+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:37:00.051431+0000","last_undegraded":"2024-06-06T08:37:00.051431+0000","last_fullsized":"2024-06-06T08:37:00.051431+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046229+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:37:00.046229+0000","last_peered":"2024-06-06T08:37:00.046229+0000","last_clean":"2024-06-06T08:37:00.046229+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:37:00.046229+0000","last_undegraded":"2024-06-06T08:37:00.046229+0000","last_fullsized":"2024-06-06T08:37:00.046229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050658+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:37:00.050658+0000","last_peered":"2024-06-06T08:37:00.050658+0000","last_clean":"2024-06-06T08:37:00.050658+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:37:00.050658+0000","last_undegraded":"2024-06-06T08:37:00.050658+0000","last_fullsized":"2024-06-06T08:37:00.050658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.051097+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:37:00.051097+0000","last_peered":"2024-06-06T08:37:00.051097+0000","last_clean":"2024-06-06T08:37:00.051097+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:37:00.051097+0000","last_undegraded":"2024-06-06T08:37:00.051097+0000","last_fullsized":"2024-06-06T08:37:00.051097+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:20.913095+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:37:20.913095+0000","last_peered":"2024-06-06T08:37:20.913095+0000","last_clean":"2024-06-06T08:37:20.913095+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:37:20.913095+0000","last_undegraded":"2024-06-06T08:37:20.913095+0000","last_fullsized":"2024-06-06T08:37:20.913095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681731+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:37:00.681731+0000","last_peered":"2024-06-06T08:37:00.681731+0000","last_clean":"2024-06-06T08:37:00.681731+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:37:00.681731+0000","last_undegraded":"2024-06-06T08:37:00.681731+0000","last_fullsized":"2024-06-06T08:37:00.681731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053427+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:37:00.053427+0000","last_peered":"2024-06-06T08:37:00.053427+0000","last_clean":"2024-06-06T08:37:00.053427+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:37:00.053427+0000","last_undegraded":"2024-06-06T08:37:00.053427+0000","last_fullsized":"2024-06-06T08:37:00.053427+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:21.916046+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:37:21.916046+0000","last_peered":"2024-06-06T08:37:21.916046+0000","last_clean":"2024-06-06T08:37:21.916046+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:37:21.916046+0000","last_undegraded":"2024-06-06T08:37:21.916046+0000","last_fullsized":"2024-06-06T08:37:21.916046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.682272+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:37:00.682272+0000","last_peered":"2024-06-06T08:37:00.682272+0000","last_clean":"2024-06-06T08:37:00.682272+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:37:00.682272+0000","last_undegraded":"2024-06-06T08:37:00.682272+0000","last_fullsized":"2024-06-06T08:37:00.682272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.450894+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:37:00.450894+0000","last_peered":"2024-06-06T08:37:00.450894+0000","last_clean":"2024-06-06T08:37:00.450894+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:37:00.450894+0000","last_undegraded":"2024-06-06T08:37:00.450894+0000","last_fullsized":"2024-06-06T08:37:00.450894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185873+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:37:00.185873+0000","last_peered":"2024-06-06T08:37:00.185873+0000","last_clean":"2024-06-06T08:37:00.185873+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:37:00.185873+0000","last_undegraded":"2024-06-06T08:37:00.185873+0000","last_fullsized":"2024-06-06T08:37:00.185873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.681596+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:37:00.681596+0000","last_peered":"2024-06-06T08:37:00.681596+0000","last_clean":"2024-06-06T08:37:00.681596+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:37:00.681596+0000","last_undegraded":"2024-06-06T08:37:00.681596+0000","last_fullsized":"2024-06-06T08:37:00.681596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.184981+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:37:00.184981+0000","last_peered":"2024-06-06T08:37:00.184981+0000","last_clean":"2024-06-06T08:37:00.184981+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:37:00.184981+0000","last_undegraded":"2024-06-06T08:37:00.184981+0000","last_fullsized":"2024-06-06T08:37:00.184981+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":52,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.066312+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:37:00.066312+0000","last_peered":"2024-06-06T08:37:00.066312+0000","last_clean":"2024-06-06T08:37:00.066312+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:37:00.066312+0000","last_undegraded":"2024-06-06T08:37:00.066312+0000","last_fullsized":"2024-06-06T08:37:00.066312+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.682066+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:37:00.682066+0000","last_peered":"2024-06-06T08:37:00.682066+0000","last_clean":"2024-06-06T08:37:00.682066+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:37:00.682066+0000","last_undegraded":"2024-06-06T08:37:00.682066+0000","last_fullsized":"2024-06-06T08:37:00.682066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049989+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:37:00.049989+0000","last_peered":"2024-06-06T08:37:00.049989+0000","last_clean":"2024-06-06T08:37:00.049989+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:37:00.049989+0000","last_undegraded":"2024-06-06T08:37:00.049989+0000","last_fullsized":"2024-06-06T08:37:00.049989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046105+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:37:00.046105+0000","last_peered":"2024-06-06T08:37:00.046105+0000","last_clean":"2024-06-06T08:37:00.046105+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:37:00.046105+0000","last_undegraded":"2024-06-06T08:37:00.046105+0000","last_fullsized":"2024-06-06T08:37:00.046105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046072+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:37:00.046072+0000","last_peered":"2024-06-06T08:37:00.046072+0000","last_clean":"2024-06-06T08:37:00.046072+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:37:00.046072+0000","last_undegraded":"2024-06-06T08:37:00.046072+0000","last_fullsized":"2024-06-06T08:37:00.046072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.451236+0000","last_change":"2024-06-06T08:36:20.903112+0000","last_active":"2024-06-06T08:37:00.451236+0000","last_peered":"2024-06-06T08:37:00.451236+0000","last_clean":"2024-06-06T08:37:00.451236+0000","last_became_active":"2024-06-06T08:36:20.902657+0000","last_became_peered":"2024-06-06T08:36:20.902657+0000","last_unstale":"2024-06-06T08:37:00.451236+0000","last_undegraded":"2024-06-06T08:37:00.451236+0000","last_fullsized":"2024-06-06T08:37:00.451236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:41:11.218419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.048584+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:37:00.048584+0000","last_peered":"2024-06-06T08:37:00.048584+0000","last_clean":"2024-06-06T08:37:00.048584+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:37:00.048584+0000","last_undegraded":"2024-06-06T08:37:00.048584+0000","last_fullsized":"2024-06-06T08:37:00.048584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049863+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:37:00.049863+0000","last_peered":"2024-06-06T08:37:00.049863+0000","last_clean":"2024-06-06T08:37:00.049863+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:37:00.049863+0000","last_undegraded":"2024-06-06T08:37:00.049863+0000","last_fullsized":"2024-06-06T08:37:00.049863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.046559+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:37:00.046559+0000","last_peered":"2024-06-06T08:37:00.046559+0000","last_clean":"2024-06-06T08:37:00.046559+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:37:00.046559+0000","last_undegraded":"2024-06-06T08:37:00.046559+0000","last_fullsized":"2024-06-06T08:37:00.046559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.332996+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:37:00.332996+0000","last_peered":"2024-06-06T08:37:00.332996+0000","last_clean":"2024-06-06T08:37:00.332996+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:37:00.332996+0000","last_undegraded":"2024-06-06T08:37:00.332996+0000","last_fullsized":"2024-06-06T08:37:00.332996+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.185862+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:37:00.185862+0000","last_peered":"2024-06-06T08:37:00.185862+0000","last_clean":"2024-06-06T08:37:00.185862+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:37:00.185862+0000","last_undegraded":"2024-06-06T08:37:00.185862+0000","last_fullsized":"2024-06-06T08:37:00.185862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050434+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:37:00.050434+0000","last_peered":"2024-06-06T08:37:00.050434+0000","last_clean":"2024-06-06T08:37:00.050434+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:37:00.050434+0000","last_undegraded":"2024-06-06T08:37:00.050434+0000","last_fullsized":"2024-06-06T08:37:00.050434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.053800+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:37:00.053800+0000","last_peered":"2024-06-06T08:37:00.053800+0000","last_clean":"2024-06-06T08:37:00.053800+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:37:00.053800+0000","last_undegraded":"2024-06-06T08:37:00.053800+0000","last_fullsized":"2024-06-06T08:37:00.053800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.049160+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:37:00.049160+0000","last_peered":"2024-06-06T08:37:00.049160+0000","last_clean":"2024-06-06T08:37:00.049160+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:37:00.049160+0000","last_undegraded":"2024-06-06T08:37:00.049160+0000","last_fullsized":"2024-06-06T08:37:00.049160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-06T08:37:00.050715+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:37:00.050715+0000","last_peered":"2024-06-06T08:37:00.050715+0000","last_clean":"2024-06-06T08:37:00.050715+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:37:00.050715+0000","last_undegraded":"2024-06-06T08:37:00.050715+0000","last_fullsized":"2024-06-06T08:37:00.050715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280848,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66070,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":203,"num_read_kb":173,"num_write":169,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":550942,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645352,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1212416,"data_stored":1200648,"data_compressed":62162,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808877,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":555252,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":551054,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1146984,"data_compressed":62162,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":546332,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":608312,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":547720,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":547592,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":546526,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":613442,"data_compressed":26441,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1142182,"data_compressed":62162,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:37:55.965 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T08:37:55.966 INFO:tasks.cephadm:Setting up client nodes... 2024-06-06T08:37:55.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-06T08:37:56.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:55 smithi027 ceph-mon[32335]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-06-06T08:37:56.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:55 smithi027 ceph-mon[32335]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:37:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:56.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:55 smithi176 ceph-mon[36786]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-06-06T08:37:56.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:55 smithi176 ceph-mon[36786]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:37:56.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:55 smithi203 ceph-mon[36579]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-06-06T08:37:56.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:55 smithi203 ceph-mon[36579]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:37:57.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 -- 172.21.15.27:0/4192228365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c100bc0 msgr2=0x7f2a0c100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:58.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 --2- 172.21.15.27:0/4192228365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c100bc0 0x7f2a0c100fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c240 tx=0x7f2a0800c550 comp rx=0 tx=0).stop 2024-06-06T08:37:58.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 -- 172.21.15.27:0/4192228365 shutdown_connections 2024-06-06T08:37:58.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 --2- 172.21.15.27:0/4192228365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a0c102760 0x7f2a0c109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 --2- 172.21.15.27:0/4192228365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c100bc0 0x7f2a0c100fe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 --2- 172.21.15.27:0/4192228365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a0c101dc0 0x7f2a0c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 -- 172.21.15.27:0/4192228365 >> 172.21.15.27:0/4192228365 conn(0x7f2a0c0fc160 msgr2=0x7f2a0c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:58.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 -- 172.21.15.27:0/4192228365 shutdown_connections 2024-06-06T08:37:58.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 -- 172.21.15.27:0/4192228365 wait complete. 2024-06-06T08:37:58.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 Processor -- start 2024-06-06T08:37:58.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.948+0000 7f2a132ae700 1 -- start start 2024-06-06T08:37:58.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a132ae700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a0c100bc0 0x7f2a0c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:58.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a132ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a0c101dc0 0x7f2a0c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:58.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a132ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c102760 0x7f2a0c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:58.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a132ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c10bf80 con 0x7f2a0c102760 2024-06-06T08:37:58.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a132ae700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c10be00 con 0x7f2a0c100bc0 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a132ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c10c100 con 0x7f2a0c101dc0 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1184b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c102760 0x7f2a0c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1184b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c102760 0x7f2a0c19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58746/0 (socket says 172.21.15.27:58746) 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1184b700 1 -- 172.21.15.27:0/2971512833 learned_addr learned my addr 172.21.15.27:0/2971512833 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1104a700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a0c100bc0 0x7f2a0c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a10849700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a0c101dc0 0x7f2a0c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1184b700 1 -- 172.21.15.27:0/2971512833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a0c101dc0 msgr2=0x7f2a0c198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1184b700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a0c101dc0 0x7f2a0c198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1184b700 1 -- 172.21.15.27:0/2971512833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a0c100bc0 msgr2=0x7f2a0c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1184b700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a0c100bc0 0x7f2a0c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1184b700 1 -- 172.21.15.27:0/2971512833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c1a02b0 con 0x7f2a0c102760 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.949+0000 7f2a1184b700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c102760 0x7f2a0c19fc10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400b580 tx=0x7f2a0400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.950+0000 7f2a027fc700 1 -- 172.21.15.27:0/2971512833 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04015020 con 0x7f2a0c102760 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.950+0000 7f2a027fc700 1 -- 172.21.15.27:0/2971512833 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a04011d10 con 0x7f2a0c102760 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.950+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c1a05a0 con 0x7f2a0c102760 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.950+0000 7f2a027fc700 1 -- 172.21.15.27:0/2971512833 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04010e00 con 0x7f2a0c102760 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.950+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c1a0bb0 con 0x7f2a0c102760 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.951+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c050a00 con 0x7f2a0c102760 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.952+0000 7f2a027fc700 1 -- 172.21.15.27:0/2971512833 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2a04018020 con 0x7f2a0c102760 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.952+0000 7f2a027fc700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29f80a2de0 0x7f29f80a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.953+0000 7f2a1104a700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29f80a2de0 0x7f29f80a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.953+0000 7f2a027fc700 1 -- 172.21.15.27:0/2971512833 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9071+0+0 (secure 0 0 0) 0x7f2a040cea50 con 0x7f2a0c102760 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.953+0000 7f2a1104a700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29f80a2de0 0x7f29f80a52a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c210 tx=0x7f2a080060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:37:58.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:57.957+0000 7f2a027fc700 1 -- 172.21.15.27:0/2971512833 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a0401f020 con 0x7f2a0c102760 2024-06-06T08:37:58.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:57 smithi027 ceph-mon[32335]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-06-06T08:37:58.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.249+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2a0c1a0fb0 con 0x7f2a0c102760 2024-06-06T08:37:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.251+0000 7f2a027fc700 1 -- 172.21.15.27:0/2971512833 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f2a0409b620 con 0x7f2a0c102760 2024-06-06T08:37:58.250 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-06-06T08:37:58.250 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBmdWFmTMLuDhAApcXPmnE1ZhKQCcz1m15pCg== 2024-06-06T08:37:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29f80a2de0 msgr2=0x7f29f80a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29f80a2de0 0x7f29f80a52a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c210 tx=0x7f2a080060c0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c102760 msgr2=0x7f2a0c19fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:37:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c102760 0x7f2a0c19fc10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400b580 tx=0x7f2a0400b890 comp rx=0 tx=0).stop 2024-06-06T08:37:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 shutdown_connections 2024-06-06T08:37:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f29f80a2de0 0x7f29f80a52a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c102760 0x7f2a0c19fc10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a0c100bc0 0x7f2a0c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 --2- 172.21.15.27:0/2971512833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a0c101dc0 0x7f2a0c198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:37:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.253+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 >> 172.21.15.27:0/2971512833 conn(0x7f2a0c0fc160 msgr2=0x7f2a0c107200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:37:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.254+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 shutdown_connections 2024-06-06T08:37:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:37:58.254+0000 7f2a132ae700 1 -- 172.21.15.27:0/2971512833 wait complete. 2024-06-06T08:37:58.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:57 smithi176 ceph-mon[36786]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-06-06T08:37:58.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:57 smithi203 ceph-mon[36579]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-06-06T08:37:59.021 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:37:59.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-06T08:37:59.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-06T08:37:59.063 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-06T08:37:59.063 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-06T08:37:59.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-06T08:37:59.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:58 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2971512833' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T08:37:59.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:37:58 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2971512833' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T08:37:59.355 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:37:59.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:58 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2971512833' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T08:37:59.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:37:58 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2971512833' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T08:37:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:58 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2971512833' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T08:37:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:37:58 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2971512833' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T08:38:00.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:00 smithi176 ceph-mon[36786]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T08:38:00.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-06-06T08:38:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-06-06T08:38:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-06-06T08:38:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-06-06T08:38:00.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-06-06T08:38:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:00 smithi027 ceph-mon[32335]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T08:38:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-06-06T08:38:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-06-06T08:38:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-06-06T08:38:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-06-06T08:38:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-06-06T08:38:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:00 smithi203 ceph-mon[36579]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-06T08:38:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-06-06T08:38:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-06-06T08:38:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-06-06T08:38:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-06-06T08:38:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-06-06T08:38:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 -- 172.21.15.27:0/2860817077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 msgr2=0x7f131c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 --2- 172.21.15.27:0/2860817077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 0x7f131c1088d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1318009990 tx=0x7f1318009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 -- 172.21.15.27:0/2860817077 shutdown_connections 2024-06-06T08:38:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 --2- 172.21.15.27:0/2860817077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f131c102f10 0x7f131c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 --2- 172.21.15.27:0/2860817077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 0x7f131c1088d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 --2- 172.21.15.27:0/2860817077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c102550 0x7f131c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 -- 172.21.15.27:0/2860817077 >> 172.21.15.27:0/2860817077 conn(0x7f131c079650 msgr2=0x7f131c079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 -- 172.21.15.27:0/2860817077 shutdown_connections 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 -- 172.21.15.27:0/2860817077 wait complete. 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 Processor -- start 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.135+0000 7f132340d700 1 -- start start 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f132340d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f131c102550 0x7f131c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f132340d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c102f10 0x7f131c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f132340d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 0x7f131c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f132340d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f131c112770 con 0x7f131c1084f0 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f132340d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f131c1125f0 con 0x7f131c102550 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f132340d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f131c1128f0 con 0x7f131c102f10 2024-06-06T08:38:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13219aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 0x7f131c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13219aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 0x7f131c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48310/0 (socket says 172.21.15.27:48310) 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13219aa700 1 -- 172.21.15.27:0/1972250975 learned_addr learned my addr 172.21.15.27:0/1972250975 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13211a9700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f131c102550 0x7f131c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13209a8700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c102f10 0x7f131c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13219aa700 1 -- 172.21.15.27:0/1972250975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c102f10 msgr2=0x7f131c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13219aa700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c102f10 0x7f131c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13219aa700 1 -- 172.21.15.27:0/1972250975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f131c102550 msgr2=0x7f131c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13219aa700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f131c102550 0x7f131c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13219aa700 1 -- 172.21.15.27:0/1972250975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c1a7100 con 0x7f131c1084f0 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13211a9700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f131c102550 0x7f131c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13219aa700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 0x7f131c075210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1314007ee0 tx=0x7f131400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:01.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.136+0000 7f13127fc700 1 -- 172.21.15.27:0/1972250975 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131400dbd0 con 0x7f131c1084f0 2024-06-06T08:38:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.137+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131c1a73f0 con 0x7f131c1084f0 2024-06-06T08:38:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.137+0000 7f13127fc700 1 -- 172.21.15.27:0/1972250975 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131400dd70 con 0x7f131c1084f0 2024-06-06T08:38:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.137+0000 7f13127fc700 1 -- 172.21.15.27:0/1972250975 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13140133f0 con 0x7f131c1084f0 2024-06-06T08:38:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.137+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131c1a7930 con 0x7f131c1084f0 2024-06-06T08:38:01.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.137+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131c103ba0 con 0x7f131c1084f0 2024-06-06T08:38:01.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.139+0000 7f13127fc700 1 -- 172.21.15.27:0/1972250975 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f131401a020 con 0x7f131c1084f0 2024-06-06T08:38:01.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.139+0000 7f13127fc700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f13080a2cb0 0x7f13080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:01.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.139+0000 7f13211a9700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f13080a2cb0 0x7f13080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:01.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.140+0000 7f13127fc700 1 -- 172.21.15.27:0/1972250975 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f13140cf380 con 0x7f131c1084f0 2024-06-06T08:38:01.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.140+0000 7f13211a9700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f13080a2cb0 0x7f13080a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f131800b5c0 tx=0x7f1318005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.143+0000 7f13127fc700 1 -- 172.21.15.27:0/1972250975 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1314010070 con 0x7f131c1084f0 2024-06-06T08:38:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:01 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-06-06T08:38:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:01 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-06-06T08:38:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:01 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-06-06T08:38:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:01 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-06-06T08:38:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:01 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-06-06T08:38:01.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:01 smithi027 ceph-mon[32335]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T08:38:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.411+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f131c02de60 con 0x7f131c1084f0 2024-06-06T08:38:01.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.415+0000 7f13127fc700 1 -- 172.21.15.27:0/1972250975 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263199 (secure 0 0 0) 0x7f131409bdb0 con 0x7f131c1084f0 2024-06-06T08:38:01.418 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:38:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.419+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f13080a2cb0 msgr2=0x7f13080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f13080a2cb0 0x7f13080a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f131800b5c0 tx=0x7f1318005a90 comp rx=0 tx=0).stop 2024-06-06T08:38:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 msgr2=0x7f131c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 0x7f131c075210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1314007ee0 tx=0x7f131400d3b0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 shutdown_connections 2024-06-06T08:38:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f13080a2cb0 0x7f13080a5170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131c1084f0 0x7f131c075210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f131c102550 0x7f131c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 --2- 172.21.15.27:0/1972250975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c102f10 0x7f131c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 >> 172.21.15.27:0/1972250975 conn(0x7f131c079650 msgr2=0x7f131c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 shutdown_connections 2024-06-06T08:38:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:01.420+0000 7f132340d700 1 -- 172.21.15.27:0/1972250975 wait complete. 2024-06-06T08:38:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:01 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-06-06T08:38:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:01 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-06-06T08:38:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:01 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-06-06T08:38:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:01 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-06-06T08:38:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:01 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-06-06T08:38:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:01 smithi203 ceph-mon[36579]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T08:38:01.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:01 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-06-06T08:38:01.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:01 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-06-06T08:38:01.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:01 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-06-06T08:38:01.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:01 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-06-06T08:38:01.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:01 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-06-06T08:38:01.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:01 smithi176 ceph-mon[36786]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T08:38:02.264 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3763593724},{"type":"v1","addr":"172.21.15.27:6801","nonce":3763593724}]},"active_addr":"172.21.15.27:6801/3763593724","active_change":"2024-06-06T08:28:58.346405+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.27:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":876705499}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":4174349888}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3053481437}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":550980984}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":2866465359}]}]} 2024-06-06T08:38:02.269 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-06T08:38:02.269 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-06T08:38:02.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:38:02.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:02 smithi027 ceph-mon[32335]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 818 B/s wr, 6 op/s 2024-06-06T08:38:02.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:02 smithi027 ceph-mon[32335]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T08:38:02.294 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:02 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1972250975' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T08:38:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:02 smithi203 ceph-mon[36579]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 818 B/s wr, 6 op/s 2024-06-06T08:38:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:02 smithi203 ceph-mon[36579]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T08:38:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:02 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1972250975' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T08:38:02.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:02 smithi176 ceph-mon[36786]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 818 B/s wr, 6 op/s 2024-06-06T08:38:02.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:02 smithi176 ceph-mon[36786]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T08:38:02.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:02 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1972250975' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T08:38:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:04.306 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:04 smithi027 ceph-mon[32335]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-06-06T08:38:04.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.314+0000 7fe0151ce700 1 -- 172.21.15.27:0/2958090827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 msgr2=0x7fe010105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.314+0000 7fe0151ce700 1 --2- 172.21.15.27:0/2958090827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 0x7fe010105800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe004009990 tx=0x7fe004009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.314+0000 7fe0151ce700 1 -- 172.21.15.27:0/2958090827 shutdown_connections 2024-06-06T08:38:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.314+0000 7fe0151ce700 1 --2- 172.21.15.27:0/2958090827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe010105f30 0x7fe01010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.314+0000 7fe0151ce700 1 --2- 172.21.15.27:0/2958090827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe010101090 0x7fe010101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.314+0000 7fe0151ce700 1 --2- 172.21.15.27:0/2958090827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 0x7fe010105800 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.314+0000 7fe0151ce700 1 -- 172.21.15.27:0/2958090827 >> 172.21.15.27:0/2958090827 conn(0x7fe0100766b0 msgr2=0x7fe010076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.314+0000 7fe0151ce700 1 -- 172.21.15.27:0/2958090827 shutdown_connections 2024-06-06T08:38:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.314+0000 7fe0151ce700 1 -- 172.21.15.27:0/2958090827 wait complete. 2024-06-06T08:38:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe0151ce700 1 Processor -- start 2024-06-06T08:38:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe0151ce700 1 -- start start 2024-06-06T08:38:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe0151ce700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe010101090 0x7fe01019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe0151ce700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 0x7fe01019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe0151ce700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe010105f30 0x7fe01019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe0151ce700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe0101105e0 con 0x7fe010101a40 2024-06-06T08:38:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe0151ce700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe010110460 con 0x7fe010105f30 2024-06-06T08:38:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe0151ce700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe010110760 con 0x7fe010101090 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe00e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 0x7fe01019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe00e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 0x7fe01019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48334/0 (socket says 172.21.15.27:48334) 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.315+0000 7fe00e59c700 1 -- 172.21.15.27:0/4093740885 learned_addr learned my addr 172.21.15.27:0/4093740885 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe00ed9d700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe010101090 0x7fe01019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe00f59e700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe010105f30 0x7fe01019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe00e59c700 1 -- 172.21.15.27:0/4093740885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe010101090 msgr2=0x7fe01019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe00e59c700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe010101090 0x7fe01019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe00e59c700 1 -- 172.21.15.27:0/4093740885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe010105f30 msgr2=0x7fe01019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe00e59c700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe010105f30 0x7fe01019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe00e59c700 1 -- 172.21.15.27:0/4093740885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01019fe60 con 0x7fe010101a40 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe00f59e700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe010105f30 0x7fe01019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:38:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe00e59c700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 0x7fe01019b390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe004006010 tx=0x7fe004009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:04.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fdff7fff700 1 -- 172.21.15.27:0/4093740885 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004027070 con 0x7fe010101a40 2024-06-06T08:38:04.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0101a00f0 con 0x7fe010101a40 2024-06-06T08:38:04.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fdff7fff700 1 -- 172.21.15.27:0/4093740885 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe004004050 con 0x7fe010101a40 2024-06-06T08:38:04.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fdff7fff700 1 -- 172.21.15.27:0/4093740885 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004022cc0 con 0x7fe010101a40 2024-06-06T08:38:04.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.316+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0101a79d0 con 0x7fe010101a40 2024-06-06T08:38:04.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.317+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe010069180 con 0x7fe010101a40 2024-06-06T08:38:04.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.318+0000 7fdff7fff700 1 -- 172.21.15.27:0/4093740885 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe00402a070 con 0x7fe010101a40 2024-06-06T08:38:04.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.319+0000 7fdff7fff700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdff80a2cb0 0x7fdff80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:04.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.319+0000 7fe00ed9d700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdff80a2cb0 0x7fdff80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:04.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.319+0000 7fdff7fff700 1 -- 172.21.15.27:0/4093740885 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fe0040d04e0 con 0x7fe010101a40 2024-06-06T08:38:04.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.320+0000 7fe00ed9d700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdff80a2cb0 0x7fdff80a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe00000bc70 tx=0x7fe00000b650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:04.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.323+0000 7fdff7fff700 1 -- 172.21.15.27:0/4093740885 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe00405e850 con 0x7fe010101a40 2024-06-06T08:38:04.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:04 smithi203 ceph-mon[36579]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-06-06T08:38:04.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:04 smithi176 ceph-mon[36786]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-06-06T08:38:04.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.533+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0101a7ba0 con 0x7fe010101a40 2024-06-06T08:38:04.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.534+0000 7fdff7fff700 1 -- 172.21.15.27:0/4093740885 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24836 (secure 0 0 0) 0x7fe004013070 con 0x7fe010101a40 2024-06-06T08:38:04.533 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:38:04.533 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:38:01.008015+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:38:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdff80a2cb0 msgr2=0x7fdff80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdff80a2cb0 0x7fdff80a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe00000bc70 tx=0x7fe00000b650 comp rx=0 tx=0).stop 2024-06-06T08:38:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 msgr2=0x7fe01019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 0x7fe01019b390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe004006010 tx=0x7fe004009f90 comp rx=0 tx=0).stop 2024-06-06T08:38:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 shutdown_connections 2024-06-06T08:38:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdff80a2cb0 0x7fdff80a5170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe010105f30 0x7fe01019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe010101090 0x7fe01019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 --2- 172.21.15.27:0/4093740885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe010101a40 0x7fe01019b390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 >> 172.21.15.27:0/4093740885 conn(0x7fe0100766b0 msgr2=0x7fe0100fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 shutdown_connections 2024-06-06T08:38:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:04.537+0000 7fe0151ce700 1 -- 172.21.15.27:0/4093740885 wait complete. 2024-06-06T08:38:05.321 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:05 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4093740885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:38:05.350 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-06T08:38:05.350 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:38:05.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:05 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4093740885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:38:05.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:05 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4093740885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:38:05.644 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:06.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:06 smithi027 ceph-mon[32335]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:06.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:06 smithi203 ceph-mon[36579]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:06.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:06 smithi176 ceph-mon[36786]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.252+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/1694885803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0101670 msgr2=0x7fc0c0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.252+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/1694885803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0101670 0x7fc0c0101af0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009990 tx=0x7fc0a8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.252+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/1694885803 shutdown_connections 2024-06-06T08:38:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.252+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/1694885803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0102030 0x7fc0c010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.252+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/1694885803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c01074e0 0x7fc0c01078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.252+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/1694885803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0101670 0x7fc0c0101af0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.252+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/1694885803 >> 172.21.15.27:0/1694885803 conn(0x7fc0c0078280 msgr2=0x7fc0c0078690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.252+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/1694885803 shutdown_connections 2024-06-06T08:38:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.252+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/1694885803 wait complete. 2024-06-06T08:38:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0c4cbc700 1 Processor -- start 2024-06-06T08:38:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0c4cbc700 1 -- start start 2024-06-06T08:38:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0c4cbc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0101670 0x7fc0c019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0c4cbc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102030 0x7fc0c019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0c4cbc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c01074e0 0x7fc0c0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0c4cbc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0111700 con 0x7fc0c0102030 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0c4cbc700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0111580 con 0x7fc0c0101670 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0c4cbc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0111880 con 0x7fc0c01074e0 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0b5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102030 0x7fc0c019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0b5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102030 0x7fc0c019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48344/0 (socket says 172.21.15.27:48344) 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0b5d9b700 1 -- 172.21.15.27:0/848501631 learned_addr learned my addr 172.21.15.27:0/848501631 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.253+0000 7fc0be59c700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0101670 0x7fc0c019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0bed9d700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c01074e0 0x7fc0c0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0b5d9b700 1 -- 172.21.15.27:0/848501631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c01074e0 msgr2=0x7fc0c0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0b5d9b700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c01074e0 0x7fc0c0198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0b5d9b700 1 -- 172.21.15.27:0/848501631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0101670 msgr2=0x7fc0c019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0b5d9b700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0101670 0x7fc0c019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0b5d9b700 1 -- 172.21.15.27:0/848501631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c01991f0 con 0x7fc0c0102030 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0be59c700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0101670 0x7fc0c019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0b5d9b700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102030 0x7fc0c019edc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8000c00 tx=0x7fc0a800bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0b7fff700 1 -- 172.21.15.27:0/848501631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8027070 con 0x7fc0c0102030 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c0199480 con 0x7fc0c0102030 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0b7fff700 1 -- 172.21.15.27:0/848501631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a8022470 con 0x7fc0c0102030 2024-06-06T08:38:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0b7fff700 1 -- 172.21.15.27:0/848501631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a800f6a0 con 0x7fc0c0102030 2024-06-06T08:38:07.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.254+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c01a5750 con 0x7fc0c0102030 2024-06-06T08:38:07.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.255+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c0102b00 con 0x7fc0c0102030 2024-06-06T08:38:07.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.256+0000 7fc0b7fff700 1 -- 172.21.15.27:0/848501631 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc0a802a070 con 0x7fc0c0102030 2024-06-06T08:38:07.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.257+0000 7fc0b7fff700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2cb0 0x7fc0ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:07.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.257+0000 7fc0be59c700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2cb0 0x7fc0ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:07.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.257+0000 7fc0b7fff700 1 -- 172.21.15.27:0/848501631 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fc0a80d03e0 con 0x7fc0c0102030 2024-06-06T08:38:07.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.258+0000 7fc0be59c700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2cb0 0x7fc0ac0a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00ffe30 tx=0x7fc0b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:07.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.262+0000 7fc0b7fff700 1 -- 172.21.15.27:0/848501631 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0a8013070 con 0x7fc0c0102030 2024-06-06T08:38:07.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.469+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0c01a5920 con 0x7fc0c0102030 2024-06-06T08:38:07.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.471+0000 7fc0b7fff700 1 -- 172.21.15.27:0/848501631 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24836 (secure 0 0 0) 0x7fc0a809cea0 con 0x7fc0c0102030 2024-06-06T08:38:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:38:07.470 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:38:01.008015+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2cb0 msgr2=0x7fc0ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2cb0 0x7fc0ac0a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00ffe30 tx=0x7fc0b000b040 comp rx=0 tx=0).stop 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102030 msgr2=0x7fc0c019edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102030 0x7fc0c019edc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8000c00 tx=0x7fc0a800bb60 comp rx=0 tx=0).stop 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 shutdown_connections 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc0ac0a2cb0 0x7fc0ac0a5170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c01074e0 0x7fc0c0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0c0101670 0x7fc0c019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 --2- 172.21.15.27:0/848501631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102030 0x7fc0c019edc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 >> 172.21.15.27:0/848501631 conn(0x7fc0c0078280 msgr2=0x7fc0c006ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 shutdown_connections 2024-06-06T08:38:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:07.473+0000 7fc0c4cbc700 1 -- 172.21.15.27:0/848501631 wait complete. 2024-06-06T08:38:08.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:08 smithi027 ceph-mon[32335]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:08.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:08 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/848501631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:38:08.139 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-06T08:38:08.139 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-06T08:38:08.140 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-06T08:38:08.140 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-06T08:38:08.141 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-06T08:38:08.141 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-06T08:38:08.141 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-06T08:38:08.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-06T08:38:08.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-06T08:38:08.143 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-06T08:38:08.143 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-06T08:38:08.144 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-06T08:38:08.292 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:08 smithi203 ceph-mon[36579]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:08.292 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:08 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/848501631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:38:08.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:08 smithi176 ceph-mon[36786]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:08.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:08 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/848501631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:38:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:08.779 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:08.800 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:10.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:10 smithi027 ceph-mon[32335]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T08:38:10.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:10 smithi203 ceph-mon[36579]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T08:38:10.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:10 smithi176 ceph-mon[36786]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T08:38:10.871 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:10.871 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:10.872 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:12.277 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:12 smithi027 ceph-mon[32335]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T08:38:12.285 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:12.286 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:12.287 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:12.308 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:12 smithi176 ceph-mon[36786]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T08:38:12.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:12 smithi203 ceph-mon[36579]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T08:38:14.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:14 smithi027 ceph-mon[32335]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-06-06T08:38:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:14 smithi203 ceph-mon[36579]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-06-06T08:38:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:14 smithi176 ceph-mon[36786]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-06-06T08:38:16.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:16 smithi027 ceph-mon[32335]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:16 smithi203 ceph-mon[36579]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:16 smithi176 ceph-mon[36786]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:18.072 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:18.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.083+0000 7fef0f3af700 1 -- 172.21.15.27:0/3261455366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef08102ef0 msgr2=0x7fef0810f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:18.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.083+0000 7fef0f3af700 1 --2- 172.21.15.27:0/3261455366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef08102ef0 0x7fef0810f3a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef0000b0d0 tx=0x7fef0000b3e0 comp rx=0 tx=0).stop 2024-06-06T08:38:18.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.083+0000 7fef0f3af700 1 -- 172.21.15.27:0/3261455366 shutdown_connections 2024-06-06T08:38:18.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.083+0000 7fef0f3af700 1 --2- 172.21.15.27:0/3261455366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef08102ef0 0x7fef0810f3a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:18.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.083+0000 7fef0f3af700 1 --2- 172.21.15.27:0/3261455366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef081085d0 0x7fef081089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:18.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.083+0000 7fef0f3af700 1 --2- 172.21.15.27:0/3261455366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef08102530 0x7fef081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:18.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.083+0000 7fef0f3af700 1 -- 172.21.15.27:0/3261455366 >> 172.21.15.27:0/3261455366 conn(0x7fef08079640 msgr2=0x7fef08079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:18.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.083+0000 7fef0f3af700 1 -- 172.21.15.27:0/3261455366 shutdown_connections 2024-06-06T08:38:18.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.083+0000 7fef0f3af700 1 -- 172.21.15.27:0/3261455366 wait complete. 2024-06-06T08:38:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0f3af700 1 Processor -- start 2024-06-06T08:38:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0f3af700 1 -- start start 2024-06-06T08:38:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0f3af700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef08102530 0x7fef08074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0f3af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef08102ef0 0x7fef08074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0f3af700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef081085d0 0x7fef08075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0f3af700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef08112810 con 0x7fef08102ef0 2024-06-06T08:38:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0f3af700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fef08112690 con 0x7fef08102530 2024-06-06T08:38:18.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0f3af700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef08112990 con 0x7fef081085d0 2024-06-06T08:38:18.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0c94a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef08102ef0 0x7fef08074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:18.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.084+0000 7fef0d14b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef08102530 0x7fef08074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:18.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.085+0000 7fef0d14b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef08102530 0x7fef08074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:35104/0 (socket says 172.21.15.27:35104) 2024-06-06T08:38:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.085+0000 7fef0d14b700 1 -- 172.21.15.27:0/634381052 learned_addr learned my addr 172.21.15.27:0/634381052 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.085+0000 7fef0d14b700 1 -- 172.21.15.27:0/634381052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef081085d0 msgr2=0x7fef08075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.085+0000 7fef0d14b700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef081085d0 0x7fef08075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.085+0000 7fef0d14b700 1 -- 172.21.15.27:0/634381052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef08102ef0 msgr2=0x7fef08074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.085+0000 7fef0d14b700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef08102ef0 0x7fef08074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.085+0000 7fef0d14b700 1 -- 172.21.15.27:0/634381052 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef081a70a0 con 0x7fef08102530 2024-06-06T08:38:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.085+0000 7fef0c94a700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef08102ef0 0x7fef08074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:38:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.085+0000 7fef0d14b700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef08102530 0x7fef08074790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef0400e9a0 tx=0x7fef0400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:18.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.086+0000 7feefe7fc700 1 -- 172.21.15.27:0/634381052 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0400cbd0 con 0x7fef08102530 2024-06-06T08:38:18.087 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:18 smithi027 ceph-mon[32335]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:18.087 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:18 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:38:18.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.086+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef081a7390 con 0x7fef08102530 2024-06-06T08:38:18.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.086+0000 7feefe7fc700 1 -- 172.21.15.27:0/634381052 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0400cd70 con 0x7fef08102530 2024-06-06T08:38:18.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.086+0000 7feefe7fc700 1 -- 172.21.15.27:0/634381052 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04018750 con 0x7fef08102530 2024-06-06T08:38:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.086+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef081a7920 con 0x7fef08102530 2024-06-06T08:38:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.087+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef08062ba0 con 0x7fef08102530 2024-06-06T08:38:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.089+0000 7feefe7fc700 1 -- 172.21.15.27:0/634381052 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fef04010430 con 0x7fef08102530 2024-06-06T08:38:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.090+0000 7feefe7fc700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7feef40a2d30 0x7feef40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.090+0000 7fef0c94a700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7feef40a2d30 0x7feef40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.091+0000 7feefe7fc700 1 -- 172.21.15.27:0/634381052 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fef040cf760 con 0x7fef08102530 2024-06-06T08:38:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.091+0000 7fef0c94a700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7feef40a2d30 0x7feef40a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef08079bd0 tx=0x7feef8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.091+0000 7feefe7fc700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] conn(0x7feef40ae090 0x7feef40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.091+0000 7feefe7fc700 1 -- 172.21.15.27:0/634381052 --> [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feef40b0c00 con 0x7feef40ae090 2024-06-06T08:38:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.091+0000 7feefe7fc700 1 -- 172.21.15.27:0/634381052 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fef040d6050 con 0x7fef08102530 2024-06-06T08:38:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.092+0000 7fef0d94c700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] conn(0x7feef40ae090 0x7feef40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:18.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.092+0000 7fef0d94c700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] conn(0x7feef40ae090 0x7feef40b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:18.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.093+0000 7feefe7fc700 1 -- 172.21.15.27:0/634381052 <== osd.1 v2:172.21.15.176:6800/790447916 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feef40b0c00 con 0x7feef40ae090 2024-06-06T08:38:18.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.113+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 --> [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef08062fa0 con 0x7feef40ae090 2024-06-06T08:38:18.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.113+0000 7feefe7fc700 1 -- 172.21.15.27:0/634381052 <== osd.1 v2:172.21.15.176:6800/790447916 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fef08062fa0 con 0x7feef40ae090 2024-06-06T08:38:18.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.113+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 >> [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] conn(0x7feef40ae090 msgr2=0x7feef40b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:18.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.113+0000 7fef0f3af700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] conn(0x7feef40ae090 0x7feef40b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:18.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.114+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7feef40a2d30 msgr2=0x7feef40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.114+0000 7fef0f3af700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7feef40a2d30 0x7feef40a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef08079bd0 tx=0x7feef8009260 comp rx=0 tx=0).stop 2024-06-06T08:38:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.114+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef08102530 msgr2=0x7fef08074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.114+0000 7fef0f3af700 1 --2- 172.21.15.27:0/634381052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef08102530 0x7fef08074790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef0400e9a0 tx=0x7fef0400ecb0 comp rx=0 tx=0).stop 2024-06-06T08:38:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.114+0000 7fef0d94c700 1 -- 172.21.15.27:0/634381052 reap_dead start 2024-06-06T08:38:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.114+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 shutdown_connections 2024-06-06T08:38:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.114+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 >> 172.21.15.27:0/634381052 conn(0x7fef08079640 msgr2=0x7fef08101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.114+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 shutdown_connections 2024-06-06T08:38:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:18.114+0000 7fef0f3af700 1 -- 172.21.15.27:0/634381052 wait complete. 2024-06-06T08:38:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:18 smithi203 ceph-mon[36579]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:18 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:38:18.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:18 smithi176 ceph-mon[36786]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:18.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:18 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:38:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:55834574939 2024-06-06T08:38:18.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-06T08:38:20.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.133+0000 7f5778baa700 1 -- 172.21.15.27:0/2815824996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5774102130 msgr2=0x7f577410e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.133+0000 7f5778baa700 1 --2- 172.21.15.27:0/2815824996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5774102130 0x7f577410e190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f576c00b0a0 tx=0x7f576c00b3b0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.133+0000 7f5778baa700 1 -- 172.21.15.27:0/2815824996 shutdown_connections 2024-06-06T08:38:20.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.133+0000 7f5778baa700 1 --2- 172.21.15.27:0/2815824996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5774102130 0x7f577410e190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.133+0000 7f5778baa700 1 --2- 172.21.15.27:0/2815824996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5774107550 0x7f5774107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.133+0000 7f5778baa700 1 --2- 172.21.15.27:0/2815824996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5774101770 0x7f5774101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.133+0000 7f5778baa700 1 -- 172.21.15.27:0/2815824996 >> 172.21.15.27:0/2815824996 conn(0x7f5774078340 msgr2=0x7f5774078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:20.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.133+0000 7f5778baa700 1 -- 172.21.15.27:0/2815824996 shutdown_connections 2024-06-06T08:38:20.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.133+0000 7f5778baa700 1 -- 172.21.15.27:0/2815824996 wait complete. 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5778baa700 1 Processor -- start 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5778baa700 1 -- start start 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5778baa700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5774101770 0x7f577419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5778baa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5774102130 0x7f577419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5778baa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5774107550 0x7f5774198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5778baa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f57741116d0 con 0x7f5774102130 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5778baa700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5774111550 con 0x7f5774101770 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5778baa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5774111850 con 0x7f5774107550 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5769d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5774102130 0x7f577419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:20.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5769d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5774102130 0x7f577419ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51076/0 (socket says 172.21.15.27:51076) 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.134+0000 7f5769d9b700 1 -- 172.21.15.27:0/124921051 learned_addr learned my addr 172.21.15.27:0/124921051 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f577259c700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5774101770 0x7f577419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f5772d9d700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5774107550 0x7f5774198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f5769d9b700 1 -- 172.21.15.27:0/124921051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5774107550 msgr2=0x7f5774198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f5769d9b700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5774107550 0x7f5774198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f5769d9b700 1 -- 172.21.15.27:0/124921051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5774101770 msgr2=0x7f577419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f5769d9b700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5774101770 0x7f577419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f5769d9b700 1 -- 172.21.15.27:0/124921051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57741990a0 con 0x7f5774102130 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f5769d9b700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5774102130 0x7f577419ecc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f575c00c9b0 tx=0x7f575c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f576bfff700 1 -- 172.21.15.27:0/124921051 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c007b00 con 0x7f5774102130 2024-06-06T08:38:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f576bfff700 1 -- 172.21.15.27:0/124921051 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f575c007ca0 con 0x7f5774102130 2024-06-06T08:38:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5774199390 con 0x7f5774102130 2024-06-06T08:38:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f576bfff700 1 -- 172.21.15.27:0/124921051 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c018730 con 0x7f5774102130 2024-06-06T08:38:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.135+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57741a5650 con 0x7f5774102130 2024-06-06T08:38:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.136+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5774100a20 con 0x7f5774102130 2024-06-06T08:38:20.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.137+0000 7f576bfff700 1 -- 172.21.15.27:0/124921051 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f575c0188d0 con 0x7f5774102130 2024-06-06T08:38:20.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.138+0000 7f576bfff700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f57600a2cb0 0x7f57600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.138+0000 7f577259c700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f57600a2cb0 0x7f57600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:20.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.138+0000 7f576bfff700 1 -- 172.21.15.27:0/124921051 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f575c0cf970 con 0x7f5774102130 2024-06-06T08:38:20.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.138+0000 7f576bfff700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] conn(0x7f57600ae000 0x7f57600b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.138+0000 7f576bfff700 1 -- 172.21.15.27:0/124921051 --> [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57600b0b70 con 0x7f57600ae000 2024-06-06T08:38:20.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.138+0000 7f576bfff700 1 -- 172.21.15.27:0/124921051 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f575c0cfda0 con 0x7f5774102130 2024-06-06T08:38:20.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.139+0000 7f5772d9d700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] conn(0x7f57600ae000 0x7f57600b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:20.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.139+0000 7f577259c700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f57600a2cb0 0x7f57600a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5774199a80 tx=0x7f5764009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:20.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.139+0000 7f5772d9d700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] conn(0x7f57600ae000 0x7f57600b04a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:20.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:20 smithi027 ceph-mon[32335]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:20.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:20 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:20 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:20 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:20 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.140+0000 7f576bfff700 1 -- 172.21.15.27:0/124921051 <== osd.9 v2:172.21.15.27:6826/657558341 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57600b0b70 con 0x7f57600ae000 2024-06-06T08:38:20.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.156+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 --> [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f577406a410 con 0x7f57600ae000 2024-06-06T08:38:20.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.156+0000 7f576bfff700 1 -- 172.21.15.27:0/124921051 <== osd.9 v2:172.21.15.27:6826/657558341 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f577406a410 con 0x7f57600ae000 2024-06-06T08:38:20.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] conn(0x7f57600ae000 msgr2=0x7f57600b04a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] conn(0x7f57600ae000 0x7f57600b04a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f57600a2cb0 msgr2=0x7f57600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f57600a2cb0 0x7f57600a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5774199a80 tx=0x7f5764009260 comp rx=0 tx=0).stop 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5774102130 msgr2=0x7f577419ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 --2- 172.21.15.27:0/124921051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5774102130 0x7f577419ecc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f575c00c9b0 tx=0x7f575c00ccc0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5772d9d700 1 -- 172.21.15.27:0/124921051 reap_dead start 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 shutdown_connections 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 >> 172.21.15.27:0/124921051 conn(0x7f5774078340 msgr2=0x7f577406ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 shutdown_connections 2024-06-06T08:38:20.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:20.157+0000 7f5778baa700 1 -- 172.21.15.27:0/124921051 wait complete. 2024-06-06T08:38:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.684+0000 7f2d6dc96700 1 -- 172.21.15.27:0/3085740767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d681085d0 msgr2=0x7f2d681089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.684+0000 7f2d6dc96700 1 --2- 172.21.15.27:0/3085740767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d681085d0 0x7f2d681089b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2d580099c0 tx=0x7f2d58009cd0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.684+0000 7f2d6dc96700 1 -- 172.21.15.27:0/3085740767 shutdown_connections 2024-06-06T08:38:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.684+0000 7f2d6dc96700 1 --2- 172.21.15.27:0/3085740767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d68102ef0 0x7f2d6810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.684+0000 7f2d6dc96700 1 --2- 172.21.15.27:0/3085740767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d681085d0 0x7f2d681089b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.684+0000 7f2d6dc96700 1 --2- 172.21.15.27:0/3085740767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d68102530 0x7f2d681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.684+0000 7f2d6dc96700 1 -- 172.21.15.27:0/3085740767 >> 172.21.15.27:0/3085740767 conn(0x7f2d68079640 msgr2=0x7f2d68079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.684+0000 7f2d6dc96700 1 -- 172.21.15.27:0/3085740767 shutdown_connections 2024-06-06T08:38:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.684+0000 7f2d6dc96700 1 -- 172.21.15.27:0/3085740767 wait complete. 2024-06-06T08:38:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d6dc96700 1 Processor -- start 2024-06-06T08:38:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d6dc96700 1 -- start start 2024-06-06T08:38:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d6dc96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d68102530 0x7f2d68198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d6dc96700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d68102ef0 0x7f2d68199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d6dc96700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d681085d0 0x7f2d6819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d6dc96700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68112940 con 0x7f2d681085d0 2024-06-06T08:38:20.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d6dc96700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681127c0 con 0x7f2d68102ef0 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d6dc96700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68112ac0 con 0x7f2d68102530 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d681085d0 0x7f2d6819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d681085d0 0x7f2d6819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55824/0 (socket says 172.21.15.27:55824) 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.685+0000 7f2d67fff700 1 -- 172.21.15.27:0/2416572201 learned_addr learned my addr 172.21.15.27:0/2416572201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d67fff700 1 -- 172.21.15.27:0/2416572201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d68102530 msgr2=0x7f2d68198e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d67fff700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d68102530 0x7f2d68198e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d67fff700 1 -- 172.21.15.27:0/2416572201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d68102ef0 msgr2=0x7f2d68199390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d67fff700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d68102ef0 0x7f2d68199390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d67fff700 1 -- 172.21.15.27:0/2416572201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6819de60 con 0x7f2d681085d0 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d67fff700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d681085d0 0x7f2d6819d720 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000b780 tx=0x7f2d6000bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d64ff9700 1 -- 172.21.15.27:0/2416572201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000f840 con 0x7f2d681085d0 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d64ff9700 1 -- 172.21.15.27:0/2416572201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60017400 con 0x7f2d681085d0 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d64ff9700 1 -- 172.21.15.27:0/2416572201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000d630 con 0x7f2d681085d0 2024-06-06T08:38:20.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6819e150 con 0x7f2d681085d0 2024-06-06T08:38:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.686+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681a59d0 con 0x7f2d681085d0 2024-06-06T08:38:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.687+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d68062ba0 con 0x7f2d681085d0 2024-06-06T08:38:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.688+0000 7f2d64ff9700 1 -- 172.21.15.27:0/2416572201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2d600175b0 con 0x7f2d681085d0 2024-06-06T08:38:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.689+0000 7f2d64ff9700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2d500a2cb0 0x7f2d500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.689+0000 7f2d677fe700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2d500a2cb0 0x7f2d500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.689+0000 7f2d64ff9700 1 -- 172.21.15.27:0/2416572201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f2d600cf800 con 0x7f2d681085d0 2024-06-06T08:38:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.689+0000 7f2d64ff9700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] conn(0x7f2d500ae010 0x7f2d500b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:20.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.689+0000 7f2d64ff9700 1 -- 172.21.15.27:0/2416572201 --> [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d60019e50 con 0x7f2d500ae010 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.689+0000 7f2d64ff9700 1 -- 172.21.15.27:0/2416572201 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f2d600cfc30 con 0x7f2d681085d0 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.689+0000 7f2d677fe700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2d500a2cb0 0x7f2d500a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800b580 tx=0x7f2d580095d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.689+0000 7f2d66ffd700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] conn(0x7f2d500ae010 0x7f2d500b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.690+0000 7f2d66ffd700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] conn(0x7f2d500ae010 0x7f2d500b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.691+0000 7f2d64ff9700 1 -- 172.21.15.27:0/2416572201 <== osd.8 v2:172.21.15.203:6816/2693614915 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d60019e50 con 0x7f2d500ae010 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.707+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 --> [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d68062fa0 con 0x7f2d500ae010 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.707+0000 7f2d64ff9700 1 -- 172.21.15.27:0/2416572201 <== osd.8 v2:172.21.15.203:6816/2693614915 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2d68062fa0 con 0x7f2d500ae010 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 >> [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] conn(0x7f2d500ae010 msgr2=0x7f2d500b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] conn(0x7f2d500ae010 0x7f2d500b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2d500a2cb0 msgr2=0x7f2d500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f2d500a2cb0 0x7f2d500a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800b580 tx=0x7f2d580095d0 comp rx=0 tx=0).stop 2024-06-06T08:38:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d681085d0 msgr2=0x7f2d6819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 --2- 172.21.15.27:0/2416572201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d681085d0 0x7f2d6819d720 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000b780 tx=0x7f2d6000bb40 comp rx=0 tx=0).stop 2024-06-06T08:38:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d67fff700 1 -- 172.21.15.27:0/2416572201 reap_dead start 2024-06-06T08:38:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 shutdown_connections 2024-06-06T08:38:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 >> 172.21.15.27:0/2416572201 conn(0x7f2d68079640 msgr2=0x7f2d68101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 shutdown_connections 2024-06-06T08:38:20.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:19.708+0000 7f2d6dc96700 1 -- 172.21.15.27:0/2416572201 wait complete. 2024-06-06T08:38:20.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:20 smithi203 ceph-mon[36579]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:20.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:20 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:20 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:20 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:20 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:20 smithi176 ceph-mon[36786]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:20.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:20 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:20 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:20 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:20.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:20 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:38:22.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:22 smithi027 ceph-mon[32335]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:22.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:22 smithi203 ceph-mon[36579]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:22.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:22 smithi176 ceph-mon[36786]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.114+0000 7f6f192ed700 1 -- 172.21.15.27:0/2012961434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f141073c0 msgr2=0x7f6f141077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.114+0000 7f6f192ed700 1 --2- 172.21.15.27:0/2012961434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f141073c0 0x7f6f141077a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009990 tx=0x7f6f04009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.114+0000 7f6f192ed700 1 -- 172.21.15.27:0/2012961434 shutdown_connections 2024-06-06T08:38:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.114+0000 7f6f192ed700 1 --2- 172.21.15.27:0/2012961434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6f14101d40 0x7f6f1410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.114+0000 7f6f192ed700 1 --2- 172.21.15.27:0/2012961434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f141073c0 0x7f6f141077a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.114+0000 7f6f192ed700 1 --2- 172.21.15.27:0/2012961434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f14101380 0x7f6f14101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.114+0000 7f6f192ed700 1 -- 172.21.15.27:0/2012961434 >> 172.21.15.27:0/2012961434 conn(0x7f6f14078330 msgr2=0x7f6f14078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.114+0000 7f6f192ed700 1 -- 172.21.15.27:0/2012961434 shutdown_connections 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.114+0000 7f6f192ed700 1 -- 172.21.15.27:0/2012961434 wait complete. 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.115+0000 7f6f192ed700 1 Processor -- start 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.115+0000 7f6f192ed700 1 -- start start 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.115+0000 7f6f192ed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f14101380 0x7f6f1419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.115+0000 7f6f192ed700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f14101d40 0x7f6f1419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.115+0000 7f6f192ed700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6f141073c0 0x7f6f14198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.115+0000 7f6f192ed700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141116b0 con 0x7f6f14101380 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.115+0000 7f6f192ed700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6f14111530 con 0x7f6f141073c0 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.115+0000 7f6f192ed700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6f14111830 con 0x7f6f14101d40 2024-06-06T08:38:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f14101380 0x7f6f1419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f14101380 0x7f6f1419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51116/0 (socket says 172.21.15.27:51116) 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f12ffd700 1 -- 172.21.15.27:0/842963508 learned_addr learned my addr 172.21.15.27:0/842963508 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f12ffd700 1 -- 172.21.15.27:0/842963508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f14101d40 msgr2=0x7f6f1419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f12ffd700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f14101d40 0x7f6f1419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f12ffd700 1 -- 172.21.15.27:0/842963508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6f141073c0 msgr2=0x7f6f14198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f12ffd700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6f141073c0 0x7f6f14198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f12ffd700 1 -- 172.21.15.27:0/842963508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f14199190 con 0x7f6f14101380 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f12ffd700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f14101380 0x7f6f1419e870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005910 tx=0x7f6f0400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6efbfff700 1 -- 172.21.15.27:0/842963508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04027070 con 0x7f6f14101380 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f14199420 con 0x7f6f14101380 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6efbfff700 1 -- 172.21.15.27:0/842963508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f04003d70 con 0x7f6f14101380 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6efbfff700 1 -- 172.21.15.27:0/842963508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400f6c0 con 0x7f6f14101380 2024-06-06T08:38:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.116+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f141a5740 con 0x7f6f14101380 2024-06-06T08:38:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.117+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f14062ba0 con 0x7f6f14101380 2024-06-06T08:38:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.119+0000 7f6efbfff700 1 -- 172.21.15.27:0/842963508 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6f0402a070 con 0x7f6f14101380 2024-06-06T08:38:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.119+0000 7f6efbfff700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6efc0a2cb0 0x7f6efc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.120+0000 7f6efbfff700 1 -- 172.21.15.27:0/842963508 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f6f040d1180 con 0x7f6f14101380 2024-06-06T08:38:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.120+0000 7f6f127fc700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6efc0a2cb0 0x7f6efc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.120+0000 7f6efbfff700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] conn(0x7f6efc0ae010 0x7f6efc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.120+0000 7f6efbfff700 1 -- 172.21.15.27:0/842963508 --> [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f0400b0b0 con 0x7f6efc0ae010 2024-06-06T08:38:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.120+0000 7f6efbfff700 1 -- 172.21.15.27:0/842963508 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f6f040d15b0 con 0x7f6f14101380 2024-06-06T08:38:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.120+0000 7f6f127fc700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6efc0a2cb0 0x7f6efc0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6f08005950 tx=0x7f6f080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.120+0000 7f6f137fe700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] conn(0x7f6efc0ae010 0x7f6efc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.121+0000 7f6f137fe700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] conn(0x7f6efc0ae010 0x7f6efc0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.122+0000 7f6efbfff700 1 -- 172.21.15.27:0/842963508 <== osd.4 v2:172.21.15.176:6808/275393367 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f0400b0b0 con 0x7f6efc0ae010 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.139+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 --> [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f14062fe0 con 0x7f6efc0ae010 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.139+0000 7f6efbfff700 1 -- 172.21.15.27:0/842963508 <== osd.4 v2:172.21.15.176:6808/275393367 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6f14062fe0 con 0x7f6efc0ae010 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 >> [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] conn(0x7f6efc0ae010 msgr2=0x7f6efc0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] conn(0x7f6efc0ae010 0x7f6efc0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6efc0a2cb0 msgr2=0x7f6efc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6efc0a2cb0 0x7f6efc0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6f08005950 tx=0x7f6f080058e0 comp rx=0 tx=0).stop 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f14101380 msgr2=0x7f6f1419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 --2- 172.21.15.27:0/842963508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f14101380 0x7f6f1419e870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005910 tx=0x7f6f0400bb60 comp rx=0 tx=0).stop 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f137fe700 1 -- 172.21.15.27:0/842963508 reap_dead start 2024-06-06T08:38:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 shutdown_connections 2024-06-06T08:38:22.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 >> 172.21.15.27:0/842963508 conn(0x7f6f14078330 msgr2=0x7f6f14100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:22.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 shutdown_connections 2024-06-06T08:38:22.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.140+0000 7f6f192ed700 1 -- 172.21.15.27:0/842963508 wait complete. 2024-06-06T08:38:23.720 INFO:teuthology.orchestra.run.smithi027.stdout:253403070519 2024-06-06T08:38:23.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-06T08:38:23.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 -- 172.21.15.27:0/4277457777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7714101380 msgr2=0x7f7714101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 --2- 172.21.15.27:0/4277457777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7714101380 0x7f7714101800 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7704009990 tx=0x7f7704009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 -- 172.21.15.27:0/4277457777 shutdown_connections 2024-06-06T08:38:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 --2- 172.21.15.27:0/4277457777 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7714101d40 0x7f771410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 --2- 172.21.15.27:0/4277457777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77141073c0 0x7f77141077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 --2- 172.21.15.27:0/4277457777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7714101380 0x7f7714101800 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 -- 172.21.15.27:0/4277457777 >> 172.21.15.27:0/4277457777 conn(0x7f7714078330 msgr2=0x7f7714078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 -- 172.21.15.27:0/4277457777 shutdown_connections 2024-06-06T08:38:23.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 -- 172.21.15.27:0/4277457777 wait complete. 2024-06-06T08:38:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.909+0000 7f771c0ab700 1 Processor -- start 2024-06-06T08:38:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f771c0ab700 1 -- start start 2024-06-06T08:38:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f771c0ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714101380 0x7f7714199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f771c0ab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7714101d40 0x7f7714199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f771c0ab700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77141073c0 0x7f77141a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f771c0ab700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7714111630 con 0x7f7714101d40 2024-06-06T08:38:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f771c0ab700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f77141114b0 con 0x7f77141073c0 2024-06-06T08:38:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f771c0ab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77141117b0 con 0x7f7714101380 2024-06-06T08:38:23.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f7719646700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7714101d40 0x7f7714199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f7719646700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7714101d40 0x7f7714199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51138/0 (socket says 172.21.15.27:51138) 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f7719646700 1 -- 172.21.15.27:0/3647440256 learned_addr learned my addr 172.21.15.27:0/3647440256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f7719e47700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714101380 0x7f7714199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f771a648700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77141073c0 0x7f77141a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f7719646700 1 -- 172.21.15.27:0/3647440256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714101380 msgr2=0x7f7714199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f7719646700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7714101380 0x7f7714199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f7719646700 1 -- 172.21.15.27:0/3647440256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77141073c0 msgr2=0x7f77141a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f7719646700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77141073c0 0x7f77141a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f7719646700 1 -- 172.21.15.27:0/3647440256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77141a5050 con 0x7f7714101d40 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.910+0000 7f771a648700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77141073c0 0x7f77141a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:23.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.911+0000 7f7719646700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7714101d40 0x7f7714199fd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7704000c00 tx=0x7f77040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.911+0000 7f770affd700 1 -- 172.21.15.27:0/3647440256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704027070 con 0x7f7714101d40 2024-06-06T08:38:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.911+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77141a5220 con 0x7f7714101d40 2024-06-06T08:38:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.911+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77141a56d0 con 0x7f7714101d40 2024-06-06T08:38:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.911+0000 7f770affd700 1 -- 172.21.15.27:0/3647440256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704003780 con 0x7f7714101d40 2024-06-06T08:38:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.911+0000 7f770affd700 1 -- 172.21.15.27:0/3647440256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704022dd0 con 0x7f7714101d40 2024-06-06T08:38:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.912+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f76f8001030 con 0x7f7714101d40 2024-06-06T08:38:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.914+0000 7f770affd700 1 -- 172.21.15.27:0/3647440256 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f770402a070 con 0x7f7714101d40 2024-06-06T08:38:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.914+0000 7f770affd700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f77000a2d30 0x7f77000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.915+0000 7f770affd700 1 -- 172.21.15.27:0/3647440256 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f77040d0910 con 0x7f7714101d40 2024-06-06T08:38:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.915+0000 7f7719e47700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f77000a2d30 0x7f77000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.915+0000 7f770affd700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] conn(0x7f77000ae090 0x7f77000b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.915+0000 7f770affd700 1 -- 172.21.15.27:0/3647440256 --> [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7704001d30 con 0x7f77000ae090 2024-06-06T08:38:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.915+0000 7f770affd700 1 -- 172.21.15.27:0/3647440256 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f77040d0d40 con 0x7f7714101d40 2024-06-06T08:38:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.915+0000 7f771a648700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] conn(0x7f77000ae090 0x7f77000b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:23.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.916+0000 7f7719e47700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f77000a2d30 0x7f77000a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7710005950 tx=0x7f771000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:23.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.916+0000 7f771a648700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] conn(0x7f77000ae090 0x7f77000b0530 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:23.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.916+0000 7f770affd700 1 -- 172.21.15.27:0/3647440256 <== osd.0 v2:172.21.15.27:6802/1837835073 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7704001d30 con 0x7f77000ae090 2024-06-06T08:38:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.935+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 --> [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f76f8002e60 con 0x7f77000ae090 2024-06-06T08:38:23.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f770affd700 1 -- 172.21.15.27:0/3647440256 <== osd.0 v2:172.21.15.27:6802/1837835073 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f76f8002e60 con 0x7f77000ae090 2024-06-06T08:38:23.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] conn(0x7f77000ae090 msgr2=0x7f77000b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:23.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] conn(0x7f77000ae090 0x7f77000b0530 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f77000a2d30 msgr2=0x7f77000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f77000a2d30 0x7f77000a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7710005950 tx=0x7f771000b410 comp rx=0 tx=0).stop 2024-06-06T08:38:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7714101d40 msgr2=0x7f7714199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 --2- 172.21.15.27:0/3647440256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7714101d40 0x7f7714199fd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7704000c00 tx=0x7f77040047b0 comp rx=0 tx=0).stop 2024-06-06T08:38:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771a648700 1 -- 172.21.15.27:0/3647440256 reap_dead start 2024-06-06T08:38:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 shutdown_connections 2024-06-06T08:38:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 >> 172.21.15.27:0/3647440256 conn(0x7f7714078330 msgr2=0x7f77141000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 shutdown_connections 2024-06-06T08:38:23.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:23.936+0000 7f771c0ab700 1 -- 172.21.15.27:0/3647440256 wait complete. 2024-06-06T08:38:24.200 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:24 smithi027 ceph-mon[32335]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:38:24.273 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.308+0000 7f379ca8c700 1 -- 172.21.15.27:0/584196723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798101df0 msgr2=0x7f37981021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.308+0000 7f379ca8c700 1 --2- 172.21.15.27:0/584196723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798101df0 0x7f37981021d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f37800099c0 tx=0x7f3780009cd0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.308+0000 7f379ca8c700 1 -- 172.21.15.27:0/584196723 shutdown_connections 2024-06-06T08:38:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.308+0000 7f379ca8c700 1 --2- 172.21.15.27:0/584196723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3798106750 0x7f379810e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.308+0000 7f379ca8c700 1 --2- 172.21.15.27:0/584196723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798101df0 0x7f37981021d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.308+0000 7f379ca8c700 1 --2- 172.21.15.27:0/584196723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37981027a0 0x7f3798106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.308+0000 7f379ca8c700 1 -- 172.21.15.27:0/584196723 >> 172.21.15.27:0/584196723 conn(0x7f3798077a00 msgr2=0x7f37980fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.308+0000 7f379ca8c700 1 -- 172.21.15.27:0/584196723 shutdown_connections 2024-06-06T08:38:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.308+0000 7f379ca8c700 1 -- 172.21.15.27:0/584196723 wait complete. 2024-06-06T08:38:24.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.309+0000 7f379ca8c700 1 Processor -- start 2024-06-06T08:38:24.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.309+0000 7f379ca8c700 1 -- start start 2024-06-06T08:38:24.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.309+0000 7f379ca8c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798101df0 0x7f379819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.309+0000 7f379ca8c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37981027a0 0x7f379819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.309+0000 7f379ca8c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3798106750 0x7f3798198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379ca8c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37981116e0 con 0x7f3798101df0 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379ca8c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3798111560 con 0x7f37981027a0 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379ca8c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3798111860 con 0x7f3798106750 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798101df0 0x7f379819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798101df0 0x7f379819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51156/0 (socket says 172.21.15.27:51156) 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379659c700 1 -- 172.21.15.27:0/1280945946 learned_addr learned my addr 172.21.15.27:0/1280945946 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f3795d9b700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37981027a0 0x7f379819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f3796d9d700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3798106750 0x7f3798198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379659c700 1 -- 172.21.15.27:0/1280945946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3798106750 msgr2=0x7f3798198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379659c700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3798106750 0x7f3798198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379659c700 1 -- 172.21.15.27:0/1280945946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37981027a0 msgr2=0x7f379819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379659c700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f37981027a0 0x7f379819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379659c700 1 -- 172.21.15.27:0/1280945946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3798199160 con 0x7f3798101df0 2024-06-06T08:38:24.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f3796d9d700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3798106750 0x7f3798198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:24.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f379659c700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798101df0 0x7f379819e840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3780006010 tx=0x7f37800038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:24.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.310+0000 7f378f7fe700 1 -- 172.21.15.27:0/1280945946 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780027070 con 0x7f3798101df0 2024-06-06T08:38:24.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.311+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37981993f0 con 0x7f3798101df0 2024-06-06T08:38:24.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.311+0000 7f378f7fe700 1 -- 172.21.15.27:0/1280945946 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3780004050 con 0x7f3798101df0 2024-06-06T08:38:24.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.311+0000 7f378f7fe700 1 -- 172.21.15.27:0/1280945946 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780022e40 con 0x7f3798101df0 2024-06-06T08:38:24.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.311+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37981a5710 con 0x7f3798101df0 2024-06-06T08:38:24.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.311+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3798062be0 con 0x7f3798101df0 2024-06-06T08:38:24.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.313+0000 7f378f7fe700 1 -- 172.21.15.27:0/1280945946 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f378002a070 con 0x7f3798101df0 2024-06-06T08:38:24.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.313+0000 7f378f7fe700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f37840a2cb0 0x7f37840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.314+0000 7f378f7fe700 1 -- 172.21.15.27:0/1280945946 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f37800d1210 con 0x7f3798101df0 2024-06-06T08:38:24.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.314+0000 7f3795d9b700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f37840a2cb0 0x7f37840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.314+0000 7f378f7fe700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] conn(0x7f37840ae010 0x7f37840b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.314+0000 7f378f7fe700 1 -- 172.21.15.27:0/1280945946 --> [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3780010ec0 con 0x7f37840ae010 2024-06-06T08:38:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.314+0000 7f378f7fe700 1 -- 172.21.15.27:0/1280945946 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f37800d1640 con 0x7f3798101df0 2024-06-06T08:38:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.314+0000 7f3795d9b700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f37840a2cb0 0x7f37840a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3798104010 tx=0x7f3788009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.314+0000 7f3796d9d700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] conn(0x7f37840ae010 0x7f37840b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.315+0000 7f3796d9d700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] conn(0x7f37840ae010 0x7f37840b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:24.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.316+0000 7f378f7fe700 1 -- 172.21.15.27:0/1280945946 <== osd.2 v2:172.21.15.203:6800/252294479 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3780010ec0 con 0x7f37840ae010 2024-06-06T08:38:24.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.332+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 --> [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3798062fe0 con 0x7f37840ae010 2024-06-06T08:38:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f378f7fe700 1 -- 172.21.15.27:0/1280945946 <== osd.2 v2:172.21.15.203:6800/252294479 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3798062fe0 con 0x7f37840ae010 2024-06-06T08:38:24.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 >> [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] conn(0x7f37840ae010 msgr2=0x7f37840b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] conn(0x7f37840ae010 0x7f37840b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f37840a2cb0 msgr2=0x7f37840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f37840a2cb0 0x7f37840a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3798104010 tx=0x7f3788009190 comp rx=0 tx=0).stop 2024-06-06T08:38:24.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798101df0 msgr2=0x7f379819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 --2- 172.21.15.27:0/1280945946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798101df0 0x7f379819e840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3780006010 tx=0x7f37800038b0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f3796d9d700 1 -- 172.21.15.27:0/1280945946 reap_dead start 2024-06-06T08:38:24.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 shutdown_connections 2024-06-06T08:38:24.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 >> 172.21.15.27:0/1280945946 conn(0x7f3798077a00 msgr2=0x7f3798100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:24.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 shutdown_connections 2024-06-06T08:38:24.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:24.333+0000 7f379ca8c700 1 -- 172.21.15.27:0/1280945946 wait complete. 2024-06-06T08:38:24.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:24 smithi203 ceph-mon[36579]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:38:24.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:24 smithi176 ceph-mon[36786]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:38:24.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 -- 172.21.15.27:0/138606189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f01073f0 msgr2=0x7fb5f01077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 --2- 172.21.15.27:0/138606189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f01073f0 0x7fb5f01077d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009990 tx=0x7fb5ec009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 -- 172.21.15.27:0/138606189 shutdown_connections 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 --2- 172.21.15.27:0/138606189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f0101d40 0x7fb5f010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 --2- 172.21.15.27:0/138606189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f01073f0 0x7fb5f01077d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 --2- 172.21.15.27:0/138606189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f0101380 0x7fb5f0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 -- 172.21.15.27:0/138606189 >> 172.21.15.27:0/138606189 conn(0x7fb5f0078300 msgr2=0x7fb5f0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 -- 172.21.15.27:0/138606189 shutdown_connections 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 -- 172.21.15.27:0/138606189 wait complete. 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:21.999+0000 7fb5f739c700 1 Processor -- start 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f739c700 1 -- start start 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f739c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f0101380 0x7fb5f0103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f739c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f0101d40 0x7fb5f01035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f739c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f01073f0 0x7fb5f0103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f739c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0111690 con 0x7fb5f0101380 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f739c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0111510 con 0x7fb5f0101d40 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f739c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0111810 con 0x7fb5f01073f0 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5138700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f0101380 0x7fb5f0103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5138700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f0101380 0x7fb5f0103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51092/0 (socket says 172.21.15.27:51092) 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5138700 1 -- 172.21.15.27:0/3732160827 learned_addr learned my addr 172.21.15.27:0/3732160827 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f4937700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f0101d40 0x7fb5f01035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5939700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f01073f0 0x7fb5f0103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5138700 1 -- 172.21.15.27:0/3732160827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f01073f0 msgr2=0x7fb5f0103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5138700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f01073f0 0x7fb5f0103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5138700 1 -- 172.21.15.27:0/3732160827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f0101d40 msgr2=0x7fb5f01035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5138700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f0101d40 0x7fb5f01035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5138700 1 -- 172.21.15.27:0/3732160827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f01a7130 con 0x7fb5f0101380 2024-06-06T08:38:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.000+0000 7fb5f5138700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f0101380 0x7fb5f0103060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00bae0 tx=0x7fb5ec00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:24.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.001+0000 7fb5e67fc700 1 -- 172.21.15.27:0/3732160827 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec026070 con 0x7fb5f0101380 2024-06-06T08:38:24.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.001+0000 7fb5e67fc700 1 -- 172.21.15.27:0/3732160827 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ec003d70 con 0x7fb5f0101380 2024-06-06T08:38:24.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.001+0000 7fb5e67fc700 1 -- 172.21.15.27:0/3732160827 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec021410 con 0x7fb5f0101380 2024-06-06T08:38:24.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.001+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f01a73c0 con 0x7fb5f0101380 2024-06-06T08:38:24.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.001+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f01a78a0 con 0x7fb5f0101380 2024-06-06T08:38:24.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.002+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5f0105430 con 0x7fb5f0101380 2024-06-06T08:38:24.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.003+0000 7fb5e67fc700 1 -- 172.21.15.27:0/3732160827 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb5ec029070 con 0x7fb5f0101380 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.004+0000 7fb5e67fc700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb5dc0a2cb0 0x7fb5dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.004+0000 7fb5f4937700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb5dc0a2cb0 0x7fb5dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.004+0000 7fb5e67fc700 1 -- 172.21.15.27:0/3732160827 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fb5ec0d0b60 con 0x7fb5f0101380 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.004+0000 7fb5e67fc700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] conn(0x7fb5dc0ae010 0x7fb5dc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.004+0000 7fb5e67fc700 1 -- 172.21.15.27:0/3732160827 --> [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5ec023ec0 con 0x7fb5dc0ae010 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.004+0000 7fb5e67fc700 1 -- 172.21.15.27:0/3732160827 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fb5ec0d0f90 con 0x7fb5f0101380 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.004+0000 7fb5f4937700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb5dc0a2cb0 0x7fb5dc0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0078870 tx=0x7fb5e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.004+0000 7fb5f5939700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] conn(0x7fb5dc0ae010 0x7fb5dc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.005+0000 7fb5f5939700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] conn(0x7fb5dc0ae010 0x7fb5dc0b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.006+0000 7fb5e67fc700 1 -- 172.21.15.27:0/3732160827 <== osd.10 v2:172.21.15.176:6824/985198618 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5ec023ec0 con 0x7fb5dc0ae010 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.040+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 --> [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb5f0102aa0 con 0x7fb5dc0ae010 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5e67fc700 1 -- 172.21.15.27:0/3732160827 <== osd.10 v2:172.21.15.176:6824/985198618 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb5f0102aa0 con 0x7fb5dc0ae010 2024-06-06T08:38:24.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 >> [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] conn(0x7fb5dc0ae010 msgr2=0x7fb5dc0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] conn(0x7fb5dc0ae010 0x7fb5dc0b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb5dc0a2cb0 msgr2=0x7fb5dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb5dc0a2cb0 0x7fb5dc0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0078870 tx=0x7fb5e0009260 comp rx=0 tx=0).stop 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f0101380 msgr2=0x7fb5f0103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 --2- 172.21.15.27:0/3732160827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f0101380 0x7fb5f0103060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00bae0 tx=0x7fb5ec00bb10 comp rx=0 tx=0).stop 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f5939700 1 -- 172.21.15.27:0/3732160827 reap_dead start 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 shutdown_connections 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 >> 172.21.15.27:0/3732160827 conn(0x7fb5f0078300 msgr2=0x7fb5f0100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 shutdown_connections 2024-06-06T08:38:24.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:22.041+0000 7fb5f739c700 1 -- 172.21.15.27:0/3732160827 wait complete. 2024-06-06T08:38:25.636 INFO:teuthology.orchestra.run.smithi027.stdout:287762808883 2024-06-06T08:38:25.637 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-06T08:38:26.349 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:26 smithi027 ceph-mon[32335]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:26.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:26 smithi203 ceph-mon[36579]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:26.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:26 smithi176 ceph-mon[36786]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.072+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/1725068436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f34101770 msgr2=0x7f0f34101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.072+0000 7f0f3b2cb700 1 --2- 172.21.15.27:0/1725068436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f34101770 0x7f0f34101bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009990 tx=0x7f0f24009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.072+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/1725068436 shutdown_connections 2024-06-06T08:38:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.072+0000 7f0f3b2cb700 1 --2- 172.21.15.27:0/1725068436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f34102130 0x7f0f3410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.072+0000 7f0f3b2cb700 1 --2- 172.21.15.27:0/1725068436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f34107550 0x7f0f34107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.072+0000 7f0f3b2cb700 1 --2- 172.21.15.27:0/1725068436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f34101770 0x7f0f34101bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.072+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/1725068436 >> 172.21.15.27:0/1725068436 conn(0x7f0f34078340 msgr2=0x7f0f34078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.072+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/1725068436 shutdown_connections 2024-06-06T08:38:27.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.072+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/1725068436 wait complete. 2024-06-06T08:38:27.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f3b2cb700 1 Processor -- start 2024-06-06T08:38:27.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f3b2cb700 1 -- start start 2024-06-06T08:38:27.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f3b2cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f34101770 0x7f0f3419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f3b2cb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f34102130 0x7f0f3419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f3b2cb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f34107550 0x7f0f341989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f3b2cb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34111770 con 0x7f0f34101770 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f3b2cb700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341115f0 con 0x7f0f34102130 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f3b2cb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341118f0 con 0x7f0f34107550 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39067700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f34101770 0x7f0f3419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39067700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f34101770 0x7f0f3419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51186/0 (socket says 172.21.15.27:51186) 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39067700 1 -- 172.21.15.27:0/2272429026 learned_addr learned my addr 172.21.15.27:0/2272429026 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39868700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f34107550 0x7f0f341989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39067700 1 -- 172.21.15.27:0/2272429026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f34107550 msgr2=0x7f0f341989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39067700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f34107550 0x7f0f341989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:27.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39067700 1 -- 172.21.15.27:0/2272429026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f34102130 msgr2=0x7f0f3419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f38866700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f34102130 0x7f0f3419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39067700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f34102130 0x7f0f3419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39067700 1 -- 172.21.15.27:0/2272429026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f34199140 con 0x7f0f34101770 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f39868700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f34107550 0x7f0f341989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.073+0000 7f0f38866700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f34102130 0x7f0f3419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.074+0000 7f0f39067700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f34101770 0x7f0f3419e820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c9b0 tx=0x7f0f3000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.074+0000 7f0f2a7fc700 1 -- 172.21.15.27:0/2272429026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30007b00 con 0x7f0f34101770 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.074+0000 7f0f2a7fc700 1 -- 172.21.15.27:0/2272429026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f30007ca0 con 0x7f0f34101770 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.074+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f34199430 con 0x7f0f34101770 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.074+0000 7f0f2a7fc700 1 -- 172.21.15.27:0/2272429026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30018780 con 0x7f0f34101770 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.074+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f341a56f0 con 0x7f0f34101770 2024-06-06T08:38:27.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.075+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f3406aeb0 con 0x7f0f34101770 2024-06-06T08:38:27.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.076+0000 7f0f2a7fc700 1 -- 172.21.15.27:0/2272429026 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0f3000f450 con 0x7f0f34101770 2024-06-06T08:38:27.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.077+0000 7f0f2a7fc700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f200a2cb0 0x7f0f200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.077+0000 7f0f38866700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f200a2cb0 0x7f0f200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.077+0000 7f0f2a7fc700 1 -- 172.21.15.27:0/2272429026 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f0f300cfa80 con 0x7f0f34101770 2024-06-06T08:38:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.077+0000 7f0f2a7fc700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] conn(0x7f0f200ae010 0x7f0f200b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.077+0000 7f0f2a7fc700 1 -- 172.21.15.27:0/2272429026 --> [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f30015e70 con 0x7f0f200ae010 2024-06-06T08:38:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.077+0000 7f0f2a7fc700 1 -- 172.21.15.27:0/2272429026 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f0f300cfeb0 con 0x7f0f34101770 2024-06-06T08:38:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.077+0000 7f0f38866700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f200a2cb0 0x7f0f200a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400b5c0 tx=0x7f0f24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.077+0000 7f0f39868700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] conn(0x7f0f200ae010 0x7f0f200b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:27.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.078+0000 7f0f39868700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] conn(0x7f0f200ae010 0x7f0f200b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:27.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.079+0000 7f0f2a7fc700 1 -- 172.21.15.27:0/2272429026 <== osd.5 v2:172.21.15.203:6808/3782177772 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f30015e70 con 0x7f0f200ae010 2024-06-06T08:38:27.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.095+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 --> [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f3406a410 con 0x7f0f200ae010 2024-06-06T08:38:27.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f2a7fc700 1 -- 172.21.15.27:0/2272429026 <== osd.5 v2:172.21.15.203:6808/3782177772 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0f3406a410 con 0x7f0f200ae010 2024-06-06T08:38:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 >> [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] conn(0x7f0f200ae010 msgr2=0x7f0f200b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] conn(0x7f0f200ae010 0x7f0f200b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f200a2cb0 msgr2=0x7f0f200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f200a2cb0 0x7f0f200a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400b5c0 tx=0x7f0f24005fb0 comp rx=0 tx=0).stop 2024-06-06T08:38:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f34101770 msgr2=0x7f0f3419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 --2- 172.21.15.27:0/2272429026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f34101770 0x7f0f3419e820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c9b0 tx=0x7f0f3000cd70 comp rx=0 tx=0).stop 2024-06-06T08:38:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f39868700 1 -- 172.21.15.27:0/2272429026 reap_dead start 2024-06-06T08:38:27.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 shutdown_connections 2024-06-06T08:38:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 >> 172.21.15.27:0/2272429026 conn(0x7f0f34078340 msgr2=0x7f0f34105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 shutdown_connections 2024-06-06T08:38:27.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:27.096+0000 7f0f3b2cb700 1 -- 172.21.15.27:0/2272429026 wait complete. 2024-06-06T08:38:27.806 INFO:teuthology.orchestra.run.smithi027.stdout:141733920845 2024-06-06T08:38:27.806 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-06T08:38:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.115+0000 7fa46e69b700 1 -- 172.21.15.27:0/3746809785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4681073f0 msgr2=0x7fa4681077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.115+0000 7fa46e69b700 1 --2- 172.21.15.27:0/3746809785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4681073f0 0x7fa4681077d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa458009990 tx=0x7fa458009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.116+0000 7fa46e69b700 1 -- 172.21.15.27:0/3746809785 shutdown_connections 2024-06-06T08:38:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.116+0000 7fa46e69b700 1 --2- 172.21.15.27:0/3746809785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468101d40 0x7fa46810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.116+0000 7fa46e69b700 1 --2- 172.21.15.27:0/3746809785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4681073f0 0x7fa4681077d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.116+0000 7fa46e69b700 1 --2- 172.21.15.27:0/3746809785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa468101380 0x7fa468101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.116+0000 7fa46e69b700 1 -- 172.21.15.27:0/3746809785 >> 172.21.15.27:0/3746809785 conn(0x7fa468078300 msgr2=0x7fa468078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.116+0000 7fa46e69b700 1 -- 172.21.15.27:0/3746809785 shutdown_connections 2024-06-06T08:38:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.116+0000 7fa46e69b700 1 -- 172.21.15.27:0/3746809785 wait complete. 2024-06-06T08:38:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.116+0000 7fa46e69b700 1 Processor -- start 2024-06-06T08:38:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.116+0000 7fa46e69b700 1 -- start start 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46e69b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa468101380 0x7fa46819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46e69b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468101d40 0x7fa46819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46e69b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4681073f0 0x7fa468198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46e69b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa468111660 con 0x7fa4681073f0 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46e69b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681114e0 con 0x7fa468101380 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46e69b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681117e0 con 0x7fa468101d40 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46cc38700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4681073f0 0x7fa468198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46cc38700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4681073f0 0x7fa468198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51200/0 (socket says 172.21.15.27:51200) 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46cc38700 1 -- 172.21.15.27:0/1522014054 learned_addr learned my addr 172.21.15.27:0/1522014054 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa467fff700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa468101380 0x7fa46819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa4677fe700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468101d40 0x7fa46819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46cc38700 1 -- 172.21.15.27:0/1522014054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468101d40 msgr2=0x7fa46819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46cc38700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468101d40 0x7fa46819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46cc38700 1 -- 172.21.15.27:0/1522014054 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa468101380 msgr2=0x7fa46819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46cc38700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa468101380 0x7fa46819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46cc38700 1 -- 172.21.15.27:0/1522014054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa468199180 con 0x7fa4681073f0 2024-06-06T08:38:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.117+0000 7fa46cc38700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4681073f0 0x7fa468198a20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa460007d70 tx=0x7fa46000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.118+0000 7fa4657fa700 1 -- 172.21.15.27:0/1522014054 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460016020 con 0x7fa4681073f0 2024-06-06T08:38:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.118+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa468199470 con 0x7fa4681073f0 2024-06-06T08:38:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.118+0000 7fa4657fa700 1 -- 172.21.15.27:0/1522014054 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46000ad40 con 0x7fa4681073f0 2024-06-06T08:38:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.118+0000 7fa4657fa700 1 -- 172.21.15.27:0/1522014054 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460011600 con 0x7fa4681073f0 2024-06-06T08:38:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.118+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4681a5730 con 0x7fa4681073f0 2024-06-06T08:38:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.118+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa4681054b0 con 0x7fa4681073f0 2024-06-06T08:38:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.120+0000 7fa4657fa700 1 -- 172.21.15.27:0/1522014054 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa460019020 con 0x7fa4681073f0 2024-06-06T08:38:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.121+0000 7fa4657fa700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4500a2cb0 0x7fa4500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.121+0000 7fa467fff700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4500a2cb0 0x7fa4500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.121+0000 7fa4657fa700 1 -- 172.21.15.27:0/1522014054 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fa4600cf940 con 0x7fa4681073f0 2024-06-06T08:38:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.121+0000 7fa4657fa700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] conn(0x7fa4500ae010 0x7fa4500b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.121+0000 7fa4657fa700 1 -- 172.21.15.27:0/1522014054 --> [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4500b0b80 con 0x7fa4500ae010 2024-06-06T08:38:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.121+0000 7fa4657fa700 1 -- 172.21.15.27:0/1522014054 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fa4600cfd70 con 0x7fa4681073f0 2024-06-06T08:38:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.121+0000 7fa467fff700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4500a2cb0 0x7fa4500a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa468199b60 tx=0x7fa458005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.122+0000 7fa4677fe700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] conn(0x7fa4500ae010 0x7fa4500b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.123+0000 7fa4677fe700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] conn(0x7fa4500ae010 0x7fa4500b04b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.123+0000 7fa4657fa700 1 -- 172.21.15.27:0/1522014054 <== osd.7 v2:172.21.15.176:6816/3681627280 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4500b0b80 con 0x7fa4500ae010 2024-06-06T08:38:28.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.140+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 --> [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa468102aa0 con 0x7fa4500ae010 2024-06-06T08:38:28.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa4657fa700 1 -- 172.21.15.27:0/1522014054 <== osd.7 v2:172.21.15.176:6816/3681627280 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa468102aa0 con 0x7fa4500ae010 2024-06-06T08:38:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 >> [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] conn(0x7fa4500ae010 msgr2=0x7fa4500b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] conn(0x7fa4500ae010 0x7fa4500b04b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4500a2cb0 msgr2=0x7fa4500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4500a2cb0 0x7fa4500a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa468199b60 tx=0x7fa458005960 comp rx=0 tx=0).stop 2024-06-06T08:38:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4681073f0 msgr2=0x7fa468198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 --2- 172.21.15.27:0/1522014054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4681073f0 0x7fa468198a20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa460007d70 tx=0x7fa46000a340 comp rx=0 tx=0).stop 2024-06-06T08:38:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46cc38700 1 -- 172.21.15.27:0/1522014054 reap_dead start 2024-06-06T08:38:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 shutdown_connections 2024-06-06T08:38:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 >> 172.21.15.27:0/1522014054 conn(0x7fa468078300 msgr2=0x7fa4681006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 shutdown_connections 2024-06-06T08:38:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:28.141+0000 7fa46e69b700 1 -- 172.21.15.27:0/1522014054 wait complete. 2024-06-06T08:38:28.352 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:28 smithi027 ceph-mon[32335]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:28.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:28 smithi203 ceph-mon[36579]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:28.502 INFO:teuthology.orchestra.run.smithi027.stdout:309237645358 2024-06-06T08:38:28.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-06T08:38:28.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:28 smithi176 ceph-mon[36786]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:30.349 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:30 smithi027 ceph-mon[32335]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:30.384 INFO:teuthology.orchestra.run.smithi027.stdout:219043332158 2024-06-06T08:38:30.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-06T08:38:30.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:30 smithi203 ceph-mon[36579]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:30.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:30 smithi176 ceph-mon[36786]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:30.795 INFO:teuthology.orchestra.run.smithi027.stdout:77309411416 2024-06-06T08:38:30.795 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-06T08:38:31.199 INFO:teuthology.orchestra.run.smithi027.stdout:34359738465 2024-06-06T08:38:31.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-06T08:38:31.319 INFO:teuthology.orchestra.run.smithi027.stdout:163208757320 2024-06-06T08:38:31.320 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-06T08:38:32.325 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:32 smithi027 ceph-mon[32335]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:32.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.433+0000 7fd1411af700 1 -- 172.21.15.27:0/4162698430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd13c1027a0 msgr2=0x7fd13c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:32.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.433+0000 7fd1411af700 1 --2- 172.21.15.27:0/4162698430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd13c1027a0 0x7fd13c102b80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0099c0 tx=0x7fd12c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:38:32.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.433+0000 7fd1411af700 1 -- 172.21.15.27:0/4162698430 shutdown_connections 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.433+0000 7fd1411af700 1 --2- 172.21.15.27:0/4162698430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c107870 0x7fd13c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.433+0000 7fd1411af700 1 --2- 172.21.15.27:0/4162698430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd13c1027a0 0x7fd13c102b80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.433+0000 7fd1411af700 1 --2- 172.21.15.27:0/4162698430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd13c103150 0x7fd13c107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.433+0000 7fd1411af700 1 -- 172.21.15.27:0/4162698430 >> 172.21.15.27:0/4162698430 conn(0x7fd13c079570 msgr2=0x7fd13c079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.433+0000 7fd1411af700 1 -- 172.21.15.27:0/4162698430 shutdown_connections 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.433+0000 7fd1411af700 1 -- 172.21.15.27:0/4162698430 wait complete. 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.434+0000 7fd1411af700 1 Processor -- start 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.434+0000 7fd1411af700 1 -- start start 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.434+0000 7fd1411af700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd13c1027a0 0x7fd13c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.434+0000 7fd1411af700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c103150 0x7fd13c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.434+0000 7fd1411af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd13c107870 0x7fd13c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:32.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.434+0000 7fd1411af700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c112790 con 0x7fd13c107870 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.434+0000 7fd1411af700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c112610 con 0x7fd13c1027a0 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd1411af700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c112910 con 0x7fd13c103150 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd13c107870 0x7fd13c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd13c107870 0x7fd13c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33482/0 (socket says 172.21.15.27:33482) 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13b59e700 1 -- 172.21.15.27:0/4058096264 learned_addr learned my addr 172.21.15.27:0/4058096264 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13259c700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c103150 0x7fd13c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13ad9d700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd13c1027a0 0x7fd13c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13b59e700 1 -- 172.21.15.27:0/4058096264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c103150 msgr2=0x7fd13c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13b59e700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c103150 0x7fd13c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13b59e700 1 -- 172.21.15.27:0/4058096264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd13c1027a0 msgr2=0x7fd13c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13b59e700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd13c1027a0 0x7fd13c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13b59e700 1 -- 172.21.15.27:0/4058096264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c199110 con 0x7fd13c107870 2024-06-06T08:38:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13ad9d700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd13c1027a0 0x7fd13c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd13b59e700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd13c107870 0x7fd13c1989b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd134007d70 tx=0x7fd13400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd138d99700 1 -- 172.21.15.27:0/4058096264 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134016020 con 0x7fd13c107870 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c199400 con 0x7fd13c107870 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd138d99700 1 -- 172.21.15.27:0/4058096264 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd13400ad40 con 0x7fd13c107870 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.435+0000 7fd138d99700 1 -- 172.21.15.27:0/4058096264 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1340115c0 con 0x7fd13c107870 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.436+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c1a5670 con 0x7fd13c107870 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.436+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd13c062c20 con 0x7fd13c107870 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.438+0000 7fd138d99700 1 -- 172.21.15.27:0/4058096264 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd134019020 con 0x7fd13c107870 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.438+0000 7fd138d99700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1280a2cb0 0x7fd1280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.439+0000 7fd13ad9d700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1280a2cb0 0x7fd1280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.439+0000 7fd138d99700 1 -- 172.21.15.27:0/4058096264 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fd1340cf980 con 0x7fd13c107870 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.439+0000 7fd138d99700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] conn(0x7fd1280ae010 0x7fd1280b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.439+0000 7fd138d99700 1 -- 172.21.15.27:0/4058096264 --> [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd134018e70 con 0x7fd1280ae010 2024-06-06T08:38:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.439+0000 7fd138d99700 1 -- 172.21.15.27:0/4058096264 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fd1340cfdb0 con 0x7fd13c107870 2024-06-06T08:38:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.439+0000 7fd13259c700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] conn(0x7fd1280ae010 0x7fd1280b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.439+0000 7fd13ad9d700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1280a2cb0 0x7fd1280a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd12c000c00 tx=0x7fd12c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.439+0000 7fd13259c700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] conn(0x7fd1280ae010 0x7fd1280b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.440+0000 7fd138d99700 1 -- 172.21.15.27:0/4058096264 <== osd.6 v2:172.21.15.27:6818/1876109947 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd134018e70 con 0x7fd1280ae010 2024-06-06T08:38:32.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.462+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 --> [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd13c063020 con 0x7fd1280ae010 2024-06-06T08:38:32.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.465+0000 7fd138d99700 1 -- 172.21.15.27:0/4058096264 <== osd.6 v2:172.21.15.27:6818/1876109947 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd13c063020 con 0x7fd1280ae010 2024-06-06T08:38:32.465 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:32.472 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:32.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.473+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] conn(0x7fd1280ae010 msgr2=0x7fd1280b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:32.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.473+0000 7fd1411af700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] conn(0x7fd1280ae010 0x7fd1280b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:32.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.475+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1280a2cb0 msgr2=0x7fd1280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:32.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.475+0000 7fd1411af700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd1280a2cb0 0x7fd1280a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd12c000c00 tx=0x7fd12c005ab0 comp rx=0 tx=0).stop 2024-06-06T08:38:32.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.475+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd13c107870 msgr2=0x7fd13c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.475+0000 7fd1411af700 1 --2- 172.21.15.27:0/4058096264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd13c107870 0x7fd13c1989b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd134007d70 tx=0x7fd13400a340 comp rx=0 tx=0).stop 2024-06-06T08:38:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.476+0000 7fd13b59e700 1 -- 172.21.15.27:0/4058096264 reap_dead start 2024-06-06T08:38:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.476+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 shutdown_connections 2024-06-06T08:38:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.476+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 >> 172.21.15.27:0/4058096264 conn(0x7fd13c079570 msgr2=0x7fd13c1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.480+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 shutdown_connections 2024-06-06T08:38:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:32.480+0000 7fd1411af700 1 -- 172.21.15.27:0/4058096264 wait complete. 2024-06-06T08:38:32.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:32 smithi203 ceph-mon[36579]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:32.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:32 smithi176 ceph-mon[36786]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:34.150 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:34 smithi027 ceph-mon[32335]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-06T08:38:34.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.402+0000 7f9270ef1700 1 -- 172.21.15.27:0/1906897085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c101df0 msgr2=0x7f926c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:34.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.402+0000 7f9270ef1700 1 --2- 172.21.15.27:0/1906897085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c101df0 0x7f926c1021d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f925c0099c0 tx=0x7f925c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:38:34.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.402+0000 7f9270ef1700 1 -- 172.21.15.27:0/1906897085 shutdown_connections 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.402+0000 7f9270ef1700 1 --2- 172.21.15.27:0/1906897085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f926c106750 0x7f926c10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.402+0000 7f9270ef1700 1 --2- 172.21.15.27:0/1906897085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c101df0 0x7f926c1021d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.402+0000 7f9270ef1700 1 --2- 172.21.15.27:0/1906897085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f926c1027a0 0x7f926c106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.402+0000 7f9270ef1700 1 -- 172.21.15.27:0/1906897085 >> 172.21.15.27:0/1906897085 conn(0x7f926c077a00 msgr2=0x7f926c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.402+0000 7f9270ef1700 1 -- 172.21.15.27:0/1906897085 shutdown_connections 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.402+0000 7f9270ef1700 1 -- 172.21.15.27:0/1906897085 wait complete. 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.403+0000 7f9270ef1700 1 Processor -- start 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.403+0000 7f9270ef1700 1 -- start start 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.403+0000 7f9270ef1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f926c101df0 0x7f926c0fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.403+0000 7f9270ef1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f926c1027a0 0x7f926c0fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.403+0000 7f9270ef1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c106750 0x7f926c0fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f9270ef1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f926c111690 con 0x7f926c106750 2024-06-06T08:38:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f9270ef1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f926c111510 con 0x7f926c1027a0 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f9270ef1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f926c111810 con 0x7f926c101df0 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f926a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f926c101df0 0x7f926c0fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f926a59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f926c101df0 0x7f926c0fce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:53978/0 (socket says 172.21.15.27:53978) 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f926a59c700 1 -- 172.21.15.27:0/1172584289 learned_addr learned my addr 172.21.15.27:0/1172584289 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f9269d9b700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f926c1027a0 0x7f926c0fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f926ad9d700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c106750 0x7f926c0fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f926a59c700 1 -- 172.21.15.27:0/1172584289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f926c1027a0 msgr2=0x7f926c0fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f926a59c700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f926c1027a0 0x7f926c0fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f926a59c700 1 -- 172.21.15.27:0/1172584289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c106750 msgr2=0x7f926c0fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f926a59c700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c106750 0x7f926c0fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.404+0000 7f926a59c700 1 -- 172.21.15.27:0/1172584289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c1a7130 con 0x7f926c101df0 2024-06-06T08:38:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.405+0000 7f926a59c700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f926c101df0 0x7f926c0fce70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f925c006010 tx=0x7f925c009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.405+0000 7f925b7fe700 1 -- 172.21.15.27:0/1172584289 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c026070 con 0x7f926c101df0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.405+0000 7f925b7fe700 1 -- 172.21.15.27:0/1172584289 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c004cd0 con 0x7f926c101df0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.405+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c1a73c0 con 0x7f926c101df0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.405+0000 7f925b7fe700 1 -- 172.21.15.27:0/1172584289 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c003ba0 con 0x7f926c101df0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.405+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c1a7810 con 0x7f926c101df0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.406+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f926c0ff240 con 0x7f926c101df0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.408+0000 7f925b7fe700 1 -- 172.21.15.27:0/1172584289 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f925c029070 con 0x7f926c101df0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.409+0000 7f925b7fe700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f92540a2cb0 0x7f92540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.409+0000 7f925b7fe700 1 -- 172.21.15.27:0/1172584289 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f925c0d05f0 con 0x7f926c101df0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.409+0000 7f9269d9b700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f92540a2cb0 0x7f92540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.409+0000 7f925b7fe700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] conn(0x7f92540ae010 0x7f92540b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.409+0000 7f925b7fe700 1 -- 172.21.15.27:0/1172584289 --> [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92540b0b80 con 0x7f92540ae010 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.409+0000 7f925b7fe700 1 -- 172.21.15.27:0/1172584289 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f925c0d0a20 con 0x7f926c101df0 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.409+0000 7f926ad9d700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] conn(0x7f92540ae010 0x7f92540b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.409+0000 7f9269d9b700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f92540a2cb0 0x7f92540a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f926c107f60 tx=0x7f926000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.410+0000 7f926ad9d700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] conn(0x7f92540ae010 0x7f92540b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.411+0000 7f925b7fe700 1 -- 172.21.15.27:0/1172584289 <== osd.11 v2:172.21.15.203:6824/1209624343 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92540b0b80 con 0x7f92540ae010 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.430+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 --> [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f926c102ba0 con 0x7f92540ae010 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.430+0000 7f925b7fe700 1 -- 172.21.15.27:0/1172584289 <== osd.11 v2:172.21.15.203:6824/1209624343 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f926c102ba0 con 0x7f92540ae010 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.430+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 >> [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] conn(0x7f92540ae010 msgr2=0x7f92540b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.430+0000 7f9270ef1700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] conn(0x7f92540ae010 0x7f92540b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.431+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f92540a2cb0 msgr2=0x7f92540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.431+0000 7f9270ef1700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f92540a2cb0 0x7f92540a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f926c107f60 tx=0x7f926000a380 comp rx=0 tx=0).stop 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.431+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f926c101df0 msgr2=0x7f926c0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.431+0000 7f9270ef1700 1 --2- 172.21.15.27:0/1172584289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f926c101df0 0x7f926c0fce70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f925c006010 tx=0x7f925c009f80 comp rx=0 tx=0).stop 2024-06-06T08:38:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.431+0000 7f926ad9d700 1 -- 172.21.15.27:0/1172584289 reap_dead start 2024-06-06T08:38:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.431+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 shutdown_connections 2024-06-06T08:38:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.431+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 >> 172.21.15.27:0/1172584289 conn(0x7f926c077a00 msgr2=0x7f926c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.431+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 shutdown_connections 2024-06-06T08:38:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:34.431+0000 7f9270ef1700 1 -- 172.21.15.27:0/1172584289 wait complete. 2024-06-06T08:38:34.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:34 smithi203 ceph-mon[36579]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-06T08:38:34.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:34 smithi176 ceph-mon[36786]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-06T08:38:35.083 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:35.117 INFO:teuthology.orchestra.run.smithi027.stdout:343597383723 2024-06-06T08:38:35.118 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-06T08:38:35.523 INFO:teuthology.orchestra.run.smithi027.stdout:197568495685 2024-06-06T08:38:35.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-06T08:38:36.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.118+0000 7fb023623700 1 -- 172.21.15.27:0/335513264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c06a170 msgr2=0x7fb01c06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.118+0000 7fb023623700 1 --2- 172.21.15.27:0/335513264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c06a170 0x7fb01c06a5f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb010009990 tx=0x7fb010009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.118+0000 7fb023623700 1 -- 172.21.15.27:0/335513264 shutdown_connections 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.118+0000 7fb023623700 1 --2- 172.21.15.27:0/335513264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb01c06ab30 0x7fb01c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.118+0000 7fb023623700 1 --2- 172.21.15.27:0/335513264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb01c108500 0x7fb01c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.118+0000 7fb023623700 1 --2- 172.21.15.27:0/335513264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c06a170 0x7fb01c06a5f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.118+0000 7fb023623700 1 -- 172.21.15.27:0/335513264 >> 172.21.15.27:0/335513264 conn(0x7fb01c078340 msgr2=0x7fb01c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.118+0000 7fb023623700 1 -- 172.21.15.27:0/335513264 shutdown_connections 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.118+0000 7fb023623700 1 -- 172.21.15.27:0/335513264 wait complete. 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb023623700 1 Processor -- start 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb023623700 1 -- start start 2024-06-06T08:38:36.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb023623700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb01c06a170 0x7fb01c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb023623700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c06ab30 0x7fb01c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb023623700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb01c108500 0x7fb01c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb023623700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c112760 con 0x7fb01c06ab30 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb023623700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1125e0 con 0x7fb01c108500 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb023623700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c1128e0 con 0x7fb01c06a170 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb020bbe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c06ab30 0x7fb01c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb020bbe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c06ab30 0x7fb01c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33526/0 (socket says 172.21.15.27:33526) 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb020bbe700 1 -- 172.21.15.27:0/884323706 learned_addr learned my addr 172.21.15.27:0/884323706 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb021bc0700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb01c108500 0x7fb01c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb0213bf700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb01c06a170 0x7fb01c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb020bbe700 1 -- 172.21.15.27:0/884323706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb01c06a170 msgr2=0x7fb01c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:36.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb020bbe700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb01c06a170 0x7fb01c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb020bbe700 1 -- 172.21.15.27:0/884323706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb01c108500 msgr2=0x7fb01c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb020bbe700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb01c108500 0x7fb01c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb020bbe700 1 -- 172.21.15.27:0/884323706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c19fe10 con 0x7fb01c06ab30 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb021bc0700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb01c108500 0x7fb01c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb0213bf700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb01c06a170 0x7fb01c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.119+0000 7fb020bbe700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c06ab30 0x7fb01c19b340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb010000c00 tx=0x7fb0100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.120+0000 7fb01a7fc700 1 -- 172.21.15.27:0/884323706 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010027070 con 0x7fb01c06ab30 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.120+0000 7fb01a7fc700 1 -- 172.21.15.27:0/884323706 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb010004050 con 0x7fb01c06ab30 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.120+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c1a00a0 con 0x7fb01c06ab30 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.120+0000 7fb01a7fc700 1 -- 172.21.15.27:0/884323706 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01000f460 con 0x7fb01c06ab30 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.120+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c1a7980 con 0x7fb01c06ab30 2024-06-06T08:38:36.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.121+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb01c10e0b0 con 0x7fb01c06ab30 2024-06-06T08:38:36.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.122+0000 7fb01a7fc700 1 -- 172.21.15.27:0/884323706 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb01002a070 con 0x7fb01c06ab30 2024-06-06T08:38:36.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.123+0000 7fb01a7fc700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0040a2d30 0x7fb0040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:36.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.123+0000 7fb0213bf700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0040a2d30 0x7fb0040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:36.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.123+0000 7fb01a7fc700 1 -- 172.21.15.27:0/884323706 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fb0100d0490 con 0x7fb01c06ab30 2024-06-06T08:38:36.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.123+0000 7fb01a7fc700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] conn(0x7fb0040ae090 0x7fb0040b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:36.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.123+0000 7fb01a7fc700 1 -- 172.21.15.27:0/884323706 --> [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb010001d30 con 0x7fb0040ae090 2024-06-06T08:38:36.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.123+0000 7fb01a7fc700 1 -- 172.21.15.27:0/884323706 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fb0100d7050 con 0x7fb01c06ab30 2024-06-06T08:38:36.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.123+0000 7fb021bc0700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] conn(0x7fb0040ae090 0x7fb0040b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:36.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.123+0000 7fb0213bf700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0040a2d30 0x7fb0040a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009d50 tx=0x7fb00c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:36.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.123+0000 7fb021bc0700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] conn(0x7fb0040ae090 0x7fb0040b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:36.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.124+0000 7fb01a7fc700 1 -- 172.21.15.27:0/884323706 <== osd.3 v2:172.21.15.27:6810/3079841108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb010001d30 con 0x7fb0040ae090 2024-06-06T08:38:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.140+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 --> [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb01c103bc0 con 0x7fb0040ae090 2024-06-06T08:38:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb01a7fc700 1 -- 172.21.15.27:0/884323706 <== osd.3 v2:172.21.15.27:6810/3079841108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb01c103bc0 con 0x7fb0040ae090 2024-06-06T08:38:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] conn(0x7fb0040ae090 msgr2=0x7fb0040b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] conn(0x7fb0040ae090 0x7fb0040b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0040a2d30 msgr2=0x7fb0040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0040a2d30 0x7fb0040a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009d50 tx=0x7fb00c009040 comp rx=0 tx=0).stop 2024-06-06T08:38:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c06ab30 msgr2=0x7fb01c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 --2- 172.21.15.27:0/884323706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb01c06ab30 0x7fb01c19b340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb010000c00 tx=0x7fb0100047b0 comp rx=0 tx=0).stop 2024-06-06T08:38:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb021bc0700 1 -- 172.21.15.27:0/884323706 reap_dead start 2024-06-06T08:38:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 shutdown_connections 2024-06-06T08:38:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 >> 172.21.15.27:0/884323706 conn(0x7fb01c078340 msgr2=0x7fb01c10d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:36.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 shutdown_connections 2024-06-06T08:38:36.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:36.141+0000 7fb023623700 1 -- 172.21.15.27:0/884323706 wait complete. 2024-06-06T08:38:36.328 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:36 smithi027 ceph-mon[32335]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-06T08:38:36.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:36 smithi203 ceph-mon[36579]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-06T08:38:36.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:36 smithi176 ceph-mon[36786]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-06T08:38:36.937 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:36.938 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:37.196 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:38.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:38 smithi027 ceph-mon[32335]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:38.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:38 smithi203 ceph-mon[36579]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:38.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:38 smithi176 ceph-mon[36786]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:39.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 -- 172.21.15.27:0/20974968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 msgr2=0x7f4bec10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:39.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/20974968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 0x7f4bec10e9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009990 tx=0x7f4bd8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 -- 172.21.15.27:0/20974968 shutdown_connections 2024-06-06T08:38:39.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/20974968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec10ef00 0x7f4bec1112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/20974968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bec0746c0 0x7f4bec074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/20974968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 0x7f4bec10e9c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 -- 172.21.15.27:0/20974968 >> 172.21.15.27:0/20974968 conn(0x7f4bec079650 msgr2=0x7f4bec079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 -- 172.21.15.27:0/20974968 shutdown_connections 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 -- 172.21.15.27:0/20974968 wait complete. 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 Processor -- start 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.394+0000 7f4bf2c39700 1 -- start start 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bf2c39700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec0746c0 0x7f4bec19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bf2c39700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 0x7f4bec19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bf2c39700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bec10ef00 0x7f4bec19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bf2c39700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1149f0 con 0x7f4bec074fe0 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bf2c39700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec114870 con 0x7f4bec10ef00 2024-06-06T08:38:39.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bf2c39700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec114b70 con 0x7f4bec0746c0 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bebfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 0x7f4bec19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bebfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 0x7f4bec19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33544/0 (socket says 172.21.15.27:33544) 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bebfff700 1 -- 172.21.15.27:0/1450934242 learned_addr learned my addr 172.21.15.27:0/1450934242 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bf09d5700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec0746c0 0x7f4bec19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bf11d6700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bec10ef00 0x7f4bec19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bebfff700 1 -- 172.21.15.27:0/1450934242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec0746c0 msgr2=0x7f4bec19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bebfff700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec0746c0 0x7f4bec19aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bebfff700 1 -- 172.21.15.27:0/1450934242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bec10ef00 msgr2=0x7f4bec19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bebfff700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bec10ef00 0x7f4bec19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bebfff700 1 -- 172.21.15.27:0/1450934242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec19fef0 con 0x7f4bec074fe0 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4bebfff700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 0x7f4bec19b420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80038b0 tx=0x7f4bd8009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.395+0000 7f4be9ffb700 1 -- 172.21.15.27:0/1450934242 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8027070 con 0x7f4bec074fe0 2024-06-06T08:38:39.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.396+0000 7f4be9ffb700 1 -- 172.21.15.27:0/1450934242 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd8004050 con 0x7f4bec074fe0 2024-06-06T08:38:39.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.396+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec1a0180 con 0x7f4bec074fe0 2024-06-06T08:38:39.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.396+0000 7f4be9ffb700 1 -- 172.21.15.27:0/1450934242 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8022d40 con 0x7f4bec074fe0 2024-06-06T08:38:39.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.396+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec1a7a60 con 0x7f4bec074fe0 2024-06-06T08:38:39.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.397+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec069180 con 0x7f4bec074fe0 2024-06-06T08:38:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.398+0000 7f4be9ffb700 1 -- 172.21.15.27:0/1450934242 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4bd802a070 con 0x7f4bec074fe0 2024-06-06T08:38:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.399+0000 7f4be9ffb700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4bdc0a2cb0 0x7f4bdc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.399+0000 7f4bf09d5700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4bdc0a2cb0 0x7f4bdc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.399+0000 7f4be9ffb700 1 -- 172.21.15.27:0/1450934242 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f4bd80d0ac0 con 0x7f4bec074fe0 2024-06-06T08:38:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.399+0000 7f4bf09d5700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4bdc0a2cb0 0x7f4bdc0a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009a10 tx=0x7f4be0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:39.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.402+0000 7f4be9ffb700 1 -- 172.21.15.27:0/1450934242 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4bd8031020 con 0x7f4bec074fe0 2024-06-06T08:38:39.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.613+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4bec02dca0 con 0x7f4bec074fe0 2024-06-06T08:38:39.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.614+0000 7f4be9ffb700 1 -- 172.21.15.27:0/1450934242 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4bd8013070 con 0x7f4bec074fe0 2024-06-06T08:38:39.613 INFO:teuthology.orchestra.run.smithi027.stdout:55834574944 2024-06-06T08:38:39.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4bdc0a2cb0 msgr2=0x7f4bdc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4bdc0a2cb0 0x7f4bdc0a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009a10 tx=0x7f4be0009190 comp rx=0 tx=0).stop 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 msgr2=0x7f4bec19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 0x7f4bec19b420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80038b0 tx=0x7f4bd8009f50 comp rx=0 tx=0).stop 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 shutdown_connections 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f4bdc0a2cb0 0x7f4bdc0a5170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4bec10ef00 0x7f4bec19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec0746c0 0x7f4bec19aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 --2- 172.21.15.27:0/1450934242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bec074fe0 0x7f4bec19b420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 >> 172.21.15.27:0/1450934242 conn(0x7f4bec079650 msgr2=0x7f4bec0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 shutdown_connections 2024-06-06T08:38:39.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:39.616+0000 7f4bf2c39700 1 -- 172.21.15.27:0/1450934242 wait complete. 2024-06-06T08:38:40.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:40 smithi027 ceph-mon[32335]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:40.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:40 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1450934242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T08:38:40.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:40 smithi203 ceph-mon[36579]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:40.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:40 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1450934242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T08:38:40.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:40 smithi176 ceph-mon[36786]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:40.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:40 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1450934242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T08:38:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.849+0000 7f46cbef0700 1 -- 172.21.15.27:0/962017120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c4108500 msgr2=0x7f46c41088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.849+0000 7f46cbef0700 1 --2- 172.21.15.27:0/962017120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c4108500 0x7f46c41088e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009990 tx=0x7f46c0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.849+0000 7f46cbef0700 1 -- 172.21.15.27:0/962017120 shutdown_connections 2024-06-06T08:38:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.849+0000 7f46cbef0700 1 --2- 172.21.15.27:0/962017120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46c406ab30 0x7f46c410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.849+0000 7f46cbef0700 1 --2- 172.21.15.27:0/962017120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c4108500 0x7f46c41088e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.849+0000 7f46cbef0700 1 --2- 172.21.15.27:0/962017120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46c406a170 0x7f46c406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.849+0000 7f46cbef0700 1 -- 172.21.15.27:0/962017120 >> 172.21.15.27:0/962017120 conn(0x7f46c4078340 msgr2=0x7f46c4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.849+0000 7f46cbef0700 1 -- 172.21.15.27:0/962017120 shutdown_connections 2024-06-06T08:38:41.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.849+0000 7f46cbef0700 1 -- 172.21.15.27:0/962017120 wait complete. 2024-06-06T08:38:41.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46cbef0700 1 Processor -- start 2024-06-06T08:38:41.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46cbef0700 1 -- start start 2024-06-06T08:38:41.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46cbef0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46c406a170 0x7f46c4101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:41.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46cbef0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c406ab30 0x7f46c41019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:41.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46cbef0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46c4108500 0x7f46c4101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:41.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46cbef0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4112780 con 0x7f46c406ab30 2024-06-06T08:38:41.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46cbef0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4112600 con 0x7f46c406a170 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46cbef0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4112900 con 0x7f46c4108500 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46c948b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c406ab30 0x7f46c41019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46c948b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c406ab30 0x7f46c41019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56384/0 (socket says 172.21.15.27:56384) 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46c948b700 1 -- 172.21.15.27:0/4094972066 learned_addr learned my addr 172.21.15.27:0/4094972066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46c9c8c700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46c406a170 0x7f46c4101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.850+0000 7f46ca48d700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46c4108500 0x7f46c4101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46c948b700 1 -- 172.21.15.27:0/4094972066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46c4108500 msgr2=0x7f46c4101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46c948b700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46c4108500 0x7f46c4101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46c948b700 1 -- 172.21.15.27:0/4094972066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46c406a170 msgr2=0x7f46c4101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46c948b700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46c406a170 0x7f46c4101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46c948b700 1 -- 172.21.15.27:0/4094972066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c41a7200 con 0x7f46c406ab30 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46c948b700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c406ab30 0x7f46c41019a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c970 tx=0x7f46b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46baffd700 1 -- 172.21.15.27:0/4094972066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b40078d0 con 0x7f46c406ab30 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46baffd700 1 -- 172.21.15.27:0/4094972066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b4018480 con 0x7f46c406ab30 2024-06-06T08:38:41.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c41a74f0 con 0x7f46c406ab30 2024-06-06T08:38:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46baffd700 1 -- 172.21.15.27:0/4094972066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b400f6b0 con 0x7f46c406ab30 2024-06-06T08:38:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.851+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c41a79e0 con 0x7f46c406ab30 2024-06-06T08:38:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.852+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c410a9e0 con 0x7f46c406ab30 2024-06-06T08:38:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.853+0000 7f46baffd700 1 -- 172.21.15.27:0/4094972066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f46b4018630 con 0x7f46c406ab30 2024-06-06T08:38:41.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.854+0000 7f46baffd700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f46b00a2cb0 0x7f46b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.854+0000 7f46c9c8c700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f46b00a2cb0 0x7f46b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.854+0000 7f46baffd700 1 -- 172.21.15.27:0/4094972066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f46b40cf620 con 0x7f46c406ab30 2024-06-06T08:38:41.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.854+0000 7f46c9c8c700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f46b00a2cb0 0x7f46b00a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f46c000b5c0 tx=0x7f46c0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:41.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:41.859+0000 7f46baffd700 1 -- 172.21.15.27:0/4094972066 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f46b409c0e0 con 0x7f46c406ab30 2024-06-06T08:38:42.076 INFO:teuthology.orchestra.run.smithi027.stdout:141733920849 2024-06-06T08:38:42.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.077+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f46c4102990 con 0x7f46c406ab30 2024-06-06T08:38:42.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.077+0000 7f46baffd700 1 -- 172.21.15.27:0/4094972066 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f46b409b9e0 con 0x7f46c406ab30 2024-06-06T08:38:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f46b00a2cb0 msgr2=0x7f46b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f46b00a2cb0 0x7f46b00a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f46c000b5c0 tx=0x7f46c0005960 comp rx=0 tx=0).stop 2024-06-06T08:38:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c406ab30 msgr2=0x7f46c41019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c406ab30 0x7f46c41019a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c970 tx=0x7f46b400cc80 comp rx=0 tx=0).stop 2024-06-06T08:38:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 shutdown_connections 2024-06-06T08:38:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f46b00a2cb0 0x7f46b00a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46c4108500 0x7f46c4101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f46c406a170 0x7f46c4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 --2- 172.21.15.27:0/4094972066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46c406ab30 0x7f46c41019a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 >> 172.21.15.27:0/4094972066 conn(0x7f46c4078340 msgr2=0x7f46c4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 shutdown_connections 2024-06-06T08:38:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:42.080+0000 7f46cbef0700 1 -- 172.21.15.27:0/4094972066 wait complete. 2024-06-06T08:38:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:42 smithi027 ceph-mon[32335]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:42 smithi203 ceph-mon[36579]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:42.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:42 smithi176 ceph-mon[36786]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:42.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574939 got 55834574944 for osd.1 2024-06-06T08:38:42.584 DEBUG:teuthology.parallel:result is None 2024-06-06T08:38:43.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:43 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4094972066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T08:38:43.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:43 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4094972066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T08:38:43.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:43 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4094972066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T08:38:43.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.818+0000 7faeb7889700 1 -- 172.21.15.27:0/1359901857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb0107840 msgr2=0x7faeb010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:43.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.818+0000 7faeb7889700 1 --2- 172.21.15.27:0/1359901857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb0107840 0x7faeb010f3c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faea800b0d0 tx=0x7faea800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:38:43.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.818+0000 7faeb7889700 1 -- 172.21.15.27:0/1359901857 shutdown_connections 2024-06-06T08:38:43.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.818+0000 7faeb7889700 1 --2- 172.21.15.27:0/1359901857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb0107840 0x7faeb010f3c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:43.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.818+0000 7faeb7889700 1 --2- 172.21.15.27:0/1359901857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faeb0102740 0x7faeb0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:43.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.818+0000 7faeb7889700 1 --2- 172.21.15.27:0/1359901857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb01030f0 0x7faeb0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:43.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.818+0000 7faeb7889700 1 -- 172.21.15.27:0/1359901857 >> 172.21.15.27:0/1359901857 conn(0x7faeb00795f0 msgr2=0x7faeb0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 -- 172.21.15.27:0/1359901857 shutdown_connections 2024-06-06T08:38:43.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 -- 172.21.15.27:0/1359901857 wait complete. 2024-06-06T08:38:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 Processor -- start 2024-06-06T08:38:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 -- start start 2024-06-06T08:38:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0102740 0x7faeb019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01030f0 0x7faeb019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:43.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faeb0107840 0x7faeb0198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faeb01127d0 con 0x7faeb01030f0 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0112650 con 0x7faeb0107840 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.819+0000 7faeb7889700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0112950 con 0x7faeb0102740 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb4e24700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01030f0 0x7faeb019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb4e24700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01030f0 0x7faeb019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56396/0 (socket says 172.21.15.27:56396) 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb4e24700 1 -- 172.21.15.27:0/598024966 learned_addr learned my addr 172.21.15.27:0/598024966 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb5625700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0102740 0x7faeb019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb5e26700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faeb0107840 0x7faeb0198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb4e24700 1 -- 172.21.15.27:0/598024966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0102740 msgr2=0x7faeb019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb4e24700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0102740 0x7faeb019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb4e24700 1 -- 172.21.15.27:0/598024966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faeb0107840 msgr2=0x7faeb0198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb4e24700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faeb0107840 0x7faeb0198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb4e24700 1 -- 172.21.15.27:0/598024966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb01990f0 con 0x7faeb01030f0 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb4e24700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01030f0 0x7faeb019ed10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faea000b530 tx=0x7faea000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:43.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faea67fc700 1 -- 172.21.15.27:0/598024966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0015020 con 0x7faeb01030f0 2024-06-06T08:38:43.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faea67fc700 1 -- 172.21.15.27:0/598024966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea0011d10 con 0x7faeb01030f0 2024-06-06T08:38:43.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb01993e0 con 0x7faeb01030f0 2024-06-06T08:38:43.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faea67fc700 1 -- 172.21.15.27:0/598024966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea000f2e0 con 0x7faeb01030f0 2024-06-06T08:38:43.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.820+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb01a56a0 con 0x7faeb01030f0 2024-06-06T08:38:43.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.821+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb0069180 con 0x7faeb01030f0 2024-06-06T08:38:43.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.823+0000 7faea67fc700 1 -- 172.21.15.27:0/598024966 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faea0018020 con 0x7faeb01030f0 2024-06-06T08:38:43.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.823+0000 7faea67fc700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae9c0a2cb0 0x7fae9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:43.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.823+0000 7faeb5625700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae9c0a2cb0 0x7fae9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:43.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.823+0000 7faea67fc700 1 -- 172.21.15.27:0/598024966 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7faea00cebb0 con 0x7faeb01030f0 2024-06-06T08:38:43.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.824+0000 7faeb5625700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae9c0a2cb0 0x7fae9c0a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faeb0199ad0 tx=0x7faeac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:43.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:43.828+0000 7faea67fc700 1 -- 172.21.15.27:0/598024966 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faea001f020 con 0x7faeb01030f0 2024-06-06T08:38:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.038+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7faeb0199ec0 con 0x7faeb01030f0 2024-06-06T08:38:44.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.039+0000 7faea67fc700 1 -- 172.21.15.27:0/598024966 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faeb0199ec0 con 0x7faeb01030f0 2024-06-06T08:38:44.038 INFO:teuthology.orchestra.run.smithi027.stdout:253403070524 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae9c0a2cb0 msgr2=0x7fae9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae9c0a2cb0 0x7fae9c0a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faeb0199ad0 tx=0x7faeac00b410 comp rx=0 tx=0).stop 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01030f0 msgr2=0x7faeb019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01030f0 0x7faeb019ed10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faea000b530 tx=0x7faea000b840 comp rx=0 tx=0).stop 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 shutdown_connections 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae9c0a2cb0 0x7fae9c0a5170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faeb0107840 0x7faeb0198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0102740 0x7faeb019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 --2- 172.21.15.27:0/598024966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01030f0 0x7faeb019ed10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 >> 172.21.15.27:0/598024966 conn(0x7faeb00795f0 msgr2=0x7faeb0100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:44.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 shutdown_connections 2024-06-06T08:38:44.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:44.041+0000 7faeb7889700 1 -- 172.21.15.27:0/598024966 wait complete. 2024-06-06T08:38:44.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:44 smithi027 ceph-mon[32335]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:38:44.207 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:44 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/598024966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T08:38:44.269 INFO:teuthology.orchestra.run.smithi027.stdout:107374182485 2024-06-06T08:38:44.270 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-06T08:38:44.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:44 smithi203 ceph-mon[36579]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:38:44.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:44 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/598024966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T08:38:44.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:44 smithi176 ceph-mon[36786]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:38:44.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:44 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/598024966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T08:38:45.295 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070519 got 253403070524 for osd.8 2024-06-06T08:38:45.295 DEBUG:teuthology.parallel:result is None 2024-06-06T08:38:46.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:46 smithi027 ceph-mon[32335]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-06T08:38:46.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920845 got 141733920849 for osd.4 2024-06-06T08:38:46.463 DEBUG:teuthology.parallel:result is None 2024-06-06T08:38:46.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:46 smithi203 ceph-mon[36579]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-06T08:38:46.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:46 smithi176 ceph-mon[36786]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.932+0000 7f64cf5e7700 1 -- 172.21.15.27:0/169782280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 msgr2=0x7f64c81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.932+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/169782280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 0x7f64c81088d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009990 tx=0x7f64c4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.932+0000 7f64cf5e7700 1 -- 172.21.15.27:0/169782280 shutdown_connections 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.932+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/169782280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64c806b3c0 0x7f64c810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.932+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/169782280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 0x7f64c81088d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.932+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/169782280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64c806aa00 0x7f64c806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.932+0000 7f64cf5e7700 1 -- 172.21.15.27:0/169782280 >> 172.21.15.27:0/169782280 conn(0x7f64c8078300 msgr2=0x7f64c8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.932+0000 7f64cf5e7700 1 -- 172.21.15.27:0/169782280 shutdown_connections 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.932+0000 7f64cf5e7700 1 -- 172.21.15.27:0/169782280 wait complete. 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cf5e7700 1 Processor -- start 2024-06-06T08:38:47.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cf5e7700 1 -- start start 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cf5e7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64c806aa00 0x7f64c819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cf5e7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64c806b3c0 0x7f64c819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cf5e7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 0x7f64c8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cf5e7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f64c8112750 con 0x7f64c81084f0 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cf5e7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f64c81125d0 con 0x7f64c806b3c0 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cf5e7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64c81128d0 con 0x7f64c806aa00 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cdb84700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 0x7f64c8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cdb84700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 0x7f64c8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56412/0 (socket says 172.21.15.27:56412) 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.933+0000 7f64cdb84700 1 -- 172.21.15.27:0/3349082704 learned_addr learned my addr 172.21.15.27:0/3349082704 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cd383700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64c806aa00 0x7f64c819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64ccb82700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64c806b3c0 0x7f64c819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:47.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cdb84700 1 -- 172.21.15.27:0/3349082704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64c806aa00 msgr2=0x7f64c819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:47.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cdb84700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64c806aa00 0x7f64c819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cdb84700 1 -- 172.21.15.27:0/3349082704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64c806b3c0 msgr2=0x7f64c819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:47.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cdb84700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64c806b3c0 0x7f64c819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cdb84700 1 -- 172.21.15.27:0/3349082704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c8199210 con 0x7f64c81084f0 2024-06-06T08:38:47.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64ccb82700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64c806b3c0 0x7f64c819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:38:47.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cd383700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64c806aa00 0x7f64c819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cdb84700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 0x7f64c8198ab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f64c0007ee0 tx=0x7f64c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64ba7fc700 1 -- 172.21.15.27:0/3349082704 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c000dbd0 con 0x7f64c81084f0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64ba7fc700 1 -- 172.21.15.27:0/3349082704 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64c000dd70 con 0x7f64c81084f0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64ba7fc700 1 -- 172.21.15.27:0/3349082704 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c00133f0 con 0x7f64c81084f0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c8199500 con 0x7f64c81084f0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.934+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c81a57c0 con 0x7f64c81084f0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.935+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c810a9d0 con 0x7f64c81084f0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.937+0000 7f64ba7fc700 1 -- 172.21.15.27:0/3349082704 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f64c001a020 con 0x7f64c81084f0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.937+0000 7f64ba7fc700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f64b40a2cb0 0x7f64b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.937+0000 7f64cd383700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f64b40a2cb0 0x7f64b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.937+0000 7f64ba7fc700 1 -- 172.21.15.27:0/3349082704 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f64c00cf3b0 con 0x7f64c81084f0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.938+0000 7f64cd383700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f64b40a2cb0 0x7f64b40a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f64c4006010 tx=0x7f64c400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:47.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:46.941+0000 7f64ba7fc700 1 -- 172.21.15.27:0/3349082704 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f64c0010070 con 0x7f64c81084f0 2024-06-06T08:38:47.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.151+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f64c8199fe0 con 0x7f64c81084f0 2024-06-06T08:38:47.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.151+0000 7f64ba7fc700 1 -- 172.21.15.27:0/3349082704 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f64c009bde0 con 0x7f64c81084f0 2024-06-06T08:38:47.150 INFO:teuthology.orchestra.run.smithi027.stdout:219043332162 2024-06-06T08:38:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f64b40a2cb0 msgr2=0x7f64b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f64b40a2cb0 0x7f64b40a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f64c4006010 tx=0x7f64c400b540 comp rx=0 tx=0).stop 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 msgr2=0x7f64c8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 0x7f64c8198ab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f64c0007ee0 tx=0x7f64c000d3b0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 shutdown_connections 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f64b40a2cb0 0x7f64b40a5170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64c81084f0 0x7f64c8198ab0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64c806aa00 0x7f64c819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 --2- 172.21.15.27:0/3349082704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64c806b3c0 0x7f64c819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 >> 172.21.15.27:0/3349082704 conn(0x7f64c8078300 msgr2=0x7f64c8100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 shutdown_connections 2024-06-06T08:38:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:47.154+0000 7f64cf5e7700 1 -- 172.21.15.27:0/3349082704 wait complete. 2024-06-06T08:38:47.888 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332158 got 219043332162 for osd.7 2024-06-06T08:38:47.889 DEBUG:teuthology.parallel:result is None 2024-06-06T08:38:48.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:48 smithi027 ceph-mon[32335]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:48.153 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:48 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3349082704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T08:38:48.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:48 smithi203 ceph-mon[36579]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:48.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:48 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3349082704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T08:38:48.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:48 smithi176 ceph-mon[36786]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:48.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:48 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3349082704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T08:38:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:49.080 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.971+0000 7fc7c3320700 1 -- 172.21.15.27:0/1369059654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc100c40 msgr2=0x7fc7bc101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.971+0000 7fc7c3320700 1 --2- 172.21.15.27:0/1369059654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc100c40 0x7fc7bc101020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80099c0 tx=0x7fc7b8009cd0 comp rx=0 tx=0).stop 2024-06-06T08:38:49.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.971+0000 7fc7c3320700 1 -- 172.21.15.27:0/1369059654 shutdown_connections 2024-06-06T08:38:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.971+0000 7fc7c3320700 1 --2- 172.21.15.27:0/1369059654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc107880 0x7fc7bc10f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.971+0000 7fc7c3320700 1 --2- 172.21.15.27:0/1369059654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc100c40 0x7fc7bc101020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.971+0000 7fc7c3320700 1 --2- 172.21.15.27:0/1369059654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7bc1015f0 0x7fc7bc107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.971+0000 7fc7c3320700 1 -- 172.21.15.27:0/1369059654 >> 172.21.15.27:0/1369059654 conn(0x7fc7bc077990 msgr2=0x7fc7bc0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.971+0000 7fc7c3320700 1 -- 172.21.15.27:0/1369059654 shutdown_connections 2024-06-06T08:38:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.971+0000 7fc7c3320700 1 -- 172.21.15.27:0/1369059654 wait complete. 2024-06-06T08:38:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.972+0000 7fc7c3320700 1 Processor -- start 2024-06-06T08:38:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.972+0000 7fc7c3320700 1 -- start start 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.972+0000 7fc7c3320700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc100c40 0x7fc7bc101bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.972+0000 7fc7c3320700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc1015f0 0x7fc7bc102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.972+0000 7fc7c3320700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7bc107880 0x7fc7bc102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.972+0000 7fc7c3320700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc112860 con 0x7fc7bc1015f0 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.972+0000 7fc7c3320700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1126e0 con 0x7fc7bc100c40 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.972+0000 7fc7c3320700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc1129e0 con 0x7fc7bc107880 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.972+0000 7fc7c08bb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc1015f0 0x7fc7bc102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c08bb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc1015f0 0x7fc7bc102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59472/0 (socket says 172.21.15.27:59472) 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c08bb700 1 -- 172.21.15.27:0/3025455471 learned_addr learned my addr 172.21.15.27:0/3025455471 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:49.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c18bd700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7bc107880 0x7fc7bc102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c10bc700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc100c40 0x7fc7bc101bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c08bb700 1 -- 172.21.15.27:0/3025455471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7bc107880 msgr2=0x7fc7bc102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c08bb700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7bc107880 0x7fc7bc102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c08bb700 1 -- 172.21.15.27:0/3025455471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc100c40 msgr2=0x7fc7bc101bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c08bb700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc100c40 0x7fc7bc101bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c08bb700 1 -- 172.21.15.27:0/3025455471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc1a7170 con 0x7fc7bc1015f0 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c08bb700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc1015f0 0x7fc7bc102100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00b530 tx=0x7fc7ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7b27fc700 1 -- 172.21.15.27:0/3025455471 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac015020 con 0x7fc7bc1015f0 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc1a7460 con 0x7fc7bc1015f0 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7b27fc700 1 -- 172.21.15.27:0/3025455471 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ac011d10 con 0x7fc7bc1015f0 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7b27fc700 1 -- 172.21.15.27:0/3025455471 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00f2e0 con 0x7fc7bc1015f0 2024-06-06T08:38:49.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.973+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc1a79f0 con 0x7fc7bc1015f0 2024-06-06T08:38:49.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.974+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc10aac0 con 0x7fc7bc1015f0 2024-06-06T08:38:49.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.975+0000 7fc7b27fc700 1 -- 172.21.15.27:0/3025455471 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc7ac018020 con 0x7fc7bc1015f0 2024-06-06T08:38:49.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.976+0000 7fc7b27fc700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc7a80a2cb0 0x7fc7a80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:49.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.976+0000 7fc7c10bc700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc7a80a2cb0 0x7fc7a80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:49.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.976+0000 7fc7b27fc700 1 -- 172.21.15.27:0/3025455471 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fc7ac0cf0f0 con 0x7fc7bc1015f0 2024-06-06T08:38:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.977+0000 7fc7c10bc700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc7a80a2cb0 0x7fc7a80a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8000c00 tx=0x7fc7b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:49.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:49.980+0000 7fc7b27fc700 1 -- 172.21.15.27:0/3025455471 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc7ac01f020 con 0x7fc7bc1015f0 2024-06-06T08:38:50.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.189+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc7bc10a4a0 con 0x7fc7bc1015f0 2024-06-06T08:38:50.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.190+0000 7fc7b27fc700 1 -- 172.21.15.27:0/3025455471 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc7ac09bc30 con 0x7fc7bc1015f0 2024-06-06T08:38:50.189 INFO:teuthology.orchestra.run.smithi027.stdout:309237645364 2024-06-06T08:38:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc7a80a2cb0 msgr2=0x7fc7a80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc7a80a2cb0 0x7fc7a80a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8000c00 tx=0x7fc7b8005fb0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc1015f0 msgr2=0x7fc7bc102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc1015f0 0x7fc7bc102100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00b530 tx=0x7fc7ac00b840 comp rx=0 tx=0).stop 2024-06-06T08:38:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 shutdown_connections 2024-06-06T08:38:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc7a80a2cb0 0x7fc7a80a5170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7bc107880 0x7fc7bc102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc100c40 0x7fc7bc101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 --2- 172.21.15.27:0/3025455471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7bc1015f0 0x7fc7bc102100 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 >> 172.21.15.27:0/3025455471 conn(0x7fc7bc077990 msgr2=0x7fc7bc105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 shutdown_connections 2024-06-06T08:38:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.192+0000 7fc7c3320700 1 -- 172.21.15.27:0/3025455471 wait complete. 2024-06-06T08:38:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.198+0000 7fc22cc1d700 1 -- 172.21.15.27:0/881092197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 msgr2=0x7fc228102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.198+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/881092197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 0x7fc228102c30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc210009990 tx=0x7fc210009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.198+0000 7fc22cc1d700 1 -- 172.21.15.27:0/881092197 shutdown_connections 2024-06-06T08:38:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.198+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/881092197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc228103170 0x7fc22810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.198+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/881092197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228108650 0x7fc228108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.198+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/881092197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 0x7fc228102c30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.198+0000 7fc22cc1d700 1 -- 172.21.15.27:0/881092197 >> 172.21.15.27:0/881092197 conn(0x7fc22807a420 msgr2=0x7fc22807a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.198+0000 7fc22cc1d700 1 -- 172.21.15.27:0/881092197 shutdown_connections 2024-06-06T08:38:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.198+0000 7fc22cc1d700 1 -- 172.21.15.27:0/881092197 wait complete. 2024-06-06T08:38:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22cc1d700 1 Processor -- start 2024-06-06T08:38:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22cc1d700 1 -- start start 2024-06-06T08:38:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22cc1d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 0x7fc228074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22cc1d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc228103170 0x7fc228074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22cc1d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228108650 0x7fc228075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22cc1d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc2281128d0 con 0x7fc2281027b0 2024-06-06T08:38:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22cc1d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc228112750 con 0x7fc228108650 2024-06-06T08:38:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22cc1d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc228112a50 con 0x7fc228103170 2024-06-06T08:38:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 0x7fc228074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 0x7fc228074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59498/0 (socket says 172.21.15.27:59498) 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.199+0000 7fc22659c700 1 -- 172.21.15.27:0/4093541152 learned_addr learned my addr 172.21.15.27:0/4093541152 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc226d9d700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228108650 0x7fc228075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc225d9b700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc228103170 0x7fc228074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc22659c700 1 -- 172.21.15.27:0/4093541152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc228103170 msgr2=0x7fc228074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc22659c700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc228103170 0x7fc228074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc22659c700 1 -- 172.21.15.27:0/4093541152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228108650 msgr2=0x7fc228075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc22659c700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228108650 0x7fc228075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc22659c700 1 -- 172.21.15.27:0/4093541152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2281a71f0 con 0x7fc2281027b0 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc22659c700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 0x7fc228074790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c920 tx=0x7fc21800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc21f7fe700 1 -- 172.21.15.27:0/4093541152 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218004a30 con 0x7fc2281027b0 2024-06-06T08:38:50.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc21f7fe700 1 -- 172.21.15.27:0/4093541152 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc218005500 con 0x7fc2281027b0 2024-06-06T08:38:50.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2281a74e0 con 0x7fc2281027b0 2024-06-06T08:38:50.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc21f7fe700 1 -- 172.21.15.27:0/4093541152 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218007d50 con 0x7fc2281027b0 2024-06-06T08:38:50.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.200+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281a7a20 con 0x7fc2281027b0 2024-06-06T08:38:50.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.201+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc228069180 con 0x7fc2281027b0 2024-06-06T08:38:50.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.203+0000 7fc21f7fe700 1 -- 172.21.15.27:0/4093541152 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc218004d60 con 0x7fc2281027b0 2024-06-06T08:38:50.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.204+0000 7fc21f7fe700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc2140a2cb0 0x7fc2140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:50.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.204+0000 7fc225d9b700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc2140a2cb0 0x7fc2140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:50.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.204+0000 7fc21f7fe700 1 -- 172.21.15.27:0/4093541152 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fc2180cf5b0 con 0x7fc2281027b0 2024-06-06T08:38:50.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.205+0000 7fc225d9b700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc2140a2cb0 0x7fc2140a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc22807a920 tx=0x7fc210005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:50.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.209+0000 7fc21f7fe700 1 -- 172.21.15.27:0/4093541152 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc218014030 con 0x7fc2281027b0 2024-06-06T08:38:50.348 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:50 smithi027 ceph-mon[32335]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:50.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.427+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc228075cc0 con 0x7fc2281027b0 2024-06-06T08:38:50.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.428+0000 7fc21f7fe700 1 -- 172.21.15.27:0/4093541152 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc21809bfe0 con 0x7fc2281027b0 2024-06-06T08:38:50.427 INFO:teuthology.orchestra.run.smithi027.stdout:163208757325 2024-06-06T08:38:50.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc2140a2cb0 msgr2=0x7fc2140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc2140a2cb0 0x7fc2140a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc22807a920 tx=0x7fc210005fd0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 msgr2=0x7fc228074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 0x7fc228074790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c920 tx=0x7fc21800cc30 comp rx=0 tx=0).stop 2024-06-06T08:38:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 shutdown_connections 2024-06-06T08:38:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fc2140a2cb0 0x7fc2140a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228108650 0x7fc228075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2281027b0 0x7fc228074790 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 --2- 172.21.15.27:0/4093541152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc228103170 0x7fc228074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 >> 172.21.15.27:0/4093541152 conn(0x7fc22807a420 msgr2=0x7fc228101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 shutdown_connections 2024-06-06T08:38:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:50.430+0000 7fc22cc1d700 1 -- 172.21.15.27:0/4093541152 wait complete. 2024-06-06T08:38:50.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:50 smithi203 ceph-mon[36579]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:50.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:50 smithi176 ceph-mon[36786]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:38:50.979 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645358 got 309237645364 for osd.10 2024-06-06T08:38:50.979 DEBUG:teuthology.parallel:result is None 2024-06-06T08:38:51.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:51 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3025455471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T08:38:51.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:51 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4093541152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T08:38:51.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:51 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3025455471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T08:38:51.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:51 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4093541152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T08:38:51.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:51 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3025455471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T08:38:51.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:51 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4093541152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T08:38:52.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:52 smithi027 ceph-mon[32335]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:38:52.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:52 smithi203 ceph-mon[36579]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:52.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:52 smithi176 ceph-mon[36786]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:38:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.679+0000 7fe4d4327700 1 -- 172.21.15.27:0/2212963221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 msgr2=0x7fe4cc1076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.679+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2212963221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 0x7fe4cc1076f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009990 tx=0x7fe4c8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.680+0000 7fe4d4327700 1 -- 172.21.15.27:0/2212963221 shutdown_connections 2024-06-06T08:38:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.680+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2212963221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4cc1066f0 0x7fe4cc10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.680+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2212963221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 0x7fe4cc1076f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.680+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2212963221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4cc107cc0 0x7fe4cc106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.680+0000 7fe4d4327700 1 -- 172.21.15.27:0/2212963221 >> 172.21.15.27:0/2212963221 conn(0x7fe4cc076600 msgr2=0x7fe4cc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.680+0000 7fe4d4327700 1 -- 172.21.15.27:0/2212963221 shutdown_connections 2024-06-06T08:38:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.680+0000 7fe4d4327700 1 -- 172.21.15.27:0/2212963221 wait complete. 2024-06-06T08:38:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.680+0000 7fe4d4327700 1 Processor -- start 2024-06-06T08:38:52.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.680+0000 7fe4d4327700 1 -- start start 2024-06-06T08:38:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d4327700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4cc1066f0 0x7fe4cc19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:52.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d4327700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 0x7fe4cc19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d4327700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4cc107cc0 0x7fe4cc19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d4327700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc111730 con 0x7fe4cc107310 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d4327700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc1115b0 con 0x7fe4cc107cc0 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d4327700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc1118b0 con 0x7fe4cc1066f0 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d18c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 0x7fe4cc19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d18c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 0x7fe4cc19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59506/0 (socket says 172.21.15.27:59506) 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d18c2700 1 -- 172.21.15.27:0/2787014976 learned_addr learned my addr 172.21.15.27:0/2787014976 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d20c3700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4cc1066f0 0x7fe4cc19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d28c4700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4cc107cc0 0x7fe4cc19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d18c2700 1 -- 172.21.15.27:0/2787014976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4cc1066f0 msgr2=0x7fe4cc19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:52.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d18c2700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4cc1066f0 0x7fe4cc19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d18c2700 1 -- 172.21.15.27:0/2787014976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4cc107cc0 msgr2=0x7fe4cc19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d18c2700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4cc107cc0 0x7fe4cc19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d18c2700 1 -- 172.21.15.27:0/2787014976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc19fe80 con 0x7fe4cc107310 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d18c2700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 0x7fe4cc19b3b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00b500 tx=0x7fe4bc00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4c37fe700 1 -- 172.21.15.27:0/2787014976 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc015020 con 0x7fe4cc107310 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4c37fe700 1 -- 172.21.15.27:0/2787014976 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc011d10 con 0x7fe4cc107310 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc1a0170 con 0x7fe4cc107310 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4c37fe700 1 -- 172.21.15.27:0/2787014976 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00f2e0 con 0x7fe4cc107310 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.681+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1a79f0 con 0x7fe4cc107310 2024-06-06T08:38:52.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.683+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc069180 con 0x7fe4cc107310 2024-06-06T08:38:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.684+0000 7fe4c37fe700 1 -- 172.21.15.27:0/2787014976 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe4bc018020 con 0x7fe4cc107310 2024-06-06T08:38:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.684+0000 7fe4c37fe700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4b80a2cb0 0x7fe4b80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.684+0000 7fe4d20c3700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4b80a2cb0 0x7fe4b80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:52.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.684+0000 7fe4c37fe700 1 -- 172.21.15.27:0/2787014976 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fe4bc0ced10 con 0x7fe4cc107310 2024-06-06T08:38:52.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.685+0000 7fe4d20c3700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4b80a2cb0 0x7fe4b80a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009750 tx=0x7fe4c800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:52.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.689+0000 7fe4c37fe700 1 -- 172.21.15.27:0/2787014976 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe4bc01f020 con 0x7fe4cc107310 2024-06-06T08:38:52.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.912+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe4cc02dca0 con 0x7fe4cc107310 2024-06-06T08:38:52.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.912+0000 7fe4c37fe700 1 -- 172.21.15.27:0/2787014976 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe4bc09b850 con 0x7fe4cc107310 2024-06-06T08:38:52.911 INFO:teuthology.orchestra.run.smithi027.stdout:77309411421 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4b80a2cb0 msgr2=0x7fe4b80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4b80a2cb0 0x7fe4b80a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009750 tx=0x7fe4c800b540 comp rx=0 tx=0).stop 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 msgr2=0x7fe4cc19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 0x7fe4cc19b3b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00b500 tx=0x7fe4bc00b8c0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 shutdown_connections 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4b80a2cb0 0x7fe4b80a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4cc107cc0 0x7fe4cc19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4cc1066f0 0x7fe4cc19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 --2- 172.21.15.27:0/2787014976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4cc107310 0x7fe4cc19b3b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 >> 172.21.15.27:0/2787014976 conn(0x7fe4cc076600 msgr2=0x7fe4cc10dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:52.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 shutdown_connections 2024-06-06T08:38:52.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:52.915+0000 7fe4d4327700 1 -- 172.21.15.27:0/2787014976 wait complete. 2024-06-06T08:38:53.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:53 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2787014976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T08:38:53.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:53 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2787014976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T08:38:53.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:53 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2787014976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T08:38:54.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:54 smithi027 ceph-mon[32335]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:38:54.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.479+0000 7fd26d40b700 1 -- 172.21.15.27:0/3431332342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26806abc0 msgr2=0x7fd26806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.479+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3431332342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26806abc0 0x7fd26806b040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009990 tx=0x7fd25c009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.479+0000 7fd26d40b700 1 -- 172.21.15.27:0/3431332342 shutdown_connections 2024-06-06T08:38:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.479+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3431332342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd26806b580 0x7fd26810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.479+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3431332342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd268108500 0x7fd2681088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.479+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3431332342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26806abc0 0x7fd26806b040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.479+0000 7fd26d40b700 1 -- 172.21.15.27:0/3431332342 >> 172.21.15.27:0/3431332342 conn(0x7fd268078340 msgr2=0x7fd268078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.479+0000 7fd26d40b700 1 -- 172.21.15.27:0/3431332342 shutdown_connections 2024-06-06T08:38:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.480+0000 7fd26d40b700 1 -- 172.21.15.27:0/3431332342 wait complete. 2024-06-06T08:38:54.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.480+0000 7fd26d40b700 1 Processor -- start 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd26d40b700 1 -- start start 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd26d40b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd26806abc0 0x7fd26819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd26d40b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26806b580 0x7fd26819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd26d40b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd268108500 0x7fd2681989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd26d40b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd268112780 con 0x7fd26806b580 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd26d40b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd268112600 con 0x7fd26806abc0 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd26d40b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd268112900 con 0x7fd268108500 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd2667fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26806b580 0x7fd26819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd2667fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26806b580 0x7fd26819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59524/0 (socket says 172.21.15.27:59524) 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd2667fc700 1 -- 172.21.15.27:0/3748352807 learned_addr learned my addr 172.21.15.27:0/3748352807 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd266ffd700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd26806abc0 0x7fd26819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.481+0000 7fd2677fe700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd268108500 0x7fd2681989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd266ffd700 1 -- 172.21.15.27:0/3748352807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd268108500 msgr2=0x7fd2681989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd266ffd700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd268108500 0x7fd2681989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd266ffd700 1 -- 172.21.15.27:0/3748352807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26806b580 msgr2=0x7fd26819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd266ffd700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26806b580 0x7fd26819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd266ffd700 1 -- 172.21.15.27:0/3748352807 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd268199140 con 0x7fd26806abc0 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd266ffd700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd26806abc0 0x7fd26819e820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c970 tx=0x7fd25800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd24ffff700 1 -- 172.21.15.27:0/3748352807 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258014070 con 0x7fd26806abc0 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd268199430 con 0x7fd26806abc0 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd24ffff700 1 -- 172.21.15.27:0/3748352807 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd258004440 con 0x7fd26806abc0 2024-06-06T08:38:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd24ffff700 1 -- 172.21.15.27:0/3748352807 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258007b60 con 0x7fd26806abc0 2024-06-06T08:38:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.482+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2681a56f0 con 0x7fd26806abc0 2024-06-06T08:38:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.483+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd268069180 con 0x7fd26806abc0 2024-06-06T08:38:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.485+0000 7fd24ffff700 1 -- 172.21.15.27:0/3748352807 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd258017070 con 0x7fd26806abc0 2024-06-06T08:38:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.485+0000 7fd24ffff700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2500a2d30 0x7fd2500a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.486+0000 7fd24ffff700 1 -- 172.21.15.27:0/3748352807 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fd2580cf090 con 0x7fd26806abc0 2024-06-06T08:38:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.486+0000 7fd2667fc700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2500a2d30 0x7fd2500a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.486+0000 7fd2667fc700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2500a2d30 0x7fd2500a51f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd268078940 tx=0x7fd25c009630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:54.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.491+0000 7fd24ffff700 1 -- 172.21.15.27:0/3748352807 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd25801e020 con 0x7fd26806abc0 2024-06-06T08:38:54.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:54 smithi203 ceph-mon[36579]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:38:54.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:54 smithi176 ceph-mon[36786]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:38:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.681+0000 7fb138e13700 1 -- 172.21.15.27:0/3329886854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1341073c0 msgr2=0x7fb1341077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.681+0000 7fb138e13700 1 --2- 172.21.15.27:0/3329886854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1341073c0 0x7fb1341077a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb124009990 tx=0x7fb124009ca0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.681+0000 7fb138e13700 1 -- 172.21.15.27:0/3329886854 shutdown_connections 2024-06-06T08:38:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.681+0000 7fb138e13700 1 --2- 172.21.15.27:0/3329886854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb134101d40 0x7fb13410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.681+0000 7fb138e13700 1 --2- 172.21.15.27:0/3329886854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1341073c0 0x7fb1341077a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.681+0000 7fb138e13700 1 --2- 172.21.15.27:0/3329886854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb134101380 0x7fb134101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.681+0000 7fb138e13700 1 -- 172.21.15.27:0/3329886854 >> 172.21.15.27:0/3329886854 conn(0x7fb134078330 msgr2=0x7fb134078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.681+0000 7fb138e13700 1 -- 172.21.15.27:0/3329886854 shutdown_connections 2024-06-06T08:38:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.681+0000 7fb138e13700 1 -- 172.21.15.27:0/3329886854 wait complete. 2024-06-06T08:38:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb138e13700 1 Processor -- start 2024-06-06T08:38:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb138e13700 1 -- start start 2024-06-06T08:38:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb138e13700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134101380 0x7fb13419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb138e13700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb134101d40 0x7fb13419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb138e13700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1341073c0 0x7fb134198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb138e13700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb134111630 con 0x7fb134101380 2024-06-06T08:38:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb138e13700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341114b0 con 0x7fb134101d40 2024-06-06T08:38:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb138e13700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341117b0 con 0x7fb1341073c0 2024-06-06T08:38:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb13259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134101380 0x7fb13419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb13259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134101380 0x7fb13419e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59544/0 (socket says 172.21.15.27:59544) 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.682+0000 7fb13259c700 1 -- 172.21.15.27:0/1933676553 learned_addr learned my addr 172.21.15.27:0/1933676553 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb132d9d700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1341073c0 0x7fb134198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb131d9b700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb134101d40 0x7fb13419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb13259c700 1 -- 172.21.15.27:0/1933676553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1341073c0 msgr2=0x7fb134198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb13259c700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1341073c0 0x7fb134198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb13259c700 1 -- 172.21.15.27:0/1933676553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb134101d40 msgr2=0x7fb13419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb13259c700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb134101d40 0x7fb13419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb13259c700 1 -- 172.21.15.27:0/1933676553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb134199060 con 0x7fb134101380 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb132d9d700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1341073c0 0x7fb134198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb131d9b700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb134101d40 0x7fb13419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb13259c700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134101380 0x7fb13419e740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb1240038b0 tx=0x7fb124005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb1237fe700 1 -- 172.21.15.27:0/1933676553 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124027070 con 0x7fb134101380 2024-06-06T08:38:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb1237fe700 1 -- 172.21.15.27:0/1933676553 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1240040d0 con 0x7fb134101380 2024-06-06T08:38:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb1237fe700 1 -- 172.21.15.27:0/1933676553 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124022dd0 con 0x7fb134101380 2024-06-06T08:38:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.683+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1341992f0 con 0x7fb134101380 2024-06-06T08:38:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.684+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1341a5610 con 0x7fb134101380 2024-06-06T08:38:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.686+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb134102a70 con 0x7fb134101380 2024-06-06T08:38:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.686+0000 7fb1237fe700 1 -- 172.21.15.27:0/1933676553 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb12402a070 con 0x7fb134101380 2024-06-06T08:38:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.687+0000 7fb1237fe700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb11c0a2cb0 0x7fb11c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.687+0000 7fb1237fe700 1 -- 172.21.15.27:0/1933676553 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fb1240d01b0 con 0x7fb134101380 2024-06-06T08:38:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.688+0000 7fb131d9b700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb11c0a2cb0 0x7fb11c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.688+0000 7fb131d9b700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb11c0a2cb0 0x7fb11c0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb128005fd0 tx=0x7fb128005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:54.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.696+0000 7fb1237fe700 1 -- 172.21.15.27:0/1933676553 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb124013070 con 0x7fb134101380 2024-06-06T08:38:54.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.708+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd268199f10 con 0x7fd26806abc0 2024-06-06T08:38:54.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.708+0000 7fd24ffff700 1 -- 172.21.15.27:0/3748352807 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd25809bac0 con 0x7fd26806abc0 2024-06-06T08:38:54.707 INFO:teuthology.orchestra.run.smithi027.stdout:34359738471 2024-06-06T08:38:54.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2500a2d30 msgr2=0x7fd2500a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2500a2d30 0x7fd2500a51f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd268078940 tx=0x7fd25c009630 comp rx=0 tx=0).stop 2024-06-06T08:38:54.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd26806abc0 msgr2=0x7fd26819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd26806abc0 0x7fd26819e820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c970 tx=0x7fd25800cc80 comp rx=0 tx=0).stop 2024-06-06T08:38:54.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 shutdown_connections 2024-06-06T08:38:54.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2500a2d30 0x7fd2500a51f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd268108500 0x7fd2681989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd26806abc0 0x7fd26819e820 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 --2- 172.21.15.27:0/3748352807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26806b580 0x7fd26819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 >> 172.21.15.27:0/3748352807 conn(0x7fd268078340 msgr2=0x7fd268100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:54.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 shutdown_connections 2024-06-06T08:38:54.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.711+0000 7fd26d40b700 1 -- 172.21.15.27:0/3748352807 wait complete. 2024-06-06T08:38:54.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.914+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb1341a57e0 con 0x7fb134101380 2024-06-06T08:38:54.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.914+0000 7fb1237fe700 1 -- 172.21.15.27:0/1933676553 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb1240bc020 con 0x7fb134101380 2024-06-06T08:38:54.913 INFO:teuthology.orchestra.run.smithi027.stdout:343597383727 2024-06-06T08:38:54.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb11c0a2cb0 msgr2=0x7fb11c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb11c0a2cb0 0x7fb11c0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb128005fd0 tx=0x7fb128005e20 comp rx=0 tx=0).stop 2024-06-06T08:38:54.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134101380 msgr2=0x7fb13419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134101380 0x7fb13419e740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb1240038b0 tx=0x7fb124005590 comp rx=0 tx=0).stop 2024-06-06T08:38:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 shutdown_connections 2024-06-06T08:38:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb11c0a2cb0 0x7fb11c0a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1341073c0 0x7fb134198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134101380 0x7fb13419e740 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 --2- 172.21.15.27:0/1933676553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb134101d40 0x7fb13419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 >> 172.21.15.27:0/1933676553 conn(0x7fb134078330 msgr2=0x7fb134100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 shutdown_connections 2024-06-06T08:38:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:54.917+0000 7fb138e13700 1 -- 172.21.15.27:0/1933676553 wait complete. 2024-06-06T08:38:55.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:55 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3748352807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T08:38:55.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:55 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1933676553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T08:38:55.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:55 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3748352807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T08:38:55.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:55 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1933676553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T08:38:55.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:55 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3748352807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T08:38:55.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:55 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1933676553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T08:38:55.579 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757320 got 163208757325 for osd.5 2024-06-06T08:38:55.579 DEBUG:teuthology.parallel:result is None 2024-06-06T08:38:56.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:56 smithi027 ceph-mon[32335]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:56.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:56 smithi203 ceph-mon[36579]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:56.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:56 smithi176 ceph-mon[36786]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:38:56.832 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738465 got 34359738471 for osd.0 2024-06-06T08:38:56.832 DEBUG:teuthology.parallel:result is None 2024-06-06T08:38:57.574 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411416 got 77309411421 for osd.2 2024-06-06T08:38:57.574 DEBUG:teuthology.parallel:result is None 2024-06-06T08:38:58.000 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383723 got 343597383727 for osd.11 2024-06-06T08:38:58.000 DEBUG:teuthology.parallel:result is None 2024-06-06T08:38:58.233 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:58 smithi027 ceph-mon[32335]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:58.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:58 smithi203 ceph-mon[36579]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:58.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:58 smithi176 ceph-mon[36786]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:38:58.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.509+0000 7fcca6b5f700 1 -- 172.21.15.27:0/504408819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca006ab30 msgr2=0x7fcca010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.509+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/504408819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca006ab30 0x7fcca010f320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800b0a0 tx=0x7fcc9800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.509+0000 7fcca6b5f700 1 -- 172.21.15.27:0/504408819 shutdown_connections 2024-06-06T08:38:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.509+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/504408819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca006ab30 0x7fcca010f320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.509+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/504408819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca0108500 0x7fcca01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.509+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/504408819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcca006a170 0x7fcca006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.509+0000 7fcca6b5f700 1 -- 172.21.15.27:0/504408819 >> 172.21.15.27:0/504408819 conn(0x7fcca0078340 msgr2=0x7fcca0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.509+0000 7fcca6b5f700 1 -- 172.21.15.27:0/504408819 shutdown_connections 2024-06-06T08:38:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.509+0000 7fcca6b5f700 1 -- 172.21.15.27:0/504408819 wait complete. 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca6b5f700 1 Processor -- start 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca6b5f700 1 -- start start 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca6b5f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcca006a170 0x7fcca019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca6b5f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca006ab30 0x7fcca019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca6b5f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca0108500 0x7fcca0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca6b5f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcca01127d0 con 0x7fcca0108500 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca6b5f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcca0112650 con 0x7fcca006a170 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca6b5f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcca0112950 con 0x7fcca006ab30 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca50fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca0108500 0x7fcca0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca50fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca0108500 0x7fcca0198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59558/0 (socket says 172.21.15.27:59558) 2024-06-06T08:38:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca50fc700 1 -- 172.21.15.27:0/3002318080 learned_addr learned my addr 172.21.15.27:0/3002318080 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcca48fb700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcca006a170 0x7fcca019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.510+0000 7fcc9ffff700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca006ab30 0x7fcca019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcca50fc700 1 -- 172.21.15.27:0/3002318080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca006ab30 msgr2=0x7fcca019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcca50fc700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca006ab30 0x7fcca019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcca50fc700 1 -- 172.21.15.27:0/3002318080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcca006a170 msgr2=0x7fcca019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcca50fc700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcca006a170 0x7fcca019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcca50fc700 1 -- 172.21.15.27:0/3002318080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca0199190 con 0x7fcca0108500 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcc9ffff700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca006ab30 0x7fcca019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcca50fc700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca0108500 0x7fcca0198a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800b070 tx=0x7fcc98004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcc9dffb700 1 -- 172.21.15.27:0/3002318080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98021070 con 0x7fcca0108500 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcc9dffb700 1 -- 172.21.15.27:0/3002318080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc98003c00 con 0x7fcca0108500 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca0199420 con 0x7fcca0108500 2024-06-06T08:38:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcc9dffb700 1 -- 172.21.15.27:0/3002318080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98004850 con 0x7fcca0108500 2024-06-06T08:38:58.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.511+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca0101390 con 0x7fcca0108500 2024-06-06T08:38:58.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.512+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca010a9e0 con 0x7fcca0108500 2024-06-06T08:38:58.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.514+0000 7fcc9dffb700 1 -- 172.21.15.27:0/3002318080 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcc98003db0 con 0x7fcca0108500 2024-06-06T08:38:58.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.514+0000 7fcc9dffb700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcc900a2d30 0x7fcc900a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:38:58.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.514+0000 7fcca48fb700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcc900a2d30 0x7fcc900a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:38:58.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.514+0000 7fcc9dffb700 1 -- 172.21.15.27:0/3002318080 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fcc9800f020 con 0x7fcca0108500 2024-06-06T08:38:58.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.515+0000 7fcca48fb700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcc900a2d30 0x7fcc900a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcca0199b70 tx=0x7fcc9400f7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:38:58.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.519+0000 7fcc9dffb700 1 -- 172.21.15.27:0/3002318080 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcc9802a020 con 0x7fcca0108500 2024-06-06T08:38:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.729+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fcca0199f60 con 0x7fcca0108500 2024-06-06T08:38:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.729+0000 7fcc9dffb700 1 -- 172.21.15.27:0/3002318080 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcc9809d490 con 0x7fcca0108500 2024-06-06T08:38:58.729 INFO:teuthology.orchestra.run.smithi027.stdout:287762808890 2024-06-06T08:38:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.731+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcc900a2d30 msgr2=0x7fcc900a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.731+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcc900a2d30 0x7fcc900a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcca0199b70 tx=0x7fcc9400f7d0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.731+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca0108500 msgr2=0x7fcca0198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:38:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.731+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca0108500 0x7fcca0198a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800b070 tx=0x7fcc98004cf0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.732+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 shutdown_connections 2024-06-06T08:38:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.732+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fcc900a2d30 0x7fcc900a51f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.732+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcca0108500 0x7fcca0198a30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.732+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcca006a170 0x7fcca019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.732+0000 7fcca6b5f700 1 --2- 172.21.15.27:0/3002318080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca006ab30 0x7fcca019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:38:58.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.732+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 >> 172.21.15.27:0/3002318080 conn(0x7fcca0078340 msgr2=0x7fcca00fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:38:58.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.732+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 shutdown_connections 2024-06-06T08:38:58.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:38:58.732+0000 7fcca6b5f700 1 -- 172.21.15.27:0/3002318080 wait complete. 2024-06-06T08:38:59.397 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:38:59 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3002318080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T08:38:59.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:38:59 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3002318080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T08:38:59.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:38:59 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3002318080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T08:38:59.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808883 got 287762808890 for osd.9 2024-06-06T08:38:59.988 DEBUG:teuthology.parallel:result is None 2024-06-06T08:39:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.184+0000 7ff6c0e35700 1 -- 172.21.15.27:0/4000364327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101d50 msgr2=0x7ff6bc10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.184+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/4000364327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101d50 0x7ff6bc10e240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009990 tx=0x7ff6b4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.184+0000 7ff6c0e35700 1 -- 172.21.15.27:0/4000364327 shutdown_connections 2024-06-06T08:39:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.184+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/4000364327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101d50 0x7ff6bc10e240 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.184+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/4000364327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6bc107400 0x7ff6bc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.184+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/4000364327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6bc101390 0x7ff6bc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.184+0000 7ff6c0e35700 1 -- 172.21.15.27:0/4000364327 >> 172.21.15.27:0/4000364327 conn(0x7ff6bc078340 msgr2=0x7ff6bc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.184+0000 7ff6c0e35700 1 -- 172.21.15.27:0/4000364327 shutdown_connections 2024-06-06T08:39:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.184+0000 7ff6c0e35700 1 -- 172.21.15.27:0/4000364327 wait complete. 2024-06-06T08:39:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6c0e35700 1 Processor -- start 2024-06-06T08:39:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6c0e35700 1 -- start start 2024-06-06T08:39:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6c0e35700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101390 0x7ff6bc103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6c0e35700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6bc101d50 0x7ff6bc1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6c0e35700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6bc107400 0x7ff6bc103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:00.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6c0e35700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc111670 con 0x7ff6bc101390 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6c0e35700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1114f0 con 0x7ff6bc101d50 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6c0e35700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1117f0 con 0x7ff6bc107400 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6ba59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101390 0x7ff6bc103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6ba59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101390 0x7ff6bc103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40736/0 (socket says 172.21.15.27:40736) 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.185+0000 7ff6ba59c700 1 -- 172.21.15.27:0/237655804 learned_addr learned my addr 172.21.15.27:0/237655804 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.186+0000 7ff6bad9d700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6bc107400 0x7ff6bc103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.186+0000 7ff6b1d9b700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6bc101d50 0x7ff6bc1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.186+0000 7ff6ba59c700 1 -- 172.21.15.27:0/237655804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6bc107400 msgr2=0x7ff6bc103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.186+0000 7ff6ba59c700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6bc107400 0x7ff6bc103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.186+0000 7ff6ba59c700 1 -- 172.21.15.27:0/237655804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6bc101d50 msgr2=0x7ff6bc1035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.186+0000 7ff6ba59c700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6bc101d50 0x7ff6bc1035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.186+0000 7ff6ba59c700 1 -- 172.21.15.27:0/237655804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc1a70f0 con 0x7ff6bc101390 2024-06-06T08:39:00.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.186+0000 7ff6b1d9b700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6bc101d50 0x7ff6bc1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:00.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.186+0000 7ff6bad9d700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6bc107400 0x7ff6bc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:00.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.187+0000 7ff6ba59c700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101390 0x7ff6bc103070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00ada0 tx=0x7ff6ac00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:00.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.187+0000 7ff6b3fff700 1 -- 172.21.15.27:0/237655804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac017020 con 0x7ff6bc101390 2024-06-06T08:39:00.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.187+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6bc1a73e0 con 0x7ff6bc101390 2024-06-06T08:39:00.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.187+0000 7ff6b3fff700 1 -- 172.21.15.27:0/237655804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac004bb0 con 0x7ff6bc101390 2024-06-06T08:39:00.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.187+0000 7ff6b3fff700 1 -- 172.21.15.27:0/237655804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0055d0 con 0x7ff6bc101390 2024-06-06T08:39:00.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.187+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6bc1a7970 con 0x7ff6bc101390 2024-06-06T08:39:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.188+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6bc069180 con 0x7ff6bc101390 2024-06-06T08:39:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.189+0000 7ff6b3fff700 1 -- 172.21.15.27:0/237655804 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff6ac005790 con 0x7ff6bc101390 2024-06-06T08:39:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.190+0000 7ff6b3fff700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff6a80a2ca0 0x7ff6a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.190+0000 7ff6b3fff700 1 -- 172.21.15.27:0/237655804 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7ff6ac0cf1e0 con 0x7ff6bc101390 2024-06-06T08:39:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.190+0000 7ff6b1d9b700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff6a80a2ca0 0x7ff6a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.191+0000 7ff6b1d9b700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff6a80a2ca0 0x7ff6a80a5160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005fd0 tx=0x7ff6a4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:00.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.195+0000 7ff6b3fff700 1 -- 172.21.15.27:0/237655804 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff6ac021020 con 0x7ff6bc101390 2024-06-06T08:39:00.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:00 smithi027 ceph-mon[32335]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-06-06T08:39:00.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:00 smithi027 ceph-mon[32335]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T08:39:00.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:39:00.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:00 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:39:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.404+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff6bc1a7bd0 con 0x7ff6bc101390 2024-06-06T08:39:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.404+0000 7ff6b3fff700 1 -- 172.21.15.27:0/237655804 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff6ac09bdc0 con 0x7ff6bc101390 2024-06-06T08:39:00.404 INFO:teuthology.orchestra.run.smithi027.stdout:197568495690 2024-06-06T08:39:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff6a80a2ca0 msgr2=0x7ff6a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff6a80a2ca0 0x7ff6a80a5160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005fd0 tx=0x7ff6a4005e30 comp rx=0 tx=0).stop 2024-06-06T08:39:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101390 msgr2=0x7ff6bc103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101390 0x7ff6bc103070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00ada0 tx=0x7ff6ac00c330 comp rx=0 tx=0).stop 2024-06-06T08:39:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 shutdown_connections 2024-06-06T08:39:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7ff6a80a2ca0 0x7ff6a80a5160 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6bc107400 0x7ff6bc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6bc101390 0x7ff6bc103070 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 --2- 172.21.15.27:0/237655804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6bc101d50 0x7ff6bc1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.406+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 >> 172.21.15.27:0/237655804 conn(0x7ff6bc078340 msgr2=0x7ff6bc1004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.407+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 shutdown_connections 2024-06-06T08:39:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:00.407+0000 7ff6c0e35700 1 -- 172.21.15.27:0/237655804 wait complete. 2024-06-06T08:39:00.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:00 smithi203 ceph-mon[36579]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-06-06T08:39:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:00 smithi203 ceph-mon[36579]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T08:39:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:39:00.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:00 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:39:00.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:00 smithi176 ceph-mon[36786]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-06-06T08:39:00.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:00 smithi176 ceph-mon[36786]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T08:39:00.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:39:00.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:00 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:39:01.269 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:01 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/237655804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T08:39:01.297 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495685 got 197568495690 for osd.6 2024-06-06T08:39:01.297 DEBUG:teuthology.parallel:result is None 2024-06-06T08:39:01.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:01 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/237655804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T08:39:01.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:01 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/237655804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 -- 172.21.15.27:0/1439589438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa470107860 msgr2=0x7fa47010f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 --2- 172.21.15.27:0/1439589438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa470107860 0x7fa47010f3e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa46800b0d0 tx=0x7fa46800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 -- 172.21.15.27:0/1439589438 shutdown_connections 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 --2- 172.21.15.27:0/1439589438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa470107860 0x7fa47010f3e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 --2- 172.21.15.27:0/1439589438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa47006adc0 0x7fa47006b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 --2- 172.21.15.27:0/1439589438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa47006b770 0x7fa470107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 -- 172.21.15.27:0/1439589438 >> 172.21.15.27:0/1439589438 conn(0x7fa470078340 msgr2=0x7fa470078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 -- 172.21.15.27:0/1439589438 shutdown_connections 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 -- 172.21.15.27:0/1439589438 wait complete. 2024-06-06T08:39:01.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.815+0000 7fa475c0b700 1 Processor -- start 2024-06-06T08:39:01.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa475c0b700 1 -- start start 2024-06-06T08:39:01.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa475c0b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa47006adc0 0x7fa470101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:01.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa475c0b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa47006b770 0x7fa470101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:01.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa475c0b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa470107860 0x7fa4701022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:01.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa475c0b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa470112780 con 0x7fa47006adc0 2024-06-06T08:39:01.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa475c0b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa470112600 con 0x7fa47006b770 2024-06-06T08:39:01.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa475c0b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa470112900 con 0x7fa470107860 2024-06-06T08:39:01.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa47006adc0 0x7fa470101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:01.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa47006adc0 0x7fa470101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40752/0 (socket says 172.21.15.27:40752) 2024-06-06T08:39:01.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46f7fe700 1 -- 172.21.15.27:0/3425075106 learned_addr learned my addr 172.21.15.27:0/3425075106 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:01.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46effd700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa47006b770 0x7fa470101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:01.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46ffff700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa470107860 0x7fa4701022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:01.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46f7fe700 1 -- 172.21.15.27:0/3425075106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa470107860 msgr2=0x7fa4701022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:01.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46f7fe700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa470107860 0x7fa4701022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:01.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46f7fe700 1 -- 172.21.15.27:0/3425075106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa47006b770 msgr2=0x7fa470101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:01.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46f7fe700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa47006b770 0x7fa470101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:01.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.816+0000 7fa46f7fe700 1 -- 172.21.15.27:0/3425075106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4701a7020 con 0x7fa47006adc0 2024-06-06T08:39:01.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.817+0000 7fa46f7fe700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa47006adc0 0x7fa470101840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c9b0 tx=0x7fa46000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:01.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.817+0000 7fa46cff9700 1 -- 172.21.15.27:0/3425075106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460007b00 con 0x7fa47006adc0 2024-06-06T08:39:01.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.817+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4701a7310 con 0x7fa47006adc0 2024-06-06T08:39:01.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.817+0000 7fa46cff9700 1 -- 172.21.15.27:0/3425075106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46000ce50 con 0x7fa47006adc0 2024-06-06T08:39:01.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.817+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4701a78a0 con 0x7fa47006adc0 2024-06-06T08:39:01.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.817+0000 7fa46cff9700 1 -- 172.21.15.27:0/3425075106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460018660 con 0x7fa47006adc0 2024-06-06T08:39:01.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.818+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa470103be0 con 0x7fa47006adc0 2024-06-06T08:39:01.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.819+0000 7fa46cff9700 1 -- 172.21.15.27:0/3425075106 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa46000f450 con 0x7fa47006adc0 2024-06-06T08:39:01.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.820+0000 7fa46cff9700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4580a2ca0 0x7fa4580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:01.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.820+0000 7fa46effd700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4580a2ca0 0x7fa4580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:01.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.820+0000 7fa46cff9700 1 -- 172.21.15.27:0/3425075106 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fa4600cf530 con 0x7fa47006adc0 2024-06-06T08:39:01.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.820+0000 7fa46effd700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4580a2ca0 0x7fa4580a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa470108b40 tx=0x7fa46400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:01.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:01.823+0000 7fa46cff9700 1 -- 172.21.15.27:0/3425075106 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa460014030 con 0x7fa47006adc0 2024-06-06T08:39:02.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.036+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa4701a7c80 con 0x7fa47006adc0 2024-06-06T08:39:02.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.037+0000 7fa46cff9700 1 -- 172.21.15.27:0/3425075106 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa46009c1d0 con 0x7fa47006adc0 2024-06-06T08:39:02.036 INFO:teuthology.orchestra.run.smithi027.stdout:107374182490 2024-06-06T08:39:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4580a2ca0 msgr2=0x7fa4580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4580a2ca0 0x7fa4580a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa470108b40 tx=0x7fa46400b410 comp rx=0 tx=0).stop 2024-06-06T08:39:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa47006adc0 msgr2=0x7fa470101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa47006adc0 0x7fa470101840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c9b0 tx=0x7fa46000ccc0 comp rx=0 tx=0).stop 2024-06-06T08:39:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 shutdown_connections 2024-06-06T08:39:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fa4580a2ca0 0x7fa4580a5160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa470107860 0x7fa4701022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa47006adc0 0x7fa470101840 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 --2- 172.21.15.27:0/3425075106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa47006b770 0x7fa470101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 >> 172.21.15.27:0/3425075106 conn(0x7fa470078340 msgr2=0x7fa4700ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 shutdown_connections 2024-06-06T08:39:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:02.039+0000 7fa475c0b700 1 -- 172.21.15.27:0/3425075106 wait complete. 2024-06-06T08:39:02.422 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:02 smithi027 ceph-mon[32335]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-06T08:39:02.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:39:02.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:39:02.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:39:02.423 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:02 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3425075106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T08:39:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:02 smithi203 ceph-mon[36579]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-06T08:39:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:39:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:39:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:39:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:02 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3425075106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T08:39:02.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:02 smithi176 ceph-mon[36786]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-06T08:39:02.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:39:02.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:39:02.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:39:02.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:02 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3425075106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T08:39:02.700 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182485 got 107374182490 for osd.3 2024-06-06T08:39:02.700 DEBUG:teuthology.parallel:result is None 2024-06-06T08:39:02.700 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T08:39:02.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T08:39:02.954 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:04.356 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:04 smithi027 ceph-mon[32335]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-06T08:39:04.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:04 smithi203 ceph-mon[36579]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-06T08:39:04.644 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:04 smithi176 ceph-mon[36786]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.613+0000 7f496b53c700 1 -- 172.21.15.27:0/1102822268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101d50 msgr2=0x7f496410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.613+0000 7f496b53c700 1 --2- 172.21.15.27:0/1102822268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101d50 0x7f496410e240 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f495c00b0a0 tx=0x7f495c00b3b0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.613+0000 7f496b53c700 1 -- 172.21.15.27:0/1102822268 shutdown_connections 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.613+0000 7f496b53c700 1 --2- 172.21.15.27:0/1102822268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101d50 0x7f496410e240 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.613+0000 7f496b53c700 1 --2- 172.21.15.27:0/1102822268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4964107400 0x7f49641077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.613+0000 7f496b53c700 1 --2- 172.21.15.27:0/1102822268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4964101390 0x7f4964101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.613+0000 7f496b53c700 1 -- 172.21.15.27:0/1102822268 >> 172.21.15.27:0/1102822268 conn(0x7f4964078340 msgr2=0x7f4964078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.613+0000 7f496b53c700 1 -- 172.21.15.27:0/1102822268 shutdown_connections 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.613+0000 7f496b53c700 1 -- 172.21.15.27:0/1102822268 wait complete. 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.614+0000 7f496b53c700 1 Processor -- start 2024-06-06T08:39:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.614+0000 7f496b53c700 1 -- start start 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f496b53c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101390 0x7f49641999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f496b53c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4964101d50 0x7f4964199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f496b53c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4964107400 0x7f49641a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f496b53c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4964111690 con 0x7f4964101390 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f496b53c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4964111510 con 0x7f4964107400 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f496b53c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4964111810 con 0x7f4964101d50 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f49692d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101390 0x7f49641999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f49692d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101390 0x7f49641999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40776/0 (socket says 172.21.15.27:40776) 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f49692d8700 1 -- 172.21.15.27:0/1341719093 learned_addr learned my addr 172.21.15.27:0/1341719093 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f4968ad7700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4964101d50 0x7f4964199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f4969ad9700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4964107400 0x7f49641a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f49692d8700 1 -- 172.21.15.27:0/1341719093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4964101d50 msgr2=0x7f4964199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f49692d8700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4964101d50 0x7f4964199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f49692d8700 1 -- 172.21.15.27:0/1341719093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4964107400 msgr2=0x7f49641a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f49692d8700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4964107400 0x7f49641a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.615+0000 7f49692d8700 1 -- 172.21.15.27:0/1341719093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49641a4fb0 con 0x7f4964101390 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.616+0000 7f49692d8700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101390 0x7f49641999f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f496000d980 tx=0x7f496000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.616+0000 7f495a7fc700 1 -- 172.21.15.27:0/1341719093 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960017070 con 0x7f4964101390 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.616+0000 7f495a7fc700 1 -- 172.21.15.27:0/1341719093 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4960005020 con 0x7f4964101390 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.616+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49641a51e0 con 0x7f4964101390 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.616+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49641a5660 con 0x7f4964101390 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.617+0000 7f495a7fc700 1 -- 172.21.15.27:0/1341719093 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960009bc0 con 0x7f4964101390 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.617+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4964069180 con 0x7f4964101390 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.618+0000 7f495a7fc700 1 -- 172.21.15.27:0/1341719093 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4960004a30 con 0x7f4964101390 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.619+0000 7f495a7fc700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f49500a2d70 0x7f49500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.619+0000 7f4968ad7700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f49500a2d70 0x7f49500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:04.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.619+0000 7f495a7fc700 1 -- 172.21.15.27:0/1341719093 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f49600cf200 con 0x7f4964101390 2024-06-06T08:39:04.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.620+0000 7f4968ad7700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f49500a2d70 0x7f49500a5230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f49640788d0 tx=0x7f495400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:04.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.624+0000 7f495a7fc700 1 -- 172.21.15.27:0/1341719093 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4960020020 con 0x7f4964101390 2024-06-06T08:39:04.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.827+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49640628d0 con 0x7f49500a2d70 2024-06-06T08:39:04.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.837+0000 7f495a7fc700 1 -- 172.21.15.27:0/1341719093 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340889 (secure 0 0 0) 0x7f49640628d0 con 0x7f49500a2d70 2024-06-06T08:39:04.844 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:04.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f49500a2d70 msgr2=0x7f49500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:04.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f49500a2d70 0x7f49500a5230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f49640788d0 tx=0x7f495400a300 comp rx=0 tx=0).stop 2024-06-06T08:39:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101390 msgr2=0x7f49641999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101390 0x7f49641999f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f496000d980 tx=0x7f496000dc90 comp rx=0 tx=0).stop 2024-06-06T08:39:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 shutdown_connections 2024-06-06T08:39:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f49500a2d70 0x7f49500a5230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4964107400 0x7f49641a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4964101390 0x7f49641999f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 --2- 172.21.15.27:0/1341719093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4964101d50 0x7f4964199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 >> 172.21.15.27:0/1341719093 conn(0x7f4964078340 msgr2=0x7f49640ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 shutdown_connections 2024-06-06T08:39:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:04.840+0000 7f496b53c700 1 -- 172.21.15.27:0/1341719093 wait complete. 2024-06-06T08:39:04.851 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:39:05.727 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-06-06T08:39:04.477082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657062,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":761,"num_read_kb":504,"num_write":359,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358308,"kb_used_data":13316,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583516,"statfs":{"total":1151940427776,"available":1151573520384,"internally_reserved":0,"allocated":13635584,"data_stored":8786168,"data_compressed":495807,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003704"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469315+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:38:58.469315+0000","last_peered":"2024-06-06T08:38:58.469315+0000","last_clean":"2024-06-06T08:38:58.469315+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:38:58.469315+0000","last_undegraded":"2024-06-06T08:38:58.469315+0000","last_fullsized":"2024-06-06T08:38:58.469315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.470122+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:38:58.470122+0000","last_peered":"2024-06-06T08:38:58.470122+0000","last_clean":"2024-06-06T08:38:58.470122+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:38:58.470122+0000","last_undegraded":"2024-06-06T08:38:58.470122+0000","last_fullsized":"2024-06-06T08:38:58.470122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.474795+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:38:58.474795+0000","last_peered":"2024-06-06T08:38:58.474795+0000","last_clean":"2024-06-06T08:38:58.474795+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:38:58.474795+0000","last_undegraded":"2024-06-06T08:38:58.474795+0000","last_fullsized":"2024-06-06T08:38:58.474795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.457919+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:38:58.457919+0000","last_peered":"2024-06-06T08:38:58.457919+0000","last_clean":"2024-06-06T08:38:58.457919+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:38:58.457919+0000","last_undegraded":"2024-06-06T08:38:58.457919+0000","last_fullsized":"2024-06-06T08:38:58.457919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.465850+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:38:58.465850+0000","last_peered":"2024-06-06T08:38:58.465850+0000","last_clean":"2024-06-06T08:38:58.465850+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:38:58.465850+0000","last_undegraded":"2024-06-06T08:38:58.465850+0000","last_fullsized":"2024-06-06T08:38:58.465850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.452855+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:38:58.452855+0000","last_peered":"2024-06-06T08:38:58.452855+0000","last_clean":"2024-06-06T08:38:58.452855+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:38:58.452855+0000","last_undegraded":"2024-06-06T08:38:58.452855+0000","last_fullsized":"2024-06-06T08:38:58.452855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.460268+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:38:58.460268+0000","last_peered":"2024-06-06T08:38:58.460268+0000","last_clean":"2024-06-06T08:38:58.460268+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:38:58.460268+0000","last_undegraded":"2024-06-06T08:38:58.460268+0000","last_fullsized":"2024-06-06T08:38:58.460268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":20,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.474082+0000","last_change":"2024-06-06T08:38:01.335419+0000","last_active":"2024-06-06T08:38:58.474082+0000","last_peered":"2024-06-06T08:38:58.474082+0000","last_clean":"2024-06-06T08:38:58.474082+0000","last_became_active":"2024-06-06T08:38:01.335191+0000","last_became_peered":"2024-06-06T08:38:01.335191+0000","last_unstale":"2024-06-06T08:38:58.474082+0000","last_undegraded":"2024-06-06T08:38:58.474082+0000","last_fullsized":"2024-06-06T08:38:58.474082+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:13.253838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.472465+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:38:58.472465+0000","last_peered":"2024-06-06T08:38:58.472465+0000","last_clean":"2024-06-06T08:38:58.472465+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:38:58.472465+0000","last_undegraded":"2024-06-06T08:38:58.472465+0000","last_fullsized":"2024-06-06T08:38:58.472465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.451391+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:38:58.451391+0000","last_peered":"2024-06-06T08:38:58.451391+0000","last_clean":"2024-06-06T08:38:58.451391+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:38:58.451391+0000","last_undegraded":"2024-06-06T08:38:58.451391+0000","last_fullsized":"2024-06-06T08:38:58.451391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473681+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:38:58.473681+0000","last_peered":"2024-06-06T08:38:58.473681+0000","last_clean":"2024-06-06T08:38:58.473681+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:38:58.473681+0000","last_undegraded":"2024-06-06T08:38:58.473681+0000","last_fullsized":"2024-06-06T08:38:58.473681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.458115+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:38:58.458115+0000","last_peered":"2024-06-06T08:38:58.458115+0000","last_clean":"2024-06-06T08:38:58.458115+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:38:58.458115+0000","last_undegraded":"2024-06-06T08:38:58.458115+0000","last_fullsized":"2024-06-06T08:38:58.458115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.457851+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:38:58.457851+0000","last_peered":"2024-06-06T08:38:58.457851+0000","last_clean":"2024-06-06T08:38:58.457851+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:38:58.457851+0000","last_undegraded":"2024-06-06T08:38:58.457851+0000","last_fullsized":"2024-06-06T08:38:58.457851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.476045+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:38:58.476045+0000","last_peered":"2024-06-06T08:38:58.476045+0000","last_clean":"2024-06-06T08:38:58.476045+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:38:58.476045+0000","last_undegraded":"2024-06-06T08:38:58.476045+0000","last_fullsized":"2024-06-06T08:38:58.476045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469729+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:38:58.469729+0000","last_peered":"2024-06-06T08:38:58.469729+0000","last_clean":"2024-06-06T08:38:58.469729+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:38:58.469729+0000","last_undegraded":"2024-06-06T08:38:58.469729+0000","last_fullsized":"2024-06-06T08:38:58.469729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475711+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:38:58.475711+0000","last_peered":"2024-06-06T08:38:58.475711+0000","last_clean":"2024-06-06T08:38:58.475711+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:38:58.475711+0000","last_undegraded":"2024-06-06T08:38:58.475711+0000","last_fullsized":"2024-06-06T08:38:58.475711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469372+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:38:58.469372+0000","last_peered":"2024-06-06T08:38:58.469372+0000","last_clean":"2024-06-06T08:38:58.469372+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:38:58.469372+0000","last_undegraded":"2024-06-06T08:38:58.469372+0000","last_fullsized":"2024-06-06T08:38:58.469372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475214+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:38:58.475214+0000","last_peered":"2024-06-06T08:38:58.475214+0000","last_clean":"2024-06-06T08:38:58.475214+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:38:58.475214+0000","last_undegraded":"2024-06-06T08:38:58.475214+0000","last_fullsized":"2024-06-06T08:38:58.475214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.461118+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:38:58.461118+0000","last_peered":"2024-06-06T08:38:58.461118+0000","last_clean":"2024-06-06T08:38:58.461118+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:38:58.461118+0000","last_undegraded":"2024-06-06T08:38:58.461118+0000","last_fullsized":"2024-06-06T08:38:58.461118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544879+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:38:01.544879+0000","last_peered":"2024-06-06T08:38:01.544879+0000","last_clean":"2024-06-06T08:38:01.544879+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:38:01.544879+0000","last_undegraded":"2024-06-06T08:38:01.544879+0000","last_fullsized":"2024-06-06T08:38:01.544879+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015106+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:38:01.015106+0000","last_peered":"2024-06-06T08:38:01.015106+0000","last_clean":"2024-06-06T08:38:01.015106+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:38:01.015106+0000","last_undegraded":"2024-06-06T08:38:01.015106+0000","last_fullsized":"2024-06-06T08:38:01.015106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.466121+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:38:58.466121+0000","last_peered":"2024-06-06T08:38:58.466121+0000","last_clean":"2024-06-06T08:38:58.466121+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:38:58.466121+0000","last_undegraded":"2024-06-06T08:38:58.466121+0000","last_fullsized":"2024-06-06T08:38:58.466121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726764+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:38:01.726764+0000","last_peered":"2024-06-06T08:38:01.726764+0000","last_clean":"2024-06-06T08:38:01.726764+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:38:01.726764+0000","last_undegraded":"2024-06-06T08:38:01.726764+0000","last_fullsized":"2024-06-06T08:38:01.726764+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.370495+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:38:01.370495+0000","last_peered":"2024-06-06T08:38:01.370495+0000","last_clean":"2024-06-06T08:38:01.370495+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:38:01.370495+0000","last_undegraded":"2024-06-06T08:38:01.370495+0000","last_fullsized":"2024-06-06T08:38:01.370495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.476257+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:38:58.476257+0000","last_peered":"2024-06-06T08:38:58.476257+0000","last_clean":"2024-06-06T08:38:58.476257+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:38:58.476257+0000","last_undegraded":"2024-06-06T08:38:58.476257+0000","last_fullsized":"2024-06-06T08:38:58.476257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727107+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:38:01.727107+0000","last_peered":"2024-06-06T08:38:01.727107+0000","last_clean":"2024-06-06T08:38:01.727107+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:38:01.727107+0000","last_undegraded":"2024-06-06T08:38:01.727107+0000","last_fullsized":"2024-06-06T08:38:01.727107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012608+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:38:01.012608+0000","last_peered":"2024-06-06T08:38:01.012608+0000","last_clean":"2024-06-06T08:38:01.012608+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:38:01.012608+0000","last_undegraded":"2024-06-06T08:38:01.012608+0000","last_fullsized":"2024-06-06T08:38:01.012608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.472702+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:38:58.472702+0000","last_peered":"2024-06-06T08:38:58.472702+0000","last_clean":"2024-06-06T08:38:58.472702+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:38:58.472702+0000","last_undegraded":"2024-06-06T08:38:58.472702+0000","last_fullsized":"2024-06-06T08:38:58.472702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726959+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:38:01.726959+0000","last_peered":"2024-06-06T08:38:01.726959+0000","last_clean":"2024-06-06T08:38:01.726959+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:38:01.726959+0000","last_undegraded":"2024-06-06T08:38:01.726959+0000","last_fullsized":"2024-06-06T08:38:01.726959+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012568+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:38:01.012568+0000","last_peered":"2024-06-06T08:38:01.012568+0000","last_clean":"2024-06-06T08:38:01.012568+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:38:01.012568+0000","last_undegraded":"2024-06-06T08:38:01.012568+0000","last_fullsized":"2024-06-06T08:38:01.012568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":26,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.471846+0000","last_change":"2024-06-06T08:38:01.728398+0000","last_active":"2024-06-06T08:38:58.471846+0000","last_peered":"2024-06-06T08:38:58.471846+0000","last_clean":"2024-06-06T08:38:58.471846+0000","last_became_active":"2024-06-06T08:38:01.728004+0000","last_became_peered":"2024-06-06T08:38:01.728004+0000","last_unstale":"2024-06-06T08:38:58.471846+0000","last_undegraded":"2024-06-06T08:38:58.471846+0000","last_fullsized":"2024-06-06T08:38:58.471846+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:02:03.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544841+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:38:01.544841+0000","last_peered":"2024-06-06T08:38:01.544841+0000","last_clean":"2024-06-06T08:38:01.544841+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:38:01.544841+0000","last_undegraded":"2024-06-06T08:38:01.544841+0000","last_fullsized":"2024-06-06T08:38:01.544841+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":150,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:00.977727+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:39:00.977727+0000","last_peered":"2024-06-06T08:39:00.977727+0000","last_clean":"2024-06-06T08:39:00.977727+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:39:00.977727+0000","last_undegraded":"2024-06-06T08:39:00.977727+0000","last_fullsized":"2024-06-06T08:39:00.977727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.454797+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:38:58.454797+0000","last_peered":"2024-06-06T08:38:58.454797+0000","last_clean":"2024-06-06T08:38:58.454797+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:38:58.454797+0000","last_undegraded":"2024-06-06T08:38:58.454797+0000","last_fullsized":"2024-06-06T08:38:58.454797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544291+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:38:01.544291+0000","last_peered":"2024-06-06T08:38:01.544291+0000","last_clean":"2024-06-06T08:38:01.544291+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:38:01.544291+0000","last_undegraded":"2024-06-06T08:38:01.544291+0000","last_fullsized":"2024-06-06T08:38:01.544291+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.334462+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:38:01.334462+0000","last_peered":"2024-06-06T08:38:01.334462+0000","last_clean":"2024-06-06T08:38:01.334462+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:38:01.334462+0000","last_undegraded":"2024-06-06T08:38:01.334462+0000","last_fullsized":"2024-06-06T08:38:01.334462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473557+0000","last_change":"2024-06-06T08:38:01.728238+0000","last_active":"2024-06-06T08:38:58.473557+0000","last_peered":"2024-06-06T08:38:58.473557+0000","last_clean":"2024-06-06T08:38:58.473557+0000","last_became_active":"2024-06-06T08:38:01.727844+0000","last_became_peered":"2024-06-06T08:38:01.727844+0000","last_unstale":"2024-06-06T08:38:58.473557+0000","last_undegraded":"2024-06-06T08:38:58.473557+0000","last_fullsized":"2024-06-06T08:38:58.473557+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:19:46.825321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726847+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:38:01.726847+0000","last_peered":"2024-06-06T08:38:01.726847+0000","last_clean":"2024-06-06T08:38:01.726847+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:38:01.726847+0000","last_undegraded":"2024-06-06T08:38:01.726847+0000","last_fullsized":"2024-06-06T08:38:01.726847+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.150302+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:38:01.150302+0000","last_peered":"2024-06-06T08:38:01.150302+0000","last_clean":"2024-06-06T08:38:01.150302+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:38:01.150302+0000","last_undegraded":"2024-06-06T08:38:01.150302+0000","last_fullsized":"2024-06-06T08:38:01.150302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.470392+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:38:58.470392+0000","last_peered":"2024-06-06T08:38:58.470392+0000","last_clean":"2024-06-06T08:38:58.470392+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:38:58.470392+0000","last_undegraded":"2024-06-06T08:38:58.470392+0000","last_fullsized":"2024-06-06T08:38:58.470392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.011847+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:38:01.011847+0000","last_peered":"2024-06-06T08:38:01.011847+0000","last_clean":"2024-06-06T08:38:01.011847+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:38:01.011847+0000","last_undegraded":"2024-06-06T08:38:01.011847+0000","last_fullsized":"2024-06-06T08:38:01.011847+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015710+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:38:01.015710+0000","last_peered":"2024-06-06T08:38:01.015710+0000","last_clean":"2024-06-06T08:38:01.015710+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:38:01.015710+0000","last_undegraded":"2024-06-06T08:38:01.015710+0000","last_fullsized":"2024-06-06T08:38:01.015710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.474208+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:38:58.474208+0000","last_peered":"2024-06-06T08:38:58.474208+0000","last_clean":"2024-06-06T08:38:58.474208+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:38:58.474208+0000","last_undegraded":"2024-06-06T08:38:58.474208+0000","last_fullsized":"2024-06-06T08:38:58.474208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.369716+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:38:01.369716+0000","last_peered":"2024-06-06T08:38:01.369716+0000","last_clean":"2024-06-06T08:38:01.369716+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:38:01.369716+0000","last_undegraded":"2024-06-06T08:38:01.369716+0000","last_fullsized":"2024-06-06T08:38:01.369716+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":147,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:01.978723+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:39:01.978723+0000","last_peered":"2024-06-06T08:39:01.978723+0000","last_clean":"2024-06-06T08:39:01.978723+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:39:01.978723+0000","last_undegraded":"2024-06-06T08:39:01.978723+0000","last_fullsized":"2024-06-06T08:39:01.978723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473111+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:38:58.473111+0000","last_peered":"2024-06-06T08:38:58.473111+0000","last_clean":"2024-06-06T08:38:58.473111+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:38:58.473111+0000","last_undegraded":"2024-06-06T08:38:58.473111+0000","last_fullsized":"2024-06-06T08:38:58.473111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.014302+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:38:01.014302+0000","last_peered":"2024-06-06T08:38:01.014302+0000","last_clean":"2024-06-06T08:38:01.014302+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:38:01.014302+0000","last_undegraded":"2024-06-06T08:38:01.014302+0000","last_fullsized":"2024-06-06T08:38:01.014302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012898+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:38:01.012898+0000","last_peered":"2024-06-06T08:38:01.012898+0000","last_clean":"2024-06-06T08:38:01.012898+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:38:01.012898+0000","last_undegraded":"2024-06-06T08:38:01.012898+0000","last_fullsized":"2024-06-06T08:38:01.012898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.470283+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:38:58.470283+0000","last_peered":"2024-06-06T08:38:58.470283+0000","last_clean":"2024-06-06T08:38:58.470283+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:38:58.470283+0000","last_undegraded":"2024-06-06T08:38:58.470283+0000","last_fullsized":"2024-06-06T08:38:58.470283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727272+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:38:01.727272+0000","last_peered":"2024-06-06T08:38:01.727272+0000","last_clean":"2024-06-06T08:38:01.727272+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:38:01.727272+0000","last_undegraded":"2024-06-06T08:38:01.727272+0000","last_fullsized":"2024-06-06T08:38:01.727272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015670+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:38:01.015670+0000","last_peered":"2024-06-06T08:38:01.015670+0000","last_clean":"2024-06-06T08:38:01.015670+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:38:01.015670+0000","last_undegraded":"2024-06-06T08:38:01.015670+0000","last_fullsized":"2024-06-06T08:38:01.015670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473370+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:38:58.473370+0000","last_peered":"2024-06-06T08:38:58.473370+0000","last_clean":"2024-06-06T08:38:58.473370+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:38:58.473370+0000","last_undegraded":"2024-06-06T08:38:58.473370+0000","last_fullsized":"2024-06-06T08:38:58.473370+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.334196+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:38:01.334196+0000","last_peered":"2024-06-06T08:38:01.334196+0000","last_clean":"2024-06-06T08:38:01.334196+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:38:01.334196+0000","last_undegraded":"2024-06-06T08:38:01.334196+0000","last_fullsized":"2024-06-06T08:38:01.334196+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"99'36","reported_seq":306,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:02.980202+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:39:02.980202+0000","last_peered":"2024-06-06T08:39:02.980202+0000","last_clean":"2024-06-06T08:39:02.980202+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:39:02.980202+0000","last_undegraded":"2024-06-06T08:39:02.980202+0000","last_fullsized":"2024-06-06T08:39:02.980202+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62319,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":209,"num_read_kb":166,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.458423+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:38:58.458423+0000","last_peered":"2024-06-06T08:38:58.458423+0000","last_clean":"2024-06-06T08:38:58.458423+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:38:58.458423+0000","last_undegraded":"2024-06-06T08:38:58.458423+0000","last_fullsized":"2024-06-06T08:38:58.458423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.016007+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:38:01.016007+0000","last_peered":"2024-06-06T08:38:01.016007+0000","last_clean":"2024-06-06T08:38:01.016007+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:38:01.016007+0000","last_undegraded":"2024-06-06T08:38:01.016007+0000","last_fullsized":"2024-06-06T08:38:01.016007+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.543888+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:38:01.543888+0000","last_peered":"2024-06-06T08:38:01.543888+0000","last_clean":"2024-06-06T08:38:01.543888+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:38:01.543888+0000","last_undegraded":"2024-06-06T08:38:01.543888+0000","last_fullsized":"2024-06-06T08:38:01.543888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.470820+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:38:58.470820+0000","last_peered":"2024-06-06T08:38:58.470820+0000","last_clean":"2024-06-06T08:38:58.470820+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:38:58.470820+0000","last_undegraded":"2024-06-06T08:38:58.470820+0000","last_fullsized":"2024-06-06T08:38:58.470820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726450+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:38:01.726450+0000","last_peered":"2024-06-06T08:38:01.726450+0000","last_clean":"2024-06-06T08:38:01.726450+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:38:01.726450+0000","last_undegraded":"2024-06-06T08:38:01.726450+0000","last_fullsized":"2024-06-06T08:38:01.726450+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015505+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:38:01.015505+0000","last_peered":"2024-06-06T08:38:01.015505+0000","last_clean":"2024-06-06T08:38:01.015505+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:38:01.015505+0000","last_undegraded":"2024-06-06T08:38:01.015505+0000","last_fullsized":"2024-06-06T08:38:01.015505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.467248+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:38:58.467248+0000","last_peered":"2024-06-06T08:38:58.467248+0000","last_clean":"2024-06-06T08:38:58.467248+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:38:58.467248+0000","last_undegraded":"2024-06-06T08:38:58.467248+0000","last_fullsized":"2024-06-06T08:38:58.467248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.014926+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:38:01.014926+0000","last_peered":"2024-06-06T08:38:01.014926+0000","last_clean":"2024-06-06T08:38:01.014926+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:38:01.014926+0000","last_undegraded":"2024-06-06T08:38:01.014926+0000","last_fullsized":"2024-06-06T08:38:01.014926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'5","reported_seq":153,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:01.979568+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:39:01.979568+0000","last_peered":"2024-06-06T08:39:01.979568+0000","last_clean":"2024-06-06T08:39:01.979568+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:39:01.979568+0000","last_undegraded":"2024-06-06T08:39:01.979568+0000","last_fullsized":"2024-06-06T08:39:01.979568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469107+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:38:58.469107+0000","last_peered":"2024-06-06T08:38:58.469107+0000","last_clean":"2024-06-06T08:38:58.469107+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:38:58.469107+0000","last_undegraded":"2024-06-06T08:38:58.469107+0000","last_fullsized":"2024-06-06T08:38:58.469107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.150985+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:38:01.150985+0000","last_peered":"2024-06-06T08:38:01.150985+0000","last_clean":"2024-06-06T08:38:01.150985+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:38:01.150985+0000","last_undegraded":"2024-06-06T08:38:01.150985+0000","last_fullsized":"2024-06-06T08:38:01.150985+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012072+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:38:01.012072+0000","last_peered":"2024-06-06T08:38:01.012072+0000","last_clean":"2024-06-06T08:38:01.012072+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:38:01.012072+0000","last_undegraded":"2024-06-06T08:38:01.012072+0000","last_fullsized":"2024-06-06T08:38:01.012072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.436368+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:38:58.436368+0000","last_peered":"2024-06-06T08:38:58.436368+0000","last_clean":"2024-06-06T08:38:58.436368+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:38:58.436368+0000","last_undegraded":"2024-06-06T08:38:58.436368+0000","last_fullsized":"2024-06-06T08:38:58.436368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015179+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:38:01.015179+0000","last_peered":"2024-06-06T08:38:01.015179+0000","last_clean":"2024-06-06T08:38:01.015179+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:38:01.015179+0000","last_undegraded":"2024-06-06T08:38:01.015179+0000","last_fullsized":"2024-06-06T08:38:01.015179+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012651+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:38:01.012651+0000","last_peered":"2024-06-06T08:38:01.012651+0000","last_clean":"2024-06-06T08:38:01.012651+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:38:01.012651+0000","last_undegraded":"2024-06-06T08:38:01.012651+0000","last_fullsized":"2024-06-06T08:38:01.012651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.449668+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:38:58.449668+0000","last_peered":"2024-06-06T08:38:58.449668+0000","last_clean":"2024-06-06T08:38:58.449668+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:38:58.449668+0000","last_undegraded":"2024-06-06T08:38:58.449668+0000","last_fullsized":"2024-06-06T08:38:58.449668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012185+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:38:01.012185+0000","last_peered":"2024-06-06T08:38:01.012185+0000","last_clean":"2024-06-06T08:38:01.012185+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:38:01.012185+0000","last_undegraded":"2024-06-06T08:38:01.012185+0000","last_fullsized":"2024-06-06T08:38:01.012185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.014647+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:38:01.014647+0000","last_peered":"2024-06-06T08:38:01.014647+0000","last_clean":"2024-06-06T08:38:01.014647+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:38:01.014647+0000","last_undegraded":"2024-06-06T08:38:01.014647+0000","last_fullsized":"2024-06-06T08:38:01.014647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.452152+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:38:58.452152+0000","last_peered":"2024-06-06T08:38:58.452152+0000","last_clean":"2024-06-06T08:38:58.452152+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:38:58.452152+0000","last_undegraded":"2024-06-06T08:38:58.452152+0000","last_fullsized":"2024-06-06T08:38:58.452152+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.150745+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:38:01.150745+0000","last_peered":"2024-06-06T08:38:01.150745+0000","last_clean":"2024-06-06T08:38:01.150745+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:38:01.150745+0000","last_undegraded":"2024-06-06T08:38:01.150745+0000","last_fullsized":"2024-06-06T08:38:01.150745+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012568+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:38:01.012568+0000","last_peered":"2024-06-06T08:38:01.012568+0000","last_clean":"2024-06-06T08:38:01.012568+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:38:01.012568+0000","last_undegraded":"2024-06-06T08:38:01.012568+0000","last_fullsized":"2024-06-06T08:38:01.012568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.465379+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:38:58.465379+0000","last_peered":"2024-06-06T08:38:58.465379+0000","last_clean":"2024-06-06T08:38:58.465379+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:38:58.465379+0000","last_undegraded":"2024-06-06T08:38:58.465379+0000","last_fullsized":"2024-06-06T08:38:58.465379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.150599+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:38:01.150599+0000","last_peered":"2024-06-06T08:38:01.150599+0000","last_clean":"2024-06-06T08:38:01.150599+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:38:01.150599+0000","last_undegraded":"2024-06-06T08:38:01.150599+0000","last_fullsized":"2024-06-06T08:38:01.150599+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.450970+0000","last_change":"2024-06-06T08:38:01.545781+0000","last_active":"2024-06-06T08:38:58.450970+0000","last_peered":"2024-06-06T08:38:58.450970+0000","last_clean":"2024-06-06T08:38:58.450970+0000","last_became_active":"2024-06-06T08:38:01.545462+0000","last_became_peered":"2024-06-06T08:38:01.545462+0000","last_unstale":"2024-06-06T08:38:58.450970+0000","last_undegraded":"2024-06-06T08:38:58.450970+0000","last_fullsized":"2024-06-06T08:38:58.450970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:14:36.722960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":150,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:02.980193+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:39:02.980193+0000","last_peered":"2024-06-06T08:39:02.980193+0000","last_clean":"2024-06-06T08:39:02.980193+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:39:02.980193+0000","last_undegraded":"2024-06-06T08:39:02.980193+0000","last_fullsized":"2024-06-06T08:39:02.980193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012769+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:38:01.012769+0000","last_peered":"2024-06-06T08:38:01.012769+0000","last_clean":"2024-06-06T08:38:01.012769+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:38:01.012769+0000","last_undegraded":"2024-06-06T08:38:01.012769+0000","last_fullsized":"2024-06-06T08:38:01.012769+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.466378+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:38:58.466378+0000","last_peered":"2024-06-06T08:38:58.466378+0000","last_clean":"2024-06-06T08:38:58.466378+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:38:58.466378+0000","last_undegraded":"2024-06-06T08:38:58.466378+0000","last_fullsized":"2024-06-06T08:38:58.466378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726698+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:38:01.726698+0000","last_peered":"2024-06-06T08:38:01.726698+0000","last_clean":"2024-06-06T08:38:01.726698+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:38:01.726698+0000","last_undegraded":"2024-06-06T08:38:01.726698+0000","last_fullsized":"2024-06-06T08:38:01.726698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":375,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012193+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:38:01.012193+0000","last_peered":"2024-06-06T08:38:01.012193+0000","last_clean":"2024-06-06T08:38:01.012193+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:38:01.012193+0000","last_undegraded":"2024-06-06T08:38:01.012193+0000","last_fullsized":"2024-06-06T08:38:01.012193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015842+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:38:01.015842+0000","last_peered":"2024-06-06T08:38:01.015842+0000","last_clean":"2024-06-06T08:38:01.015842+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:38:01.015842+0000","last_undegraded":"2024-06-06T08:38:01.015842+0000","last_fullsized":"2024-06-06T08:38:01.015842+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.456642+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:38:58.456642+0000","last_peered":"2024-06-06T08:38:58.456642+0000","last_clean":"2024-06-06T08:38:58.456642+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:38:58.456642+0000","last_undegraded":"2024-06-06T08:38:58.456642+0000","last_fullsized":"2024-06-06T08:38:58.456642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726240+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:38:01.726240+0000","last_peered":"2024-06-06T08:38:01.726240+0000","last_clean":"2024-06-06T08:38:01.726240+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:38:01.726240+0000","last_undegraded":"2024-06-06T08:38:01.726240+0000","last_fullsized":"2024-06-06T08:38:01.726240+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726454+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:38:01.726454+0000","last_peered":"2024-06-06T08:38:01.726454+0000","last_clean":"2024-06-06T08:38:01.726454+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:38:01.726454+0000","last_undegraded":"2024-06-06T08:38:01.726454+0000","last_fullsized":"2024-06-06T08:38:01.726454+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726661+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:38:01.726661+0000","last_peered":"2024-06-06T08:38:01.726661+0000","last_clean":"2024-06-06T08:38:01.726661+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:38:01.726661+0000","last_undegraded":"2024-06-06T08:38:01.726661+0000","last_fullsized":"2024-06-06T08:38:01.726661+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.454450+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:38:58.454450+0000","last_peered":"2024-06-06T08:38:58.454450+0000","last_clean":"2024-06-06T08:38:58.454450+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:38:58.454450+0000","last_undegraded":"2024-06-06T08:38:58.454450+0000","last_fullsized":"2024-06-06T08:38:58.454450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.369696+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:38:01.369696+0000","last_peered":"2024-06-06T08:38:01.369696+0000","last_clean":"2024-06-06T08:38:01.369696+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:38:01.369696+0000","last_undegraded":"2024-06-06T08:38:01.369696+0000","last_fullsized":"2024-06-06T08:38:01.369696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'13","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:27.796156+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:38:27.796156+0000","last_peered":"2024-06-06T08:38:27.796156+0000","last_clean":"2024-06-06T08:38:27.796156+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:38:27.796156+0000","last_undegraded":"2024-06-06T08:38:27.796156+0000","last_fullsized":"2024-06-06T08:38:27.796156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.468600+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:38:58.468600+0000","last_peered":"2024-06-06T08:38:58.468600+0000","last_clean":"2024-06-06T08:38:58.468600+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:38:58.468600+0000","last_undegraded":"2024-06-06T08:38:58.468600+0000","last_fullsized":"2024-06-06T08:38:58.468600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544520+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:38:01.544520+0000","last_peered":"2024-06-06T08:38:01.544520+0000","last_clean":"2024-06-06T08:38:01.544520+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:38:01.544520+0000","last_undegraded":"2024-06-06T08:38:01.544520+0000","last_fullsized":"2024-06-06T08:38:01.544520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726536+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:38:01.726536+0000","last_peered":"2024-06-06T08:38:01.726536+0000","last_clean":"2024-06-06T08:38:01.726536+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:38:01.726536+0000","last_undegraded":"2024-06-06T08:38:01.726536+0000","last_fullsized":"2024-06-06T08:38:01.726536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.476213+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:38:58.476213+0000","last_peered":"2024-06-06T08:38:58.476213+0000","last_clean":"2024-06-06T08:38:58.476213+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:38:58.476213+0000","last_undegraded":"2024-06-06T08:38:58.476213+0000","last_fullsized":"2024-06-06T08:38:58.476213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727328+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:38:01.727328+0000","last_peered":"2024-06-06T08:38:01.727328+0000","last_clean":"2024-06-06T08:38:01.727328+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:38:01.727328+0000","last_undegraded":"2024-06-06T08:38:01.727328+0000","last_fullsized":"2024-06-06T08:38:01.727328+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.391691+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:38:01.391691+0000","last_peered":"2024-06-06T08:38:01.391691+0000","last_clean":"2024-06-06T08:38:01.391691+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:38:01.391691+0000","last_undegraded":"2024-06-06T08:38:01.391691+0000","last_fullsized":"2024-06-06T08:38:01.391691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.425634+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:38:58.425634+0000","last_peered":"2024-06-06T08:38:58.425634+0000","last_clean":"2024-06-06T08:38:58.425634+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:38:58.425634+0000","last_undegraded":"2024-06-06T08:38:58.425634+0000","last_fullsized":"2024-06-06T08:38:58.425634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015038+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:38:01.015038+0000","last_peered":"2024-06-06T08:38:01.015038+0000","last_clean":"2024-06-06T08:38:01.015038+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:38:01.015038+0000","last_undegraded":"2024-06-06T08:38:01.015038+0000","last_fullsized":"2024-06-06T08:38:01.015038+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.370344+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:38:01.370344+0000","last_peered":"2024-06-06T08:38:01.370344+0000","last_clean":"2024-06-06T08:38:01.370344+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:38:01.370344+0000","last_undegraded":"2024-06-06T08:38:01.370344+0000","last_fullsized":"2024-06-06T08:38:01.370344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.468584+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:38:58.468584+0000","last_peered":"2024-06-06T08:38:58.468584+0000","last_clean":"2024-06-06T08:38:58.468584+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:38:58.468584+0000","last_undegraded":"2024-06-06T08:38:58.468584+0000","last_fullsized":"2024-06-06T08:38:58.468584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727456+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:38:01.727456+0000","last_peered":"2024-06-06T08:38:01.727456+0000","last_clean":"2024-06-06T08:38:01.727456+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:38:01.727456+0000","last_undegraded":"2024-06-06T08:38:01.727456+0000","last_fullsized":"2024-06-06T08:38:01.727456+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.543888+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:38:01.543888+0000","last_peered":"2024-06-06T08:38:01.543888+0000","last_clean":"2024-06-06T08:38:01.543888+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:38:01.543888+0000","last_undegraded":"2024-06-06T08:38:01.543888+0000","last_fullsized":"2024-06-06T08:38:01.543888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473941+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:38:58.473941+0000","last_peered":"2024-06-06T08:38:58.473941+0000","last_clean":"2024-06-06T08:38:58.473941+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:38:58.473941+0000","last_undegraded":"2024-06-06T08:38:58.473941+0000","last_fullsized":"2024-06-06T08:38:58.473941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727282+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:38:01.727282+0000","last_peered":"2024-06-06T08:38:01.727282+0000","last_clean":"2024-06-06T08:38:01.727282+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:38:01.727282+0000","last_undegraded":"2024-06-06T08:38:01.727282+0000","last_fullsized":"2024-06-06T08:38:01.727282+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012243+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:38:01.012243+0000","last_peered":"2024-06-06T08:38:01.012243+0000","last_clean":"2024-06-06T08:38:01.012243+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:38:01.012243+0000","last_undegraded":"2024-06-06T08:38:01.012243+0000","last_fullsized":"2024-06-06T08:38:01.012243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.471338+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:38:58.471338+0000","last_peered":"2024-06-06T08:38:58.471338+0000","last_clean":"2024-06-06T08:38:58.471338+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:38:58.471338+0000","last_undegraded":"2024-06-06T08:38:58.471338+0000","last_fullsized":"2024-06-06T08:38:58.471338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.391711+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:38:01.391711+0000","last_peered":"2024-06-06T08:38:01.391711+0000","last_clean":"2024-06-06T08:38:01.391711+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:38:01.391711+0000","last_undegraded":"2024-06-06T08:38:01.391711+0000","last_fullsized":"2024-06-06T08:38:01.391711+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544786+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:38:01.544786+0000","last_peered":"2024-06-06T08:38:01.544786+0000","last_clean":"2024-06-06T08:38:01.544786+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:38:01.544786+0000","last_undegraded":"2024-06-06T08:38:01.544786+0000","last_fullsized":"2024-06-06T08:38:01.544786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473736+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:38:58.473736+0000","last_peered":"2024-06-06T08:38:58.473736+0000","last_clean":"2024-06-06T08:38:58.473736+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:38:58.473736+0000","last_undegraded":"2024-06-06T08:38:58.473736+0000","last_fullsized":"2024-06-06T08:38:58.473736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.011706+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:38:01.011706+0000","last_peered":"2024-06-06T08:38:01.011706+0000","last_clean":"2024-06-06T08:38:01.011706+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:38:01.011706+0000","last_undegraded":"2024-06-06T08:38:01.011706+0000","last_fullsized":"2024-06-06T08:38:01.011706+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015846+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:38:01.015846+0000","last_peered":"2024-06-06T08:38:01.015846+0000","last_clean":"2024-06-06T08:38:01.015846+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:38:01.015846+0000","last_undegraded":"2024-06-06T08:38:01.015846+0000","last_fullsized":"2024-06-06T08:38:01.015846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.465807+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:38:58.465807+0000","last_peered":"2024-06-06T08:38:58.465807+0000","last_clean":"2024-06-06T08:38:58.465807+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:38:58.465807+0000","last_undegraded":"2024-06-06T08:38:58.465807+0000","last_fullsized":"2024-06-06T08:38:58.465807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015077+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:38:01.015077+0000","last_peered":"2024-06-06T08:38:01.015077+0000","last_clean":"2024-06-06T08:38:01.015077+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:38:01.015077+0000","last_undegraded":"2024-06-06T08:38:01.015077+0000","last_fullsized":"2024-06-06T08:38:01.015077+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.151056+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:38:01.151056+0000","last_peered":"2024-06-06T08:38:01.151056+0000","last_clean":"2024-06-06T08:38:01.151056+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:38:01.151056+0000","last_undegraded":"2024-06-06T08:38:01.151056+0000","last_fullsized":"2024-06-06T08:38:01.151056+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.467697+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:38:58.467697+0000","last_peered":"2024-06-06T08:38:58.467697+0000","last_clean":"2024-06-06T08:38:58.467697+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:38:58.467697+0000","last_undegraded":"2024-06-06T08:38:58.467697+0000","last_fullsized":"2024-06-06T08:38:58.467697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.458255+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:38:58.458255+0000","last_peered":"2024-06-06T08:38:58.458255+0000","last_clean":"2024-06-06T08:38:58.458255+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:38:58.458255+0000","last_undegraded":"2024-06-06T08:38:58.458255+0000","last_fullsized":"2024-06-06T08:38:58.458255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.458387+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:38:58.458387+0000","last_peered":"2024-06-06T08:38:58.458387+0000","last_clean":"2024-06-06T08:38:58.458387+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:38:58.458387+0000","last_undegraded":"2024-06-06T08:38:58.458387+0000","last_fullsized":"2024-06-06T08:38:58.458387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.474063+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:38:58.474063+0000","last_peered":"2024-06-06T08:38:58.474063+0000","last_clean":"2024-06-06T08:38:58.474063+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:38:58.474063+0000","last_undegraded":"2024-06-06T08:38:58.474063+0000","last_fullsized":"2024-06-06T08:38:58.474063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473465+0000","last_change":"2024-06-06T08:38:01.151003+0000","last_active":"2024-06-06T08:38:58.473465+0000","last_peered":"2024-06-06T08:38:58.473465+0000","last_clean":"2024-06-06T08:38:58.473465+0000","last_became_active":"2024-06-06T08:38:01.150557+0000","last_became_peered":"2024-06-06T08:38:01.150557+0000","last_unstale":"2024-06-06T08:38:58.473465+0000","last_undegraded":"2024-06-06T08:38:58.473465+0000","last_fullsized":"2024-06-06T08:38:58.473465+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:29:44.629894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475187+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:38:58.475187+0000","last_peered":"2024-06-06T08:38:58.475187+0000","last_clean":"2024-06-06T08:38:58.475187+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:38:58.475187+0000","last_undegraded":"2024-06-06T08:38:58.475187+0000","last_fullsized":"2024-06-06T08:38:58.475187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.450072+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:38:58.450072+0000","last_peered":"2024-06-06T08:38:58.450072+0000","last_clean":"2024-06-06T08:38:58.450072+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:38:58.450072+0000","last_undegraded":"2024-06-06T08:38:58.450072+0000","last_fullsized":"2024-06-06T08:38:58.450072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.457657+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:38:58.457657+0000","last_peered":"2024-06-06T08:38:58.457657+0000","last_clean":"2024-06-06T08:38:58.457657+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:38:58.457657+0000","last_undegraded":"2024-06-06T08:38:58.457657+0000","last_fullsized":"2024-06-06T08:38:58.457657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.451104+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:38:58.451104+0000","last_peered":"2024-06-06T08:38:58.451104+0000","last_clean":"2024-06-06T08:38:58.451104+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:38:58.451104+0000","last_undegraded":"2024-06-06T08:38:58.451104+0000","last_fullsized":"2024-06-06T08:38:58.451104+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.463786+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:38:58.463786+0000","last_peered":"2024-06-06T08:38:58.463786+0000","last_clean":"2024-06-06T08:38:58.463786+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:38:58.463786+0000","last_undegraded":"2024-06-06T08:38:58.463786+0000","last_fullsized":"2024-06-06T08:38:58.463786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469407+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:38:58.469407+0000","last_peered":"2024-06-06T08:38:58.469407+0000","last_clean":"2024-06-06T08:38:58.469407+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:38:58.469407+0000","last_undegraded":"2024-06-06T08:38:58.469407+0000","last_fullsized":"2024-06-06T08:38:58.469407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475125+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:38:58.475125+0000","last_peered":"2024-06-06T08:38:58.475125+0000","last_clean":"2024-06-06T08:38:58.475125+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:38:58.475125+0000","last_undegraded":"2024-06-06T08:38:58.475125+0000","last_fullsized":"2024-06-06T08:38:58.475125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475402+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:38:58.475402+0000","last_peered":"2024-06-06T08:38:58.475402+0000","last_clean":"2024-06-06T08:38:58.475402+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:38:58.475402+0000","last_undegraded":"2024-06-06T08:38:58.475402+0000","last_fullsized":"2024-06-06T08:38:58.475402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.455776+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:38:58.455776+0000","last_peered":"2024-06-06T08:38:58.455776+0000","last_clean":"2024-06-06T08:38:58.455776+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:38:58.455776+0000","last_undegraded":"2024-06-06T08:38:58.455776+0000","last_fullsized":"2024-06-06T08:38:58.455776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280848,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66460,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":625,"num_read_kb":385,"num_write":176,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383729,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":569943,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645367,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32388,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712764,"statfs":{"total":95995035648,"available":95961870336,"internally_reserved":0,"allocated":1241088,"data_stored":1220039,"data_compressed":68108,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808892,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":574253,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070528,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":570055,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1166126,"data_compressed":68108,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495692,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":565333,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":627562,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":566721,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":566593,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565527,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":632833,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1161183,"data_compressed":68108,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:39:05.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T08:39:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:06.400 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:06 smithi027 ceph-mon[32335]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T08:39:06.401 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:06 smithi027 ceph-mon[32335]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:39:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:06 smithi203 ceph-mon[36579]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T08:39:06.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:06 smithi203 ceph-mon[36579]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:39:06.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:06 smithi176 ceph-mon[36786]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T08:39:06.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:06 smithi176 ceph-mon[36786]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:39:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 -- 172.21.15.27:0/426008016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 msgr2=0x7f4788101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 --2- 172.21.15.27:0/426008016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 0x7f4788101840 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f477c009990 tx=0x7f477c009ca0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 -- 172.21.15.27:0/426008016 shutdown_connections 2024-06-06T08:39:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 --2- 172.21.15.27:0/426008016 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4788101d80 0x7f478810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 --2- 172.21.15.27:0/426008016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4788107440 0x7f4788107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 --2- 172.21.15.27:0/426008016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 0x7f4788101840 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 -- 172.21.15.27:0/426008016 >> 172.21.15.27:0/426008016 conn(0x7f4788077400 msgr2=0x7f4788077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 -- 172.21.15.27:0/426008016 shutdown_connections 2024-06-06T08:39:07.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 -- 172.21.15.27:0/426008016 wait complete. 2024-06-06T08:39:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 Processor -- start 2024-06-06T08:39:07.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.760+0000 7f478e577700 1 -- start start 2024-06-06T08:39:07.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f478e577700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 0x7f478819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:07.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f478e577700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4788101d80 0x7f478819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:07.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f478e577700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4788107440 0x7f4788198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:07.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f478e577700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47881116e0 con 0x7f47881013c0 2024-06-06T08:39:07.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f478e577700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4788111560 con 0x7f4788107440 2024-06-06T08:39:07.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f478e577700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4788111860 con 0x7f4788101d80 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f4787fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 0x7f478819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f4787fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 0x7f478819e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40800/0 (socket says 172.21.15.27:40800) 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f4787fff700 1 -- 172.21.15.27:0/2978342861 learned_addr learned my addr 172.21.15.27:0/2978342861 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f478cb14700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4788107440 0x7f4788198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f47877fe700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4788101d80 0x7f478819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f4787fff700 1 -- 172.21.15.27:0/2978342861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4788101d80 msgr2=0x7f478819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f4787fff700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4788101d80 0x7f478819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f4787fff700 1 -- 172.21.15.27:0/2978342861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4788107440 msgr2=0x7f4788198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f4787fff700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4788107440 0x7f4788198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f4787fff700 1 -- 172.21.15.27:0/2978342861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4788199230 con 0x7f47881013c0 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.761+0000 7f478cb14700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4788107440 0x7f4788198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.762+0000 7f4787fff700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 0x7f478819e910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f477800b530 tx=0x7f477800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:07.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.762+0000 7f47857fa700 1 -- 172.21.15.27:0/2978342861 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778015020 con 0x7f47881013c0 2024-06-06T08:39:07.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.762+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4788199520 con 0x7f47881013c0 2024-06-06T08:39:07.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.762+0000 7f47857fa700 1 -- 172.21.15.27:0/2978342861 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4778011d10 con 0x7f47881013c0 2024-06-06T08:39:07.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.762+0000 7f47857fa700 1 -- 172.21.15.27:0/2978342861 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477800f2e0 con 0x7f47881013c0 2024-06-06T08:39:07.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.762+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47881a57e0 con 0x7f47881013c0 2024-06-06T08:39:07.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.763+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4788102ab0 con 0x7f47881013c0 2024-06-06T08:39:07.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.764+0000 7f47857fa700 1 -- 172.21.15.27:0/2978342861 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4778018020 con 0x7f47881013c0 2024-06-06T08:39:07.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.765+0000 7f47857fa700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47700a2ca0 0x7f47700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:07.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.765+0000 7f47877fe700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47700a2ca0 0x7f47700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:07.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.765+0000 7f47857fa700 1 -- 172.21.15.27:0/2978342861 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f47780cec70 con 0x7f47881013c0 2024-06-06T08:39:07.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.766+0000 7f47877fe700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47700a2ca0 0x7f47700a5160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f477c00b5c0 tx=0x7f477c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:07.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.770+0000 7f47857fa700 1 -- 172.21.15.27:0/2978342861 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f477801f020 con 0x7f47881013c0 2024-06-06T08:39:07.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.970+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f478806ac00 con 0x7f47700a2ca0 2024-06-06T08:39:07.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.980+0000 7f47857fa700 1 -- 172.21.15.27:0/2978342861 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340841 (secure 0 0 0) 0x7f478806ac00 con 0x7f47700a2ca0 2024-06-06T08:39:07.982 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47700a2ca0 msgr2=0x7f47700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47700a2ca0 0x7f47700a5160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f477c00b5c0 tx=0x7f477c005960 comp rx=0 tx=0).stop 2024-06-06T08:39:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 msgr2=0x7f478819e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 0x7f478819e910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f477800b530 tx=0x7f477800b840 comp rx=0 tx=0).stop 2024-06-06T08:39:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 shutdown_connections 2024-06-06T08:39:07.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f47700a2ca0 0x7f47700a5160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4788107440 0x7f4788198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47881013c0 0x7f478819e910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 --2- 172.21.15.27:0/2978342861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4788101d80 0x7f478819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 >> 172.21.15.27:0/2978342861 conn(0x7f4788077400 msgr2=0x7f4788105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 shutdown_connections 2024-06-06T08:39:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:07.983+0000 7f478e577700 1 -- 172.21.15.27:0/2978342861 wait complete. 2024-06-06T08:39:07.986 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:39:08.306 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:08 smithi027 ceph-mon[32335]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T08:39:08.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:08 smithi203 ceph-mon[36579]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T08:39:08.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:08 smithi176 ceph-mon[36786]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T08:39:08.798 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-06-06T08:39:06.477620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657062,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":761,"num_read_kb":504,"num_write":359,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358308,"kb_used_data":13316,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583516,"statfs":{"total":1151940427776,"available":1151573520384,"internally_reserved":0,"allocated":13635584,"data_stored":8786168,"data_compressed":495807,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003701"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469315+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:38:58.469315+0000","last_peered":"2024-06-06T08:38:58.469315+0000","last_clean":"2024-06-06T08:38:58.469315+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:38:58.469315+0000","last_undegraded":"2024-06-06T08:38:58.469315+0000","last_fullsized":"2024-06-06T08:38:58.469315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.470122+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:38:58.470122+0000","last_peered":"2024-06-06T08:38:58.470122+0000","last_clean":"2024-06-06T08:38:58.470122+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:38:58.470122+0000","last_undegraded":"2024-06-06T08:38:58.470122+0000","last_fullsized":"2024-06-06T08:38:58.470122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.474795+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:38:58.474795+0000","last_peered":"2024-06-06T08:38:58.474795+0000","last_clean":"2024-06-06T08:38:58.474795+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:38:58.474795+0000","last_undegraded":"2024-06-06T08:38:58.474795+0000","last_fullsized":"2024-06-06T08:38:58.474795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.457919+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:38:58.457919+0000","last_peered":"2024-06-06T08:38:58.457919+0000","last_clean":"2024-06-06T08:38:58.457919+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:38:58.457919+0000","last_undegraded":"2024-06-06T08:38:58.457919+0000","last_fullsized":"2024-06-06T08:38:58.457919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.465850+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:38:58.465850+0000","last_peered":"2024-06-06T08:38:58.465850+0000","last_clean":"2024-06-06T08:38:58.465850+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:38:58.465850+0000","last_undegraded":"2024-06-06T08:38:58.465850+0000","last_fullsized":"2024-06-06T08:38:58.465850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.452855+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:38:58.452855+0000","last_peered":"2024-06-06T08:38:58.452855+0000","last_clean":"2024-06-06T08:38:58.452855+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:38:58.452855+0000","last_undegraded":"2024-06-06T08:38:58.452855+0000","last_fullsized":"2024-06-06T08:38:58.452855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.460268+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:38:58.460268+0000","last_peered":"2024-06-06T08:38:58.460268+0000","last_clean":"2024-06-06T08:38:58.460268+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:38:58.460268+0000","last_undegraded":"2024-06-06T08:38:58.460268+0000","last_fullsized":"2024-06-06T08:38:58.460268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":20,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.474082+0000","last_change":"2024-06-06T08:38:01.335419+0000","last_active":"2024-06-06T08:38:58.474082+0000","last_peered":"2024-06-06T08:38:58.474082+0000","last_clean":"2024-06-06T08:38:58.474082+0000","last_became_active":"2024-06-06T08:38:01.335191+0000","last_became_peered":"2024-06-06T08:38:01.335191+0000","last_unstale":"2024-06-06T08:38:58.474082+0000","last_undegraded":"2024-06-06T08:38:58.474082+0000","last_fullsized":"2024-06-06T08:38:58.474082+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:13.253838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.472465+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:38:58.472465+0000","last_peered":"2024-06-06T08:38:58.472465+0000","last_clean":"2024-06-06T08:38:58.472465+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:38:58.472465+0000","last_undegraded":"2024-06-06T08:38:58.472465+0000","last_fullsized":"2024-06-06T08:38:58.472465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.451391+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:38:58.451391+0000","last_peered":"2024-06-06T08:38:58.451391+0000","last_clean":"2024-06-06T08:38:58.451391+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:38:58.451391+0000","last_undegraded":"2024-06-06T08:38:58.451391+0000","last_fullsized":"2024-06-06T08:38:58.451391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473681+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:38:58.473681+0000","last_peered":"2024-06-06T08:38:58.473681+0000","last_clean":"2024-06-06T08:38:58.473681+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:38:58.473681+0000","last_undegraded":"2024-06-06T08:38:58.473681+0000","last_fullsized":"2024-06-06T08:38:58.473681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.458115+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:38:58.458115+0000","last_peered":"2024-06-06T08:38:58.458115+0000","last_clean":"2024-06-06T08:38:58.458115+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:38:58.458115+0000","last_undegraded":"2024-06-06T08:38:58.458115+0000","last_fullsized":"2024-06-06T08:38:58.458115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.457851+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:38:58.457851+0000","last_peered":"2024-06-06T08:38:58.457851+0000","last_clean":"2024-06-06T08:38:58.457851+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:38:58.457851+0000","last_undegraded":"2024-06-06T08:38:58.457851+0000","last_fullsized":"2024-06-06T08:38:58.457851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.476045+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:38:58.476045+0000","last_peered":"2024-06-06T08:38:58.476045+0000","last_clean":"2024-06-06T08:38:58.476045+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:38:58.476045+0000","last_undegraded":"2024-06-06T08:38:58.476045+0000","last_fullsized":"2024-06-06T08:38:58.476045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469729+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:38:58.469729+0000","last_peered":"2024-06-06T08:38:58.469729+0000","last_clean":"2024-06-06T08:38:58.469729+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:38:58.469729+0000","last_undegraded":"2024-06-06T08:38:58.469729+0000","last_fullsized":"2024-06-06T08:38:58.469729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475711+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:38:58.475711+0000","last_peered":"2024-06-06T08:38:58.475711+0000","last_clean":"2024-06-06T08:38:58.475711+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:38:58.475711+0000","last_undegraded":"2024-06-06T08:38:58.475711+0000","last_fullsized":"2024-06-06T08:38:58.475711+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469372+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:38:58.469372+0000","last_peered":"2024-06-06T08:38:58.469372+0000","last_clean":"2024-06-06T08:38:58.469372+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:38:58.469372+0000","last_undegraded":"2024-06-06T08:38:58.469372+0000","last_fullsized":"2024-06-06T08:38:58.469372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475214+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:38:58.475214+0000","last_peered":"2024-06-06T08:38:58.475214+0000","last_clean":"2024-06-06T08:38:58.475214+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:38:58.475214+0000","last_undegraded":"2024-06-06T08:38:58.475214+0000","last_fullsized":"2024-06-06T08:38:58.475214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.461118+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:38:58.461118+0000","last_peered":"2024-06-06T08:38:58.461118+0000","last_clean":"2024-06-06T08:38:58.461118+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:38:58.461118+0000","last_undegraded":"2024-06-06T08:38:58.461118+0000","last_fullsized":"2024-06-06T08:38:58.461118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544879+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:38:01.544879+0000","last_peered":"2024-06-06T08:38:01.544879+0000","last_clean":"2024-06-06T08:38:01.544879+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:38:01.544879+0000","last_undegraded":"2024-06-06T08:38:01.544879+0000","last_fullsized":"2024-06-06T08:38:01.544879+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015106+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:38:01.015106+0000","last_peered":"2024-06-06T08:38:01.015106+0000","last_clean":"2024-06-06T08:38:01.015106+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:38:01.015106+0000","last_undegraded":"2024-06-06T08:38:01.015106+0000","last_fullsized":"2024-06-06T08:38:01.015106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.466121+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:38:58.466121+0000","last_peered":"2024-06-06T08:38:58.466121+0000","last_clean":"2024-06-06T08:38:58.466121+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:38:58.466121+0000","last_undegraded":"2024-06-06T08:38:58.466121+0000","last_fullsized":"2024-06-06T08:38:58.466121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726764+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:38:01.726764+0000","last_peered":"2024-06-06T08:38:01.726764+0000","last_clean":"2024-06-06T08:38:01.726764+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:38:01.726764+0000","last_undegraded":"2024-06-06T08:38:01.726764+0000","last_fullsized":"2024-06-06T08:38:01.726764+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.370495+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:38:01.370495+0000","last_peered":"2024-06-06T08:38:01.370495+0000","last_clean":"2024-06-06T08:38:01.370495+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:38:01.370495+0000","last_undegraded":"2024-06-06T08:38:01.370495+0000","last_fullsized":"2024-06-06T08:38:01.370495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.476257+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:38:58.476257+0000","last_peered":"2024-06-06T08:38:58.476257+0000","last_clean":"2024-06-06T08:38:58.476257+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:38:58.476257+0000","last_undegraded":"2024-06-06T08:38:58.476257+0000","last_fullsized":"2024-06-06T08:38:58.476257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727107+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:38:01.727107+0000","last_peered":"2024-06-06T08:38:01.727107+0000","last_clean":"2024-06-06T08:38:01.727107+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:38:01.727107+0000","last_undegraded":"2024-06-06T08:38:01.727107+0000","last_fullsized":"2024-06-06T08:38:01.727107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012608+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:38:01.012608+0000","last_peered":"2024-06-06T08:38:01.012608+0000","last_clean":"2024-06-06T08:38:01.012608+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:38:01.012608+0000","last_undegraded":"2024-06-06T08:38:01.012608+0000","last_fullsized":"2024-06-06T08:38:01.012608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.472702+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:38:58.472702+0000","last_peered":"2024-06-06T08:38:58.472702+0000","last_clean":"2024-06-06T08:38:58.472702+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:38:58.472702+0000","last_undegraded":"2024-06-06T08:38:58.472702+0000","last_fullsized":"2024-06-06T08:38:58.472702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726959+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:38:01.726959+0000","last_peered":"2024-06-06T08:38:01.726959+0000","last_clean":"2024-06-06T08:38:01.726959+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:38:01.726959+0000","last_undegraded":"2024-06-06T08:38:01.726959+0000","last_fullsized":"2024-06-06T08:38:01.726959+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012568+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:38:01.012568+0000","last_peered":"2024-06-06T08:38:01.012568+0000","last_clean":"2024-06-06T08:38:01.012568+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:38:01.012568+0000","last_undegraded":"2024-06-06T08:38:01.012568+0000","last_fullsized":"2024-06-06T08:38:01.012568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":26,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.471846+0000","last_change":"2024-06-06T08:38:01.728398+0000","last_active":"2024-06-06T08:38:58.471846+0000","last_peered":"2024-06-06T08:38:58.471846+0000","last_clean":"2024-06-06T08:38:58.471846+0000","last_became_active":"2024-06-06T08:38:01.728004+0000","last_became_peered":"2024-06-06T08:38:01.728004+0000","last_unstale":"2024-06-06T08:38:58.471846+0000","last_undegraded":"2024-06-06T08:38:58.471846+0000","last_fullsized":"2024-06-06T08:38:58.471846+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:02:03.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544841+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:38:01.544841+0000","last_peered":"2024-06-06T08:38:01.544841+0000","last_clean":"2024-06-06T08:38:01.544841+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:38:01.544841+0000","last_undegraded":"2024-06-06T08:38:01.544841+0000","last_fullsized":"2024-06-06T08:38:01.544841+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'5","reported_seq":150,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:00.977727+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:39:00.977727+0000","last_peered":"2024-06-06T08:39:00.977727+0000","last_clean":"2024-06-06T08:39:00.977727+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:39:00.977727+0000","last_undegraded":"2024-06-06T08:39:00.977727+0000","last_fullsized":"2024-06-06T08:39:00.977727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.454797+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:38:58.454797+0000","last_peered":"2024-06-06T08:38:58.454797+0000","last_clean":"2024-06-06T08:38:58.454797+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:38:58.454797+0000","last_undegraded":"2024-06-06T08:38:58.454797+0000","last_fullsized":"2024-06-06T08:38:58.454797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544291+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:38:01.544291+0000","last_peered":"2024-06-06T08:38:01.544291+0000","last_clean":"2024-06-06T08:38:01.544291+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:38:01.544291+0000","last_undegraded":"2024-06-06T08:38:01.544291+0000","last_fullsized":"2024-06-06T08:38:01.544291+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.334462+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:38:01.334462+0000","last_peered":"2024-06-06T08:38:01.334462+0000","last_clean":"2024-06-06T08:38:01.334462+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:38:01.334462+0000","last_undegraded":"2024-06-06T08:38:01.334462+0000","last_fullsized":"2024-06-06T08:38:01.334462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473557+0000","last_change":"2024-06-06T08:38:01.728238+0000","last_active":"2024-06-06T08:38:58.473557+0000","last_peered":"2024-06-06T08:38:58.473557+0000","last_clean":"2024-06-06T08:38:58.473557+0000","last_became_active":"2024-06-06T08:38:01.727844+0000","last_became_peered":"2024-06-06T08:38:01.727844+0000","last_unstale":"2024-06-06T08:38:58.473557+0000","last_undegraded":"2024-06-06T08:38:58.473557+0000","last_fullsized":"2024-06-06T08:38:58.473557+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:19:46.825321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726847+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:38:01.726847+0000","last_peered":"2024-06-06T08:38:01.726847+0000","last_clean":"2024-06-06T08:38:01.726847+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:38:01.726847+0000","last_undegraded":"2024-06-06T08:38:01.726847+0000","last_fullsized":"2024-06-06T08:38:01.726847+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.150302+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:38:01.150302+0000","last_peered":"2024-06-06T08:38:01.150302+0000","last_clean":"2024-06-06T08:38:01.150302+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:38:01.150302+0000","last_undegraded":"2024-06-06T08:38:01.150302+0000","last_fullsized":"2024-06-06T08:38:01.150302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.470392+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:38:58.470392+0000","last_peered":"2024-06-06T08:38:58.470392+0000","last_clean":"2024-06-06T08:38:58.470392+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:38:58.470392+0000","last_undegraded":"2024-06-06T08:38:58.470392+0000","last_fullsized":"2024-06-06T08:38:58.470392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.011847+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:38:01.011847+0000","last_peered":"2024-06-06T08:38:01.011847+0000","last_clean":"2024-06-06T08:38:01.011847+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:38:01.011847+0000","last_undegraded":"2024-06-06T08:38:01.011847+0000","last_fullsized":"2024-06-06T08:38:01.011847+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015710+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:38:01.015710+0000","last_peered":"2024-06-06T08:38:01.015710+0000","last_clean":"2024-06-06T08:38:01.015710+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:38:01.015710+0000","last_undegraded":"2024-06-06T08:38:01.015710+0000","last_fullsized":"2024-06-06T08:38:01.015710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.474208+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:38:58.474208+0000","last_peered":"2024-06-06T08:38:58.474208+0000","last_clean":"2024-06-06T08:38:58.474208+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:38:58.474208+0000","last_undegraded":"2024-06-06T08:38:58.474208+0000","last_fullsized":"2024-06-06T08:38:58.474208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.369716+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:38:01.369716+0000","last_peered":"2024-06-06T08:38:01.369716+0000","last_clean":"2024-06-06T08:38:01.369716+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:38:01.369716+0000","last_undegraded":"2024-06-06T08:38:01.369716+0000","last_fullsized":"2024-06-06T08:38:01.369716+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'3","reported_seq":147,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:01.978723+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:39:01.978723+0000","last_peered":"2024-06-06T08:39:01.978723+0000","last_clean":"2024-06-06T08:39:01.978723+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:39:01.978723+0000","last_undegraded":"2024-06-06T08:39:01.978723+0000","last_fullsized":"2024-06-06T08:39:01.978723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473111+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:38:58.473111+0000","last_peered":"2024-06-06T08:38:58.473111+0000","last_clean":"2024-06-06T08:38:58.473111+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:38:58.473111+0000","last_undegraded":"2024-06-06T08:38:58.473111+0000","last_fullsized":"2024-06-06T08:38:58.473111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.014302+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:38:01.014302+0000","last_peered":"2024-06-06T08:38:01.014302+0000","last_clean":"2024-06-06T08:38:01.014302+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:38:01.014302+0000","last_undegraded":"2024-06-06T08:38:01.014302+0000","last_fullsized":"2024-06-06T08:38:01.014302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012898+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:38:01.012898+0000","last_peered":"2024-06-06T08:38:01.012898+0000","last_clean":"2024-06-06T08:38:01.012898+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:38:01.012898+0000","last_undegraded":"2024-06-06T08:38:01.012898+0000","last_fullsized":"2024-06-06T08:38:01.012898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.470283+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:38:58.470283+0000","last_peered":"2024-06-06T08:38:58.470283+0000","last_clean":"2024-06-06T08:38:58.470283+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:38:58.470283+0000","last_undegraded":"2024-06-06T08:38:58.470283+0000","last_fullsized":"2024-06-06T08:38:58.470283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727272+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:38:01.727272+0000","last_peered":"2024-06-06T08:38:01.727272+0000","last_clean":"2024-06-06T08:38:01.727272+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:38:01.727272+0000","last_undegraded":"2024-06-06T08:38:01.727272+0000","last_fullsized":"2024-06-06T08:38:01.727272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015670+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:38:01.015670+0000","last_peered":"2024-06-06T08:38:01.015670+0000","last_clean":"2024-06-06T08:38:01.015670+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:38:01.015670+0000","last_undegraded":"2024-06-06T08:38:01.015670+0000","last_fullsized":"2024-06-06T08:38:01.015670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473370+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:38:58.473370+0000","last_peered":"2024-06-06T08:38:58.473370+0000","last_clean":"2024-06-06T08:38:58.473370+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:38:58.473370+0000","last_undegraded":"2024-06-06T08:38:58.473370+0000","last_fullsized":"2024-06-06T08:38:58.473370+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.334196+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:38:01.334196+0000","last_peered":"2024-06-06T08:38:01.334196+0000","last_clean":"2024-06-06T08:38:01.334196+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:38:01.334196+0000","last_undegraded":"2024-06-06T08:38:01.334196+0000","last_fullsized":"2024-06-06T08:38:01.334196+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"99'36","reported_seq":306,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:02.980202+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:39:02.980202+0000","last_peered":"2024-06-06T08:39:02.980202+0000","last_clean":"2024-06-06T08:39:02.980202+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:39:02.980202+0000","last_undegraded":"2024-06-06T08:39:02.980202+0000","last_fullsized":"2024-06-06T08:39:02.980202+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62319,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":209,"num_read_kb":166,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.458423+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:38:58.458423+0000","last_peered":"2024-06-06T08:38:58.458423+0000","last_clean":"2024-06-06T08:38:58.458423+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:38:58.458423+0000","last_undegraded":"2024-06-06T08:38:58.458423+0000","last_fullsized":"2024-06-06T08:38:58.458423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.016007+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:38:01.016007+0000","last_peered":"2024-06-06T08:38:01.016007+0000","last_clean":"2024-06-06T08:38:01.016007+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:38:01.016007+0000","last_undegraded":"2024-06-06T08:38:01.016007+0000","last_fullsized":"2024-06-06T08:38:01.016007+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.543888+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:38:01.543888+0000","last_peered":"2024-06-06T08:38:01.543888+0000","last_clean":"2024-06-06T08:38:01.543888+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:38:01.543888+0000","last_undegraded":"2024-06-06T08:38:01.543888+0000","last_fullsized":"2024-06-06T08:38:01.543888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.470820+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:38:58.470820+0000","last_peered":"2024-06-06T08:38:58.470820+0000","last_clean":"2024-06-06T08:38:58.470820+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:38:58.470820+0000","last_undegraded":"2024-06-06T08:38:58.470820+0000","last_fullsized":"2024-06-06T08:38:58.470820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726450+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:38:01.726450+0000","last_peered":"2024-06-06T08:38:01.726450+0000","last_clean":"2024-06-06T08:38:01.726450+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:38:01.726450+0000","last_undegraded":"2024-06-06T08:38:01.726450+0000","last_fullsized":"2024-06-06T08:38:01.726450+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015505+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:38:01.015505+0000","last_peered":"2024-06-06T08:38:01.015505+0000","last_clean":"2024-06-06T08:38:01.015505+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:38:01.015505+0000","last_undegraded":"2024-06-06T08:38:01.015505+0000","last_fullsized":"2024-06-06T08:38:01.015505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.467248+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:38:58.467248+0000","last_peered":"2024-06-06T08:38:58.467248+0000","last_clean":"2024-06-06T08:38:58.467248+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:38:58.467248+0000","last_undegraded":"2024-06-06T08:38:58.467248+0000","last_fullsized":"2024-06-06T08:38:58.467248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.014926+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:38:01.014926+0000","last_peered":"2024-06-06T08:38:01.014926+0000","last_clean":"2024-06-06T08:38:01.014926+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:38:01.014926+0000","last_undegraded":"2024-06-06T08:38:01.014926+0000","last_fullsized":"2024-06-06T08:38:01.014926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'5","reported_seq":153,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:01.979568+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:39:01.979568+0000","last_peered":"2024-06-06T08:39:01.979568+0000","last_clean":"2024-06-06T08:39:01.979568+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:39:01.979568+0000","last_undegraded":"2024-06-06T08:39:01.979568+0000","last_fullsized":"2024-06-06T08:39:01.979568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469107+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:38:58.469107+0000","last_peered":"2024-06-06T08:38:58.469107+0000","last_clean":"2024-06-06T08:38:58.469107+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:38:58.469107+0000","last_undegraded":"2024-06-06T08:38:58.469107+0000","last_fullsized":"2024-06-06T08:38:58.469107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.150985+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:38:01.150985+0000","last_peered":"2024-06-06T08:38:01.150985+0000","last_clean":"2024-06-06T08:38:01.150985+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:38:01.150985+0000","last_undegraded":"2024-06-06T08:38:01.150985+0000","last_fullsized":"2024-06-06T08:38:01.150985+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012072+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:38:01.012072+0000","last_peered":"2024-06-06T08:38:01.012072+0000","last_clean":"2024-06-06T08:38:01.012072+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:38:01.012072+0000","last_undegraded":"2024-06-06T08:38:01.012072+0000","last_fullsized":"2024-06-06T08:38:01.012072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.436368+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:38:58.436368+0000","last_peered":"2024-06-06T08:38:58.436368+0000","last_clean":"2024-06-06T08:38:58.436368+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:38:58.436368+0000","last_undegraded":"2024-06-06T08:38:58.436368+0000","last_fullsized":"2024-06-06T08:38:58.436368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015179+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:38:01.015179+0000","last_peered":"2024-06-06T08:38:01.015179+0000","last_clean":"2024-06-06T08:38:01.015179+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:38:01.015179+0000","last_undegraded":"2024-06-06T08:38:01.015179+0000","last_fullsized":"2024-06-06T08:38:01.015179+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012651+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:38:01.012651+0000","last_peered":"2024-06-06T08:38:01.012651+0000","last_clean":"2024-06-06T08:38:01.012651+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:38:01.012651+0000","last_undegraded":"2024-06-06T08:38:01.012651+0000","last_fullsized":"2024-06-06T08:38:01.012651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.449668+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:38:58.449668+0000","last_peered":"2024-06-06T08:38:58.449668+0000","last_clean":"2024-06-06T08:38:58.449668+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:38:58.449668+0000","last_undegraded":"2024-06-06T08:38:58.449668+0000","last_fullsized":"2024-06-06T08:38:58.449668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012185+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:38:01.012185+0000","last_peered":"2024-06-06T08:38:01.012185+0000","last_clean":"2024-06-06T08:38:01.012185+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:38:01.012185+0000","last_undegraded":"2024-06-06T08:38:01.012185+0000","last_fullsized":"2024-06-06T08:38:01.012185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.014647+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:38:01.014647+0000","last_peered":"2024-06-06T08:38:01.014647+0000","last_clean":"2024-06-06T08:38:01.014647+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:38:01.014647+0000","last_undegraded":"2024-06-06T08:38:01.014647+0000","last_fullsized":"2024-06-06T08:38:01.014647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.452152+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:38:58.452152+0000","last_peered":"2024-06-06T08:38:58.452152+0000","last_clean":"2024-06-06T08:38:58.452152+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:38:58.452152+0000","last_undegraded":"2024-06-06T08:38:58.452152+0000","last_fullsized":"2024-06-06T08:38:58.452152+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.150745+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:38:01.150745+0000","last_peered":"2024-06-06T08:38:01.150745+0000","last_clean":"2024-06-06T08:38:01.150745+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:38:01.150745+0000","last_undegraded":"2024-06-06T08:38:01.150745+0000","last_fullsized":"2024-06-06T08:38:01.150745+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012568+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:38:01.012568+0000","last_peered":"2024-06-06T08:38:01.012568+0000","last_clean":"2024-06-06T08:38:01.012568+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:38:01.012568+0000","last_undegraded":"2024-06-06T08:38:01.012568+0000","last_fullsized":"2024-06-06T08:38:01.012568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.465379+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:38:58.465379+0000","last_peered":"2024-06-06T08:38:58.465379+0000","last_clean":"2024-06-06T08:38:58.465379+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:38:58.465379+0000","last_undegraded":"2024-06-06T08:38:58.465379+0000","last_fullsized":"2024-06-06T08:38:58.465379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.150599+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:38:01.150599+0000","last_peered":"2024-06-06T08:38:01.150599+0000","last_clean":"2024-06-06T08:38:01.150599+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:38:01.150599+0000","last_undegraded":"2024-06-06T08:38:01.150599+0000","last_fullsized":"2024-06-06T08:38:01.150599+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.450970+0000","last_change":"2024-06-06T08:38:01.545781+0000","last_active":"2024-06-06T08:38:58.450970+0000","last_peered":"2024-06-06T08:38:58.450970+0000","last_clean":"2024-06-06T08:38:58.450970+0000","last_became_active":"2024-06-06T08:38:01.545462+0000","last_became_peered":"2024-06-06T08:38:01.545462+0000","last_unstale":"2024-06-06T08:38:58.450970+0000","last_undegraded":"2024-06-06T08:38:58.450970+0000","last_fullsized":"2024-06-06T08:38:58.450970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:14:36.722960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"99'5","reported_seq":150,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:39:02.980193+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:39:02.980193+0000","last_peered":"2024-06-06T08:39:02.980193+0000","last_clean":"2024-06-06T08:39:02.980193+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:39:02.980193+0000","last_undegraded":"2024-06-06T08:39:02.980193+0000","last_fullsized":"2024-06-06T08:39:02.980193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012769+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:38:01.012769+0000","last_peered":"2024-06-06T08:38:01.012769+0000","last_clean":"2024-06-06T08:38:01.012769+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:38:01.012769+0000","last_undegraded":"2024-06-06T08:38:01.012769+0000","last_fullsized":"2024-06-06T08:38:01.012769+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.466378+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:38:58.466378+0000","last_peered":"2024-06-06T08:38:58.466378+0000","last_clean":"2024-06-06T08:38:58.466378+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:38:58.466378+0000","last_undegraded":"2024-06-06T08:38:58.466378+0000","last_fullsized":"2024-06-06T08:38:58.466378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726698+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:38:01.726698+0000","last_peered":"2024-06-06T08:38:01.726698+0000","last_clean":"2024-06-06T08:38:01.726698+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:38:01.726698+0000","last_undegraded":"2024-06-06T08:38:01.726698+0000","last_fullsized":"2024-06-06T08:38:01.726698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":375,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012193+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:38:01.012193+0000","last_peered":"2024-06-06T08:38:01.012193+0000","last_clean":"2024-06-06T08:38:01.012193+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:38:01.012193+0000","last_undegraded":"2024-06-06T08:38:01.012193+0000","last_fullsized":"2024-06-06T08:38:01.012193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015842+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:38:01.015842+0000","last_peered":"2024-06-06T08:38:01.015842+0000","last_clean":"2024-06-06T08:38:01.015842+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:38:01.015842+0000","last_undegraded":"2024-06-06T08:38:01.015842+0000","last_fullsized":"2024-06-06T08:38:01.015842+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.456642+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:38:58.456642+0000","last_peered":"2024-06-06T08:38:58.456642+0000","last_clean":"2024-06-06T08:38:58.456642+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:38:58.456642+0000","last_undegraded":"2024-06-06T08:38:58.456642+0000","last_fullsized":"2024-06-06T08:38:58.456642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726240+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:38:01.726240+0000","last_peered":"2024-06-06T08:38:01.726240+0000","last_clean":"2024-06-06T08:38:01.726240+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:38:01.726240+0000","last_undegraded":"2024-06-06T08:38:01.726240+0000","last_fullsized":"2024-06-06T08:38:01.726240+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726454+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:38:01.726454+0000","last_peered":"2024-06-06T08:38:01.726454+0000","last_clean":"2024-06-06T08:38:01.726454+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:38:01.726454+0000","last_undegraded":"2024-06-06T08:38:01.726454+0000","last_fullsized":"2024-06-06T08:38:01.726454+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726734+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:38:01.726734+0000","last_peered":"2024-06-06T08:38:01.726734+0000","last_clean":"2024-06-06T08:38:01.726734+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:38:01.726734+0000","last_undegraded":"2024-06-06T08:38:01.726734+0000","last_fullsized":"2024-06-06T08:38:01.726734+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.454450+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:38:58.454450+0000","last_peered":"2024-06-06T08:38:58.454450+0000","last_clean":"2024-06-06T08:38:58.454450+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:38:58.454450+0000","last_undegraded":"2024-06-06T08:38:58.454450+0000","last_fullsized":"2024-06-06T08:38:58.454450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.369696+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:38:01.369696+0000","last_peered":"2024-06-06T08:38:01.369696+0000","last_clean":"2024-06-06T08:38:01.369696+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:38:01.369696+0000","last_undegraded":"2024-06-06T08:38:01.369696+0000","last_fullsized":"2024-06-06T08:38:01.369696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'13","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:27.796156+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:38:27.796156+0000","last_peered":"2024-06-06T08:38:27.796156+0000","last_clean":"2024-06-06T08:38:27.796156+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:38:27.796156+0000","last_undegraded":"2024-06-06T08:38:27.796156+0000","last_fullsized":"2024-06-06T08:38:27.796156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.468600+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:38:58.468600+0000","last_peered":"2024-06-06T08:38:58.468600+0000","last_clean":"2024-06-06T08:38:58.468600+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:38:58.468600+0000","last_undegraded":"2024-06-06T08:38:58.468600+0000","last_fullsized":"2024-06-06T08:38:58.468600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544520+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:38:01.544520+0000","last_peered":"2024-06-06T08:38:01.544520+0000","last_clean":"2024-06-06T08:38:01.544520+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:38:01.544520+0000","last_undegraded":"2024-06-06T08:38:01.544520+0000","last_fullsized":"2024-06-06T08:38:01.544520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.726536+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:38:01.726536+0000","last_peered":"2024-06-06T08:38:01.726536+0000","last_clean":"2024-06-06T08:38:01.726536+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:38:01.726536+0000","last_undegraded":"2024-06-06T08:38:01.726536+0000","last_fullsized":"2024-06-06T08:38:01.726536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.476213+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:38:58.476213+0000","last_peered":"2024-06-06T08:38:58.476213+0000","last_clean":"2024-06-06T08:38:58.476213+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:38:58.476213+0000","last_undegraded":"2024-06-06T08:38:58.476213+0000","last_fullsized":"2024-06-06T08:38:58.476213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727328+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:38:01.727328+0000","last_peered":"2024-06-06T08:38:01.727328+0000","last_clean":"2024-06-06T08:38:01.727328+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:38:01.727328+0000","last_undegraded":"2024-06-06T08:38:01.727328+0000","last_fullsized":"2024-06-06T08:38:01.727328+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.391691+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:38:01.391691+0000","last_peered":"2024-06-06T08:38:01.391691+0000","last_clean":"2024-06-06T08:38:01.391691+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:38:01.391691+0000","last_undegraded":"2024-06-06T08:38:01.391691+0000","last_fullsized":"2024-06-06T08:38:01.391691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.425634+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:38:58.425634+0000","last_peered":"2024-06-06T08:38:58.425634+0000","last_clean":"2024-06-06T08:38:58.425634+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:38:58.425634+0000","last_undegraded":"2024-06-06T08:38:58.425634+0000","last_fullsized":"2024-06-06T08:38:58.425634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015038+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:38:01.015038+0000","last_peered":"2024-06-06T08:38:01.015038+0000","last_clean":"2024-06-06T08:38:01.015038+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:38:01.015038+0000","last_undegraded":"2024-06-06T08:38:01.015038+0000","last_fullsized":"2024-06-06T08:38:01.015038+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.370344+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:38:01.370344+0000","last_peered":"2024-06-06T08:38:01.370344+0000","last_clean":"2024-06-06T08:38:01.370344+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:38:01.370344+0000","last_undegraded":"2024-06-06T08:38:01.370344+0000","last_fullsized":"2024-06-06T08:38:01.370344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.468584+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:38:58.468584+0000","last_peered":"2024-06-06T08:38:58.468584+0000","last_clean":"2024-06-06T08:38:58.468584+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:38:58.468584+0000","last_undegraded":"2024-06-06T08:38:58.468584+0000","last_fullsized":"2024-06-06T08:38:58.468584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727456+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:38:01.727456+0000","last_peered":"2024-06-06T08:38:01.727456+0000","last_clean":"2024-06-06T08:38:01.727456+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:38:01.727456+0000","last_undegraded":"2024-06-06T08:38:01.727456+0000","last_fullsized":"2024-06-06T08:38:01.727456+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.543888+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:38:01.543888+0000","last_peered":"2024-06-06T08:38:01.543888+0000","last_clean":"2024-06-06T08:38:01.543888+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:38:01.543888+0000","last_undegraded":"2024-06-06T08:38:01.543888+0000","last_fullsized":"2024-06-06T08:38:01.543888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473941+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:38:58.473941+0000","last_peered":"2024-06-06T08:38:58.473941+0000","last_clean":"2024-06-06T08:38:58.473941+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:38:58.473941+0000","last_undegraded":"2024-06-06T08:38:58.473941+0000","last_fullsized":"2024-06-06T08:38:58.473941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.727282+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:38:01.727282+0000","last_peered":"2024-06-06T08:38:01.727282+0000","last_clean":"2024-06-06T08:38:01.727282+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:38:01.727282+0000","last_undegraded":"2024-06-06T08:38:01.727282+0000","last_fullsized":"2024-06-06T08:38:01.727282+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.012243+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:38:01.012243+0000","last_peered":"2024-06-06T08:38:01.012243+0000","last_clean":"2024-06-06T08:38:01.012243+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:38:01.012243+0000","last_undegraded":"2024-06-06T08:38:01.012243+0000","last_fullsized":"2024-06-06T08:38:01.012243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.471338+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:38:58.471338+0000","last_peered":"2024-06-06T08:38:58.471338+0000","last_clean":"2024-06-06T08:38:58.471338+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:38:58.471338+0000","last_undegraded":"2024-06-06T08:38:58.471338+0000","last_fullsized":"2024-06-06T08:38:58.471338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.391711+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:38:01.391711+0000","last_peered":"2024-06-06T08:38:01.391711+0000","last_clean":"2024-06-06T08:38:01.391711+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:38:01.391711+0000","last_undegraded":"2024-06-06T08:38:01.391711+0000","last_fullsized":"2024-06-06T08:38:01.391711+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.544786+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:38:01.544786+0000","last_peered":"2024-06-06T08:38:01.544786+0000","last_clean":"2024-06-06T08:38:01.544786+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:38:01.544786+0000","last_undegraded":"2024-06-06T08:38:01.544786+0000","last_fullsized":"2024-06-06T08:38:01.544786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473736+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:38:58.473736+0000","last_peered":"2024-06-06T08:38:58.473736+0000","last_clean":"2024-06-06T08:38:58.473736+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:38:58.473736+0000","last_undegraded":"2024-06-06T08:38:58.473736+0000","last_fullsized":"2024-06-06T08:38:58.473736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.011706+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:38:01.011706+0000","last_peered":"2024-06-06T08:38:01.011706+0000","last_clean":"2024-06-06T08:38:01.011706+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:38:01.011706+0000","last_undegraded":"2024-06-06T08:38:01.011706+0000","last_fullsized":"2024-06-06T08:38:01.011706+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015846+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:38:01.015846+0000","last_peered":"2024-06-06T08:38:01.015846+0000","last_clean":"2024-06-06T08:38:01.015846+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:38:01.015846+0000","last_undegraded":"2024-06-06T08:38:01.015846+0000","last_fullsized":"2024-06-06T08:38:01.015846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.465807+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:38:58.465807+0000","last_peered":"2024-06-06T08:38:58.465807+0000","last_clean":"2024-06-06T08:38:58.465807+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:38:58.465807+0000","last_undegraded":"2024-06-06T08:38:58.465807+0000","last_fullsized":"2024-06-06T08:38:58.465807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.015077+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:38:01.015077+0000","last_peered":"2024-06-06T08:38:01.015077+0000","last_clean":"2024-06-06T08:38:01.015077+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:38:01.015077+0000","last_undegraded":"2024-06-06T08:38:01.015077+0000","last_fullsized":"2024-06-06T08:38:01.015077+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:01.151056+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:38:01.151056+0000","last_peered":"2024-06-06T08:38:01.151056+0000","last_clean":"2024-06-06T08:38:01.151056+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:38:01.151056+0000","last_undegraded":"2024-06-06T08:38:01.151056+0000","last_fullsized":"2024-06-06T08:38:01.151056+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.467697+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:38:58.467697+0000","last_peered":"2024-06-06T08:38:58.467697+0000","last_clean":"2024-06-06T08:38:58.467697+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:38:58.467697+0000","last_undegraded":"2024-06-06T08:38:58.467697+0000","last_fullsized":"2024-06-06T08:38:58.467697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.458255+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:38:58.458255+0000","last_peered":"2024-06-06T08:38:58.458255+0000","last_clean":"2024-06-06T08:38:58.458255+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:38:58.458255+0000","last_undegraded":"2024-06-06T08:38:58.458255+0000","last_fullsized":"2024-06-06T08:38:58.458255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.458387+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:38:58.458387+0000","last_peered":"2024-06-06T08:38:58.458387+0000","last_clean":"2024-06-06T08:38:58.458387+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:38:58.458387+0000","last_undegraded":"2024-06-06T08:38:58.458387+0000","last_fullsized":"2024-06-06T08:38:58.458387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.474063+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:38:58.474063+0000","last_peered":"2024-06-06T08:38:58.474063+0000","last_clean":"2024-06-06T08:38:58.474063+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:38:58.474063+0000","last_undegraded":"2024-06-06T08:38:58.474063+0000","last_fullsized":"2024-06-06T08:38:58.474063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.473465+0000","last_change":"2024-06-06T08:38:01.151003+0000","last_active":"2024-06-06T08:38:58.473465+0000","last_peered":"2024-06-06T08:38:58.473465+0000","last_clean":"2024-06-06T08:38:58.473465+0000","last_became_active":"2024-06-06T08:38:01.150557+0000","last_became_peered":"2024-06-06T08:38:01.150557+0000","last_unstale":"2024-06-06T08:38:58.473465+0000","last_undegraded":"2024-06-06T08:38:58.473465+0000","last_fullsized":"2024-06-06T08:38:58.473465+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:29:44.629894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475187+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:38:58.475187+0000","last_peered":"2024-06-06T08:38:58.475187+0000","last_clean":"2024-06-06T08:38:58.475187+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:38:58.475187+0000","last_undegraded":"2024-06-06T08:38:58.475187+0000","last_fullsized":"2024-06-06T08:38:58.475187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.450072+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:38:58.450072+0000","last_peered":"2024-06-06T08:38:58.450072+0000","last_clean":"2024-06-06T08:38:58.450072+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:38:58.450072+0000","last_undegraded":"2024-06-06T08:38:58.450072+0000","last_fullsized":"2024-06-06T08:38:58.450072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.457657+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:38:58.457657+0000","last_peered":"2024-06-06T08:38:58.457657+0000","last_clean":"2024-06-06T08:38:58.457657+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:38:58.457657+0000","last_undegraded":"2024-06-06T08:38:58.457657+0000","last_fullsized":"2024-06-06T08:38:58.457657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.451104+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:38:58.451104+0000","last_peered":"2024-06-06T08:38:58.451104+0000","last_clean":"2024-06-06T08:38:58.451104+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:38:58.451104+0000","last_undegraded":"2024-06-06T08:38:58.451104+0000","last_fullsized":"2024-06-06T08:38:58.451104+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.463786+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:38:58.463786+0000","last_peered":"2024-06-06T08:38:58.463786+0000","last_clean":"2024-06-06T08:38:58.463786+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:38:58.463786+0000","last_undegraded":"2024-06-06T08:38:58.463786+0000","last_fullsized":"2024-06-06T08:38:58.463786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.469407+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:38:58.469407+0000","last_peered":"2024-06-06T08:38:58.469407+0000","last_clean":"2024-06-06T08:38:58.469407+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:38:58.469407+0000","last_undegraded":"2024-06-06T08:38:58.469407+0000","last_fullsized":"2024-06-06T08:38:58.469407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475125+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:38:58.475125+0000","last_peered":"2024-06-06T08:38:58.475125+0000","last_clean":"2024-06-06T08:38:58.475125+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:38:58.475125+0000","last_undegraded":"2024-06-06T08:38:58.475125+0000","last_fullsized":"2024-06-06T08:38:58.475125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.475402+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:38:58.475402+0000","last_peered":"2024-06-06T08:38:58.475402+0000","last_clean":"2024-06-06T08:38:58.475402+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:38:58.475402+0000","last_undegraded":"2024-06-06T08:38:58.475402+0000","last_fullsized":"2024-06-06T08:38:58.475402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-06T08:38:58.455776+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:38:58.455776+0000","last_peered":"2024-06-06T08:38:58.455776+0000","last_clean":"2024-06-06T08:38:58.455776+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:38:58.455776+0000","last_undegraded":"2024-06-06T08:38:58.455776+0000","last_fullsized":"2024-06-06T08:38:58.455776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280848,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66460,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":625,"num_read_kb":385,"num_write":176,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383730,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":569943,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645367,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32388,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712764,"statfs":{"total":95995035648,"available":95961870336,"internally_reserved":0,"allocated":1241088,"data_stored":1220039,"data_compressed":68108,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808892,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":574253,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070529,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":570055,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332166,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1166126,"data_compressed":68108,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495692,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":565333,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":627562,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":566721,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":566593,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":565527,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":632833,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":1161183,"data_compressed":68108,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34081,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:39:08.804 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T08:39:08.804 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-06T08:39:08.804 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-06T08:39:08.804 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-06T08:39:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:09.403 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:09 smithi027 ceph-mon[32335]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:39:09.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:09 smithi203 ceph-mon[36579]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:39:09.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:09 smithi176 ceph-mon[36786]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:39:10.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:10 smithi027 ceph-mon[32335]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T08:39:10.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:10 smithi203 ceph-mon[36579]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T08:39:10.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:10 smithi176 ceph-mon[36786]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T08:39:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.742+0000 7f5569b3d700 1 -- 172.21.15.27:0/753106476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55641030f0 msgr2=0x7f5564107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.742+0000 7f5569b3d700 1 --2- 172.21.15.27:0/753106476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55641030f0 0x7f5564107110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f55580099c0 tx=0x7f5558009cd0 comp rx=0 tx=0).stop 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.742+0000 7f5569b3d700 1 -- 172.21.15.27:0/753106476 shutdown_connections 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.742+0000 7f5569b3d700 1 --2- 172.21.15.27:0/753106476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5564107840 0x7f556410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.742+0000 7f5569b3d700 1 --2- 172.21.15.27:0/753106476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5564102740 0x7f5564102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.742+0000 7f5569b3d700 1 --2- 172.21.15.27:0/753106476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55641030f0 0x7f5564107110 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.742+0000 7f5569b3d700 1 -- 172.21.15.27:0/753106476 >> 172.21.15.27:0/753106476 conn(0x7f55640795f0 msgr2=0x7f5564079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.742+0000 7f5569b3d700 1 -- 172.21.15.27:0/753106476 shutdown_connections 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.742+0000 7f5569b3d700 1 -- 172.21.15.27:0/753106476 wait complete. 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5569b3d700 1 Processor -- start 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5569b3d700 1 -- start start 2024-06-06T08:39:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5569b3d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5564102740 0x7f556419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5569b3d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55641030f0 0x7f556419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5569b3d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5564107840 0x7f5564198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5569b3d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5564112850 con 0x7f5564102740 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5569b3d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f55641126d0 con 0x7f5564107840 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5569b3d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f55641129d0 con 0x7f55641030f0 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f55637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5564102740 0x7f556419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f55637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5564102740 0x7f556419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33992/0 (socket says 172.21.15.27:33992) 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f55637fe700 1 -- 172.21.15.27:0/1909312338 learned_addr learned my addr 172.21.15.27:0/1909312338 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5563fff700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5564107840 0x7f5564198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f5562ffd700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55641030f0 0x7f556419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f55637fe700 1 -- 172.21.15.27:0/1909312338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55641030f0 msgr2=0x7f556419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.743+0000 7f55637fe700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55641030f0 0x7f556419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f55637fe700 1 -- 172.21.15.27:0/1909312338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5564107840 msgr2=0x7f5564198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f55637fe700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5564107840 0x7f5564198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f55637fe700 1 -- 172.21.15.27:0/1909312338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5564199410 con 0x7f5564102740 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f5563fff700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5564107840 0x7f5564198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f55637fe700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5564102740 0x7f556419ea60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f555400b530 tx=0x7f555400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f5560ff9700 1 -- 172.21.15.27:0/1909312338 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554015020 con 0x7f5564102740 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5564199700 con 0x7f5564102740 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f5560ff9700 1 -- 172.21.15.27:0/1909312338 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5554011d10 con 0x7f5564102740 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f5560ff9700 1 -- 172.21.15.27:0/1909312338 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554010e50 con 0x7f5564102740 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.744+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55641a59e0 con 0x7f5564102740 2024-06-06T08:39:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.745+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556410aa80 con 0x7f5564102740 2024-06-06T08:39:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.746+0000 7f5560ff9700 1 -- 172.21.15.27:0/1909312338 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5554018020 con 0x7f5564102740 2024-06-06T08:39:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.747+0000 7f5560ff9700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f554c0a2dc0 0x7f554c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.747+0000 7f5562ffd700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f554c0a2dc0 0x7f554c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.747+0000 7f5560ff9700 1 -- 172.21.15.27:0/1909312338 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f55540cec70 con 0x7f5564102740 2024-06-06T08:39:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.747+0000 7f5562ffd700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f554c0a2dc0 0x7f554c0a5280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5558006010 tx=0x7f5558000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:10.752+0000 7f5560ff9700 1 -- 172.21.15.27:0/1909312338 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f555401f020 con 0x7f5564102740 2024-06-06T08:39:11.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.008+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5564103bd0 con 0x7f5564102740 2024-06-06T08:39:11.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.008+0000 7f5560ff9700 1 -- 172.21.15.27:0/1909312338 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f555409b7c0 con 0x7f5564102740 2024-06-06T08:39:11.007 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:11.007 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-06T08:39:11.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f554c0a2dc0 msgr2=0x7f554c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:11.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f554c0a2dc0 0x7f554c0a5280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5558006010 tx=0x7f5558000bc0 comp rx=0 tx=0).stop 2024-06-06T08:39:11.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5564102740 msgr2=0x7f556419ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:11.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5564102740 0x7f556419ea60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f555400b530 tx=0x7f555400b8f0 comp rx=0 tx=0).stop 2024-06-06T08:39:11.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 shutdown_connections 2024-06-06T08:39:11.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f554c0a2dc0 0x7f554c0a5280 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:11.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5564107840 0x7f5564198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5564102740 0x7f556419ea60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 --2- 172.21.15.27:0/1909312338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55641030f0 0x7f556419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.010+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 >> 172.21.15.27:0/1909312338 conn(0x7f55640795f0 msgr2=0x7f5564100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.011+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 shutdown_connections 2024-06-06T08:39:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:11.011+0000 7f5569b3d700 1 -- 172.21.15.27:0/1909312338 wait complete. 2024-06-06T08:39:11.374 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:11 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1909312338' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T08:39:11.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:11 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1909312338' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T08:39:11.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:11 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1909312338' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T08:39:11.947 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-06T08:39:11.948 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:39:12.201 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:12.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:12 smithi027 ceph-mon[32335]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T08:39:12.407 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:12 smithi176 ceph-mon[36786]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T08:39:12.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:12 smithi203 ceph-mon[36579]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T08:39:13.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.946+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/306943212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf41024e0 msgr2=0x7f6bf4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:13.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.946+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/306943212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf41024e0 0x7f6bf4102940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009990 tx=0x7f6be4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:39:13.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.947+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/306943212 shutdown_connections 2024-06-06T08:39:13.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.947+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/306943212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf4102e80 0x7f6bf410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.947+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/306943212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41084f0 0x7f6bf41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.947+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/306943212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf41024e0 0x7f6bf4102940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.947+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/306943212 >> 172.21.15.27:0/306943212 conn(0x7f6bf4079530 msgr2=0x7f6bf4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.947+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/306943212 shutdown_connections 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.947+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/306943212 wait complete. 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.947+0000 7f6bfb4e4700 1 Processor -- start 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.947+0000 7f6bfb4e4700 1 -- start start 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bfb4e4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41024e0 0x7f6bf419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bfb4e4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf4102e80 0x7f6bf419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bfb4e4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf41084f0 0x7f6bf4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bfb4e4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41127b0 con 0x7f6bf4102e80 2024-06-06T08:39:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bfb4e4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4112630 con 0x7f6bf41084f0 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bfb4e4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4112930 con 0x7f6bf41024e0 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf8a7f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf4102e80 0x7f6bf419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf8a7f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf4102e80 0x7f6bf419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33998/0 (socket says 172.21.15.27:33998) 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf8a7f700 1 -- 172.21.15.27:0/3958312884 learned_addr learned my addr 172.21.15.27:0/3958312884 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf9a81700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf41084f0 0x7f6bf4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf9280700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41024e0 0x7f6bf419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf8a7f700 1 -- 172.21.15.27:0/3958312884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41024e0 msgr2=0x7f6bf419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf8a7f700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41024e0 0x7f6bf419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf8a7f700 1 -- 172.21.15.27:0/3958312884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf41084f0 msgr2=0x7f6bf4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf8a7f700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf41084f0 0x7f6bf4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf8a7f700 1 -- 172.21.15.27:0/3958312884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf4199230 con 0x7f6bf4102e80 2024-06-06T08:39:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bf8a7f700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf4102e80 0x7f6bf419ee50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6be400bff0 tx=0x7f6be4005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:13.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.948+0000 7f6bea7fc700 1 -- 172.21.15.27:0/3958312884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4027070 con 0x7f6bf4102e80 2024-06-06T08:39:13.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.949+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf41994c0 con 0x7f6bf4102e80 2024-06-06T08:39:13.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.949+0000 7f6bea7fc700 1 -- 172.21.15.27:0/3958312884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be4004050 con 0x7f6bf4102e80 2024-06-06T08:39:13.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.949+0000 7f6bea7fc700 1 -- 172.21.15.27:0/3958312884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4022d00 con 0x7f6bf4102e80 2024-06-06T08:39:13.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.949+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf41a57e0 con 0x7f6bf4102e80 2024-06-06T08:39:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.950+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf4069180 con 0x7f6bf4102e80 2024-06-06T08:39:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.951+0000 7f6bea7fc700 1 -- 172.21.15.27:0/3958312884 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6be402a070 con 0x7f6bf4102e80 2024-06-06T08:39:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.951+0000 7f6bea7fc700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6be00a2c20 0x7f6be00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.951+0000 7f6bf9280700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6be00a2c20 0x7f6be00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.952+0000 7f6bea7fc700 1 -- 172.21.15.27:0/3958312884 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f6be40d05e0 con 0x7f6bf4102e80 2024-06-06T08:39:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.952+0000 7f6bf9280700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6be00a2c20 0x7f6be00a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009a10 tx=0x7f6bf0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:13.956+0000 7f6bea7fc700 1 -- 172.21.15.27:0/3958312884 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6be4013070 con 0x7f6bf4102e80 2024-06-06T08:39:14.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:14 smithi027 ceph-mon[32335]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T08:39:14.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.208+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bf402dca0 con 0x7f6bf4102e80 2024-06-06T08:39:14.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.209+0000 7f6bea7fc700 1 -- 172.21.15.27:0/3958312884 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11882 (secure 0 0 0) 0x7f6be409d200 con 0x7f6bf4102e80 2024-06-06T08:39:14.208 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:14.208 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:37:27.822986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24458,24460]},"id":1}]} 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6be00a2c20 msgr2=0x7f6be00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6be00a2c20 0x7f6be00a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009a10 tx=0x7f6bf0009190 comp rx=0 tx=0).stop 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf4102e80 msgr2=0x7f6bf419ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf4102e80 0x7f6bf419ee50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6be400bff0 tx=0x7f6be4005590 comp rx=0 tx=0).stop 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 shutdown_connections 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f6be00a2c20 0x7f6be00a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6bf41084f0 0x7f6bf4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bf41024e0 0x7f6bf419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 --2- 172.21.15.27:0/3958312884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bf4102e80 0x7f6bf419ee50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 >> 172.21.15.27:0/3958312884 conn(0x7f6bf4079530 msgr2=0x7f6bf4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 shutdown_connections 2024-06-06T08:39:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:14.211+0000 7f6bfb4e4700 1 -- 172.21.15.27:0/3958312884 wait complete. 2024-06-06T08:39:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-06-06T08:39:14.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:14 smithi203 ceph-mon[36579]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T08:39:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:14 smithi176 ceph-mon[36786]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T08:39:14.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:39:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:15.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:15 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3958312884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:15.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:15 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3958312884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:15.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:15 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3958312884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:16.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:16 smithi027 ceph-mon[32335]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:39:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:16 smithi203 ceph-mon[36579]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:39:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:16 smithi176 ceph-mon[36786]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:39:16.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.944+0000 7fdad912f700 1 -- 172.21.15.27:0/462817833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad4101390 msgr2=0x7fdad41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:16.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.944+0000 7fdad912f700 1 --2- 172.21.15.27:0/462817833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad4101390 0x7fdad41017f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009990 tx=0x7fdac8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:39:16.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.944+0000 7fdad912f700 1 -- 172.21.15.27:0/462817833 shutdown_connections 2024-06-06T08:39:16.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.944+0000 7fdad912f700 1 --2- 172.21.15.27:0/462817833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdad4101d30 0x7fdad410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:16.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.944+0000 7fdad912f700 1 --2- 172.21.15.27:0/462817833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad41073d0 0x7fdad41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:16.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.944+0000 7fdad912f700 1 --2- 172.21.15.27:0/462817833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad4101390 0x7fdad41017f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:16.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.944+0000 7fdad912f700 1 -- 172.21.15.27:0/462817833 >> 172.21.15.27:0/462817833 conn(0x7fdad40782c0 msgr2=0x7fdad40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:16.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.944+0000 7fdad912f700 1 -- 172.21.15.27:0/462817833 shutdown_connections 2024-06-06T08:39:16.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.944+0000 7fdad912f700 1 -- 172.21.15.27:0/462817833 wait complete. 2024-06-06T08:39:16.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad912f700 1 Processor -- start 2024-06-06T08:39:16.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad912f700 1 -- start start 2024-06-06T08:39:16.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad912f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdad4101390 0x7fdad4103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:16.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad912f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad4101d30 0x7fdad4103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:16.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad912f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad41073d0 0x7fdad4103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad912f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4111680 con 0x7fdad41073d0 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad912f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4111500 con 0x7fdad4101390 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad912f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4111800 con 0x7fdad4101d30 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad2d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdad4101390 0x7fdad4103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad2d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdad4101390 0x7fdad4103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:60402/0 (socket says 172.21.15.27:60402) 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad41073d0 0x7fdad4103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad2d9d700 1 -- 172.21.15.27:0/2192847496 learned_addr learned my addr 172.21.15.27:0/2192847496 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.945+0000 7fdad259c700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad4101d30 0x7fdad4103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdad2d9d700 1 -- 172.21.15.27:0/2192847496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad4101d30 msgr2=0x7fdad4103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdad2d9d700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad4101d30 0x7fdad4103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdad2d9d700 1 -- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad41073d0 msgr2=0x7fdad4103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:16.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdad2d9d700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad41073d0 0x7fdad4103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:16.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdad2d9d700 1 -- 172.21.15.27:0/2192847496 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad41a7190 con 0x7fdad4101390 2024-06-06T08:39:16.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdad359e700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad41073d0 0x7fdad4103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:39:16.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdad2d9d700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdad4101390 0x7fdad4103040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdac400c970 tx=0x7fdac400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:16.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdabbfff700 1 -- 172.21.15.27:0/2192847496 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4007aa0 con 0x7fdad4101390 2024-06-06T08:39:16.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41a7480 con 0x7fdad4101390 2024-06-06T08:39:16.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdabbfff700 1 -- 172.21.15.27:0/2192847496 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac4004d10 con 0x7fdad4101390 2024-06-06T08:39:16.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdabbfff700 1 -- 172.21.15.27:0/2192847496 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac400f450 con 0x7fdad4101390 2024-06-06T08:39:16.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.946+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41a79c0 con 0x7fdad4101390 2024-06-06T08:39:16.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.947+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad4069180 con 0x7fdad4101390 2024-06-06T08:39:16.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.949+0000 7fdabbfff700 1 -- 172.21.15.27:0/2192847496 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdac4004750 con 0x7fdad4101390 2024-06-06T08:39:16.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.949+0000 7fdabbfff700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdabc0a2c20 0x7fdabc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:16.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.949+0000 7fdad259c700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdabc0a2c20 0x7fdabc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:16.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.949+0000 7fdabbfff700 1 -- 172.21.15.27:0/2192847496 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fdac40cf440 con 0x7fdad4101390 2024-06-06T08:39:16.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.950+0000 7fdad259c700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdabc0a2c20 0x7fdabc0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdac800b5c0 tx=0x7fdac8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:16.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:16.953+0000 7fdabbfff700 1 -- 172.21.15.27:0/2192847496 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdac409c0b0 con 0x7fdad4101390 2024-06-06T08:39:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.205+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdad41a7c80 con 0x7fdad4101390 2024-06-06T08:39:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.206+0000 7fdabbfff700 1 -- 172.21.15.27:0/2192847496 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11882 (secure 0 0 0) 0x7fdac4005700 con 0x7fdad4101390 2024-06-06T08:39:17.205 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:37:27.822986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24458,24460]},"id":1}]} 2024-06-06T08:39:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdabc0a2c20 msgr2=0x7fdabc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdabc0a2c20 0x7fdabc0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdac800b5c0 tx=0x7fdac8005fd0 comp rx=0 tx=0).stop 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdad4101390 msgr2=0x7fdad4103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdad4101390 0x7fdad4103040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdac400c970 tx=0x7fdac400cc80 comp rx=0 tx=0).stop 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 shutdown_connections 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fdabc0a2c20 0x7fdabc0a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad41073d0 0x7fdad4103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdad4101390 0x7fdad4103040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 --2- 172.21.15.27:0/2192847496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdad4101d30 0x7fdad4103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 >> 172.21.15.27:0/2192847496 conn(0x7fdad40782c0 msgr2=0x7fdad41006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 shutdown_connections 2024-06-06T08:39:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:17.209+0000 7fdad912f700 1 -- 172.21.15.27:0/2192847496 wait complete. 2024-06-06T08:39:17.209 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-06-06T08:39:17.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:39:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:18.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:18 smithi027 ceph-mon[32335]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:18.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:18 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2192847496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:18.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:18 smithi203 ceph-mon[36579]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:18 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2192847496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:18.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:18 smithi176 ceph-mon[36786]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:18.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:18 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2192847496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:19.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.916+0000 7f437be97700 1 -- 172.21.15.27:0/1610705236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741084f0 msgr2=0x7f43741088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:19.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.916+0000 7f437be97700 1 --2- 172.21.15.27:0/1610705236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741084f0 0x7f43741088d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f436400b0a0 tx=0x7f436400b3b0 comp rx=0 tx=0).stop 2024-06-06T08:39:19.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.916+0000 7f437be97700 1 -- 172.21.15.27:0/1610705236 shutdown_connections 2024-06-06T08:39:19.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.916+0000 7f437be97700 1 --2- 172.21.15.27:0/1610705236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4374102e80 0x7f437410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:19.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.916+0000 7f437be97700 1 --2- 172.21.15.27:0/1610705236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741084f0 0x7f43741088d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:19.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.916+0000 7f437be97700 1 --2- 172.21.15.27:0/1610705236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43741024e0 0x7f4374102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:19.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.916+0000 7f437be97700 1 -- 172.21.15.27:0/1610705236 >> 172.21.15.27:0/1610705236 conn(0x7f4374079530 msgr2=0x7f4374079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:19.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 -- 172.21.15.27:0/1610705236 shutdown_connections 2024-06-06T08:39:19.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 -- 172.21.15.27:0/1610705236 wait complete. 2024-06-06T08:39:19.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 Processor -- start 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 -- start start 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741024e0 0x7f4374074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4374102e80 0x7f4374074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43741084f0 0x7f4374075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43741127b0 con 0x7f43741024e0 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4374112630 con 0x7f4374102e80 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.917+0000 7f437be97700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4374112930 con 0x7f43741084f0 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379c33700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741024e0 0x7f4374074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379c33700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741024e0 0x7f4374074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36540/0 (socket says 172.21.15.27:36540) 2024-06-06T08:39:19.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379c33700 1 -- 172.21.15.27:0/3167382044 learned_addr learned my addr 172.21.15.27:0/3167382044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f437a434700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43741084f0 0x7f4374075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379432700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4374102e80 0x7f4374074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379c33700 1 -- 172.21.15.27:0/3167382044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43741084f0 msgr2=0x7f4374075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379c33700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43741084f0 0x7f4374075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379c33700 1 -- 172.21.15.27:0/3167382044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4374102e80 msgr2=0x7f4374074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379c33700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4374102e80 0x7f4374074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379c33700 1 -- 172.21.15.27:0/3167382044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43741a7190 con 0x7f43741024e0 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379432700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4374102e80 0x7f4374074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4379c33700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741024e0 0x7f4374074790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4364000f80 tx=0x7f436400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4372ffd700 1 -- 172.21.15.27:0/3167382044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364021070 con 0x7f43741024e0 2024-06-06T08:39:19.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4372ffd700 1 -- 172.21.15.27:0/3167382044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4364007db0 con 0x7f43741024e0 2024-06-06T08:39:19.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43741a7420 con 0x7f43741024e0 2024-06-06T08:39:19.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f4372ffd700 1 -- 172.21.15.27:0/3167382044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364005550 con 0x7f43741024e0 2024-06-06T08:39:19.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.918+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43741a7920 con 0x7f43741024e0 2024-06-06T08:39:19.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.919+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4374069180 con 0x7f43741024e0 2024-06-06T08:39:19.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.921+0000 7f4372ffd700 1 -- 172.21.15.27:0/3167382044 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4364024070 con 0x7f43741024e0 2024-06-06T08:39:19.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.921+0000 7f4372ffd700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f435c0a2ca0 0x7f435c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:19.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.921+0000 7f4379432700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f435c0a2ca0 0x7f435c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:19.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.921+0000 7f4372ffd700 1 -- 172.21.15.27:0/3167382044 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f436400f020 con 0x7f43741024e0 2024-06-06T08:39:19.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.922+0000 7f4379432700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f435c0a2ca0 0x7f435c0a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4368005fd0 tx=0x7f4368005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:19.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:19.926+0000 7f4372ffd700 1 -- 172.21.15.27:0/3167382044 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f43640d3030 con 0x7f43741024e0 2024-06-06T08:39:20.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.135+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f437402dca0 con 0x7f43741024e0 2024-06-06T08:39:20.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.136+0000 7f4372ffd700 1 -- 172.21.15.27:0/3167382044 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24836 (secure 0 0 0) 0x7f436402b060 con 0x7f43741024e0 2024-06-06T08:39:20.135 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:20.135 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:38:01.008015+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:39:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f435c0a2ca0 msgr2=0x7f435c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f435c0a2ca0 0x7f435c0a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4368005fd0 tx=0x7f4368005dc0 comp rx=0 tx=0).stop 2024-06-06T08:39:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741024e0 msgr2=0x7f4374074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:20.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741024e0 0x7f4374074790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4364000f80 tx=0x7f436400bf10 comp rx=0 tx=0).stop 2024-06-06T08:39:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 shutdown_connections 2024-06-06T08:39:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f435c0a2ca0 0x7f435c0a5160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43741084f0 0x7f4374075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43741024e0 0x7f4374074790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 --2- 172.21.15.27:0/3167382044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4374102e80 0x7f4374074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 >> 172.21.15.27:0/3167382044 conn(0x7f4374079530 msgr2=0x7f4374100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 shutdown_connections 2024-06-06T08:39:20.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:20.139+0000 7f437be97700 1 -- 172.21.15.27:0/3167382044 wait complete. 2024-06-06T08:39:20.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:20 smithi027 ceph-mon[32335]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:20.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:20 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3167382044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:39:20.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:20 smithi203 ceph-mon[36579]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:20.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:20 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3167382044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:39:20.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:20 smithi176 ceph-mon[36786]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:20.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:20 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3167382044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:39:20.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:39:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:22.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:22 smithi027 ceph-mon[32335]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:22.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:22 smithi203 ceph-mon[36579]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:22.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:22 smithi176 ceph-mon[36786]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.868+0000 7fccec65e700 1 -- 172.21.15.27:0/1487576696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4102740 msgr2=0x7fcce4102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.868+0000 7fccec65e700 1 --2- 172.21.15.27:0/1487576696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4102740 0x7fcce4102b20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcce000b0d0 tx=0x7fcce000b3e0 comp rx=0 tx=0).stop 2024-06-06T08:39:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.868+0000 7fccec65e700 1 -- 172.21.15.27:0/1487576696 shutdown_connections 2024-06-06T08:39:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.868+0000 7fccec65e700 1 --2- 172.21.15.27:0/1487576696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce4107840 0x7fcce410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.868+0000 7fccec65e700 1 --2- 172.21.15.27:0/1487576696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4102740 0x7fcce4102b20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.868+0000 7fccec65e700 1 --2- 172.21.15.27:0/1487576696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcce41030f0 0x7fcce4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.868+0000 7fccec65e700 1 -- 172.21.15.27:0/1487576696 >> 172.21.15.27:0/1487576696 conn(0x7fcce40795f0 msgr2=0x7fcce4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.868+0000 7fccec65e700 1 -- 172.21.15.27:0/1487576696 shutdown_connections 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.868+0000 7fccec65e700 1 -- 172.21.15.27:0/1487576696 wait complete. 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fccec65e700 1 Processor -- start 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fccec65e700 1 -- start start 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fccec65e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcce4102740 0x7fcce419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fccec65e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce41030f0 0x7fcce419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fccec65e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4107840 0x7fcce41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fccec65e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcce41127b0 con 0x7fcce4107840 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fccec65e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4112630 con 0x7fcce41030f0 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fccec65e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4112930 con 0x7fcce4102740 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fcceabfb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4107840 0x7fcce41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fcceabfb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4107840 0x7fcce41989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36568/0 (socket says 172.21.15.27:36568) 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fcceabfb700 1 -- 172.21.15.27:0/771471598 learned_addr learned my addr 172.21.15.27:0/771471598 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:22.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fcce9bf9700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce41030f0 0x7fcce419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fccea3fa700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcce4102740 0x7fcce419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fcceabfb700 1 -- 172.21.15.27:0/771471598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcce4102740 msgr2=0x7fcce419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fcceabfb700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcce4102740 0x7fcce419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fcceabfb700 1 -- 172.21.15.27:0/771471598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce41030f0 msgr2=0x7fcce419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fcceabfb700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce41030f0 0x7fcce419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.869+0000 7fcceabfb700 1 -- 172.21.15.27:0/771471598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce4199150 con 0x7fcce4107840 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.870+0000 7fcceabfb700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4107840 0x7fcce41989f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e940 tx=0x7fccdc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.870+0000 7fccdb7fe700 1 -- 172.21.15.27:0/771471598 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00cbd0 con 0x7fcce4107840 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.870+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce4199440 con 0x7fcce4107840 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.870+0000 7fccdb7fe700 1 -- 172.21.15.27:0/771471598 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccdc00ee90 con 0x7fcce4107840 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.870+0000 7fccdb7fe700 1 -- 172.21.15.27:0/771471598 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0056c0 con 0x7fcce4107840 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.870+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce41a56b0 con 0x7fcce4107840 2024-06-06T08:39:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.871+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce410aa80 con 0x7fcce4107840 2024-06-06T08:39:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.872+0000 7fccdb7fe700 1 -- 172.21.15.27:0/771471598 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fccdc004750 con 0x7fcce4107840 2024-06-06T08:39:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.873+0000 7fccdb7fe700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fccd00a2c20 0x7fccd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.873+0000 7fccea3fa700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fccd00a2c20 0x7fccd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.873+0000 7fccdb7fe700 1 -- 172.21.15.27:0/771471598 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fccdc0cf920 con 0x7fcce4107840 2024-06-06T08:39:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.873+0000 7fccea3fa700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fccd00a2c20 0x7fccd00a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009fd0 tx=0x7fcce000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:22.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:22.876+0000 7fccdb7fe700 1 -- 172.21.15.27:0/771471598 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fccdc053850 con 0x7fcce4107840 2024-06-06T08:39:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.131+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcce4199ed0 con 0x7fcce4107840 2024-06-06T08:39:23.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.131+0000 7fccdb7fe700 1 -- 172.21.15.27:0/771471598 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11882 (secure 0 0 0) 0x7fccdc09c4b0 con 0x7fcce4107840 2024-06-06T08:39:23.130 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:23.130 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:37:27.822986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24458,24460]},"id":1}]} 2024-06-06T08:39:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.134+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fccd00a2c20 msgr2=0x7fccd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.134+0000 7fccec65e700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fccd00a2c20 0x7fccd00a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009fd0 tx=0x7fcce000b040 comp rx=0 tx=0).stop 2024-06-06T08:39:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.134+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4107840 msgr2=0x7fcce41989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.134+0000 7fccec65e700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4107840 0x7fcce41989f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e940 tx=0x7fccdc00ec50 comp rx=0 tx=0).stop 2024-06-06T08:39:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.135+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 shutdown_connections 2024-06-06T08:39:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.135+0000 7fccec65e700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fccd00a2c20 0x7fccd00a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.135+0000 7fccec65e700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcce4107840 0x7fcce41989f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.135+0000 7fccec65e700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcce4102740 0x7fcce419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.135+0000 7fccec65e700 1 --2- 172.21.15.27:0/771471598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcce41030f0 0x7fcce419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.135+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 >> 172.21.15.27:0/771471598 conn(0x7fcce40795f0 msgr2=0x7fcce4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.135+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 shutdown_connections 2024-06-06T08:39:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:23.135+0000 7fccec65e700 1 -- 172.21.15.27:0/771471598 wait complete. 2024-06-06T08:39:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-06-06T08:39:23.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:23 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/771471598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:23.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:23 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/771471598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:23.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:23 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/771471598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:23.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T08:36:23.190518+0000', 'modified': '2024-06-06T08:37:27.822986+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24460, 'mds_2': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/2823423002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 2823423002}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 2823423002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6835/1368203014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6834', 'nonce': 1368203014}, {'type': 'v1', 'addr': '172.21.15.176:6835', 'nonce': 1368203014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/266251643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 266251643}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 266251643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.176:6837/3697558303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 3697558303}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 3697558303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.203:6839/693474346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 693474346}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 693474346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.176:6839/1702529484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 1702529484}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 1702529484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482, 24458, 24460]} 2024-06-06T08:39:23.962 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-06T08:39:23.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T08:39:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:24 smithi027 ceph-mon[32335]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:24.214 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:24.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:24 smithi203 ceph-mon[36579]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:24.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:24 smithi176 ceph-mon[36786]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:25.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 -- 172.21.15.27:0/3381386670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101380 msgr2=0x7faea4101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:25.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 --2- 172.21.15.27:0/3381386670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101380 0x7faea4101800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fae98009990 tx=0x7fae98009ca0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 -- 172.21.15.27:0/3381386670 shutdown_connections 2024-06-06T08:39:25.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 --2- 172.21.15.27:0/3381386670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea4101d40 0x7faea410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 --2- 172.21.15.27:0/3381386670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faea41073f0 0x7faea41077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 --2- 172.21.15.27:0/3381386670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101380 0x7faea4101800 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 -- 172.21.15.27:0/3381386670 >> 172.21.15.27:0/3381386670 conn(0x7faea4078300 msgr2=0x7faea4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 -- 172.21.15.27:0/3381386670 shutdown_connections 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 -- 172.21.15.27:0/3381386670 wait complete. 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.944+0000 7faeab5b6700 1 Processor -- start 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faeab5b6700 1 -- start start 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faeab5b6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faea4101380 0x7faea419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faeab5b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101d40 0x7faea419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faeab5b6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea41073f0 0x7faea41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faeab5b6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faea4111690 con 0x7faea4101d40 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faeab5b6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7faea4111510 con 0x7faea4101380 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faeab5b6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faea4111810 con 0x7faea41073f0 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea8b51700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101d40 0x7faea419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea8b51700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101d40 0x7faea419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36598/0 (socket says 172.21.15.27:36598) 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea8b51700 1 -- 172.21.15.27:0/4117285922 learned_addr learned my addr 172.21.15.27:0/4117285922 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea9352700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faea4101380 0x7faea419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea9b53700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea41073f0 0x7faea41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea8b51700 1 -- 172.21.15.27:0/4117285922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea41073f0 msgr2=0x7faea41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea8b51700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea41073f0 0x7faea41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea8b51700 1 -- 172.21.15.27:0/4117285922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faea4101380 msgr2=0x7faea419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea8b51700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faea4101380 0x7faea419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea8b51700 1 -- 172.21.15.27:0/4117285922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea4199140 con 0x7faea4101d40 2024-06-06T08:39:25.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.945+0000 7faea8b51700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101d40 0x7faea419ed60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fae98000c00 tx=0x7fae980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:25.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.946+0000 7faea27fc700 1 -- 172.21.15.27:0/4117285922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98027070 con 0x7faea4101d40 2024-06-06T08:39:25.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.946+0000 7faeab5b6700 1 -- 172.21.15.27:0/4117285922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea41993d0 con 0x7faea4101d40 2024-06-06T08:39:25.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.946+0000 7faea27fc700 1 -- 172.21.15.27:0/4117285922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae98003e40 con 0x7faea4101d40 2024-06-06T08:39:25.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.946+0000 7faea27fc700 1 -- 172.21.15.27:0/4117285922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98022d10 con 0x7faea4101d40 2024-06-06T08:39:25.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.946+0000 7faeab5b6700 1 -- 172.21.15.27:0/4117285922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea41a56f0 con 0x7faea4101d40 2024-06-06T08:39:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.948+0000 7faea27fc700 1 -- 172.21.15.27:0/4117285922 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae9802a070 con 0x7faea4101d40 2024-06-06T08:39:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.948+0000 7faea27fc700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae8c0a2c20 0x7fae8c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.949+0000 7faea9352700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae8c0a2c20 0x7fae8c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.949+0000 7faea27fc700 1 -- 172.21.15.27:0/4117285922 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fae980d0210 con 0x7faea4101d40 2024-06-06T08:39:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.949+0000 7faea9352700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae8c0a2c20 0x7fae8c0a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae94009550 tx=0x7fae940091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faeb5e39b80 1 Processor -- start 2024-06-06T08:39:25.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faeb5e39b80 1 -- start start 2024-06-06T08:39:25.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 0x556faf07f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 0x556faf079620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf079fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556faef67f10 con 0x556faf079b60 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556faefc6800 con 0x556faf07fc40 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556faef2e410 con 0x556faf07f320 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea9b53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf079fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea9b53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf079fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36610/0 (socket says 172.21.15.27:36610) 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea9b53700 1 -- 172.21.15.27:0/1255948772 learned_addr learned my addr 172.21.15.27:0/1255948772 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea8b51700 1 --2- 172.21.15.27:0/1255948772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 0x556faf079620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea9352700 1 --2- 172.21.15.27:0/1255948772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 0x556faf07f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea9b53700 1 -- 172.21.15.27:0/1255948772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 msgr2=0x556faf07f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea9b53700 1 --2- 172.21.15.27:0/1255948772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 0x556faf07f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea9b53700 1 -- 172.21.15.27:0/1255948772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 msgr2=0x556faf079620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea9b53700 1 --2- 172.21.15.27:0/1255948772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 0x556faf079620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.950+0000 7faea9b53700 1 -- 172.21.15.27:0/1255948772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556faef15880 con 0x556faf079b60 2024-06-06T08:39:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.951+0000 7faea9b53700 1 --2- 172.21.15.27:0/1255948772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf079fa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00b6c0 tx=0x7fae9c00b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c7c911c9a4ba88 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.951+0000 7fae8b7fe700 1 -- 172.21.15.27:0/1255948772 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c004a30 con 0x556faf079b60 2024-06-06T08:39:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.951+0000 7fae8b7fe700 1 -- 172.21.15.27:0/1255948772 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9c015070 con 0x556faf079b60 2024-06-06T08:39:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.951+0000 7fae8b7fe700 1 -- 172.21.15.27:0/1255948772 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c003c20 con 0x556faf079b60 2024-06-06T08:39:25.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.952+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1255948772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 msgr2=0x556faf079fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:25.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.952+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/1255948772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf079fa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00b6c0 tx=0x7fae9c00b9d0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.952+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1255948772 shutdown_connections 2024-06-06T08:39:25.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.952+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/1255948772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf079fa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.952+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/1255948772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 0x556faf07f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.952+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/1255948772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 0x556faf079620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.952+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1255948772 >> 172.21.15.27:0/1255948772 conn(0x556faefa2030 msgr2=0x556faf0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.952+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1255948772 shutdown_connections 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.952+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1255948772 wait complete. 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faeb5e39b80 1 -- start start 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faeb5e39b80 1 Processor -- start 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf079b60 0x556faf085a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07f320 0x556faf082d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07fc40 0x556faf083260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556faef66f70 con 0x556faf07fc40 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556faefc27c0 con 0x556faf07f320 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556faf086630 con 0x556faf079b60 2024-06-06T08:39:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faeb5e39b80 10 client.0 _ll_register_callbacks cb 0x556faef5c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faea9b53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07fc40 0x556faf083260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.953+0000 7faea9b53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07fc40 0x556faf083260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36618/0 (socket says 172.21.15.27:36618) 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea9b53700 1 -- 172.21.15.27:0/2707333421 learned_addr learned my addr 172.21.15.27:0/2707333421 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea9352700 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf079b60 0x556faf085a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea8b51700 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07f320 0x556faf082d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea9b53700 1 -- 172.21.15.27:0/2707333421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf079b60 msgr2=0x556faf085a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea9b53700 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf079b60 0x556faf085a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea9b53700 1 -- 172.21.15.27:0/2707333421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07f320 msgr2=0x556faf082d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea9b53700 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07f320 0x556faf082d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea9b53700 1 -- 172.21.15.27:0/2707333421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556faef8c700 con 0x556faf07fc40 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea8b51700 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07f320 0x556faf082d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faea9b53700 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07fc40 0x556faf083260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00bdd0 tx=0x7fae9c004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34741e1714f730c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7fae88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:39:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7fae88ff9700 1 -- 172.21.15.27:0/2707333421 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c028060 con 0x556faf07fc40 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7fae88ff9700 1 -- 172.21.15.27:0/2707333421 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fae9c0051c0 con 0x556faf07fc40 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7faeb5e39b80 1 -- 172.21.15.27:0/2707333421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9c015070 con 0x556faf07fc40 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.954+0000 7fae88ff9700 1 -- 172.21.15.27:0/2707333421 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fae9c02f050 con 0x556faf07fc40 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7faeb5e39b80 10 client.15021 fetch_fsmap learned FSMap version 24 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7faeb5e39b80 1 -- 172.21.15.27:0/2707333421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556faf090dd0 con 0x556faf07fc40 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7fae88ff9700 1 -- 172.21.15.27:0/2707333421 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9c015070 con 0x556faf07fc40 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7fae88ff9700 1 -- 172.21.15.27:0/2707333421 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fae9c011440 con 0x556faf07fc40 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7faeb5e39b80 10 client.15021 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7faeb5e39b80 10 client.15021 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7faeb5e39b80 20 client.15021 populate_metadata read hostname 'smithi027' 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556faf092f70 0x556faf0921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7faeb5e39b80 4 client.15021 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:39:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7faeb5e39b80 1 -- 172.21.15.27:0/2707333421 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556faefd3d00 con 0x556faf092f70 2024-06-06T08:39:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.955+0000 7faea9352700 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556faf092f70 0x556faf0921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.956+0000 7fae88ff9700 10 client.15021 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:39:25.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.956+0000 7faea9352700 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556faf092f70 0x556faf0921d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28bf55bdb030336 server_cookie=e114b72f29bdb999 in_seq=0 out_seq=0 2024-06-06T08:39:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7fae88ff9700 1 -- 172.21.15.27:0/2707333421 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556faefd3d00 con 0x556faf092f70 2024-06-06T08:39:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7fae88ff9700 10 client.15021 handle_command_reply: tid=0 2024-06-06T08:39:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 client.15021 shutdown 2024-06-06T08:39:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 -- 172.21.15.27:0/2707333421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07fc40 msgr2=0x556faf083260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07fc40 0x556faf083260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00bdd0 tx=0x7fae9c004a80 comp rx=0 tx=0).stop 2024-06-06T08:39:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 -- 172.21.15.27:0/2707333421 shutdown_connections 2024-06-06T08:39:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556faf092f70 0x556faf0921d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07fc40 0x556faf083260 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf079b60 0x556faf085a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/2707333421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07f320 0x556faf082d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7fae88ff9700 0 client.15021 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:39:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 -- 172.21.15.27:0/2707333421 >> 172.21.15.27:0/2707333421 conn(0x556faefa2030 msgr2=0x556faf06f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 -- 172.21.15.27:0/2707333421 shutdown_connections 2024-06-06T08:39:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 1 -- 172.21.15.27:0/2707333421 wait complete. 2024-06-06T08:39:25.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.957+0000 7faeb5e39b80 20 client.15021 trim_cache size 0 max 16384 2024-06-06T08:39:25.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.981+0000 7faeb5e39b80 1 Processor -- start 2024-06-06T08:39:25.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.981+0000 7faeb5e39b80 1 -- start start 2024-06-06T08:39:25.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.981+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf079b60 0x556faf07d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.981+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07f320 0x556faf06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.981+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07fc40 0x556faf070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.981+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556faf092720 con 0x556faf07f320 2024-06-06T08:39:25.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.981+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556faf092df0 con 0x556faf079b60 2024-06-06T08:39:25.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.981+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556faf0934c0 con 0x556faf07fc40 2024-06-06T08:39:25.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.981+0000 7faea8b51700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07f320 0x556faf06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea8b51700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07f320 0x556faf06fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36634/0 (socket says 172.21.15.27:36634) 2024-06-06T08:39:25.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea8b51700 1 -- 172.21.15.27:0/1138180734 learned_addr learned my addr 172.21.15.27:0/1138180734 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:25.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea9352700 1 --2- 172.21.15.27:0/1138180734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf079b60 0x556faf07d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea8b51700 1 -- 172.21.15.27:0/1138180734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07fc40 msgr2=0x556faf070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:25.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea8b51700 1 --2- 172.21.15.27:0/1138180734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07fc40 0x556faf070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea8b51700 1 -- 172.21.15.27:0/1138180734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf079b60 msgr2=0x556faf07d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:25.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea8b51700 1 --2- 172.21.15.27:0/1138180734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf079b60 0x556faf07d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea8b51700 1 -- 172.21.15.27:0/1138180734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556faf086630 con 0x556faf07f320 2024-06-06T08:39:25.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea9352700 1 --2- 172.21.15.27:0/1138180734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf079b60 0x556faf07d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:25.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7faea8b51700 1 --2- 172.21.15.27:0/1138180734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07f320 0x556faf06fe30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fae9809c6c0 tx=0x7fae980a3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0cafed37b7cb7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:25.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7fae8affd700 1 -- 172.21.15.27:0/1138180734 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9809c730 con 0x556faf07f320 2024-06-06T08:39:25.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7fae8affd700 1 -- 172.21.15.27:0/1138180734 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae980a0ae0 con 0x556faf07f320 2024-06-06T08:39:25.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.982+0000 7fae8affd700 1 -- 172.21.15.27:0/1138180734 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98003ff0 con 0x556faf07f320 2024-06-06T08:39:25.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1138180734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07f320 msgr2=0x556faf06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:25.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/1138180734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07f320 0x556faf06fe30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fae9809c6c0 tx=0x7fae980a3040 comp rx=0 tx=0).stop 2024-06-06T08:39:25.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1138180734 shutdown_connections 2024-06-06T08:39:25.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/1138180734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07fc40 0x556faf070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/1138180734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf079b60 0x556faf07d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/1138180734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf07f320 0x556faf06fe30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1138180734 >> 172.21.15.27:0/1138180734 conn(0x556faefa2030 msgr2=0x556faf07cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:25.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1138180734 shutdown_connections 2024-06-06T08:39:25.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 -- 172.21.15.27:0/1138180734 wait complete. 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 -- start start 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.983+0000 7faeb5e39b80 1 Processor -- start 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 0x556faf06f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faeb5e39b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 0x556faf06f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556faf094260 con 0x556faf079b60 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556faf093b90 con 0x556faf07fc40 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faeb5e39b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556faf094930 con 0x556faf07f320 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faeb5e39b80 10 client.0 _ll_register_callbacks cb 0x556faef5c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9352700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9352700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf06ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36636/0 (socket says 172.21.15.27:36636) 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9352700 1 -- 172.21.15.27:0/3643458289 learned_addr learned my addr 172.21.15.27:0/3643458289 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:25.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9b53700 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 0x556faf06f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea8b51700 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 0x556faf06f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9352700 1 -- 172.21.15.27:0/3643458289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 msgr2=0x556faf06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9352700 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 0x556faf06f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9352700 1 -- 172.21.15.27:0/3643458289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 msgr2=0x556faf06f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9352700 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 0x556faf06f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9352700 1 -- 172.21.15.27:0/3643458289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556faf09cad0 con 0x556faf079b60 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7faea9352700 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf06ee20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae94009890 tx=0x7fae9400b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93f9e287a563299 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7fae88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.984+0000 7fae88ff9700 1 -- 172.21.15.27:0/3643458289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9401c070 con 0x556faf079b60 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7fae88ff9700 1 -- 172.21.15.27:0/3643458289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fae94004db0 con 0x556faf079b60 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7faeb5e39b80 1 -- 172.21.15.27:0/3643458289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae980a0ae0 con 0x556faf079b60 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7fae88ff9700 1 -- 172.21.15.27:0/3643458289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fae94039050 con 0x556faf079b60 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7faeb5e39b80 10 client.15027 fetch_fsmap learned FSMap version 24 2024-06-06T08:39:25.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7faeb5e39b80 1 -- 172.21.15.27:0/3643458289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556faf09ddd0 con 0x556faf079b60 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7fae88ff9700 1 -- 172.21.15.27:0/3643458289 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae980a0ae0 con 0x556faf079b60 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7fae88ff9700 1 -- 172.21.15.27:0/3643458289 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fae94022040 con 0x556faf079b60 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7faeb5e39b80 10 client.15027 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7faeb5e39b80 10 client.15027 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7faeb5e39b80 20 client.15027 populate_metadata read hostname 'smithi027' 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556faf102c00 0x556faf101910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7faeb5e39b80 4 client.15027 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.985+0000 7faeb5e39b80 1 -- 172.21.15.27:0/3643458289 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "status"}) -- 0x556faefce280 con 0x556faf102c00 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faea8b51700 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556faf102c00 0x556faf101910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faea8b51700 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556faf102c00 0x556faf101910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ac9571b85c806a4 server_cookie=864096745bab2ec8 in_seq=0 out_seq=0 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7fae88ff9700 10 client.15027 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7fae88ff9700 1 -- 172.21.15.27:0/3643458289 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556faefce280 con 0x556faf102c00 2024-06-06T08:39:25.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7fae88ff9700 10 client.15027 handle_command_reply: tid=0 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 client.15027 shutdown 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 -- 172.21.15.27:0/3643458289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 msgr2=0x556faf06ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf06ee20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae94009890 tx=0x7fae9400b050 comp rx=0 tx=0).stop 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 -- 172.21.15.27:0/3643458289 shutdown_connections 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556faf102c00 0x556faf101910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7fae88ff9700 0 client.15027 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556faf07fc40 0x556faf06f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556faf079b60 0x556faf06ee20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 --2- 172.21.15.27:0/3643458289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556faf07f320 0x556faf06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 -- 172.21.15.27:0/3643458289 >> 172.21.15.27:0/3643458289 conn(0x556faefa2030 msgr2=0x556faf0800e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 -- 172.21.15.27:0/3643458289 shutdown_connections 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.986+0000 7faeb5e39b80 1 -- 172.21.15.27:0/3643458289 wait complete. 2024-06-06T08:39:25.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeb5e39b80 20 client.15027 trim_cache size 0 max 16384 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24458, 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 124.081431787, 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-06-06T08:39:25.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 99, 2024-06-06T08:39:25.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T08:39:25.993 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 213.63058115300001 2024-06-06T08:39:25.993 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 -- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae8c0a2c20 msgr2=0x7fae8c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae8c0a2c20 0x7fae8c0a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae94009550 tx=0x7fae940091b0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 -- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101d40 msgr2=0x7faea419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101d40 0x7faea419ed60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fae98000c00 tx=0x7fae980047b0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 -- 172.21.15.27:0/4117285922 shutdown_connections 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fae8c0a2c20 0x7fae8c0a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea41073f0 0x7faea41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faea4101380 0x7faea419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 --2- 172.21.15.27:0/4117285922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea4101d40 0x7faea419ed60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 -- 172.21.15.27:0/4117285922 >> 172.21.15.27:0/4117285922 conn(0x7faea4078300 msgr2=0x7faea406ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 -- 172.21.15.27:0/4117285922 shutdown_connections 2024-06-06T08:39:25.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:25.987+0000 7faeab5b6700 1 -- 172.21.15.27:0/4117285922 wait complete. 2024-06-06T08:39:26.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:26 smithi027 ceph-mon[32335]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:39:26.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:26 smithi203 ceph-mon[36579]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:39:26.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:26 smithi176 ceph-mon[36786]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:39:26.636 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T08:39:26.886 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:28.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:28 smithi027 ceph-mon[32335]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:28.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:28 smithi203 ceph-mon[36579]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:28.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:28 smithi176 ceph-mon[36786]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:28.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.546+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/3213522607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b41084e0 msgr2=0x7fb1b41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:28.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.546+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/3213522607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b41084e0 0x7fb1b41088c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00099c0 tx=0x7fb1b0009cd0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.546+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/3213522607 shutdown_connections 2024-06-06T08:39:28.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.546+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/3213522607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1b406ab10 0x7fb1b410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.546+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/3213522607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b41084e0 0x7fb1b41088c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.546+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/3213522607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b406a170 0x7fb1b406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.546+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/3213522607 >> 172.21.15.27:0/3213522607 conn(0x7fb1b40782c0 msgr2=0x7fb1b40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:28.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.546+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/3213522607 shutdown_connections 2024-06-06T08:39:28.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.547+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/3213522607 wait complete. 2024-06-06T08:39:28.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.547+0000 7fb1bb3a5700 1 Processor -- start 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.547+0000 7fb1bb3a5700 1 -- start start 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1bb3a5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1b406a170 0x7fb1b419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1bb3a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b406ab10 0x7fb1b419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1bb3a5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b41084e0 0x7fb1b4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1bb3a5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41127a0 con 0x7fb1b406ab10 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1bb3a5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b4112620 con 0x7fb1b406a170 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1bb3a5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b4112920 con 0x7fb1b41084e0 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b8940700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b406ab10 0x7fb1b419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b8940700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b406ab10 0x7fb1b419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36652/0 (socket says 172.21.15.27:36652) 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b8940700 1 -- 172.21.15.27:0/2716102165 learned_addr learned my addr 172.21.15.27:0/2716102165 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:28.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b9942700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b41084e0 0x7fb1b4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b9141700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1b406a170 0x7fb1b419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b8940700 1 -- 172.21.15.27:0/2716102165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b41084e0 msgr2=0x7fb1b4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b8940700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b41084e0 0x7fb1b4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b8940700 1 -- 172.21.15.27:0/2716102165 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1b406a170 msgr2=0x7fb1b419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b8940700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1b406a170 0x7fb1b419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b8940700 1 -- 172.21.15.27:0/2716102165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b4199240 con 0x7fb1b406ab10 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b9141700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1b406a170 0x7fb1b419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1b8940700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b406ab10 0x7fb1b419ee60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b530 tx=0x7fb1a400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.548+0000 7fb1aa7fc700 1 -- 172.21.15.27:0/2716102165 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4015020 con 0x7fb1b406ab10 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.549+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/2716102165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b4199530 con 0x7fb1b406ab10 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.549+0000 7fb1aa7fc700 1 -- 172.21.15.27:0/2716102165 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a4011d10 con 0x7fb1b406ab10 2024-06-06T08:39:28.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.549+0000 7fb1aa7fc700 1 -- 172.21.15.27:0/2716102165 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a400f2e0 con 0x7fb1b406ab10 2024-06-06T08:39:28.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.549+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/2716102165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b41a57f0 con 0x7fb1b406ab10 2024-06-06T08:39:28.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.552+0000 7fb1aa7fc700 1 -- 172.21.15.27:0/2716102165 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1a4018020 con 0x7fb1b406ab10 2024-06-06T08:39:28.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.552+0000 7fb1aa7fc700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb1a00a2c20 0x7fb1a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.552+0000 7fb1b9141700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb1a00a2c20 0x7fb1a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.552+0000 7fb1aa7fc700 1 -- 172.21.15.27:0/2716102165 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fb1a40cecd0 con 0x7fb1b406ab10 2024-06-06T08:39:28.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.553+0000 7fb1b9141700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb1a00a2c20 0x7fb1a00a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009fd0 tx=0x7fb1b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:28.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.553+0000 7fb1c5c28b80 1 Processor -- start 2024-06-06T08:39:28.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.553+0000 7fb1c5c28b80 1 -- start start 2024-06-06T08:39:28.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.553+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 0x55c15f5bd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.553+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5b7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.553+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.553+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c15f5097c0 con 0x55c15f5bdf10 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.553+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c15f4b16d0 con 0x55c15f5b7ed0 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.553+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c15f505780 con 0x55c15f5bd5f0 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b8940700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b8940700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5b7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36662/0 (socket says 172.21.15.27:36662) 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b8940700 1 -- 172.21.15.27:0/2653157331 learned_addr learned my addr 172.21.15.27:0/2653157331 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b9141700 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 0x55c15f5bd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b9942700 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b8940700 1 -- 172.21.15.27:0/2653157331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 msgr2=0x55c15f5bd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b8940700 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 0x55c15f5bd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b8940700 1 -- 172.21.15.27:0/2653157331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 msgr2=0x55c15f5b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:28.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b8940700 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b8940700 1 -- 172.21.15.27:0/2653157331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c15f4a7f00 con 0x55c15f5bdf10 2024-06-06T08:39:28.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b9942700 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:28.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb1b8940700 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5b7990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb1a409b0f0 tx=0x7fb1a409b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c140019b9e8df4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:28.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb19f7fe700 1 -- 172.21.15.27:0/2653157331 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a409f510 con 0x55c15f5bdf10 2024-06-06T08:39:28.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb19f7fe700 1 -- 172.21.15.27:0/2653157331 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a40a3dd0 con 0x55c15f5bdf10 2024-06-06T08:39:28.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.554+0000 7fb19f7fe700 1 -- 172.21.15.27:0/2653157331 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a409f120 con 0x55c15f5bdf10 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.555+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/2653157331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 msgr2=0x55c15f5b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.555+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5b7990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb1a409b0f0 tx=0x7fb1a409b860 comp rx=0 tx=0).stop 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.555+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/2653157331 shutdown_connections 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.555+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.555+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 0x55c15f5bd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.555+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/2653157331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5b7990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.555+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/2653157331 >> 172.21.15.27:0/2653157331 conn(0x55c15f4e2c70 msgr2=0x55c15f5b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.555+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/2653157331 shutdown_connections 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.555+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/2653157331 wait complete. 2024-06-06T08:39:28.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1c5c28b80 1 -- start start 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1c5c28b80 1 Processor -- start 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5b7ed0 0x55c15f5c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5bd5f0 0x55c15f5c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c15f5c4980 con 0x55c15f5bdf10 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c15f4a9ba0 con 0x55c15f5bd5f0 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c15f5c4b00 con 0x55c15f5b7ed0 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1c5c28b80 10 client.0 _ll_register_callbacks cb 0x55c15f4a38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1b9942700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1b9942700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36664/0 (socket says 172.21.15.27:36664) 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.556+0000 7fb1b9942700 1 -- 172.21.15.27:0/4028796486 learned_addr learned my addr 172.21.15.27:0/4028796486 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:28.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1b9141700 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5b7ed0 0x55c15f5c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1b8940700 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5bd5f0 0x55c15f5c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1b9942700 1 -- 172.21.15.27:0/4028796486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5b7ed0 msgr2=0x55c15f5c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1b9942700 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5b7ed0 0x55c15f5c3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1b9942700 1 -- 172.21.15.27:0/4028796486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5bd5f0 msgr2=0x55c15f5c0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1b9942700 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5bd5f0 0x55c15f5c0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1b9942700 1 -- 172.21.15.27:0/4028796486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c15f458880 con 0x55c15f5bdf10 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1b9942700 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac008e60 tx=0x7fb1ac010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b83363ff8b2c926 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb19cff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb19cff9700 1 -- 172.21.15.27:0/4028796486 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac010dd0 con 0x55c15f5bdf10 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb19cff9700 1 -- 172.21.15.27:0/4028796486 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fb1ac015410 con 0x55c15f5bdf10 2024-06-06T08:39:28.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/4028796486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1a40a3dd0 con 0x55c15f5bdf10 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb19cff9700 1 -- 172.21.15.27:0/4028796486 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb1ac023050 con 0x55c15f5bdf10 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1c5c28b80 10 client.15039 fetch_fsmap learned FSMap version 24 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.557+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/4028796486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c15f5d3690 con 0x55c15f5bdf10 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.558+0000 7fb19cff9700 1 -- 172.21.15.27:0/4028796486 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a40a3dd0 con 0x55c15f5bdf10 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.558+0000 7fb19cff9700 1 -- 172.21.15.27:0/4028796486 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb1ac01e6b0 con 0x55c15f5bdf10 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.558+0000 7fb1c5c28b80 10 client.15039 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.558+0000 7fb1c5c28b80 10 client.15039 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.558+0000 7fb1c5c28b80 20 client.15039 populate_metadata read hostname 'smithi027' 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.558+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c15f5d5cd0 0x55c15f5d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.558+0000 7fb1c5c28b80 4 client.15039 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.558+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/4028796486 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c15f516cc0 con 0x55c15f5d5cd0 2024-06-06T08:39:28.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.558+0000 7fb1b9141700 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c15f5d5cd0 0x55c15f5d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.559+0000 7fb1b9141700 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c15f5d5cd0 0x55c15f5d7140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=339588d26e3dfc8f server_cookie=61e10764f1d63d31 in_seq=0 out_seq=0 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.559+0000 7fb19cff9700 10 client.15039 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb19cff9700 1 -- 172.21.15.27:0/4028796486 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c15f516cc0 con 0x55c15f5d5cd0 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb19cff9700 10 client.15039 handle_command_reply: tid=0 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 client.15039 shutdown 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/4028796486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 msgr2=0x55c15f5c1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac008e60 tx=0x7fb1ac010430 comp rx=0 tx=0).stop 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/4028796486 shutdown_connections 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1500 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5b7ed0 0x55c15f5c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c15f5d5cd0 0x55c15f5d7140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/4028796486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5bd5f0 0x55c15f5c0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/4028796486 >> 172.21.15.27:0/4028796486 conn(0x55c15f4e2c70 msgr2=0x55c15f5b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:28.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb19cff9700 0 client.15039 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:39:28.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/4028796486 shutdown_connections 2024-06-06T08:39:28.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/4028796486 wait complete. 2024-06-06T08:39:28.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.560+0000 7fb1c5c28b80 20 client.15039 trim_cache size 0 max 16384 2024-06-06T08:39:28.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1c5c28b80 1 Processor -- start 2024-06-06T08:39:28.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1c5c28b80 1 -- start start 2024-06-06T08:39:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 0x55c15f5b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c15f5d5480 con 0x55c15f5bdf10 2024-06-06T08:39:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c15f5d5b50 con 0x55c15f5b7ed0 2024-06-06T08:39:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c15f5d6220 con 0x55c15f5bd5f0 2024-06-06T08:39:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1b9942700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1b9942700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36670/0 (socket says 172.21.15.27:36670) 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.582+0000 7fb1b9942700 1 -- 172.21.15.27:0/1755364080 learned_addr learned my addr 172.21.15.27:0/1755364080 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b9141700 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b8940700 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 0x55c15f5b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b9942700 1 -- 172.21.15.27:0/1755364080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 msgr2=0x55c15f5b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b9942700 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 0x55c15f5b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b9942700 1 -- 172.21.15.27:0/1755364080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 msgr2=0x55c15f5b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b9942700 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b9942700 1 -- 172.21.15.27:0/1755364080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c15f5c42c0 con 0x55c15f5bdf10 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b9141700 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b8940700 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 0x55c15f5b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb1b9942700 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1530 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac008e30 tx=0x7fb1ac007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2f828b538c943f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb19effd700 1 -- 172.21.15.27:0/1755364080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac010dd0 con 0x55c15f5bdf10 2024-06-06T08:39:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb19effd700 1 -- 172.21.15.27:0/1755364080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1ac01a070 con 0x55c15f5bdf10 2024-06-06T08:39:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.583+0000 7fb19effd700 1 -- 172.21.15.27:0/1755364080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac030af0 con 0x55c15f5bdf10 2024-06-06T08:39:28.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1755364080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 msgr2=0x55c15f5c1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1530 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac008e30 tx=0x7fb1ac007d60 comp rx=0 tx=0).stop 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1755364080 shutdown_connections 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bdf10 0x55c15f5c1530 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1755364080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bd5f0 0x55c15f5b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1755364080 >> 172.21.15.27:0/1755364080 conn(0x55c15f4e2c70 msgr2=0x55c15f5be3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1755364080 shutdown_connections 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1755364080 wait complete. 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 -- start start 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.584+0000 7fb1c5c28b80 1 Processor -- start 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5e05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bd5f0 0x55c15f5e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1c5c28b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bdf10 0x55c15f5db570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c15f5d6fc0 con 0x55c15f5bd5f0 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c15f5d68f0 con 0x55c15f5b7ed0 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1c5c28b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c15f5d7690 con 0x55c15f5bdf10 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1c5c28b80 10 client.0 _ll_register_callbacks cb 0x55c15f4a38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b8940700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bd5f0 0x55c15f5e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b8940700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bd5f0 0x55c15f5e0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36672/0 (socket says 172.21.15.27:36672) 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b8940700 1 -- 172.21.15.27:0/1512593313 learned_addr learned my addr 172.21.15.27:0/1512593313 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b9141700 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5e05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b9942700 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bdf10 0x55c15f5db570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b8940700 1 -- 172.21.15.27:0/1512593313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bdf10 msgr2=0x55c15f5db570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b8940700 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bdf10 0x55c15f5db570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b8940700 1 -- 172.21.15.27:0/1512593313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 msgr2=0x55c15f5e05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b8940700 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5e05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b8940700 1 -- 172.21.15.27:0/1512593313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c15f5dbab0 con 0x55c15f5bd5f0 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b9141700 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5e05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb1b8940700 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bd5f0 0x55c15f5e0b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400bc40 tx=0x7fb1a4010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a3c32f163ad7e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.585+0000 7fb19cff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb19cff9700 1 -- 172.21.15.27:0/1512593313 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a409af30 con 0x55c15f5bd5f0 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb19cff9700 1 -- 172.21.15.27:0/1512593313 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fb1a4099460 con 0x55c15f5bd5f0 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1512593313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1ac01a070 con 0x55c15f5bd5f0 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb19cff9700 1 -- 172.21.15.27:0/1512593313 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb1a40a3dd0 con 0x55c15f5bd5f0 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb1c5c28b80 10 client.15045 fetch_fsmap learned FSMap version 24 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1512593313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c15f641d10 con 0x55c15f5bd5f0 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb19cff9700 1 -- 172.21.15.27:0/1512593313 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1ac01a070 con 0x55c15f5bd5f0 2024-06-06T08:39:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb19cff9700 1 -- 172.21.15.27:0/1512593313 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb1a409f120 con 0x55c15f5bd5f0 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb1c5c28b80 10 client.15045 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb1c5c28b80 10 client.15045 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb1c5c28b80 20 client.15045 populate_metadata read hostname 'smithi027' 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c15f6463c0 0x55c15f6450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb1c5c28b80 4 client.15045 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.586+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1512593313 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "status"}) -- 0x55c15f5b8860 con 0x55c15f6463c0 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1b9141700 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c15f6463c0 0x55c15f6450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1b9141700 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c15f6463c0 0x55c15f6450d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77d9c15512e224c0 server_cookie=5d4eb3bebebc1885 in_seq=0 out_seq=0 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb19cff9700 10 client.15045 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb19cff9700 1 -- 172.21.15.27:0/1512593313 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c15f5b8860 con 0x55c15f6463c0 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb19cff9700 10 client.15045 handle_command_reply: tid=0 2024-06-06T08:39:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1c5c28b80 1 client.15045 shutdown 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1512593313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bd5f0 msgr2=0x55c15f5e0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bd5f0 0x55c15f5e0b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400bc40 tx=0x7fb1a4010970 comp rx=0 tx=0).stop 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1512593313 shutdown_connections 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c15f5bdf10 0x55c15f5db570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c15f6463c0 0x55c15f6450d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c15f5b7ed0 0x55c15f5e05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1c5c28b80 1 --2- 172.21.15.27:0/1512593313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c15f5bd5f0 0x55c15f5e0b00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb19cff9700 0 client.15045 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.587+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1512593313 >> 172.21.15.27:0/1512593313 conn(0x55c15f4e2c70 msgr2=0x55c15f5abdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1512593313 shutdown_connections 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1c5c28b80 1 -- 172.21.15.27:0/1512593313 wait complete. 2024-06-06T08:39:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1c5c28b80 20 client.15045 trim_cache size 0 max 16384 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24460, 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 127.685976924, 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 99, 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T08:39:28.593 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 213.42317998199999 2024-06-06T08:39:28.594 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:39:28.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb1a00a2c20 msgr2=0x7fb1a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:28.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb1a00a2c20 0x7fb1a00a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009fd0 tx=0x7fb1b0005fd0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b406ab10 msgr2=0x7fb1b419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:28.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b406ab10 0x7fb1b419ee60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b530 tx=0x7fb1a400b840 comp rx=0 tx=0).stop 2024-06-06T08:39:28.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/2716102165 shutdown_connections 2024-06-06T08:39:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb1a00a2c20 0x7fb1a00a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b41084e0 0x7fb1b4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1b406a170 0x7fb1b419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 --2- 172.21.15.27:0/2716102165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1b406ab10 0x7fb1b419ee60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/2716102165 >> 172.21.15.27:0/2716102165 conn(0x7fb1b40782c0 msgr2=0x7fb1b40ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/2716102165 shutdown_connections 2024-06-06T08:39:28.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:28.588+0000 7fb1bb3a5700 1 -- 172.21.15.27:0/2716102165 wait complete. 2024-06-06T08:39:29.274 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T08:39:29.530 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:30.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:30 smithi027 ceph-mon[32335]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:30 smithi203 ceph-mon[36579]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:30.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:30 smithi176 ceph-mon[36786]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.221+0000 7f28a3490700 1 -- 172.21.15.27:0/1265178459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 msgr2=0x7f289c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.221+0000 7f28a3490700 1 --2- 172.21.15.27:0/1265178459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 0x7f289c1067f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f28980099c0 tx=0x7f2898009cd0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.221+0000 7f28a3490700 1 -- 172.21.15.27:0/1265178459 shutdown_connections 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.221+0000 7f28a3490700 1 --2- 172.21.15.27:0/1265178459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f289c1020a0 0x7f289c10e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.221+0000 7f28a3490700 1 --2- 172.21.15.27:0/1265178459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 0x7f289c1067f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.221+0000 7f28a3490700 1 --2- 172.21.15.27:0/1265178459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f289c1016e0 0x7f289c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.221+0000 7f28a3490700 1 -- 172.21.15.27:0/1265178459 >> 172.21.15.27:0/1265178459 conn(0x7f289c077200 msgr2=0x7f289c077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.221+0000 7f28a3490700 1 -- 172.21.15.27:0/1265178459 shutdown_connections 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a3490700 1 -- 172.21.15.27:0/1265178459 wait complete. 2024-06-06T08:39:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a3490700 1 Processor -- start 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a3490700 1 -- start start 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a3490700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f289c1016e0 0x7f289c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a3490700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f289c1020a0 0x7f289c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a3490700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 0x7f289c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a3490700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f289c111790 con 0x7f289c106410 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a3490700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f289c111610 con 0x7f289c1020a0 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a3490700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f289c111910 con 0x7f289c1016e0 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.222+0000 7f28a1a2d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 0x7f289c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a1a2d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 0x7f289c197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41202/0 (socket says 172.21.15.27:41202) 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1365155350 learned_addr learned my addr 172.21.15.27:0/1365155350 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a122c700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f289c1016e0 0x7f289c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f289c1020a0 0x7f289c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1365155350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f289c1016e0 msgr2=0x7f289c19d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f289c1016e0 0x7f289c19d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1365155350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f289c1020a0 msgr2=0x7f289c19dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f289c1020a0 0x7f289c19dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1365155350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c1980d0 con 0x7f289c106410 2024-06-06T08:39:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a122c700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f289c1016e0 0x7f289c19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f289c1020a0 0x7f289c19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 0x7f289c197970 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2894007d70 tx=0x7f289400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28927fc700 1 -- 172.21.15.27:0/1365155350 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894016020 con 0x7f289c106410 2024-06-06T08:39:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28927fc700 1 -- 172.21.15.27:0/1365155350 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289400ad40 con 0x7f289c106410 2024-06-06T08:39:31.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a3490700 1 -- 172.21.15.27:0/1365155350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c1983c0 con 0x7f289c106410 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28927fc700 1 -- 172.21.15.27:0/1365155350 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28940116f0 con 0x7f289c106410 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.223+0000 7f28a3490700 1 -- 172.21.15.27:0/1365155350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c06abc0 con 0x7f289c106410 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.225+0000 7f28927fc700 1 -- 172.21.15.27:0/1365155350 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2894019020 con 0x7f289c106410 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.226+0000 7f28927fc700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f28880a2c20 0x7f28880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.226+0000 7f28a122c700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f28880a2c20 0x7f28880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.226+0000 7f28927fc700 1 -- 172.21.15.27:0/1365155350 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f28940cf490 con 0x7f289c106410 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28a122c700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f28880a2c20 0x7f28880a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2898009750 tx=0x7f2898005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28add13b80 1 Processor -- start 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28add13b80 1 -- start start 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28add13b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28add13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d94830 0x5629c8d8e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28add13b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d8eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28add13b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5629c8c80f10 con 0x5629c8d93f10 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28add13b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5629c8cdf800 con 0x5629c8d8e750 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28add13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5629c8c47410 con 0x5629c8d94830 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28a122c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28a122c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d942f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41210/0 (socket says 172.21.15.27:41210) 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.227+0000 7f28a122c700 1 -- 172.21.15.27:0/2464598489 learned_addr learned my addr 172.21.15.27:0/2464598489 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d8eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d94830 0x5629c8d8e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28a122c700 1 -- 172.21.15.27:0/2464598489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d94830 msgr2=0x5629c8d8e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28a122c700 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d94830 0x5629c8d8e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28a122c700 1 -- 172.21.15.27:0/2464598489 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 msgr2=0x5629c8d8eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28a122c700 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d8eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28a122c700 1 -- 172.21.15.27:0/2464598489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c8c2e880 con 0x5629c8d93f10 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d94830 0x5629c8d8e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28a122c700 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d942f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f28980099c0 tx=0x7f289801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41dd36553ced1c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28877fe700 1 -- 172.21.15.27:0/2464598489 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28980054b0 con 0x5629c8d93f10 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28877fe700 1 -- 172.21.15.27:0/2464598489 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898005650 con 0x5629c8d93f10 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.228+0000 7f28877fe700 1 -- 172.21.15.27:0/2464598489 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289802fb40 con 0x5629c8d93f10 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 -- 172.21.15.27:0/2464598489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 msgr2=0x5629c8d942f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d942f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f28980099c0 tx=0x7f289801d000 comp rx=0 tx=0).stop 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 -- 172.21.15.27:0/2464598489 shutdown_connections 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d8eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d942f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 --2- 172.21.15.27:0/2464598489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d94830 0x5629c8d8e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 -- 172.21.15.27:0/2464598489 >> 172.21.15.27:0/2464598489 conn(0x5629c8cbb030 msgr2=0x5629c8d891d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 -- 172.21.15.27:0/2464598489 shutdown_connections 2024-06-06T08:39:31.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 -- 172.21.15.27:0/2464598489 wait complete. 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 -- start start 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.229+0000 7f28add13b80 1 Processor -- start 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28add13b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d9a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28add13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28add13b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d97e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28add13b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5629c8c7ff70 con 0x5629c8d94830 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28add13b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5629c8cdb7c0 con 0x5629c8d8e750 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28add13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5629c8d9b1b0 con 0x5629c8d93f10 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28add13b80 10 client.0 _ll_register_callbacks cb 0x5629c8c75560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:39:31.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a1a2d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d97e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a1a2d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d97e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41220/0 (socket says 172.21.15.27:41220) 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1257063643 learned_addr learned my addr 172.21.15.27:0/1257063643 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a122c700 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d9a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1257063643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 msgr2=0x5629c8d978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1257063643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 msgr2=0x5629c8d9a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d9a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1257063643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c8ca5700 con 0x5629c8d94830 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d97e00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f289409b8b0 tx=0x7f28940193e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f71f8d5e947d1c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f2884ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.230+0000 7f2884ff9700 1 -- 172.21.15.27:0/1257063643 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289409fcd0 con 0x5629c8d94830 2024-06-06T08:39:31.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f2884ff9700 1 -- 172.21.15.27:0/1257063643 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f28940cf190 con 0x5629c8d94830 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f28add13b80 1 -- 172.21.15.27:0/1257063643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2898005650 con 0x5629c8d94830 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f2884ff9700 1 -- 172.21.15.27:0/1257063643 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f28940a4590 con 0x5629c8d94830 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f28add13b80 10 client.15057 fetch_fsmap learned FSMap version 24 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f28add13b80 1 -- 172.21.15.27:0/1257063643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5629c8da5930 con 0x5629c8d94830 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f2884ff9700 1 -- 172.21.15.27:0/1257063643 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898005650 con 0x5629c8d94830 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f2884ff9700 1 -- 172.21.15.27:0/1257063643 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f28940860b0 con 0x5629c8d94830 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f28add13b80 10 client.15057 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f28add13b80 10 client.15057 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f28add13b80 20 client.15057 populate_metadata read hostname 'smithi027' 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f28add13b80 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5629c8da7f10 0x5629c8da9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f28add13b80 4 client.15057 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.231+0000 7f28add13b80 1 -- 172.21.15.27:0/1257063643 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c8cecd00 con 0x5629c8da7f10 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.232+0000 7f28a122c700 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5629c8da7f10 0x5629c8da9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.232+0000 7f28a122c700 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5629c8da7f10 0x5629c8da9380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caac4d9e4189aa0d server_cookie=c763de8c3dc3557b in_seq=0 out_seq=0 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.232+0000 7f2884ff9700 10 client.15057 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f2884ff9700 1 -- 172.21.15.27:0/1257063643 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5629c8cecd00 con 0x5629c8da7f10 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f2884ff9700 10 client.15057 handle_command_reply: tid=0 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 client.15057 shutdown 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 -- 172.21.15.27:0/1257063643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 msgr2=0x5629c8d97e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d97e00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f289409b8b0 tx=0x7f28940193e0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 -- 172.21.15.27:0/1257063643 shutdown_connections 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5629c8da7f10 0x5629c8da9380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d97e00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d9a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f2884ff9700 0 client.15057 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:39:31.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 --2- 172.21.15.27:0/1257063643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 -- 172.21.15.27:0/1257063643 >> 172.21.15.27:0/1257063643 conn(0x5629c8cbb030 msgr2=0x5629c8d886b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:31.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 -- 172.21.15.27:0/1257063643 shutdown_connections 2024-06-06T08:39:31.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 1 -- 172.21.15.27:0/1257063643 wait complete. 2024-06-06T08:39:31.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.233+0000 7f28add13b80 20 client.15057 trim_cache size 0 max 16384 2024-06-06T08:39:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28add13b80 1 Processor -- start 2024-06-06T08:39:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28add13b80 1 -- start start 2024-06-06T08:39:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28add13b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28add13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28add13b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d88e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28add13b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5629c8da76c0 con 0x5629c8d94830 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28add13b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5629c8da7d90 con 0x5629c8d8e750 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28add13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5629c8da8460 con 0x5629c8d93f10 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a1a2d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d88e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a1a2d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d88e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41232/0 (socket says 172.21.15.27:41232) 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1453886556 learned_addr learned my addr 172.21.15.27:0/1453886556 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a122c700 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1453886556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 msgr2=0x5629c8d953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1453886556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 msgr2=0x5629c8d96860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d96860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a1a2d700 1 -- 172.21.15.27:0/1453886556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c8d95920 con 0x5629c8d94830 2024-06-06T08:39:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a122c700 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d96860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.255+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.256+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d88e30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f28940cf730 tx=0x7f289409baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0daae14864d9206 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.256+0000 7f2886ffd700 1 -- 172.21.15.27:0/1453886556 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894007aa0 con 0x5629c8d94830 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.256+0000 7f2886ffd700 1 -- 172.21.15.27:0/1453886556 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28940b4070 con 0x5629c8d94830 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.256+0000 7f2886ffd700 1 -- 172.21.15.27:0/1453886556 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28940cf190 con 0x5629c8d94830 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 -- 172.21.15.27:0/1453886556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 msgr2=0x5629c8d88e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d88e30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f28940cf730 tx=0x7f289409baa0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 -- 172.21.15.27:0/1453886556 shutdown_connections 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d94830 0x5629c8d88e30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d8e750 0x5629c8d96860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 --2- 172.21.15.27:0/1453886556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d93f10 0x5629c8d953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 -- 172.21.15.27:0/1453886556 >> 172.21.15.27:0/1453886556 conn(0x5629c8cbb030 msgr2=0x5629c8d94cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 -- 172.21.15.27:0/1453886556 shutdown_connections 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 -- 172.21.15.27:0/1453886556 wait complete. 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 -- start start 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.257+0000 7f28add13b80 1 Processor -- start 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28add13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d8e750 0x5629c8d887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28add13b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d88cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28add13b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d94830 0x5629c8d89230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28add13b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5629c8da9200 con 0x5629c8d93f10 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28add13b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5629c8da8b30 con 0x5629c8d94830 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28add13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5629c8da98d0 con 0x5629c8d8e750 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28add13b80 10 client.0 _ll_register_callbacks cb 0x5629c8c75560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28a0a2b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d88cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28a0a2b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d88cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41248/0 (socket says 172.21.15.27:41248) 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28a0a2b700 1 -- 172.21.15.27:0/2201930956 learned_addr learned my addr 172.21.15.27:0/2201930956 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28a1a2d700 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d94830 0x5629c8d89230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28a122c700 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d8e750 0x5629c8d887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28a0a2b700 1 -- 172.21.15.27:0/2201930956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d8e750 msgr2=0x5629c8d887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d8e750 0x5629c8d887b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28a0a2b700 1 -- 172.21.15.27:0/2201930956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d94830 msgr2=0x5629c8d89230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.258+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d94830 0x5629c8d89230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f28a0a2b700 1 -- 172.21.15.27:0/2201930956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c8db0f80 con 0x5629c8d93f10 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f28a122c700 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d8e750 0x5629c8d887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f28a0a2b700 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d88cf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f289c077690 tx=0x7f288c0128b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905119623770dbe7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f2884ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f2884ff9700 1 -- 172.21.15.27:0/2201930956 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c019030 con 0x5629c8d93f10 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f28add13b80 1 -- 172.21.15.27:0/2201930956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28940b4070 con 0x5629c8d93f10 2024-06-06T08:39:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f2884ff9700 1 -- 172.21.15.27:0/2201930956 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f288c00f040 con 0x5629c8d93f10 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f2884ff9700 1 -- 172.21.15.27:0/2201930956 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f288c026050 con 0x5629c8d93f10 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f28add13b80 10 client.15063 fetch_fsmap learned FSMap version 24 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f28add13b80 1 -- 172.21.15.27:0/2201930956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5629c8e13c40 con 0x5629c8d93f10 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.259+0000 7f2884ff9700 1 -- 172.21.15.27:0/2201930956 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28940b4070 con 0x5629c8d93f10 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f2884ff9700 1 -- 172.21.15.27:0/2201930956 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f288c01e7e0 con 0x5629c8d93f10 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f28add13b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f28add13b80 10 client.15063 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f28add13b80 20 client.15063 populate_metadata read hostname 'smithi027' 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f28add13b80 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5629c8e188d0 0x5629c8e175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f28add13b80 4 client.15063 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f28add13b80 1 -- 172.21.15.27:0/2201930956 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "status"}) -- 0x5629c8ce7280 con 0x5629c8e188d0 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f28a122c700 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5629c8e188d0 0x5629c8e175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f28a122c700 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5629c8e188d0 0x5629c8e175e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0616be2b6b7fa server_cookie=6d8119dc378ec629 in_seq=0 out_seq=0 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.260+0000 7f2884ff9700 10 client.15063 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:39:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f2884ff9700 1 -- 172.21.15.27:0/2201930956 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5629c8ce7280 con 0x5629c8e188d0 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f2884ff9700 10 client.15063 handle_command_reply: tid=0 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 client.15063 shutdown 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 -- 172.21.15.27:0/2201930956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 msgr2=0x5629c8d88cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d88cf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f289c077690 tx=0x7f288c0128b0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 -- 172.21.15.27:0/2201930956 shutdown_connections 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5629c8d94830 0x5629c8d89230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629c8d8e750 0x5629c8d887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5629c8e188d0 0x5629c8e175e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 --2- 172.21.15.27:0/2201930956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5629c8d93f10 0x5629c8d88cf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 -- 172.21.15.27:0/2201930956 >> 172.21.15.27:0/2201930956 conn(0x5629c8cbb030 msgr2=0x5629c8d990f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f2884ff9700 0 client.15063 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 -- 172.21.15.27:0/2201930956 shutdown_connections 2024-06-06T08:39:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 1 -- 172.21.15.27:0/2201930956 wait complete. 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.261+0000 7f28add13b80 20 client.15063 trim_cache size 0 max 16384 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24482, 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 187.361954126, 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 99, 2024-06-06T08:39:31.267 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T08:39:31.268 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 205.96487797 2024-06-06T08:39:31.268 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:39:31.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 -- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f28880a2c20 msgr2=0x7f28880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:31.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f28880a2c20 0x7f28880a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2898009750 tx=0x7f2898005fb0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 -- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 msgr2=0x7f289c197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 0x7f289c197970 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2894007d70 tx=0x7f289400a340 comp rx=0 tx=0).stop 2024-06-06T08:39:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 -- 172.21.15.27:0/1365155350 shutdown_connections 2024-06-06T08:39:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f28880a2c20 0x7f28880a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c106410 0x7f289c197970 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f289c1016e0 0x7f289c19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 --2- 172.21.15.27:0/1365155350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f289c1020a0 0x7f289c19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 -- 172.21.15.27:0/1365155350 >> 172.21.15.27:0/1365155350 conn(0x7f289c077200 msgr2=0x7f289c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 -- 172.21.15.27:0/1365155350 shutdown_connections 2024-06-06T08:39:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:31.262+0000 7f28a3490700 1 -- 172.21.15.27:0/1365155350 wait complete. 2024-06-06T08:39:31.924 INFO:tasks.cephadm:Setup complete, yielding 2024-06-06T08:39:31.924 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T08:39:31.934 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:39:31.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-06T08:39:32.187 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:32.312 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:32 smithi027 ceph-mon[32335]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:32.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:32 smithi203 ceph-mon[36579]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:32.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:32 smithi176 ceph-mon[36786]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:33.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.955+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/1436091877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24102f80 msgr2=0x7f5c24105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.955+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/1436091877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24102f80 0x7f5c24105370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009990 tx=0x7f5c20009ca0 comp rx=0 tx=0).stop 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.955+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/1436091877 shutdown_connections 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.955+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/1436091877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c241081e0 0x7f5c2410a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.955+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/1436091877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c24102f80 0x7f5c24105370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.955+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/1436091877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c241058b0 0x7f5c24107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.955+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/1436091877 >> 172.21.15.27:0/1436091877 conn(0x7f5c240fc990 msgr2=0x7f5c240fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.955+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/1436091877 shutdown_connections 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.955+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/1436091877 wait complete. 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2c3c6700 1 Processor -- start 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2c3c6700 1 -- start start 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2c3c6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102f80 0x7f5c241028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:33.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2c3c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c241058b0 0x7f5c24100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2c3c6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c241081e0 0x7f5c24101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2c3c6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2410e140 con 0x7f5c241058b0 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2c3c6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2410dfc0 con 0x7f5c241081e0 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2c3c6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2410e2c0 con 0x7f5c24102f80 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c29961700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c241058b0 0x7f5c24100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c29961700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c241058b0 0x7f5c24100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41260/0 (socket says 172.21.15.27:41260) 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c29961700 1 -- 172.21.15.27:0/4007637866 learned_addr learned my addr 172.21.15.27:0/4007637866 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2a963700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c241081e0 0x7f5c24101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c2a162700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102f80 0x7f5c241028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c29961700 1 -- 172.21.15.27:0/4007637866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102f80 msgr2=0x7f5c241028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c29961700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102f80 0x7f5c241028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c29961700 1 -- 172.21.15.27:0/4007637866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c241081e0 msgr2=0x7f5c24101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.956+0000 7f5c29961700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c241081e0 0x7f5c24101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.957+0000 7f5c29961700 1 -- 172.21.15.27:0/4007637866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c24101d20 con 0x7f5c241058b0 2024-06-06T08:39:33.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.957+0000 7f5c2a162700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102f80 0x7f5c241028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:33.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.957+0000 7f5c29961700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c241058b0 0x7f5c24100f30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c9b0 tx=0x7f5c1400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:33.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.957+0000 7f5c1b7fe700 1 -- 172.21.15.27:0/4007637866 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14007b00 con 0x7f5c241058b0 2024-06-06T08:39:33.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.957+0000 7f5c1b7fe700 1 -- 172.21.15.27:0/4007637866 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c14007ca0 con 0x7f5c241058b0 2024-06-06T08:39:33.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.957+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c241a2880 con 0x7f5c241058b0 2024-06-06T08:39:33.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.957+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c241a2e40 con 0x7f5c241058b0 2024-06-06T08:39:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.958+0000 7f5c1b7fe700 1 -- 172.21.15.27:0/4007637866 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14018730 con 0x7f5c241058b0 2024-06-06T08:39:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.958+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c24069180 con 0x7f5c241058b0 2024-06-06T08:39:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.960+0000 7f5c1b7fe700 1 -- 172.21.15.27:0/4007637866 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c140188d0 con 0x7f5c241058b0 2024-06-06T08:39:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.960+0000 7f5c1b7fe700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c100a2ca0 0x7f5c100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.961+0000 7f5c2a162700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c100a2ca0 0x7f5c100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.961+0000 7f5c1b7fe700 1 -- 172.21.15.27:0/4007637866 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f5c140cf5f0 con 0x7f5c241058b0 2024-06-06T08:39:33.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.961+0000 7f5c2a162700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c100a2ca0 0x7f5c100a5160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009750 tx=0x7f5c20005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:33.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:33.964+0000 7f5c1b7fe700 1 -- 172.21.15.27:0/4007637866 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c14014030 con 0x7f5c241058b0 2024-06-06T08:39:34.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.193+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5c240628d0 con 0x7f5c100a2ca0 2024-06-06T08:39:34.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.194+0000 7f5c1b7fe700 1 -- 172.21.15.27:0/4007637866 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5c240628d0 con 0x7f5c100a2ca0 2024-06-06T08:39:34.192 INFO:teuthology.orchestra.run.smithi027.stdout:Backend: cephadm 2024-06-06T08:39:34.193 INFO:teuthology.orchestra.run.smithi027.stdout:Available: Yes 2024-06-06T08:39:34.193 INFO:teuthology.orchestra.run.smithi027.stdout:Paused: No 2024-06-06T08:39:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.196+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c100a2ca0 msgr2=0x7f5c100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.196+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c100a2ca0 0x7f5c100a5160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009750 tx=0x7f5c20005fb0 comp rx=0 tx=0).stop 2024-06-06T08:39:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.196+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c241058b0 msgr2=0x7f5c24100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.196+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c241058b0 0x7f5c24100f30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c9b0 tx=0x7f5c1400ccc0 comp rx=0 tx=0).stop 2024-06-06T08:39:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.196+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 shutdown_connections 2024-06-06T08:39:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.196+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5c100a2ca0 0x7f5c100a5160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.196+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c241081e0 0x7f5c24101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.196+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c24102f80 0x7f5c241028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.196+0000 7f5c2c3c6700 1 --2- 172.21.15.27:0/4007637866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c241058b0 0x7f5c24100f30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.197+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 >> 172.21.15.27:0/4007637866 conn(0x7f5c240fc990 msgr2=0x7f5c241038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.197+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 shutdown_connections 2024-06-06T08:39:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:34.197+0000 7f5c2c3c6700 1 -- 172.21.15.27:0/4007637866 wait complete. 2024-06-06T08:39:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:34 smithi027 ceph-mon[32335]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:34.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:34 smithi203 ceph-mon[36579]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:34.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:34 smithi176 ceph-mon[36786]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:34.877 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-06T08:39:35.130 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:35.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:35 smithi027 ceph-mon[32335]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:35.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:35 smithi203 ceph-mon[36579]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:35.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:35 smithi176 ceph-mon[36786]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:36.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:36 smithi027 ceph-mon[32335]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:39:36.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:36 smithi203 ceph-mon[36579]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:39:36.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:36 smithi176 ceph-mon[36786]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:39:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.789+0000 7f78763dd700 1 -- 172.21.15.27:0/4047071783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78701040a0 msgr2=0x7f787010a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.789+0000 7f78763dd700 1 --2- 172.21.15.27:0/4047071783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78701040a0 0x7f787010a950 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f786800b0d0 tx=0x7f786800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:39:36.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.789+0000 7f78763dd700 1 -- 172.21.15.27:0/4047071783 shutdown_connections 2024-06-06T08:39:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.789+0000 7f78763dd700 1 --2- 172.21.15.27:0/4047071783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78701040a0 0x7f787010a950 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.789+0000 7f78763dd700 1 --2- 172.21.15.27:0/4047071783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7870102500 0x7f7870102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.789+0000 7f78763dd700 1 --2- 172.21.15.27:0/4047071783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7870103700 0x7f7870103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.789+0000 7f78763dd700 1 -- 172.21.15.27:0/4047071783 >> 172.21.15.27:0/4047071783 conn(0x7f78700fdaa0 msgr2=0x7f78700ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.789+0000 7f78763dd700 1 -- 172.21.15.27:0/4047071783 shutdown_connections 2024-06-06T08:39:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.789+0000 7f78763dd700 1 -- 172.21.15.27:0/4047071783 wait complete. 2024-06-06T08:39:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f78763dd700 1 Processor -- start 2024-06-06T08:39:36.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f78763dd700 1 -- start start 2024-06-06T08:39:36.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f78763dd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7870102500 0x7f78701985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f78763dd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7870103700 0x7f7870198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f78763dd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78701040a0 0x7f787019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f78763dd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f787010e0f0 con 0x7f7870102500 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f78763dd700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f787010df70 con 0x7f7870103700 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f78763dd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f787010e270 con 0x7f78701040a0 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f786ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7870102500 0x7f78701985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f786ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7870102500 0x7f78701985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41268/0 (socket says 172.21.15.27:41268) 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.790+0000 7f786ffff700 1 -- 172.21.15.27:0/586964473 learned_addr learned my addr 172.21.15.27:0/586964473 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f787497a700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78701040a0 0x7f787019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786f7fe700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7870103700 0x7f7870198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786ffff700 1 -- 172.21.15.27:0/586964473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78701040a0 msgr2=0x7f787019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786ffff700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78701040a0 0x7f787019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:36.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786ffff700 1 -- 172.21.15.27:0/586964473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7870103700 msgr2=0x7f7870198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786ffff700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7870103700 0x7f7870198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786ffff700 1 -- 172.21.15.27:0/586964473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78701a0250 con 0x7f7870102500 2024-06-06T08:39:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786ffff700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7870102500 0x7f78701985f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f786000d9b0 tx=0x7f786000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786d7fa700 1 -- 172.21.15.27:0/586964473 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860017070 con 0x7f7870102500 2024-06-06T08:39:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78701a0540 con 0x7f7870102500 2024-06-06T08:39:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786d7fa700 1 -- 172.21.15.27:0/586964473 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7860009e10 con 0x7f7870102500 2024-06-06T08:39:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f786d7fa700 1 -- 172.21.15.27:0/586964473 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786000f620 con 0x7f7870102500 2024-06-06T08:39:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.791+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78701a0a80 con 0x7f7870102500 2024-06-06T08:39:36.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.792+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7870069180 con 0x7f7870102500 2024-06-06T08:39:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.794+0000 7f786d7fa700 1 -- 172.21.15.27:0/586964473 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7860009930 con 0x7f7870102500 2024-06-06T08:39:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.794+0000 7f786d7fa700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f78580a2ca0 0x7f78580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.794+0000 7f786d7fa700 1 -- 172.21.15.27:0/586964473 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f78600cef80 con 0x7f7870102500 2024-06-06T08:39:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.794+0000 7f786f7fe700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f78580a2ca0 0x7f78580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:36.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.795+0000 7f786f7fe700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f78580a2ca0 0x7f78580a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7864005ea0 tx=0x7f7864005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:36.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:36.799+0000 7f786d7fa700 1 -- 172.21.15.27:0/586964473 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f786009bb10 con 0x7f7870102500 2024-06-06T08:39:37.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.015+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f78700628d0 con 0x7f78580a2ca0 2024-06-06T08:39:37.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.029+0000 7f786d7fa700 1 -- 172.21.15.27:0/586964473 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f78700628d0 con 0x7f78580a2ca0 2024-06-06T08:39:37.029 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-06T08:39:37.029 INFO:teuthology.orchestra.run.smithi027.stdout:mds.a smithi027 running (3m) 37s ago 3m 14.1M - 19.0.0-2946-g3d89accf b045ac545f82 bf14b0f8c55e 2024-06-06T08:39:37.029 INFO:teuthology.orchestra.run.smithi027.stdout:mds.b smithi176 running (3m) 77s ago 3m 28.8M - 19.0.0-2946-g3d89accf b045ac545f82 8210361d2b03 2024-06-06T08:39:37.029 INFO:teuthology.orchestra.run.smithi027.stdout:mds.c smithi203 running (3m) 77s ago 3m 20.5M - 19.0.0-2946-g3d89accf b045ac545f82 5c47540cd5cd 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mds.d smithi027 running (3m) 37s ago 3m 14.5M - 19.0.0-2946-g3d89accf b045ac545f82 c86825e1d59d 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mds.e smithi176 running (3m) 77s ago 3m 18.7M - 19.0.0-2946-g3d89accf b045ac545f82 58df65fbc510 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mds.f smithi203 running (3m) 77s ago 3m 18.7M - 19.0.0-2946-g3d89accf b045ac545f82 aaca192b1d3e 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mds.g smithi027 running (3m) 37s ago 3m 15.7M - 19.0.0-2946-g3d89accf b045ac545f82 95bb7bab2ce7 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mds.h smithi176 running (3m) 77s ago 3m 17.6M - 19.0.0-2946-g3d89accf b045ac545f82 b19b4f193920 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mds.i smithi203 running (3m) 77s ago 3m 18.3M - 19.0.0-2946-g3d89accf b045ac545f82 6ec074f172ee 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mds.j smithi027 running (3m) 37s ago 3m 13.6M - 19.0.0-2946-g3d89accf b045ac545f82 7753269c0d96 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mds.k smithi176 running (4m) 77s ago 4m 14.3M - 19.0.0-2946-g3d89accf b045ac545f82 2543aa97c256 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mds.l smithi203 running (4m) 77s ago 4m 13.8M - 19.0.0-2946-g3d89accf b045ac545f82 ca8f63a505d1 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.x smithi027 *:9283,8765 running (11m) 37s ago 11m 518M - 19.0.0-2946-g3d89accf b045ac545f82 e42a772f7b3d 2024-06-06T08:39:37.030 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.y smithi176 *:8443,8765 running (9m) 77s ago 9m 456M - 19.0.0-2946-g3d89accf b045ac545f82 26d069ee88a7 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.z smithi203 *:8443,8765 running (9m) 77s ago 9m 455M - 19.0.0-2946-g3d89accf b045ac545f82 a7190f80265a 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a smithi027 running (11m) 37s ago 11m 50.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 ddea9b723a2d 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b smithi176 running (9m) 77s ago 9m 40.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 e636c7c3892f 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c smithi203 running (9m) 77s ago 9m 42.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 276655564d2f 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (9m) 37s ago 9m 48.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 041bb80bb98c 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi176 running (8m) 77s ago 8m 56.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 993160e51dd3 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi203 running (8m) 77s ago 8m 58.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 b9ee16b5d62a 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi027 running (7m) 37s ago 7m 47.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e93a91fcb967 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi176 running (7m) 77s ago 7m 44.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d2a49b94d24f 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi203 running (7m) 77s ago 7m 49.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 8d1eb7db8a88 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 smithi027 running (6m) 37s ago 6m 44.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 537977a48b3c 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 smithi176 running (6m) 77s ago 6m 47.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 c2f208249487 2024-06-06T08:39:37.031 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 smithi203 running (5m) 77s ago 5m 43.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 3875a46c8e4c 2024-06-06T08:39:37.032 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 smithi027 running (5m) 37s ago 5m 43.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 6ffe3e694f3c 2024-06-06T08:39:37.032 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 smithi176 running (4m) 77s ago 4m 54.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 870e9db97e28 2024-06-06T08:39:37.032 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 smithi203 running (4m) 77s ago 4m 54.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 fc156673dc66 2024-06-06T08:39:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f78580a2ca0 msgr2=0x7f78580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f78580a2ca0 0x7f78580a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7864005ea0 tx=0x7f7864005e30 comp rx=0 tx=0).stop 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7870102500 msgr2=0x7f78701985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7870102500 0x7f78701985f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f786000d9b0 tx=0x7f786000dcc0 comp rx=0 tx=0).stop 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 shutdown_connections 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f78580a2ca0 0x7f78580a5160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78701040a0 0x7f787019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7870102500 0x7f78701985f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 --2- 172.21.15.27:0/586964473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7870103700 0x7f7870198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 >> 172.21.15.27:0/586964473 conn(0x7f78700fdaa0 msgr2=0x7f7870106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 shutdown_connections 2024-06-06T08:39:37.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:37.032+0000 7f78763dd700 1 -- 172.21.15.27:0/586964473 wait complete. 2024-06-06T08:39:37.734 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-06T08:39:37.987 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:38.362 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:38 smithi027 ceph-mon[32335]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:38.362 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:38 smithi027 ceph-mon[32335]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:38.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:38 smithi203 ceph-mon[36579]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:38.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:38 smithi203 ceph-mon[36579]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:38.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:38 smithi176 ceph-mon[36786]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:38.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:38 smithi176 ceph-mon[36786]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:39.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 -- 172.21.15.27:0/4248441948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c104130 msgr2=0x7f865c10a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:39.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 --2- 172.21.15.27:0/4248441948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c104130 0x7f865c10a9e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f865400b0a0 tx=0x7f865400b3b0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 -- 172.21.15.27:0/4248441948 shutdown_connections 2024-06-06T08:39:39.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 --2- 172.21.15.27:0/4248441948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c104130 0x7f865c10a9e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 --2- 172.21.15.27:0/4248441948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f865c1007b0 0x7f865c100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 --2- 172.21.15.27:0/4248441948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f865c103790 0x7f865c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 -- 172.21.15.27:0/4248441948 >> 172.21.15.27:0/4248441948 conn(0x7f865c0fc0f0 msgr2=0x7f865c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:39.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 -- 172.21.15.27:0/4248441948 shutdown_connections 2024-06-06T08:39:39.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 -- 172.21.15.27:0/4248441948 wait complete. 2024-06-06T08:39:39.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.752+0000 7f8661d55700 1 Processor -- start 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f8661d55700 1 -- start start 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f8661d55700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c1007b0 0x7f865c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f8661d55700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f865c103790 0x7f865c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f8661d55700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f865c104130 0x7f865c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f8661d55700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f865c10e120 con 0x7f865c1007b0 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f8661d55700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f865c10dfa0 con 0x7f865c104130 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f8661d55700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f865c10e2a0 con 0x7f865c103790 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c1007b0 0x7f865c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c1007b0 0x7f865c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41290/0 (socket says 172.21.15.27:41290) 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865b7fe700 1 -- 172.21.15.27:0/1312874377 learned_addr learned my addr 172.21.15.27:0/1312874377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865bfff700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f865c104130 0x7f865c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:39.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865affd700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f865c103790 0x7f865c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:39.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865b7fe700 1 -- 172.21.15.27:0/1312874377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f865c103790 msgr2=0x7f865c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:39.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865b7fe700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f865c103790 0x7f865c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865b7fe700 1 -- 172.21.15.27:0/1312874377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f865c104130 msgr2=0x7f865c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:39.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865b7fe700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f865c104130 0x7f865c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.753+0000 7f865b7fe700 1 -- 172.21.15.27:0/1312874377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c1a0340 con 0x7f865c1007b0 2024-06-06T08:39:39.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.754+0000 7f865b7fe700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c1007b0 0x7f865c1986e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c920 tx=0x7f864c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:39.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.754+0000 7f8658ff9700 1 -- 172.21.15.27:0/1312874377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c004220 con 0x7f865c1007b0 2024-06-06T08:39:39.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.754+0000 7f8658ff9700 1 -- 172.21.15.27:0/1312874377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f864c007da0 con 0x7f865c1007b0 2024-06-06T08:39:39.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.754+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c1a0630 con 0x7f865c1007b0 2024-06-06T08:39:39.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.754+0000 7f8658ff9700 1 -- 172.21.15.27:0/1312874377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c003d10 con 0x7f865c1007b0 2024-06-06T08:39:39.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.754+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c1a0b70 con 0x7f865c1007b0 2024-06-06T08:39:39.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.755+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c069180 con 0x7f865c1007b0 2024-06-06T08:39:39.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.756+0000 7f8658ff9700 1 -- 172.21.15.27:0/1312874377 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f864c0043c0 con 0x7f865c1007b0 2024-06-06T08:39:39.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.757+0000 7f8658ff9700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f86440a2ca0 0x7f86440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:39.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.757+0000 7f8658ff9700 1 -- 172.21.15.27:0/1312874377 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f864c0cef80 con 0x7f865c1007b0 2024-06-06T08:39:39.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.757+0000 7f865affd700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f86440a2ca0 0x7f86440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:39.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.757+0000 7f865affd700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f86440a2ca0 0x7f86440a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8650005950 tx=0x7f86500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:39.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.761+0000 7f8658ff9700 1 -- 172.21.15.27:0/1312874377 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f864c014030 con 0x7f865c1007b0 2024-06-06T08:39:39.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.976+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f865c0628d0 con 0x7f86440a2ca0 2024-06-06T08:39:39.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.982+0000 7f8658ff9700 1 -- 172.21.15.27:0/1312874377 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f865c0628d0 con 0x7f86440a2ca0 2024-06-06T08:39:39.982 INFO:teuthology.orchestra.run.smithi027.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-06T08:39:39.982 INFO:teuthology.orchestra.run.smithi027.stdout:mds.all 12/12 80s ago 4m smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-06-06T08:39:39.982 INFO:teuthology.orchestra.run.smithi027.stdout:mgr 3/3 80s ago 9m smithi027=x;smithi176=y;smithi203=z;count:3 2024-06-06T08:39:39.982 INFO:teuthology.orchestra.run.smithi027.stdout:mon 3/3 80s ago 10m smithi027:172.21.15.27=a;smithi176:172.21.15.176=b;smithi203:172.21.15.203=c;count:3 2024-06-06T08:39:39.982 INFO:teuthology.orchestra.run.smithi027.stdout:osd 12 80s ago - 2024-06-06T08:39:39.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f86440a2ca0 msgr2=0x7f86440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:39.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f86440a2ca0 0x7f86440a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8650005950 tx=0x7f86500058e0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c1007b0 msgr2=0x7f865c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:39.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c1007b0 0x7f865c1986e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c920 tx=0x7f864c00cc30 comp rx=0 tx=0).stop 2024-06-06T08:39:39.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 shutdown_connections 2024-06-06T08:39:39.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f86440a2ca0 0x7f86440a5160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f865c104130 0x7f865c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f865c1007b0 0x7f865c1986e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 --2- 172.21.15.27:0/1312874377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f865c103790 0x7f865c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:39.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.984+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 >> 172.21.15.27:0/1312874377 conn(0x7f865c0fc0f0 msgr2=0x7f865c106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:39.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.985+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 shutdown_connections 2024-06-06T08:39:39.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:39.985+0000 7f8661d55700 1 -- 172.21.15.27:0/1312874377 wait complete. 2024-06-06T08:39:40.361 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:40 smithi027 ceph-mon[32335]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:40.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:40 smithi203 ceph-mon[36579]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:40.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:40 smithi176 ceph-mon[36786]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:40.692 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-06T08:39:40.944 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:41.325 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:41 smithi027 ceph-mon[32335]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:41.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:41 smithi203 ceph-mon[36579]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:41.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:41 smithi176 ceph-mon[36786]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:42.366 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:42 smithi027 ceph-mon[32335]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:42.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:42 smithi203 ceph-mon[36579]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:42.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:42 smithi176 ceph-mon[36786]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.629+0000 7f71e1e63700 1 -- 172.21.15.27:0/4236775092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 msgr2=0x7f71dc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.629+0000 7f71e1e63700 1 --2- 172.21.15.27:0/4236775092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 0x7f71dc101810 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00b0d0 tx=0x7f71cc00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.629+0000 7f71e1e63700 1 -- 172.21.15.27:0/4236775092 shutdown_connections 2024-06-06T08:39:42.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.629+0000 7f71e1e63700 1 --2- 172.21.15.27:0/4236775092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71dc102f90 0x7f71dc109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.629+0000 7f71e1e63700 1 --2- 172.21.15.27:0/4236775092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 0x7f71dc101810 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.629+0000 7f71e1e63700 1 --2- 172.21.15.27:0/4236775092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc1025f0 0x7f71dc102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.629+0000 7f71e1e63700 1 -- 172.21.15.27:0/4236775092 >> 172.21.15.27:0/4236775092 conn(0x7f71dc0fc990 msgr2=0x7f71dc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.629+0000 7f71e1e63700 1 -- 172.21.15.27:0/4236775092 shutdown_connections 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.629+0000 7f71e1e63700 1 -- 172.21.15.27:0/4236775092 wait complete. 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71e1e63700 1 Processor -- start 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71e1e63700 1 -- start start 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71e1e63700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 0x7f71dc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71e1e63700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71dc1025f0 0x7f71dc198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71e1e63700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc102f90 0x7f71dc19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71e1e63700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10d030 con 0x7f71dc1013f0 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71e1e63700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10ceb0 con 0x7f71dc1025f0 2024-06-06T08:39:42.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71e1e63700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10d1b0 con 0x7f71dc102f90 2024-06-06T08:39:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 0x7f71dc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 0x7f71dc198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33846/0 (socket says 172.21.15.27:33846) 2024-06-06T08:39:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.630+0000 7f71db7fe700 1 -- 172.21.15.27:0/315450322 learned_addr learned my addr 172.21.15.27:0/315450322 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71dbfff700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc102f90 0x7f71dc19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71daffd700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71dc1025f0 0x7f71dc198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71db7fe700 1 -- 172.21.15.27:0/315450322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc102f90 msgr2=0x7f71dc19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71db7fe700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc102f90 0x7f71dc19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71db7fe700 1 -- 172.21.15.27:0/315450322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71dc1025f0 msgr2=0x7f71dc198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:42.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71db7fe700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71dc1025f0 0x7f71dc198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71db7fe700 1 -- 172.21.15.27:0/315450322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc1a03f0 con 0x7f71dc1013f0 2024-06-06T08:39:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71dbfff700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc102f90 0x7f71dc19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71daffd700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71dc1025f0 0x7f71dc198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71db7fe700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 0x7f71dc198790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71cc007d90 tx=0x7f71cc007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71d8ff9700 1 -- 172.21.15.27:0/315450322 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc01f070 con 0x7f71dc1013f0 2024-06-06T08:39:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71d8ff9700 1 -- 172.21.15.27:0/315450322 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71cc005530 con 0x7f71dc1013f0 2024-06-06T08:39:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc1a0680 con 0x7f71dc1013f0 2024-06-06T08:39:42.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71d8ff9700 1 -- 172.21.15.27:0/315450322 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc01b880 con 0x7f71dc1013f0 2024-06-06T08:39:42.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.631+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc1a0bc0 con 0x7f71dc1013f0 2024-06-06T08:39:42.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.632+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71dc069180 con 0x7f71dc1013f0 2024-06-06T08:39:42.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.634+0000 7f71d8ff9700 1 -- 172.21.15.27:0/315450322 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f71cc022070 con 0x7f71dc1013f0 2024-06-06T08:39:42.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.634+0000 7f71d8ff9700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f71c80a2c20 0x7f71c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:42.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.634+0000 7f71daffd700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f71c80a2c20 0x7f71c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.634+0000 7f71d8ff9700 1 -- 172.21.15.27:0/315450322 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f71cc00f020 con 0x7f71dc1013f0 2024-06-06T08:39:42.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.635+0000 7f71daffd700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f71c80a2c20 0x7f71c80a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f71c400bcf0 tx=0x7f71c400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:42.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.638+0000 7f71d8ff9700 1 -- 172.21.15.27:0/315450322 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f71cc029020 con 0x7f71dc1013f0 2024-06-06T08:39:42.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.849+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f71dc0feb60 con 0x7f71c80a2c20 2024-06-06T08:39:42.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.851+0000 7f71d8ff9700 1 -- 172.21.15.27:0/315450322 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f71dc0feb60 con 0x7f71c80a2c20 2024-06-06T08:39:42.850 INFO:teuthology.orchestra.run.smithi027.stdout:HOST ADDR LABELS STATUS 2024-06-06T08:39:42.850 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 172.21.15.27 2024-06-06T08:39:42.850 INFO:teuthology.orchestra.run.smithi027.stdout:smithi176 172.21.15.176 2024-06-06T08:39:42.850 INFO:teuthology.orchestra.run.smithi027.stdout:smithi203 172.21.15.203 2024-06-06T08:39:42.850 INFO:teuthology.orchestra.run.smithi027.stdout:3 hosts in cluster 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f71c80a2c20 msgr2=0x7f71c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f71c80a2c20 0x7f71c80a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f71c400bcf0 tx=0x7f71c400b2c0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 msgr2=0x7f71dc198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 0x7f71dc198790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71cc007d90 tx=0x7f71cc007e70 comp rx=0 tx=0).stop 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 shutdown_connections 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f71c80a2c20 0x7f71c80a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71dc102f90 0x7f71dc19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71dc1013f0 0x7f71dc198790 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 --2- 172.21.15.27:0/315450322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71dc1025f0 0x7f71dc198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.853+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 >> 172.21.15.27:0/315450322 conn(0x7f71dc0fc990 msgr2=0x7f71dc0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:42.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.854+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 shutdown_connections 2024-06-06T08:39:42.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:42.854+0000 7f71e1e63700 1 -- 172.21.15.27:0/315450322 wait complete. 2024-06-06T08:39:43.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-06T08:39:43.786 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:44 smithi027 ceph-mon[32335]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:44 smithi027 ceph-mon[32335]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:44.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:44 smithi203 ceph-mon[36579]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:44.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:44 smithi203 ceph-mon[36579]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:44.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:44 smithi176 ceph-mon[36786]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:44.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:44 smithi176 ceph-mon[36786]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.437+0000 7f7d93cc1700 1 -- 172.21.15.27:0/819032118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c102720 msgr2=0x7f7d8c108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.437+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/819032118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c102720 0x7f7d8c108fd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400b0d0 tx=0x7f7d8400b3e0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.437+0000 7f7d93cc1700 1 -- 172.21.15.27:0/819032118 shutdown_connections 2024-06-06T08:39:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.437+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/819032118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c102720 0x7f7d8c108fd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.437+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/819032118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8c100b80 0x7f7d8c100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.437+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/819032118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d8c101d80 0x7f7d8c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.437+0000 7f7d93cc1700 1 -- 172.21.15.27:0/819032118 >> 172.21.15.27:0/819032118 conn(0x7f7d8c0fc100 msgr2=0x7f7d8c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.437+0000 7f7d93cc1700 1 -- 172.21.15.27:0/819032118 shutdown_connections 2024-06-06T08:39:45.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.437+0000 7f7d93cc1700 1 -- 172.21.15.27:0/819032118 wait complete. 2024-06-06T08:39:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d93cc1700 1 Processor -- start 2024-06-06T08:39:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d93cc1700 1 -- start start 2024-06-06T08:39:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d93cc1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8c100b80 0x7f7d8c104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:45.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d93cc1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c101d80 0x7f7d8c103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d93cc1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d8c102720 0x7f7d8c1036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d93cc1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c10bec0 con 0x7f7d8c101d80 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d93cc1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c10bd40 con 0x7f7d8c102720 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d93cc1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c10c040 con 0x7f7d8c100b80 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d9125c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c101d80 0x7f7d8c103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d9125c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c101d80 0x7f7d8c103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33872/0 (socket says 172.21.15.27:33872) 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d9125c700 1 -- 172.21.15.27:0/136993697 learned_addr learned my addr 172.21.15.27:0/136993697 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d91a5d700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8c100b80 0x7f7d8c104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.438+0000 7f7d9225e700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d8c102720 0x7f7d8c1036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d9125c700 1 -- 172.21.15.27:0/136993697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8c100b80 msgr2=0x7f7d8c104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d9125c700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8c100b80 0x7f7d8c104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d9125c700 1 -- 172.21.15.27:0/136993697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d8c102720 msgr2=0x7f7d8c1036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:45.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d9125c700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d8c102720 0x7f7d8c1036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d9125c700 1 -- 172.21.15.27:0/136993697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c103f50 con 0x7f7d8c101d80 2024-06-06T08:39:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d9125c700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c101d80 0x7f7d8c103160 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00d950 tx=0x7f7d7c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d82ffd700 1 -- 172.21.15.27:0/136993697 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c016070 con 0x7f7d8c101d80 2024-06-06T08:39:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c1a2810 con 0x7f7d8c101d80 2024-06-06T08:39:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d82ffd700 1 -- 172.21.15.27:0/136993697 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d7c00fd70 con 0x7f7d8c101d80 2024-06-06T08:39:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d82ffd700 1 -- 172.21.15.27:0/136993697 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c00e610 con 0x7f7d8c101d80 2024-06-06T08:39:45.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.439+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c1a2d50 con 0x7f7d8c101d80 2024-06-06T08:39:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.440+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8c069180 con 0x7f7d8c101d80 2024-06-06T08:39:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.441+0000 7f7d82ffd700 1 -- 172.21.15.27:0/136993697 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d7c019070 con 0x7f7d8c101d80 2024-06-06T08:39:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.442+0000 7f7d82ffd700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f7d780a2c20 0x7f7d780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.442+0000 7f7d91a5d700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f7d780a2c20 0x7f7d780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.442+0000 7f7d82ffd700 1 -- 172.21.15.27:0/136993697 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f7d7c0ceec0 con 0x7f7d8c101d80 2024-06-06T08:39:45.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.442+0000 7f7d91a5d700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f7d780a2c20 0x7f7d780a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c101be0 tx=0x7f7d8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:45.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.447+0000 7f7d82ffd700 1 -- 172.21.15.27:0/136993697 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7d7c09ba50 con 0x7f7d8c101d80 2024-06-06T08:39:45.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.660+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f7d8c1088e0 con 0x7f7d780a2c20 2024-06-06T08:39:45.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.662+0000 7f7d82ffd700 1 -- 172.21.15.27:0/136993697 <== mgr.14152 v2:172.21.15.27:6800/3763593724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f7d8c1088e0 con 0x7f7d780a2c20 2024-06-06T08:39:45.660 INFO:teuthology.orchestra.run.smithi027.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-06T08:39:45.661 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-06T08:39:45.661 INFO:teuthology.orchestra.run.smithi027.stdout:smithi176 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330099400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-06T08:39:45.661 INFO:teuthology.orchestra.run.smithi027.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-06T08:39:45.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.664+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f7d780a2c20 msgr2=0x7f7d780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:45.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.664+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f7d780a2c20 0x7f7d780a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c101be0 tx=0x7f7d8800b410 comp rx=0 tx=0).stop 2024-06-06T08:39:45.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.664+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c101d80 msgr2=0x7f7d8c103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:45.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.664+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c101d80 0x7f7d8c103160 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00d950 tx=0x7f7d7c00dc60 comp rx=0 tx=0).stop 2024-06-06T08:39:45.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.665+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 shutdown_connections 2024-06-06T08:39:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.665+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f7d780a2c20 0x7f7d780a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.665+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d8c102720 0x7f7d8c1036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.665+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8c100b80 0x7f7d8c104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.665+0000 7f7d93cc1700 1 --2- 172.21.15.27:0/136993697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d8c101d80 0x7f7d8c103160 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.665+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 >> 172.21.15.27:0/136993697 conn(0x7f7d8c0fc100 msgr2=0x7f7d8c1071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.665+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 shutdown_connections 2024-06-06T08:39:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:45.665+0000 7f7d93cc1700 1 -- 172.21.15.27:0/136993697 wait complete. 2024-06-06T08:39:46.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:46 smithi027 ceph-mon[32335]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:39:46.352 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T08:39:46.362 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:39:46.362 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-06T08:39:46.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:46 smithi203 ceph-mon[36579]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:39:46.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:46 smithi176 ceph-mon[36786]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T08:39:46.615 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:47.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:47 smithi203 ceph-mon[36579]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:47.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:47 smithi176 ceph-mon[36786]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:47.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:47 smithi027 ceph-mon[32335]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:39:48.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.231+0000 7fd943fb3700 1 -- 172.21.15.27:0/1120015276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1013f0 msgr2=0x7fd93c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:48.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.231+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1120015276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1013f0 0x7fd93c101810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd9380099c0 tx=0x7fd938009cd0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.231+0000 7fd943fb3700 1 -- 172.21.15.27:0/1120015276 shutdown_connections 2024-06-06T08:39:48.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.231+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1120015276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd93c102f90 0x7fd93c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.231+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1120015276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1013f0 0x7fd93c101810 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.231+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1120015276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd93c1025f0 0x7fd93c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.231+0000 7fd943fb3700 1 -- 172.21.15.27:0/1120015276 >> 172.21.15.27:0/1120015276 conn(0x7fd93c0fc990 msgr2=0x7fd93c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.231+0000 7fd943fb3700 1 -- 172.21.15.27:0/1120015276 shutdown_connections 2024-06-06T08:39:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.231+0000 7fd943fb3700 1 -- 172.21.15.27:0/1120015276 wait complete. 2024-06-06T08:39:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.232+0000 7fd943fb3700 1 Processor -- start 2024-06-06T08:39:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.232+0000 7fd943fb3700 1 -- start start 2024-06-06T08:39:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.232+0000 7fd943fb3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd93c1013f0 0x7fd93c06ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.232+0000 7fd943fb3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1025f0 0x7fd93c19d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.232+0000 7fd943fb3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd93c102f90 0x7fd93c19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd943fb3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c10d010 con 0x7fd93c1025f0 2024-06-06T08:39:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd943fb3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c10ce90 con 0x7fd93c102f90 2024-06-06T08:39:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd943fb3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c10d190 con 0x7fd93c1013f0 2024-06-06T08:39:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd94154e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1025f0 0x7fd93c19d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd94154e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1025f0 0x7fd93c19d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33892/0 (socket says 172.21.15.27:33892) 2024-06-06T08:39:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd94154e700 1 -- 172.21.15.27:0/1518941935 learned_addr learned my addr 172.21.15.27:0/1518941935 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd941d4f700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd93c1013f0 0x7fd93c06ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd942550700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd93c102f90 0x7fd93c19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd94154e700 1 -- 172.21.15.27:0/1518941935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd93c1013f0 msgr2=0x7fd93c06ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd94154e700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd93c1013f0 0x7fd93c06ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd94154e700 1 -- 172.21.15.27:0/1518941935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd93c102f90 msgr2=0x7fd93c19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd94154e700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd93c102f90 0x7fd93c19fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd94154e700 1 -- 172.21.15.27:0/1518941935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c1a0440 con 0x7fd93c1025f0 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd94154e700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1025f0 0x7fd93c19d4e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00d950 tx=0x7fd92c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd932ffd700 1 -- 172.21.15.27:0/1518941935 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c016070 con 0x7fd93c1025f0 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd932ffd700 1 -- 172.21.15.27:0/1518941935 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd92c00fd70 con 0x7fd93c1025f0 2024-06-06T08:39:48.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93c1a0730 con 0x7fd93c1025f0 2024-06-06T08:39:48.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.233+0000 7fd932ffd700 1 -- 172.21.15.27:0/1518941935 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00e610 con 0x7fd93c1025f0 2024-06-06T08:39:48.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.234+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93c1a0cc0 con 0x7fd93c1025f0 2024-06-06T08:39:48.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.235+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93c069180 con 0x7fd93c1025f0 2024-06-06T08:39:48.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.236+0000 7fd932ffd700 1 -- 172.21.15.27:0/1518941935 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd92c019070 con 0x7fd93c1025f0 2024-06-06T08:39:48.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.236+0000 7fd932ffd700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd9280a2ca0 0x7fd9280a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:48.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.237+0000 7fd941d4f700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd9280a2ca0 0x7fd9280a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:48.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.237+0000 7fd932ffd700 1 -- 172.21.15.27:0/1518941935 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fd92c0cee40 con 0x7fd93c1025f0 2024-06-06T08:39:48.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.237+0000 7fd941d4f700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd9280a2ca0 0x7fd9280a5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd938000c00 tx=0x7fd938005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:48.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.241+0000 7fd932ffd700 1 -- 172.21.15.27:0/1518941935 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd92c09b9d0 con 0x7fd93c1025f0 2024-06-06T08:39:48.241 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:48 smithi027 ceph-mon[32335]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:48.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:48 smithi203 ceph-mon[36579]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:48.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.495+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd93c0630a0 con 0x7fd93c1025f0 2024-06-06T08:39:48.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.496+0000 7fd932ffd700 1 -- 172.21.15.27:0/1518941935 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3058 (secure 0 0 0) 0x7fd92c09b2d0 con 0x7fd93c1025f0 2024-06-06T08:39:48.494 INFO:teuthology.orchestra.run.smithi027.stdout:e24 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 24 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-06-06T08:36:23.190518+0000 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-06-06T08:37:27.822986+0000 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2024-06-06T08:39:48.495 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 300 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:max_xattr_size 65536 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 3 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1,2 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24482,1=24460,2=24458} 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2024-06-06T08:39:48.496 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3,4] 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:bal_rank_mask -1 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 0 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:qdb_cluster leader: 24482 members: 24482,24458,24460 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b{0:24482} state up:active seq 6 addr [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c{0:24445} state up:standby-replay seq 1 addr [v2:172.21.15.203:6836/266251643,v1:172.21.15.203:6837/266251643] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.i{1:24460} state up:active seq 23 addr [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.176:6834/1368203014,v1:172.21.15.176:6835/1368203014] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.h{2:24458} state up:active seq 24 addr [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.497 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f{2:24433} state up:standby-replay seq 1 addr [v2:172.21.15.203:6834/2823423002,v1:172.21.15.203:6835/2823423002] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.j{-1:14511} state up:standby seq 1 addr [v2:172.21.15.27:6834/1517816957,v1:172.21.15.27:6835/1517816957] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d{-1:14517} state up:standby seq 1 addr [v2:172.21.15.27:6836/1670386995,v1:172.21.15.27:6837/1670386995] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.g{-1:14562} state up:standby seq 1 addr [v2:172.21.15.27:6838/2317631267,v1:172.21.15.27:6839/2317631267] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a{-1:14568} state up:standby seq 1 addr [v2:172.21.15.27:6840/1085633536,v1:172.21.15.27:6841/1085633536] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.l{-1:24397} state up:standby seq 1 addr [v2:172.21.15.203:6832/3789595568,v1:172.21.15.203:6833/3789595568] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.498 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.k{-1:24425} state up:standby seq 1 addr [v2:172.21.15.176:6832/1587702098,v1:172.21.15.176:6833/1587702098] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T08:39:48.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd9280a2ca0 msgr2=0x7fd9280a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:48.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd9280a2ca0 0x7fd9280a5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd938000c00 tx=0x7fd938005fb0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1025f0 msgr2=0x7fd93c19d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:48.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1025f0 0x7fd93c19d4e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00d950 tx=0x7fd92c00dc60 comp rx=0 tx=0).stop 2024-06-06T08:39:48.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 shutdown_connections 2024-06-06T08:39:48.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd9280a2ca0 0x7fd9280a5160 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd93c102f90 0x7fd93c19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd93c1013f0 0x7fd93c06ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 --2- 172.21.15.27:0/1518941935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd93c1025f0 0x7fd93c19d4e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:48.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 >> 172.21.15.27:0/1518941935 conn(0x7fd93c0fc990 msgr2=0x7fd93c1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:48.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 shutdown_connections 2024-06-06T08:39:48.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:48.498+0000 7fd943fb3700 1 -- 172.21.15.27:0/1518941935 wait complete. 2024-06-06T08:39:48.500 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-06-06T08:39:48.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:48 smithi176 ceph-mon[36786]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:39:49.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-06T08:39:49.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:49 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1518941935' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T08:39:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:49.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:49 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1518941935' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T08:39:49.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:49 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1518941935' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T08:39:50.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:50 smithi203 ceph-mon[36579]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:50.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:50 smithi176 ceph-mon[36786]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:50.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:50 smithi027 ceph-mon[32335]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:39:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.183+0000 7f152170c700 1 -- 172.21.15.27:0/2423192152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c1058b0 msgr2=0x7f151c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.183+0000 7f152170c700 1 --2- 172.21.15.27:0/2423192152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c1058b0 0x7f151c107ca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1510009990 tx=0x7f1510009ca0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.183+0000 7f152170c700 1 -- 172.21.15.27:0/2423192152 shutdown_connections 2024-06-06T08:39:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.183+0000 7f152170c700 1 --2- 172.21.15.27:0/2423192152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f151c1081e0 0x7f151c10a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.183+0000 7f152170c700 1 --2- 172.21.15.27:0/2423192152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f151c102f80 0x7f151c105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.183+0000 7f152170c700 1 --2- 172.21.15.27:0/2423192152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c1058b0 0x7f151c107ca0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.183+0000 7f152170c700 1 -- 172.21.15.27:0/2423192152 >> 172.21.15.27:0/2423192152 conn(0x7f151c0fc9b0 msgr2=0x7f151c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.183+0000 7f152170c700 1 -- 172.21.15.27:0/2423192152 shutdown_connections 2024-06-06T08:39:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.183+0000 7f152170c700 1 -- 172.21.15.27:0/2423192152 wait complete. 2024-06-06T08:39:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f152170c700 1 Processor -- start 2024-06-06T08:39:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f152170c700 1 -- start start 2024-06-06T08:39:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f152170c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c102f80 0x7f151c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f152170c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f151c1058b0 0x7f151c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f152170c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f151c1081e0 0x7f151c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f152170c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f151c10e100 con 0x7f151c102f80 2024-06-06T08:39:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f152170c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f151c10df80 con 0x7f151c1058b0 2024-06-06T08:39:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f152170c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f151c10e280 con 0x7f151c1081e0 2024-06-06T08:39:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f151affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c102f80 0x7f151c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f151affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c102f80 0x7f151c198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39188/0 (socket says 172.21.15.27:39188) 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f151affd700 1 -- 172.21.15.27:0/994711584 learned_addr learned my addr 172.21.15.27:0/994711584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.184+0000 7f151a7fc700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f151c1058b0 0x7f151c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f151b7fe700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f151c1081e0 0x7f151c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f151affd700 1 -- 172.21.15.27:0/994711584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f151c1081e0 msgr2=0x7f151c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f151affd700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f151c1081e0 0x7f151c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f151affd700 1 -- 172.21.15.27:0/994711584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f151c1058b0 msgr2=0x7f151c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f151affd700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f151c1058b0 0x7f151c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f151affd700 1 -- 172.21.15.27:0/994711584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c1a02e0 con 0x7f151c102f80 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f151b7fe700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f151c1081e0 0x7f151c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f151affd700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c102f80 0x7f151c198680 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f150c00d980 tx=0x7f150c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f1503fff700 1 -- 172.21.15.27:0/994711584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c016070 con 0x7f151c102f80 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c1a05d0 con 0x7f151c102f80 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f1503fff700 1 -- 172.21.15.27:0/994711584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150c00de70 con 0x7f151c102f80 2024-06-06T08:39:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f1503fff700 1 -- 172.21.15.27:0/994711584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c00f3c0 con 0x7f151c102f80 2024-06-06T08:39:51.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.185+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c1a0b10 con 0x7f151c102f80 2024-06-06T08:39:51.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.187+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151c069180 con 0x7f151c102f80 2024-06-06T08:39:51.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.188+0000 7f1503fff700 1 -- 172.21.15.27:0/994711584 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f150c019070 con 0x7f151c102f80 2024-06-06T08:39:51.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.188+0000 7f1503fff700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f15040a2c20 0x7f15040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:51.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.188+0000 7f151a7fc700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f15040a2c20 0x7f15040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:51.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.188+0000 7f1503fff700 1 -- 172.21.15.27:0/994711584 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f150c0cefa0 con 0x7f151c102f80 2024-06-06T08:39:51.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.189+0000 7f151a7fc700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f15040a2c20 0x7f15040a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f151000b5c0 tx=0x7f1510003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.193+0000 7f1503fff700 1 -- 172.21.15.27:0/994711584 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f150c09bb30 con 0x7f151c102f80 2024-06-06T08:39:51.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.402+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f151c0630a0 con 0x7f151c102f80 2024-06-06T08:39:51.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.404+0000 7f1503fff700 1 -- 172.21.15.27:0/994711584 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v99) ==== 56+0+5701 (secure 0 0 0) 0x7f150c09b430 con 0x7f151c102f80 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 99 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-06-06T08:28:03.871746+0000 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-06-06T08:38:01.008015+0000 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 27 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client luminous 2024-06-06T08:39:51.403 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release squid 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 95 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 12 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 8 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6802/1837835073,v1:172.21.15.27:6803/1837835073] [v2:172.21.15.27:6804/1837835073,v1:172.21.15.27:6805/1837835073] exists,up 70143ff8-83c2-42ee-b943-52fd8388c1c0 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6800/790447916,v1:172.21.15.176:6801/790447916] [v2:172.21.15.176:6802/790447916,v1:172.21.15.176:6803/790447916] exists,up ab40e570-fc0d-4f20-840d-fb0e56483710 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6800/252294479,v1:172.21.15.203:6801/252294479] [v2:172.21.15.203:6802/252294479,v1:172.21.15.203:6803/252294479] exists,up 8c91227d-9be5-4fea-81bc-e6696441777f 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 25 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6810/3079841108,v1:172.21.15.27:6811/3079841108] [v2:172.21.15.27:6812/3079841108,v1:172.21.15.27:6813/3079841108] exists,up aa6a8c4e-0080-4151-aee8-052550a92b6c 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 33 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6808/275393367,v1:172.21.15.176:6809/275393367] [v2:172.21.15.176:6810/275393367,v1:172.21.15.176:6811/275393367] exists,up b29f8a35-7dd1-4418-9aa3-dca189a351b6 2024-06-06T08:39:51.404 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 38 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6808/3782177772,v1:172.21.15.203:6809/3782177772] [v2:172.21.15.203:6810/3782177772,v1:172.21.15.203:6811/3782177772] exists,up a98cab95-b7f2-405a-ba80-8814c9a60ad9 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 up in weight 1 up_from 46 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6818/1876109947,v1:172.21.15.27:6819/1876109947] [v2:172.21.15.27:6820/1876109947,v1:172.21.15.27:6821/1876109947] exists,up c4261660-83ed-4dbc-9789-405ffbb4389a 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6816/3681627280,v1:172.21.15.176:6817/3681627280] [v2:172.21.15.176:6818/3681627280,v1:172.21.15.176:6819/3681627280] exists,up e8f52b13-40bd-443c-a8e1-d3ce46bb4397 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 up in weight 1 up_from 59 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6816/2693614915,v1:172.21.15.203:6817/2693614915] [v2:172.21.15.203:6818/2693614915,v1:172.21.15.203:6819/2693614915] exists,up fd6572ce-6f94-49c6-9602-07e0196dada4 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 up in weight 1 up_from 67 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6826/657558341,v1:172.21.15.27:6827/657558341] [v2:172.21.15.27:6828/657558341,v1:172.21.15.27:6829/657558341] exists,up 3bece157-d4cb-4072-a56d-a3e3d3c09f85 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 up in weight 1 up_from 72 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6824/985198618,v1:172.21.15.176:6825/985198618] [v2:172.21.15.176:6826/985198618,v1:172.21.15.176:6827/985198618] exists,up 1044fbd0-7574-4faf-a3b1-da2a76a54e73 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 up in weight 1 up_from 80 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6824/1209624343,v1:172.21.15.203:6825/1209624343] [v2:172.21.15.203:6826/1209624343,v1:172.21.15.203:6827/1209624343] exists,up 0c7f5eef-bc5d-4858-abf0-42d889571c21 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.0 [8,4,7,11] 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.5 [8,11] 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.e [2,1] 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.14 [2,4] 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.0 [4,6] 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.1b [5,9] 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.1d [5,9] 2024-06-06T08:39:51.405 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.23 [4,7] 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.27 [4,8] 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.38 [3,9] 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.e [9,11] 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.11 [5,8] 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.12 [9,10] 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.14 [9,8] 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1905066984 expires 2024-06-07T08:28:58.346290+0000 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/2182206949 expires 2024-06-07T08:28:58.346290+0000 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/2182206949 expires 2024-06-07T08:28:58.346290+0000 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1183901040 expires 2024-06-07T08:28:31.314626+0000 2024-06-06T08:39:51.406 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1624199721 expires 2024-06-07T08:28:31.314626+0000 2024-06-06T08:39:51.407 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/897818490 expires 2024-06-07T08:28:31.314626+0000 2024-06-06T08:39:51.407 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/897818490 expires 2024-06-07T08:28:31.314626+0000 2024-06-06T08:39:51.407 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3611712452 expires 2024-06-07T08:28:58.346290+0000 2024-06-06T08:39:51.407 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1620281536 expires 2024-06-07T08:28:31.314626+0000 2024-06-06T08:39:51.407 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/752935431 expires 2024-06-07T08:28:58.346290+0000 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f15040a2c20 msgr2=0x7f15040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f15040a2c20 0x7f15040a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f151000b5c0 tx=0x7f1510003960 comp rx=0 tx=0).stop 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c102f80 msgr2=0x7f151c198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c102f80 0x7f151c198680 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f150c00d980 tx=0x7f150c00dc90 comp rx=0 tx=0).stop 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 shutdown_connections 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f15040a2c20 0x7f15040a50e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f151c1081e0 0x7f151c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f151c102f80 0x7f151c198680 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 --2- 172.21.15.27:0/994711584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f151c1058b0 0x7f151c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 >> 172.21.15.27:0/994711584 conn(0x7f151c0fc9b0 msgr2=0x7f151c1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 shutdown_connections 2024-06-06T08:39:51.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:51.406+0000 7f152170c700 1 -- 172.21.15.27:0/994711584 wait complete. 2024-06-06T08:39:51.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:51 smithi027 ceph-mon[32335]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:51.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:51 smithi203 ceph-mon[36579]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:51.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:51 smithi176 ceph-mon[36786]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T08:39:52.122 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-06T08:39:52.137 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:39:52.390 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:52.642 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:52 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/994711584' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T08:39:52.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:52 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/994711584' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T08:39:52.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:52 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/994711584' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T08:39:53.495 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:53 smithi027 ceph-mon[32335]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:53.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:53 smithi203 ceph-mon[36579]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:53.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:53 smithi176 ceph-mon[36786]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:39:54.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.022+0000 7f884d3d0700 1 -- 172.21.15.27:0/3717298618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 msgr2=0x7f88481070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.022+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3717298618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 0x7f88481070d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f883c0099c0 tx=0x7f883c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.022+0000 7f884d3d0700 1 -- 172.21.15.27:0/3717298618 shutdown_connections 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.022+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3717298618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88481077b0 0x7f884810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.022+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3717298618 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f884806ae60 0x7f884806b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.022+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3717298618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 0x7f88481070d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.022+0000 7f884d3d0700 1 -- 172.21.15.27:0/3717298618 >> 172.21.15.27:0/3717298618 conn(0x7f88480773e0 msgr2=0x7f88480777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.022+0000 7f884d3d0700 1 -- 172.21.15.27:0/3717298618 shutdown_connections 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.022+0000 7f884d3d0700 1 -- 172.21.15.27:0/3717298618 wait complete. 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.023+0000 7f884d3d0700 1 Processor -- start 2024-06-06T08:39:54.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.023+0000 7f884d3d0700 1 -- start start 2024-06-06T08:39:54.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f884d3d0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f884806ae60 0x7f884819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:54.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f884d3d0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 0x7f884819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:54.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f884d3d0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88481077b0 0x7f884819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:54.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f884d3d0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f88481127d0 con 0x7f884806b810 2024-06-06T08:39:54.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f884d3d0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8848112650 con 0x7f884806ae60 2024-06-06T08:39:54.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f884d3d0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8848112950 con 0x7f88481077b0 2024-06-06T08:39:54.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88467fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 0x7f884819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:54.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88467fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 0x7f884819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39212/0 (socket says 172.21.15.27:39212) 2024-06-06T08:39:54.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88467fc700 1 -- 172.21.15.27:0/3194371042 learned_addr learned my addr 172.21.15.27:0/3194371042 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f8846ffd700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f884806ae60 0x7f884819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88477fe700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88481077b0 0x7f884819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88467fc700 1 -- 172.21.15.27:0/3194371042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88481077b0 msgr2=0x7f884819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88467fc700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88481077b0 0x7f884819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88467fc700 1 -- 172.21.15.27:0/3194371042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f884806ae60 msgr2=0x7f884819ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88467fc700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f884806ae60 0x7f884819ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88467fc700 1 -- 172.21.15.27:0/3194371042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884819fe90 con 0x7f884806b810 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f88467fc700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 0x7f884819b3c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c010 tx=0x7f883c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f882ffff700 1 -- 172.21.15.27:0/3194371042 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c004830 con 0x7f884806b810 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.024+0000 7f882ffff700 1 -- 172.21.15.27:0/3194371042 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c005020 con 0x7f884806b810 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.025+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88481a0120 con 0x7f884806b810 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.025+0000 7f882ffff700 1 -- 172.21.15.27:0/3194371042 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c003cb0 con 0x7f884806b810 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.025+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88481a7a00 con 0x7f884806b810 2024-06-06T08:39:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.026+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8848069180 con 0x7f884806b810 2024-06-06T08:39:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.027+0000 7f882ffff700 1 -- 172.21.15.27:0/3194371042 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f883c0049d0 con 0x7f884806b810 2024-06-06T08:39:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.027+0000 7f882ffff700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f88300a2ca0 0x7f88300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.028+0000 7f8846ffd700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f88300a2ca0 0x7f88300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.028+0000 7f882ffff700 1 -- 172.21.15.27:0/3194371042 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f883c0d06f0 con 0x7f884806b810 2024-06-06T08:39:54.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.028+0000 7f8846ffd700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f88300a2ca0 0x7f88300a5160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8838005950 tx=0x7f883800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:54.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.031+0000 7f882ffff700 1 -- 172.21.15.27:0/3194371042 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f883c013070 con 0x7f884806b810 2024-06-06T08:39:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.284+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88481a7bd0 con 0x7f884806b810 2024-06-06T08:39:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.284+0000 7f882ffff700 1 -- 172.21.15.27:0/3194371042 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11882 (secure 0 0 0) 0x7f883c0095a0 con 0x7f884806b810 2024-06-06T08:39:54.283 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:54.283 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:37:27.822986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24458,24460]},"id":1}]} 2024-06-06T08:39:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f88300a2ca0 msgr2=0x7f88300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f88300a2ca0 0x7f88300a5160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8838005950 tx=0x7f883800b410 comp rx=0 tx=0).stop 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 msgr2=0x7f884819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 0x7f884819b3c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c010 tx=0x7f883c0047b0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 shutdown_connections 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f88300a2ca0 0x7f88300a5160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88481077b0 0x7f884819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f884806ae60 0x7f884819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 --2- 172.21.15.27:0/3194371042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f884806b810 0x7f884819b3c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 >> 172.21.15.27:0/3194371042 conn(0x7f88480773e0 msgr2=0x7f884810e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 shutdown_connections 2024-06-06T08:39:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:54.287+0000 7f884d3d0700 1 -- 172.21.15.27:0/3194371042 wait complete. 2024-06-06T08:39:54.287 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-06-06T08:39:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:54 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3194371042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:54.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:54 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3194371042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:54.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:54 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3194371042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:55.081 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:39:55.340 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:55.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:55 smithi027 ceph-mon[32335]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T08:39:55.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:55 smithi203 ceph-mon[36579]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T08:39:55.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:55 smithi176 ceph-mon[36786]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T08:39:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.066+0000 7f66b5c09700 1 -- 172.21.15.27:0/3454651499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 msgr2=0x7f66b0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.066+0000 7f66b5c09700 1 --2- 172.21.15.27:0/3454651499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 0x7f66b0101810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009990 tx=0x7f66a4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.066+0000 7f66b5c09700 1 -- 172.21.15.27:0/3454651499 shutdown_connections 2024-06-06T08:39:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.066+0000 7f66b5c09700 1 --2- 172.21.15.27:0/3454651499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66b0101d50 0x7f66b010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.066+0000 7f66b5c09700 1 --2- 172.21.15.27:0/3454651499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b0107400 0x7f66b01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.066+0000 7f66b5c09700 1 --2- 172.21.15.27:0/3454651499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 0x7f66b0101810 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.066+0000 7f66b5c09700 1 -- 172.21.15.27:0/3454651499 >> 172.21.15.27:0/3454651499 conn(0x7f66b0078340 msgr2=0x7f66b0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.066+0000 7f66b5c09700 1 -- 172.21.15.27:0/3454651499 shutdown_connections 2024-06-06T08:39:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.066+0000 7f66b5c09700 1 -- 172.21.15.27:0/3454651499 wait complete. 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.067+0000 7f66b5c09700 1 Processor -- start 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.067+0000 7f66b5c09700 1 -- start start 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.067+0000 7f66b5c09700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 0x7f66b0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.067+0000 7f66b5c09700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66b0101d50 0x7f66b0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.067+0000 7f66b5c09700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b0107400 0x7f66b01a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.067+0000 7f66b5c09700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01116f0 con 0x7f66b0101390 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.067+0000 7f66b5c09700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0111570 con 0x7f66b0107400 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.067+0000 7f66b5c09700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0111870 con 0x7f66b0101d50 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66af7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 0x7f66b0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66af7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 0x7f66b0199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39234/0 (socket says 172.21.15.27:39234) 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66af7fe700 1 -- 172.21.15.27:0/4141651849 learned_addr learned my addr 172.21.15.27:0/4141651849 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66aeffd700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66b0101d50 0x7f66b0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66af7fe700 1 -- 172.21.15.27:0/4141651849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66b0101d50 msgr2=0x7f66b0199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66af7fe700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66b0101d50 0x7f66b0199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66affff700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b0107400 0x7f66b01a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66af7fe700 1 -- 172.21.15.27:0/4141651849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b0107400 msgr2=0x7f66b01a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66af7fe700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b0107400 0x7f66b01a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66af7fe700 1 -- 172.21.15.27:0/4141651849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b01a5050 con 0x7f66b0101390 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66af7fe700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 0x7f66b0199a90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f66a000b530 tx=0x7f66a000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66acff9700 1 -- 172.21.15.27:0/4141651849 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0015020 con 0x7f66b0101390 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66acff9700 1 -- 172.21.15.27:0/4141651849 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a0011d10 con 0x7f66b0101390 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b01a5280 con 0x7f66b0101390 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66acff9700 1 -- 172.21.15.27:0/4141651849 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000f2e0 con 0x7f66b0101390 2024-06-06T08:39:57.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.068+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b01a57e0 con 0x7f66b0101390 2024-06-06T08:39:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.069+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b0109970 con 0x7f66b0101390 2024-06-06T08:39:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.071+0000 7f66acff9700 1 -- 172.21.15.27:0/4141651849 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66a0018020 con 0x7f66b0101390 2024-06-06T08:39:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.071+0000 7f66acff9700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f66980a2ca0 0x7f66980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.071+0000 7f66aeffd700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f66980a2ca0 0x7f66980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.071+0000 7f66acff9700 1 -- 172.21.15.27:0/4141651849 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f66a00cf030 con 0x7f66b0101390 2024-06-06T08:39:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.072+0000 7f66aeffd700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f66980a2ca0 0x7f66980a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f66a400b5c0 tx=0x7f66a4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:57.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.075+0000 7f66acff9700 1 -- 172.21.15.27:0/4141651849 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f66a001f020 con 0x7f66b0101390 2024-06-06T08:39:57.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.330+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66b0069180 con 0x7f66b0101390 2024-06-06T08:39:57.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.331+0000 7f66acff9700 1 -- 172.21.15.27:0/4141651849 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11882 (secure 0 0 0) 0x7f66a009bbc0 con 0x7f66b0101390 2024-06-06T08:39:57.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:39:57.330 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:37:27.822986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24458,24460]},"id":1}]} 2024-06-06T08:39:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f66980a2ca0 msgr2=0x7f66980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f66980a2ca0 0x7f66980a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f66a400b5c0 tx=0x7f66a4005960 comp rx=0 tx=0).stop 2024-06-06T08:39:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 msgr2=0x7f66b0199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 0x7f66b0199a90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f66a000b530 tx=0x7f66a000b840 comp rx=0 tx=0).stop 2024-06-06T08:39:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 shutdown_connections 2024-06-06T08:39:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f66980a2ca0 0x7f66980a5160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b0107400 0x7f66b01a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66b0101390 0x7f66b0199a90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 --2- 172.21.15.27:0/4141651849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66b0101d50 0x7f66b0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 >> 172.21.15.27:0/4141651849 conn(0x7f66b0078340 msgr2=0x7f66b00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 shutdown_connections 2024-06-06T08:39:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:57.333+0000 7f66b5c09700 1 -- 172.21.15.27:0/4141651849 wait complete. 2024-06-06T08:39:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-06-06T08:39:57.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:57 smithi027 ceph-mon[32335]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-06-06T08:39:57.696 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:57 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4141651849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:57.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:57 smithi203 ceph-mon[36579]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-06-06T08:39:57.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:57 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4141651849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:58.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:39:58.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:57 smithi176 ceph-mon[36786]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-06-06T08:39:58.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:57 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4141651849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:39:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:39:59.689 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:39:59 smithi027 ceph-mon[32335]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:39:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 -- 172.21.15.27:0/4283166732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 msgr2=0x7f5f14102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/4283166732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 0x7f5f14102940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009990 tx=0x7f5f08009ca0 comp rx=0 tx=0).stop 2024-06-06T08:39:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 -- 172.21.15.27:0/4283166732 shutdown_connections 2024-06-06T08:39:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/4283166732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f14102e80 0x7f5f1410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/4283166732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f14108650 0x7f5f14108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/4283166732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 0x7f5f14102940 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 -- 172.21.15.27:0/4283166732 >> 172.21.15.27:0/4283166732 conn(0x7f5f140795f0 msgr2=0x7f5f14079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 -- 172.21.15.27:0/4283166732 shutdown_connections 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 -- 172.21.15.27:0/4283166732 wait complete. 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 Processor -- start 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.961+0000 7f5f1aa33700 1 -- start start 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f1aa33700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 0x7f5f1419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f1aa33700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f14102e80 0x7f5f1419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f1aa33700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f14108650 0x7f5f14198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f1aa33700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f141127c0 con 0x7f5f141024e0 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f1aa33700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5f14112640 con 0x7f5f14102e80 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f1aa33700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5f14112940 con 0x7f5f14108650 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f13fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 0x7f5f1419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f13fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 0x7f5f1419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55276/0 (socket says 172.21.15.27:55276) 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f13fff700 1 -- 172.21.15.27:0/2606675142 learned_addr learned my addr 172.21.15.27:0/2606675142 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f137fe700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f14102e80 0x7f5f1419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f18fd0700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f14108650 0x7f5f14198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f13fff700 1 -- 172.21.15.27:0/2606675142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f14108650 msgr2=0x7f5f14198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f13fff700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f14108650 0x7f5f14198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f13fff700 1 -- 172.21.15.27:0/2606675142 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f14102e80 msgr2=0x7f5f1419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:39:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f13fff700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f14102e80 0x7f5f1419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f13fff700 1 -- 172.21.15.27:0/2606675142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f14199220 con 0x7f5f141024e0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.962+0000 7f5f13fff700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 0x7f5f1419e900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400c970 tx=0x7f5f0400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.963+0000 7f5f117fa700 1 -- 172.21.15.27:0/2606675142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f040078d0 con 0x7f5f141024e0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.963+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f14199510 con 0x7f5f141024e0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.963+0000 7f5f117fa700 1 -- 172.21.15.27:0/2606675142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f04018480 con 0x7f5f141024e0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.963+0000 7f5f117fa700 1 -- 172.21.15.27:0/2606675142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0400f6b0 con 0x7f5f141024e0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.963+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f141a57d0 con 0x7f5f141024e0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.964+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f14069180 con 0x7f5f141024e0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.965+0000 7f5f117fa700 1 -- 172.21.15.27:0/2606675142 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f04018630 con 0x7f5f141024e0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.966+0000 7f5f117fa700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5efc0a2c20 0x7f5efc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.966+0000 7f5f137fe700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5efc0a2c20 0x7f5efc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.966+0000 7f5f117fa700 1 -- 172.21.15.27:0/2606675142 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f5f040cf3c0 con 0x7f5f141024e0 2024-06-06T08:39:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.966+0000 7f5f137fe700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5efc0a2c20 0x7f5efc0a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800b5c0 tx=0x7f5f08005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:39:59.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:39:59.971+0000 7f5f117fa700 1 -- 172.21.15.27:0/2606675142 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5f0409c080 con 0x7f5f141024e0 2024-06-06T08:39:59.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:39:59 smithi203 ceph-mon[36579]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:40:00.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:39:59 smithi176 ceph-mon[36786]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:40:00.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.181+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f14199ff0 con 0x7f5f141024e0 2024-06-06T08:40:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.183+0000 7f5f117fa700 1 -- 172.21.15.27:0/2606675142 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24836 (secure 0 0 0) 0x7f5f0409b980 con 0x7f5f141024e0 2024-06-06T08:40:00.182 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:00.182 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":99,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:38:01.008015+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:40:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5efc0a2c20 msgr2=0x7f5efc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5efc0a2c20 0x7f5efc0a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800b5c0 tx=0x7f5f08005960 comp rx=0 tx=0).stop 2024-06-06T08:40:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 msgr2=0x7f5f1419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 0x7f5f1419e900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400c970 tx=0x7f5f0400cc80 comp rx=0 tx=0).stop 2024-06-06T08:40:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 shutdown_connections 2024-06-06T08:40:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f5efc0a2c20 0x7f5efc0a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f14108650 0x7f5f14198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f141024e0 0x7f5f1419e900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 --2- 172.21.15.27:0/2606675142 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f14102e80 0x7f5f1419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.185+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 >> 172.21.15.27:0/2606675142 conn(0x7f5f140795f0 msgr2=0x7f5f141011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.186+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 shutdown_connections 2024-06-06T08:40:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:00.186+0000 7f5f1aa33700 1 -- 172.21.15.27:0/2606675142 wait complete. 2024-06-06T08:40:00.853 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:00 smithi027 ceph-mon[32335]: overall HEALTH_OK 2024-06-06T08:40:00.853 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:00 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2606675142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:00.886 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T08:36:23.190518+0000', 'modified': '2024-06-06T08:37:27.822986+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24460, 'mds_2': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/2823423002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 2823423002}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 2823423002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6835/1368203014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6834', 'nonce': 1368203014}, {'type': 'v1', 'addr': '172.21.15.176:6835', 'nonce': 1368203014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/266251643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 266251643}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 266251643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.176:6837/3697558303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 3697558303}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 3697558303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.203:6839/693474346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 693474346}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 693474346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.176:6839/1702529484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 1702529484}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 1702529484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482, 24458, 24460]} 2024-06-06T08:40:00.886 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-06T08:40:00.886 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T08:40:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:00 smithi203 ceph-mon[36579]: overall HEALTH_OK 2024-06-06T08:40:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:00 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2606675142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:00 smithi176 ceph-mon[36786]: overall HEALTH_OK 2024-06-06T08:40:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:00 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2606675142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:01.141 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:01.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:01 smithi027 ceph-mon[32335]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-06-06T08:40:01.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:01 smithi203 ceph-mon[36579]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-06-06T08:40:02.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:01 smithi176 ceph-mon[36786]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-06-06T08:40:02.871 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:02 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.906+0000 7f0f58151700 1 -- 172.21.15.27:0/1550585185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50101d50 msgr2=0x7f0f5010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.906+0000 7f0f58151700 1 --2- 172.21.15.27:0/1550585185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50101d50 0x7f0f5010e240 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800b0a0 tx=0x7f0f4800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.906+0000 7f0f58151700 1 -- 172.21.15.27:0/1550585185 shutdown_connections 2024-06-06T08:40:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.906+0000 7f0f58151700 1 --2- 172.21.15.27:0/1550585185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50101d50 0x7f0f5010e240 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.906+0000 7f0f58151700 1 --2- 172.21.15.27:0/1550585185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f50107400 0x7f0f501077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.906+0000 7f0f58151700 1 --2- 172.21.15.27:0/1550585185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f50101390 0x7f0f50101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.906+0000 7f0f58151700 1 -- 172.21.15.27:0/1550585185 >> 172.21.15.27:0/1550585185 conn(0x7f0f50078340 msgr2=0x7f0f50078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:02.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.906+0000 7f0f58151700 1 -- 172.21.15.27:0/1550585185 shutdown_connections 2024-06-06T08:40:02.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.906+0000 7f0f58151700 1 -- 172.21.15.27:0/1550585185 wait complete. 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f58151700 1 Processor -- start 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f58151700 1 -- start start 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f58151700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f50101390 0x7f0f50199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f58151700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f50101d50 0x7f0f50199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f58151700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50107400 0x7f0f501a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f58151700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50111670 con 0x7f0f50107400 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f58151700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501114f0 con 0x7f0f50101d50 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f58151700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f501117f0 con 0x7f0f50101390 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f566ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50107400 0x7f0f501a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f566ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50107400 0x7f0f501a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55294/0 (socket says 172.21.15.27:55294) 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f556ec700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f50101d50 0x7f0f50199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f566ee700 1 -- 172.21.15.27:0/2551390767 learned_addr learned my addr 172.21.15.27:0/2551390767 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.907+0000 7f0f55eed700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f50101390 0x7f0f50199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f566ee700 1 -- 172.21.15.27:0/2551390767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f50101390 msgr2=0x7f0f50199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f566ee700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f50101390 0x7f0f50199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f566ee700 1 -- 172.21.15.27:0/2551390767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f50101d50 msgr2=0x7f0f50199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f566ee700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f50101d50 0x7f0f50199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f566ee700 1 -- 172.21.15.27:0/2551390767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f501a5010 con 0x7f0f50107400 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f556ec700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f50101d50 0x7f0f50199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f566ee700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50107400 0x7f0f501a4ad0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800b780 tx=0x7f0f48004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f46ffd700 1 -- 172.21.15.27:0/2551390767 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48021070 con 0x7f0f50107400 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f46ffd700 1 -- 172.21.15.27:0/2551390767 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f48007b80 con 0x7f0f50107400 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f46ffd700 1 -- 172.21.15.27:0/2551390767 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480052a0 con 0x7f0f50107400 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f58151700 1 -- 172.21.15.27:0/2551390767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f501a51e0 con 0x7f0f50107400 2024-06-06T08:40:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.908+0000 7f0f58151700 1 -- 172.21.15.27:0/2551390767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f501a55e0 con 0x7f0f50107400 2024-06-06T08:40:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.910+0000 7f0f46ffd700 1 -- 172.21.15.27:0/2551390767 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f48024070 con 0x7f0f50107400 2024-06-06T08:40:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.911+0000 7f0f46ffd700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f3c0a2c20 0x7f0f3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.911+0000 7f0f55eed700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f3c0a2c20 0x7f0f3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.911+0000 7f0f46ffd700 1 -- 172.21.15.27:0/2551390767 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f0f480d0370 con 0x7f0f50107400 2024-06-06T08:40:02.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.912+0000 7f0f55eed700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f3c0a2c20 0x7f0f3c0a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009be0 tx=0x7f0f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.912+0000 7f0f629d4b80 1 Processor -- start 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.912+0000 7f0f629d4b80 1 -- start start 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d6355f0 0x55bb2d6359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d635f10 0x55bb2d62f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d62fed0 0x55bb2d6302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d5817c0 con 0x55bb2d6355f0 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d5296d0 con 0x55bb2d62fed0 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d57d780 con 0x55bb2d635f10 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f55eed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d6355f0 0x55bb2d6359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f55eed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d6355f0 0x55bb2d6359d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55310/0 (socket says 172.21.15.27:55310) 2024-06-06T08:40:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f55eed700 1 -- 172.21.15.27:0/1959414138 learned_addr learned my addr 172.21.15.27:0/1959414138 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f556ec700 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d635f10 0x55bb2d62f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f566ee700 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d62fed0 0x55bb2d6302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f566ee700 1 -- 172.21.15.27:0/1959414138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d635f10 msgr2=0x55bb2d62f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f566ee700 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d635f10 0x55bb2d62f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f566ee700 1 -- 172.21.15.27:0/1959414138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d6355f0 msgr2=0x55bb2d6359d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f566ee700 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d6355f0 0x55bb2d6359d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f566ee700 1 -- 172.21.15.27:0/1959414138 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb2d51ff00 con 0x55bb2d62fed0 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f55eed700 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d6355f0 0x55bb2d6359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f566ee700 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d62fed0 0x55bb2d6302f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f4801dfd0 tx=0x7f0f4801dcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7e42186451d02bb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.913+0000 7f0f3bfff700 1 -- 172.21.15.27:0/1959414138 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4802b070 con 0x55bb2d62fed0 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.914+0000 7f0f3bfff700 1 -- 172.21.15.27:0/1959414138 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f480d0120 con 0x55bb2d62fed0 2024-06-06T08:40:02.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.914+0000 7f0f3bfff700 1 -- 172.21.15.27:0/1959414138 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48087070 con 0x55bb2d62fed0 2024-06-06T08:40:02.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.914+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1959414138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d62fed0 msgr2=0x55bb2d6302f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d62fed0 0x55bb2d6302f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f4801dfd0 tx=0x7f0f4801dcd0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1959414138 shutdown_connections 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d62fed0 0x55bb2d6302f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d6355f0 0x55bb2d6359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/1959414138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d635f10 0x55bb2d62f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1959414138 >> 172.21.15.27:0/1959414138 conn(0x55bb2d55ac70 msgr2=0x55bb2d62a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1959414138 shutdown_connections 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1959414138 wait complete. 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 -- start start 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.915+0000 7f0f629d4b80 1 Processor -- start 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 0x55bb2d63bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d638fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d639500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d63c6a0 con 0x55bb2d635f10 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d521ba0 con 0x55bb2d6355f0 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d63c820 con 0x55bb2d62fed0 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f629d4b80 10 client.0 _ll_register_callbacks cb 0x55bb2d51b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f556ec700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d638fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f566ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d639500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f55eed700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 0x55bb2d63bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f556ec700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d638fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:34480/0 (socket says 172.21.15.27:34480) 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f556ec700 1 -- 172.21.15.27:0/552114493 learned_addr learned my addr 172.21.15.27:0/552114493 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f556ec700 1 -- 172.21.15.27:0/552114493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 msgr2=0x55bb2d63bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f556ec700 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 0x55bb2d63bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f556ec700 1 -- 172.21.15.27:0/552114493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 msgr2=0x55bb2d639500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f556ec700 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d639500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f556ec700 1 -- 172.21.15.27:0/552114493 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb2d4d0880 con 0x55bb2d6355f0 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f55eed700 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 0x55bb2d63bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.916+0000 7f0f566ee700 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d639500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.917+0000 7f0f556ec700 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d638fc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0f400116f0 tx=0x7f0f40011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944fbb7f2c009362 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.917+0000 7f0f397fa700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.917+0000 7f0f397fa700 1 -- 172.21.15.27:0/552114493 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40004320 con 0x55bb2d6355f0 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.917+0000 7f0f397fa700 1 -- 172.21.15.27:0/552114493 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f0f400044c0 con 0x55bb2d6355f0 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.917+0000 7f0f629d4b80 1 -- 172.21.15.27:0/552114493 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f480d0120 con 0x55bb2d6355f0 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.917+0000 7f0f397fa700 1 -- 172.21.15.27:0/552114493 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0f40024050 con 0x55bb2d6355f0 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.917+0000 7f0f629d4b80 10 client.24845 fetch_fsmap learned FSMap version 24 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.917+0000 7f0f629d4b80 1 -- 172.21.15.27:0/552114493 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bb2d646fd0 con 0x55bb2d6355f0 2024-06-06T08:40:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.917+0000 7f0f397fa700 1 -- 172.21.15.27:0/552114493 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f480d0120 con 0x55bb2d6355f0 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f397fa700 1 -- 172.21.15.27:0/552114493 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0f4001b900 con 0x55bb2d6355f0 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f629d4b80 10 client.24845 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f629d4b80 10 client.24845 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f629d4b80 20 client.24845 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55bb2d649620 0x55bb2d64aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f629d4b80 4 client.24845 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f629d4b80 1 -- 172.21.15.27:0/552114493 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb2d58ecc0 con 0x55bb2d649620 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f55eed700 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55bb2d649620 0x55bb2d64aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f55eed700 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55bb2d649620 0x55bb2d64aa90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9fd9691f75a5f11d server_cookie=76b56bc3a69834ab in_seq=0 out_seq=0 2024-06-06T08:40:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.918+0000 7f0f397fa700 10 client.24845 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:40:02.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f397fa700 1 -- 172.21.15.27:0/552114493 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bb2d58ecc0 con 0x55bb2d649620 2024-06-06T08:40:02.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f397fa700 10 client.24845 handle_command_reply: tid=0 2024-06-06T08:40:02.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 client.24845 shutdown 2024-06-06T08:40:02.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 -- 172.21.15.27:0/552114493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 msgr2=0x55bb2d638fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:02.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d638fc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0f400116f0 tx=0x7f0f40011a00 comp rx=0 tx=0).stop 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 -- 172.21.15.27:0/552114493 shutdown_connections 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55bb2d649620 0x55bb2d64aa90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d639500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 0x55bb2d63bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f397fa700 0 client.24845 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/552114493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d638fc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 -- 172.21.15.27:0/552114493 >> 172.21.15.27:0/552114493 conn(0x55bb2d55ac70 msgr2=0x55bb2d62e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 -- 172.21.15.27:0/552114493 shutdown_connections 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.919+0000 7f0f629d4b80 1 -- 172.21.15.27:0/552114493 wait complete. 2024-06-06T08:40:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.920+0000 7f0f629d4b80 20 client.24845 trim_cache size 0 max 16384 2024-06-06T08:40:02.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.965+0000 7f0f629d4b80 1 Processor -- start 2024-06-06T08:40:02.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.965+0000 7f0f629d4b80 1 -- start start 2024-06-06T08:40:02.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.965+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 0x55bb2d62f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.965+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d639530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.965+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d653ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.965+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d648dd0 con 0x55bb2d635f10 2024-06-06T08:40:02.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d6494a0 con 0x55bb2d6355f0 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d649b70 con 0x55bb2d62fed0 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f566ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d653ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f566ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d653ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55322/0 (socket says 172.21.15.27:55322) 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f566ee700 1 -- 172.21.15.27:0/1799978890 learned_addr learned my addr 172.21.15.27:0/1799978890 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f556ec700 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d639530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f55eed700 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 0x55bb2d62f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f566ee700 1 -- 172.21.15.27:0/1799978890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 msgr2=0x55bb2d62f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f566ee700 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 0x55bb2d62f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f566ee700 1 -- 172.21.15.27:0/1799978890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 msgr2=0x55bb2d639530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f566ee700 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d639530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f566ee700 1 -- 172.21.15.27:0/1799978890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb2d637530 con 0x55bb2d635f10 2024-06-06T08:40:02.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f556ec700 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d639530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:02.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f566ee700 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d653ef0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009850 tx=0x7f0f480099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cdbb416a4c136e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:02.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f3b7fe700 1 -- 172.21.15.27:0/1799978890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480ba070 con 0x55bb2d635f10 2024-06-06T08:40:02.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.966+0000 7f0f3b7fe700 1 -- 172.21.15.27:0/1799978890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f480d0120 con 0x55bb2d635f10 2024-06-06T08:40:02.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f3b7fe700 1 -- 172.21.15.27:0/1799978890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4802b070 con 0x55bb2d635f10 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1799978890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 msgr2=0x55bb2d653ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d653ef0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009850 tx=0x7f0f480099e0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1799978890 shutdown_connections 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d635f10 0x55bb2d653ef0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d62fed0 0x55bb2d62f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/1799978890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d6355f0 0x55bb2d639530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1799978890 >> 172.21.15.27:0/1799978890 conn(0x55bb2d55ac70 msgr2=0x55bb2d633070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1799978890 shutdown_connections 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.967+0000 7f0f629d4b80 1 -- 172.21.15.27:0/1799978890 wait complete. 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f629d4b80 1 -- start start 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f629d4b80 1 Processor -- start 2024-06-06T08:40:02.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d62fed0 0x55bb2d638920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d6355f0 0x55bb2d638e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f629d4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d635f10 0x55bb2d6393a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d64a910 con 0x55bb2d62fed0 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d64a240 con 0x55bb2d635f10 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f629d4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bb2d64afe0 con 0x55bb2d6355f0 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f629d4b80 10 client.0 _ll_register_callbacks cb 0x55bb2d51b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f55eed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d62fed0 0x55bb2d638920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f55eed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d62fed0 0x55bb2d638920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55332/0 (socket says 172.21.15.27:55332) 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f556ec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d6355f0 0x55bb2d638e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f55eed700 1 -- 172.21.15.27:0/4071955773 learned_addr learned my addr 172.21.15.27:0/4071955773 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.968+0000 7f0f566ee700 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d635f10 0x55bb2d6393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f55eed700 1 -- 172.21.15.27:0/4071955773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d6355f0 msgr2=0x55bb2d638e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:02.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f55eed700 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d6355f0 0x55bb2d638e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f55eed700 1 -- 172.21.15.27:0/4071955773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d635f10 msgr2=0x55bb2d6393a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f55eed700 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d635f10 0x55bb2d6393a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f55eed700 1 -- 172.21.15.27:0/4071955773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb2d652380 con 0x55bb2d62fed0 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f556ec700 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d6355f0 0x55bb2d638e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f55eed700 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d62fed0 0x55bb2d638920 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005950 tx=0x7f0f4c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d2a6ee38541dcf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f397fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f397fa700 1 -- 172.21.15.27:0/4071955773 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c01f980 con 0x55bb2d62fed0 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f397fa700 1 -- 172.21.15.27:0/4071955773 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f0f4c009040 con 0x55bb2d62fed0 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f629d4b80 1 -- 172.21.15.27:0/4071955773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f480d0120 con 0x55bb2d62fed0 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f397fa700 1 -- 172.21.15.27:0/4071955773 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0f4c03b050 con 0x55bb2d62fed0 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f629d4b80 10 client.15141 fetch_fsmap learned FSMap version 24 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.969+0000 7f0f629d4b80 1 -- 172.21.15.27:0/4071955773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bb2d6b5110 con 0x55bb2d62fed0 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.970+0000 7f0f397fa700 1 -- 172.21.15.27:0/4071955773 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f480d0120 con 0x55bb2d62fed0 2024-06-06T08:40:02.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.970+0000 7f0f397fa700 1 -- 172.21.15.27:0/4071955773 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0f4c02b3f0 con 0x55bb2d62fed0 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.970+0000 7f0f629d4b80 10 client.15141 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.970+0000 7f0f629d4b80 10 client.15141 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.970+0000 7f0f629d4b80 20 client.15141 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.970+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55bb2d6b9d10 0x55bb2d6b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.970+0000 7f0f629d4b80 4 client.15141 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.970+0000 7f0f629d4b80 1 -- 172.21.15.27:0/4071955773 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "status"}) -- 0x55bb2d589240 con 0x55bb2d6b9d10 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.970+0000 7f0f556ec700 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55bb2d6b9d10 0x55bb2d6b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f556ec700 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55bb2d6b9d10 0x55bb2d6b8a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ece9e0bda99fb46d server_cookie=d8c9a1be5810ab6d in_seq=0 out_seq=0 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f397fa700 10 client.15141 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f397fa700 1 -- 172.21.15.27:0/4071955773 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bb2d589240 con 0x55bb2d6b9d10 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f397fa700 10 client.15141 handle_command_reply: tid=0 2024-06-06T08:40:02.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 client.15141 shutdown 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 -- 172.21.15.27:0/4071955773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d62fed0 msgr2=0x55bb2d638920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d62fed0 0x55bb2d638920 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005950 tx=0x7f0f4c0047c0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 -- 172.21.15.27:0/4071955773 shutdown_connections 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55bb2d6b9d10 0x55bb2d6b8a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2d635f10 0x55bb2d6393a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bb2d62fed0 0x55bb2d638920 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 --2- 172.21.15.27:0/4071955773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bb2d6355f0 0x55bb2d638e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f397fa700 0 client.15141 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 -- 172.21.15.27:0/4071955773 >> 172.21.15.27:0/4071955773 conn(0x55bb2d55ac70 msgr2=0x55bb2d623db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 -- 172.21.15.27:0/4071955773 shutdown_connections 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 1 -- 172.21.15.27:0/4071955773 wait complete. 2024-06-06T08:40:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.971+0000 7f0f629d4b80 20 client.15141 trim_cache size 0 max 16384 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24458, 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 161.06619970899999, 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 99, 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 250.61535029999999 2024-06-06T08:40:02.976 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:40:02.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 -- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f3c0a2c20 msgr2=0x7f0f3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:02.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f3c0a2c20 0x7f0f3c0a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009be0 tx=0x7f0f4c009260 comp rx=0 tx=0).stop 2024-06-06T08:40:02.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 -- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50107400 msgr2=0x7f0f501a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:02.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50107400 0x7f0f501a4ad0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800b780 tx=0x7f0f48004d00 comp rx=0 tx=0).stop 2024-06-06T08:40:02.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 -- 172.21.15.27:0/2551390767 shutdown_connections 2024-06-06T08:40:02.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f0f3c0a2c20 0x7f0f3c0a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f50107400 0x7f0f501a4ad0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f50101390 0x7f0f50199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 --2- 172.21.15.27:0/2551390767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f50101d50 0x7f0f50199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:02.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 -- 172.21.15.27:0/2551390767 >> 172.21.15.27:0/2551390767 conn(0x7f0f50078340 msgr2=0x7f0f500ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:02.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 -- 172.21.15.27:0/2551390767 shutdown_connections 2024-06-06T08:40:02.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:02.972+0000 7f0f58151700 1 -- 172.21.15.27:0/2551390767 wait complete. 2024-06-06T08:40:02.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:02 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:03.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:02 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:03.660 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-06-06T08:40:03.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:03.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:03.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:03 smithi027 ceph-mon[32335]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T08:40:03.940 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-06-06T08:40:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:03.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:03.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:03.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:03 smithi203 ceph-mon[36579]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:04.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-06-06T08:40:04.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:04.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:04.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:04.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:04.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:04.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:04.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:04.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:04.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:03 smithi176 ceph-mon[36786]: from='mgr.14152 172.21.15.27:0/2495026823' entity='mgr.x' 2024-06-06T08:40:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.690+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3120208911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc107cc0 msgr2=0x7f23dc106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.690+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3120208911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc107cc0 0x7f23dc106010 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009990 tx=0x7f23c8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.690+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3120208911 shutdown_connections 2024-06-06T08:40:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.690+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3120208911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23dc1066f0 0x7f23dc10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.690+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3120208911 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23dc107310 0x7f23dc1076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.690+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3120208911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc107cc0 0x7f23dc106010 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.690+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3120208911 >> 172.21.15.27:0/3120208911 conn(0x7f23dc076600 msgr2=0x7f23dc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:05.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.690+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3120208911 shutdown_connections 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.690+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3120208911 wait complete. 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.691+0000 7f23e2ca0700 1 Processor -- start 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.691+0000 7f23e2ca0700 1 -- start start 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.691+0000 7f23e2ca0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc1066f0 0x7f23dc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.691+0000 7f23e2ca0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23dc107310 0x7f23dc19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.691+0000 7f23e2ca0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23dc107cc0 0x7f23dc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.691+0000 7f23e2ca0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc111710 con 0x7f23dc1066f0 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.691+0000 7f23e2ca0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc111590 con 0x7f23dc107cc0 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.691+0000 7f23e2ca0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc111890 con 0x7f23dc107310 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e0a3c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc1066f0 0x7f23dc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e0a3c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc1066f0 0x7f23dc19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55358/0 (socket says 172.21.15.27:55358) 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e0a3c700 1 -- 172.21.15.27:0/3054330779 learned_addr learned my addr 172.21.15.27:0/3054330779 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:05.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e123d700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23dc107cc0 0x7f23dc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23dbfff700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23dc107310 0x7f23dc19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e0a3c700 1 -- 172.21.15.27:0/3054330779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23dc107310 msgr2=0x7f23dc19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23dc107310 0x7f23dc19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e0a3c700 1 -- 172.21.15.27:0/3054330779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23dc107cc0 msgr2=0x7f23dc198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23dc107cc0 0x7f23dc198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e0a3c700 1 -- 172.21.15.27:0/3054330779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc199260 con 0x7f23dc1066f0 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc1066f0 0x7f23dc19e940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c920 tx=0x7f23d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23d9ffb700 1 -- 172.21.15.27:0/3054330779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0007b00 con 0x7f23dc1066f0 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23d9ffb700 1 -- 172.21.15.27:0/3054330779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23d0004d10 con 0x7f23dc1066f0 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23d9ffb700 1 -- 172.21.15.27:0/3054330779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d000f450 con 0x7f23dc1066f0 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3054330779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc199550 con 0x7f23dc1066f0 2024-06-06T08:40:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.692+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3054330779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc1a5810 con 0x7f23dc1066f0 2024-06-06T08:40:05.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.695+0000 7f23d9ffb700 1 -- 172.21.15.27:0/3054330779 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f23d0004750 con 0x7f23dc1066f0 2024-06-06T08:40:05.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.695+0000 7f23d9ffb700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f23cc0a2ca0 0x7f23cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.695+0000 7f23d9ffb700 1 -- 172.21.15.27:0/3054330779 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f23d00cf3d0 con 0x7f23dc1066f0 2024-06-06T08:40:05.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.695+0000 7f23dbfff700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f23cc0a2ca0 0x7f23cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.696+0000 7f23dbfff700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f23cc0a2ca0 0x7f23cc0a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006010 tx=0x7f23c800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:05.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.696+0000 7f23ed523b80 1 Processor -- start 2024-06-06T08:40:05.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.696+0000 7f23ed523b80 1 -- start start 2024-06-06T08:40:05.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 0x55c5fb1179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb111990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 0x55c5fb1122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23ed523b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb0637c0 con 0x55c5fb117f10 2024-06-06T08:40:05.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23ed523b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb00b6d0 con 0x55c5fb111ed0 2024-06-06T08:40:05.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23ed523b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb05f780 con 0x55c5fb1175f0 2024-06-06T08:40:05.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23dbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb111990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23dbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb111990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55374/0 (socket says 172.21.15.27:55374) 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23dbfff700 1 -- 172.21.15.27:0/2506565941 learned_addr learned my addr 172.21.15.27:0/2506565941 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 0x55c5fb1179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23e123d700 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 0x55c5fb1122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23dbfff700 1 -- 172.21.15.27:0/2506565941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 msgr2=0x55c5fb1179d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23dbfff700 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 0x55c5fb1179d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23dbfff700 1 -- 172.21.15.27:0/2506565941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 msgr2=0x55c5fb1122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23dbfff700 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 0x55c5fb1122f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23dbfff700 1 -- 172.21.15.27:0/2506565941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5fb001f00 con 0x55c5fb117f10 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23e123d700 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 0x55c5fb1122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23dbfff700 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb111990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f23c8015000 tx=0x7f23c801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0aca866ac3cb02 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:05.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23c6ffd700 1 -- 172.21.15.27:0/2506565941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c800bdc0 con 0x55c5fb117f10 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23c6ffd700 1 -- 172.21.15.27:0/2506565941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23c800f940 con 0x55c5fb117f10 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.697+0000 7f23c6ffd700 1 -- 172.21.15.27:0/2506565941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c80249d0 con 0x55c5fb117f10 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.698+0000 7f23ed523b80 1 -- 172.21.15.27:0/2506565941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 msgr2=0x55c5fb111990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.698+0000 7f23ed523b80 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb111990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f23c8015000 tx=0x7f23c801f000 comp rx=0 tx=0).stop 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.698+0000 7f23ed523b80 1 -- 172.21.15.27:0/2506565941 shutdown_connections 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.698+0000 7f23ed523b80 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 0x55c5fb1122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.698+0000 7f23ed523b80 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 0x55c5fb1179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.698+0000 7f23ed523b80 1 --2- 172.21.15.27:0/2506565941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb111990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.698+0000 7f23ed523b80 1 -- 172.21.15.27:0/2506565941 >> 172.21.15.27:0/2506565941 conn(0x55c5fb03cc70 msgr2=0x55c5fb10c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.698+0000 7f23ed523b80 1 -- 172.21.15.27:0/2506565941 shutdown_connections 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.698+0000 7f23ed523b80 1 -- 172.21.15.27:0/2506565941 wait complete. 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23ed523b80 1 -- start start 2024-06-06T08:40:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23ed523b80 1 Processor -- start 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb111ed0 0x55c5fb11dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 0x55c5fb11afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb117f10 0x55c5fb11b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23ed523b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb11e8c0 con 0x55c5fb111ed0 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23ed523b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb003ba0 con 0x55c5fb117f10 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23ed523b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb11ea40 con 0x55c5fb1175f0 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23ed523b80 10 client.0 _ll_register_callbacks cb 0x55c5faffd8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23e0a3c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb111ed0 0x55c5fb11dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23e0a3c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb111ed0 0x55c5fb11dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55390/0 (socket says 172.21.15.27:55390) 2024-06-06T08:40:05.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23e0a3c700 1 -- 172.21.15.27:0/4120526590 learned_addr learned my addr 172.21.15.27:0/4120526590 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23e123d700 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb117f10 0x55c5fb11b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.699+0000 7f23dbfff700 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 0x55c5fb11afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23e0a3c700 1 -- 172.21.15.27:0/4120526590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 msgr2=0x55c5fb11afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 0x55c5fb11afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23e0a3c700 1 -- 172.21.15.27:0/4120526590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb117f10 msgr2=0x55c5fb11b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb117f10 0x55c5fb11b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23e0a3c700 1 -- 172.21.15.27:0/4120526590 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5fafb2880 con 0x55c5fb111ed0 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb111ed0 0x55c5fb11dcc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23d00074a0 tx=0x7f23d009b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2240b45702fd4c95 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23bbfff700 1 -- 172.21.15.27:0/4120526590 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d00b4060 con 0x55c5fb111ed0 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23bbfff700 1 -- 172.21.15.27:0/4120526590 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f23d0099b60 con 0x55c5fb111ed0 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23ed523b80 1 -- 172.21.15.27:0/4120526590 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23c800f940 con 0x55c5fb111ed0 2024-06-06T08:40:05.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23bbfff700 1 -- 172.21.15.27:0/4120526590 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f23d001d040 con 0x55c5fb111ed0 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23ed523b80 10 client.15153 fetch_fsmap learned FSMap version 24 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23ed523b80 1 -- 172.21.15.27:0/4120526590 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c5fb12d5e0 con 0x55c5fb111ed0 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23bbfff700 1 -- 172.21.15.27:0/4120526590 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23c800f940 con 0x55c5fb111ed0 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.700+0000 7f23bbfff700 1 -- 172.21.15.27:0/4120526590 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f23d00cf0b0 con 0x55c5fb111ed0 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.701+0000 7f23ed523b80 10 client.15153 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.701+0000 7f23ed523b80 10 client.15153 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.701+0000 7f23ed523b80 20 client.15153 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.701+0000 7f23ed523b80 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c5fb12faf0 0x55c5fb130f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.701+0000 7f23ed523b80 4 client.15153 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.701+0000 7f23ed523b80 1 -- 172.21.15.27:0/4120526590 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5fb070cc0 con 0x55c5fb12faf0 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.701+0000 7f23dbfff700 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c5fb12faf0 0x55c5fb130f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.701+0000 7f23dbfff700 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c5fb12faf0 0x55c5fb130f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a09a56e4e716bd7a server_cookie=1df3f5ee7f5e82d1 in_seq=0 out_seq=0 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.701+0000 7f23bbfff700 10 client.15153 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:40:05.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.702+0000 7f23bbfff700 1 -- 172.21.15.27:0/4120526590 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c5fb070cc0 con 0x55c5fb12faf0 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.702+0000 7f23bbfff700 10 client.15153 handle_command_reply: tid=0 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.702+0000 7f23ed523b80 1 client.15153 shutdown 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.702+0000 7f23ed523b80 1 -- 172.21.15.27:0/4120526590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb111ed0 msgr2=0x55c5fb11dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.702+0000 7f23ed523b80 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb111ed0 0x55c5fb11dcc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23d00074a0 tx=0x7f23d009b3f0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.702+0000 7f23ed523b80 1 -- 172.21.15.27:0/4120526590 shutdown_connections 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.702+0000 7f23ed523b80 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c5fb12faf0 0x55c5fb130f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.702+0000 7f23bbfff700 0 client.15153 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.703+0000 7f23ed523b80 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb117f10 0x55c5fb11b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.703+0000 7f23ed523b80 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb111ed0 0x55c5fb11dcc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.703+0000 7f23ed523b80 1 --2- 172.21.15.27:0/4120526590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb1175f0 0x55c5fb11afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.703+0000 7f23ed523b80 1 -- 172.21.15.27:0/4120526590 >> 172.21.15.27:0/4120526590 conn(0x55c5fb03cc70 msgr2=0x55c5fb110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.703+0000 7f23ed523b80 1 -- 172.21.15.27:0/4120526590 shutdown_connections 2024-06-06T08:40:05.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.703+0000 7f23ed523b80 1 -- 172.21.15.27:0/4120526590 wait complete. 2024-06-06T08:40:05.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.703+0000 7f23ed523b80 20 client.15153 trim_cache size 0 max 16384 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.724+0000 7f23ed523b80 1 Processor -- start 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.724+0000 7f23ed523b80 1 -- start start 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb111ed0 0x55c5fb111990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb1175f0 0x55c5fb106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb13a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23ed523b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb12f2a0 con 0x55c5fb117f10 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23ed523b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb12f970 con 0x55c5fb1175f0 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23ed523b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb130040 con 0x55c5fb111ed0 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e123d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb13a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e123d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb13a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55392/0 (socket says 172.21.15.27:55392) 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e123d700 1 -- 172.21.15.27:0/3447374832 learned_addr learned my addr 172.21.15.27:0/3447374832 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb111ed0 0x55c5fb111990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23dbfff700 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb1175f0 0x55c5fb106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e123d700 1 -- 172.21.15.27:0/3447374832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb111ed0 msgr2=0x55c5fb111990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e123d700 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb111ed0 0x55c5fb111990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e123d700 1 -- 172.21.15.27:0/3447374832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb1175f0 msgr2=0x55c5fb106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e123d700 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb1175f0 0x55c5fb106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e123d700 1 -- 172.21.15.27:0/3447374832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5fb12f0c0 con 0x55c5fb117f10 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb111ed0 0x55c5fb111990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23e123d700 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb13a3c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23d40094c0 tx=0x7f23d40094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16d7102ea7a8ad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23c67fc700 1 -- 172.21.15.27:0/3447374832 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d401a020 con 0x55c5fb117f10 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23c67fc700 1 -- 172.21.15.27:0/3447374832 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23d4016920 con 0x55c5fb117f10 2024-06-06T08:40:05.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.725+0000 7f23c67fc700 1 -- 172.21.15.27:0/3447374832 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4015a60 con 0x55c5fb117f10 2024-06-06T08:40:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 -- 172.21.15.27:0/3447374832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 msgr2=0x55c5fb13a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb13a3c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23d40094c0 tx=0x7f23d40094f0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 -- 172.21.15.27:0/3447374832 shutdown_connections 2024-06-06T08:40:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb117f10 0x55c5fb13a3c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb111ed0 0x55c5fb111990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 --2- 172.21.15.27:0/3447374832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb1175f0 0x55c5fb106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 -- 172.21.15.27:0/3447374832 >> 172.21.15.27:0/3447374832 conn(0x55c5fb03cc70 msgr2=0x55c5fb1198b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 -- 172.21.15.27:0/3447374832 shutdown_connections 2024-06-06T08:40:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 -- 172.21.15.27:0/3447374832 wait complete. 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.726+0000 7f23ed523b80 1 -- start start 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23ed523b80 1 Processor -- start 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 0x55c5fb13a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb1175f0 0x55c5fb13a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23ed523b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb117f10 0x55c5fb135390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23ed523b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb130de0 con 0x55c5fb1175f0 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23ed523b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb130710 con 0x55c5fb111ed0 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23ed523b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c5fb1314b0 con 0x55c5fb117f10 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23ed523b80 10 client.0 _ll_register_callbacks cb 0x55c5faffd8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23dbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb1175f0 0x55c5fb13a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23dbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb1175f0 0x55c5fb13a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55404/0 (socket says 172.21.15.27:55404) 2024-06-06T08:40:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23dbfff700 1 -- 172.21.15.27:0/1370067201 learned_addr learned my addr 172.21.15.27:0/1370067201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23e123d700 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb117f10 0x55c5fb135390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 0x55c5fb13a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23dbfff700 1 -- 172.21.15.27:0/1370067201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb117f10 msgr2=0x55c5fb135390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23dbfff700 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb117f10 0x55c5fb135390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23dbfff700 1 -- 172.21.15.27:0/1370067201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 msgr2=0x55c5fb13a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23dbfff700 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 0x55c5fb13a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.727+0000 7f23dbfff700 1 -- 172.21.15.27:0/1370067201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5fb1359d0 con 0x55c5fb1175f0 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23dbfff700 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb1175f0 0x55c5fb13a920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f23c8024520 tx=0x7f23c8024410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81333f6092410377 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23bbfff700 1 -- 172.21.15.27:0/1370067201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c800fda0 con 0x55c5fb1175f0 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23bbfff700 1 -- 172.21.15.27:0/1370067201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f23c800f460 con 0x55c5fb1175f0 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23ed523b80 1 -- 172.21.15.27:0/1370067201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d4016920 con 0x55c5fb1175f0 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23bbfff700 1 -- 172.21.15.27:0/1370067201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f23c804c050 con 0x55c5fb1175f0 2024-06-06T08:40:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23ed523b80 10 client.15159 fetch_fsmap learned FSMap version 24 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23ed523b80 1 -- 172.21.15.27:0/1370067201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c5fb19bca0 con 0x55c5fb1175f0 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23bbfff700 1 -- 172.21.15.27:0/1370067201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23d4016920 con 0x55c5fb1175f0 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23bbfff700 1 -- 172.21.15.27:0/1370067201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f23c8043480 con 0x55c5fb1175f0 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.728+0000 7f23ed523b80 10 client.15159 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23ed523b80 10 client.15159 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23ed523b80 20 client.15159 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23ed523b80 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c5fb12faf0 0x55c5fb19f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23ed523b80 4 client.15159 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23ed523b80 1 -- 172.21.15.27:0/1370067201 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "status"}) -- 0x55c5fb06b240 con 0x55c5fb12faf0 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c5fb12faf0 0x55c5fb19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23e0a3c700 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c5fb12faf0 0x55c5fb19f040 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdafd1c1cd2bb7d7 server_cookie=1182ff46187eefd9 in_seq=0 out_seq=0 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23bbfff700 10 client.15159 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23bbfff700 1 -- 172.21.15.27:0/1370067201 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55c5fb06b240 con 0x55c5fb12faf0 2024-06-06T08:40:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23bbfff700 10 client.15159 handle_command_reply: tid=0 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.729+0000 7f23ed523b80 1 client.15159 shutdown 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 -- 172.21.15.27:0/1370067201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb1175f0 msgr2=0x55c5fb13a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb1175f0 0x55c5fb13a920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f23c8024520 tx=0x7f23c8024410 comp rx=0 tx=0).stop 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 -- 172.21.15.27:0/1370067201 shutdown_connections 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5fb117f10 0x55c5fb135390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55c5fb12faf0 0x55c5fb19f040 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23bbfff700 0 client.15159 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c5fb111ed0 0x55c5fb13a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 --2- 172.21.15.27:0/1370067201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c5fb1175f0 0x55c5fb13a920 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 -- 172.21.15.27:0/1370067201 >> 172.21.15.27:0/1370067201 conn(0x55c5fb03cc70 msgr2=0x55c5fb1182f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 -- 172.21.15.27:0/1370067201 shutdown_connections 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 1 -- 172.21.15.27:0/1370067201 wait complete. 2024-06-06T08:40:05.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.730+0000 7f23ed523b80 20 client.15159 trim_cache size 0 max 16384 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24460, 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 164.828232224, 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-06-06T08:40:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 99, 2024-06-06T08:40:05.735 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T08:40:05.735 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 250.565435208 2024-06-06T08:40:05.735 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:40:05.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f23cc0a2ca0 msgr2=0x7f23cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:05.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f23cc0a2ca0 0x7f23cc0a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006010 tx=0x7f23c800b540 comp rx=0 tx=0).stop 2024-06-06T08:40:05.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc1066f0 msgr2=0x7f23dc19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc1066f0 0x7f23dc19e940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c920 tx=0x7f23d000cce0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3054330779 shutdown_connections 2024-06-06T08:40:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f23cc0a2ca0 0x7f23cc0a5160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f23dc107cc0 0x7f23dc198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23dc1066f0 0x7f23dc19e940 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 --2- 172.21.15.27:0/3054330779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23dc107310 0x7f23dc19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3054330779 >> 172.21.15.27:0/3054330779 conn(0x7f23dc076600 msgr2=0x7f23dc100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3054330779 shutdown_connections 2024-06-06T08:40:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:05.731+0000 7f23e2ca0700 1 -- 172.21.15.27:0/3054330779 wait complete. 2024-06-06T08:40:05.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:05 smithi027 ceph-mon[32335]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T08:40:06.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:05 smithi203 ceph-mon[36579]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T08:40:06.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:05 smithi176 ceph-mon[36786]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-06T08:40:06.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T08:40:06.696 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:08.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:07 smithi027 ceph-mon[32335]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:40:08.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:07 smithi203 ceph-mon[36579]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:40:08.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:07 smithi176 ceph-mon[36786]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:40:08.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.433+0000 7fe44d071700 1 -- 172.21.15.27:0/3909140035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 msgr2=0x7fe44806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:08.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.433+0000 7fe44d071700 1 --2- 172.21.15.27:0/3909140035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 0x7fe44806b040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe430009990 tx=0x7fe430009ca0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.433+0000 7fe44d071700 1 -- 172.21.15.27:0/3909140035 shutdown_connections 2024-06-06T08:40:08.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.433+0000 7fe44d071700 1 --2- 172.21.15.27:0/3909140035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44806b580 0x7fe44810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.433+0000 7fe44d071700 1 --2- 172.21.15.27:0/3909140035 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe448108500 0x7fe4481088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.433+0000 7fe44d071700 1 --2- 172.21.15.27:0/3909140035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 0x7fe44806b040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.433+0000 7fe44d071700 1 -- 172.21.15.27:0/3909140035 >> 172.21.15.27:0/3909140035 conn(0x7fe448078340 msgr2=0x7fe448078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:08.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.433+0000 7fe44d071700 1 -- 172.21.15.27:0/3909140035 shutdown_connections 2024-06-06T08:40:08.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.433+0000 7fe44d071700 1 -- 172.21.15.27:0/3909140035 wait complete. 2024-06-06T08:40:08.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.434+0000 7fe44d071700 1 Processor -- start 2024-06-06T08:40:08.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.434+0000 7fe44d071700 1 -- start start 2024-06-06T08:40:08.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.434+0000 7fe44d071700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 0x7fe448101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.434+0000 7fe44d071700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe44806b580 0x7fe4481019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.434+0000 7fe44d071700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe448108500 0x7fe448101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.434+0000 7fe44d071700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe448112780 con 0x7fe44806abc0 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe44d071700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe448112600 con 0x7fe44806b580 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe44d071700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe448112900 con 0x7fe448108500 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe446d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 0x7fe448101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe446d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 0x7fe448101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55432/0 (socket says 172.21.15.27:55432) 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe446d9d700 1 -- 172.21.15.27:0/2304687952 learned_addr learned my addr 172.21.15.27:0/2304687952 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe44759e700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe448108500 0x7fe448101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe44806b580 0x7fe4481019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe446d9d700 1 -- 172.21.15.27:0/2304687952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe448108500 msgr2=0x7fe448101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe448108500 0x7fe448101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe446d9d700 1 -- 172.21.15.27:0/2304687952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe44806b580 msgr2=0x7fe4481019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe44806b580 0x7fe4481019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe446d9d700 1 -- 172.21.15.27:0/2304687952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4481a7150 con 0x7fe44806abc0 2024-06-06T08:40:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe44759e700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe448108500 0x7fe448101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 0x7fe448101490 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe43800d950 tx=0x7fe43800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe444d99700 1 -- 172.21.15.27:0/2304687952 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438016070 con 0x7fe44806abc0 2024-06-06T08:40:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe44d071700 1 -- 172.21.15.27:0/2304687952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4481a7440 con 0x7fe44806abc0 2024-06-06T08:40:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe444d99700 1 -- 172.21.15.27:0/2304687952 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43800fd70 con 0x7fe44806abc0 2024-06-06T08:40:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.435+0000 7fe444d99700 1 -- 172.21.15.27:0/2304687952 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43800e610 con 0x7fe44806abc0 2024-06-06T08:40:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.436+0000 7fe44d071700 1 -- 172.21.15.27:0/2304687952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4481a79a0 con 0x7fe44806abc0 2024-06-06T08:40:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.438+0000 7fe444d99700 1 -- 172.21.15.27:0/2304687952 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe438019070 con 0x7fe44806abc0 2024-06-06T08:40:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.439+0000 7fe444d99700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4340a2ca0 0x7fe4340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.439+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4340a2ca0 0x7fe4340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.439+0000 7fe444d99700 1 -- 172.21.15.27:0/2304687952 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fe4380cedf0 con 0x7fe44806abc0 2024-06-06T08:40:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4340a2ca0 0x7fe4340a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe43000b2e0 tx=0x7fe430005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe4578f4b80 1 Processor -- start 2024-06-06T08:40:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe4578f4b80 1 -- start start 2024-06-06T08:40:08.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc895f0 0x563d2dc899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 0x563d2dc83990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 0x563d2dc842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563d2dbd57c0 con 0x563d2dc895f0 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563d2db7d6d0 con 0x563d2dc89f10 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563d2dbd1780 con 0x563d2dc83ed0 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe446d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc895f0 0x563d2dc899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe446d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc895f0 0x563d2dc899d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55442/0 (socket says 172.21.15.27:55442) 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.440+0000 7fe446d9d700 1 -- 172.21.15.27:0/2141483122 learned_addr learned my addr 172.21.15.27:0/2141483122 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 0x563d2dc83990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe44759e700 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 0x563d2dc842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe446d9d700 1 -- 172.21.15.27:0/2141483122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 msgr2=0x563d2dc842f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 0x563d2dc842f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe446d9d700 1 -- 172.21.15.27:0/2141483122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 msgr2=0x563d2dc83990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 0x563d2dc83990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe446d9d700 1 -- 172.21.15.27:0/2141483122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2db73f00 con 0x563d2dc895f0 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 0x563d2dc83990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc895f0 0x563d2dc899d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe4380a3980 tx=0x7fe438005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b0f80c2217934e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe43d59a700 1 -- 172.21.15.27:0/2141483122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438020040 con 0x563d2dc895f0 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe43d59a700 1 -- 172.21.15.27:0/2141483122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43809f5f0 con 0x563d2dc895f0 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.441+0000 7fe43d59a700 1 -- 172.21.15.27:0/2141483122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380a3c20 con 0x563d2dc895f0 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2141483122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc895f0 msgr2=0x563d2dc899d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc895f0 0x563d2dc899d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe4380a3980 tx=0x7fe438005910 comp rx=0 tx=0).stop 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2141483122 shutdown_connections 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 0x563d2dc842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc895f0 0x563d2dc899d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2141483122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 0x563d2dc83990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2141483122 >> 172.21.15.27:0/2141483122 conn(0x563d2dbaec70 msgr2=0x563d2dc7e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:08.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2141483122 shutdown_connections 2024-06-06T08:40:08.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2141483122 wait complete. 2024-06-06T08:40:08.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 -- start start 2024-06-06T08:40:08.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.442+0000 7fe4578f4b80 1 Processor -- start 2024-06-06T08:40:08.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dc8fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 0x563d2dc8cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc89f10 0x563d2dc8d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563d2dc90920 con 0x563d2dc83ed0 2024-06-06T08:40:08.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563d2db75ba0 con 0x563d2dc895f0 2024-06-06T08:40:08.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563d2dc90aa0 con 0x563d2dc89f10 2024-06-06T08:40:08.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe4578f4b80 10 client.0 _ll_register_callbacks cb 0x563d2db6f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe446d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dc8fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe446d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dc8fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55456/0 (socket says 172.21.15.27:55456) 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe446d9d700 1 -- 172.21.15.27:0/2329586816 learned_addr learned my addr 172.21.15.27:0/2329586816 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 0x563d2dc8cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe44759e700 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc89f10 0x563d2dc8d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe446d9d700 1 -- 172.21.15.27:0/2329586816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc89f10 msgr2=0x563d2dc8d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc89f10 0x563d2dc8d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe446d9d700 1 -- 172.21.15.27:0/2329586816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 msgr2=0x563d2dc8cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 0x563d2dc8cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe446d9d700 1 -- 172.21.15.27:0/2329586816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2db24880 con 0x563d2dc83ed0 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe44759e700 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc89f10 0x563d2dc8d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:08.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 0x563d2dc8cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dc8fc50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe438012040 tx=0x7fe4380cc4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c81e37ef4f68113 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe42e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe42e7fc700 1 -- 172.21.15.27:0/2329586816 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380b9060 con 0x563d2dc83ed0 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe42e7fc700 1 -- 172.21.15.27:0/2329586816 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fe438099540 con 0x563d2dc83ed0 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.443+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2329586816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe43809f5f0 con 0x563d2dc83ed0 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe42e7fc700 1 -- 172.21.15.27:0/2329586816 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe438081050 con 0x563d2dc83ed0 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe4578f4b80 10 client.15171 fetch_fsmap learned FSMap version 24 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2329586816 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563d2dc90920 con 0x563d2dc83ed0 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe42e7fc700 1 -- 172.21.15.27:0/2329586816 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43809f5f0 con 0x563d2dc83ed0 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe42e7fc700 1 -- 172.21.15.27:0/2329586816 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe4380bb150 con 0x563d2dc83ed0 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe4578f4b80 10 client.15171 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe4578f4b80 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe4578f4b80 20 client.15171 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:08.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563d2dc9d3b0 0x563d2dc9e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe4578f4b80 4 client.15171 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2329586816 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d2dbe2cc0 con 0x563d2dc9d3b0 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.444+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563d2dc9d3b0 0x563d2dc9e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.445+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563d2dc9d3b0 0x563d2dc9e820 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47f20e5ca952cca2 server_cookie=d09be16d916d989f in_seq=0 out_seq=0 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.445+0000 7fe42e7fc700 10 client.15171 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.445+0000 7fe42e7fc700 1 -- 172.21.15.27:0/2329586816 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563d2dbe2cc0 con 0x563d2dc9d3b0 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.445+0000 7fe42e7fc700 10 client.15171 handle_command_reply: tid=0 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.445+0000 7fe4578f4b80 1 client.15171 shutdown 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2329586816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 msgr2=0x563d2dc8fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dc8fc50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe438012040 tx=0x7fe4380cc4a0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2329586816 shutdown_connections 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc89f10 0x563d2dc8d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563d2dc9d3b0 0x563d2dc9e820 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dc8fc50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe42e7fc700 0 client.15171 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:08.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2329586816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 0x563d2dc8cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2329586816 >> 172.21.15.27:0/2329586816 conn(0x563d2dbaec70 msgr2=0x563d2dc7e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:08.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2329586816 shutdown_connections 2024-06-06T08:40:08.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2329586816 wait complete. 2024-06-06T08:40:08.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.446+0000 7fe4578f4b80 20 client.15171 trim_cache size 0 max 16384 2024-06-06T08:40:08.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.467+0000 7fe4578f4b80 1 Processor -- start 2024-06-06T08:40:08.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.467+0000 7fe4578f4b80 1 -- start start 2024-06-06T08:40:08.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 0x563d2dc83990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 0x563d2dc8aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc89f10 0x563d2dca7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563d2dc9cb60 con 0x563d2dc89f10 2024-06-06T08:40:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563d2dc9d230 con 0x563d2dc895f0 2024-06-06T08:40:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563d2dc9d900 con 0x563d2dc83ed0 2024-06-06T08:40:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe44759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc89f10 0x563d2dca7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe44759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc89f10 0x563d2dca7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55458/0 (socket says 172.21.15.27:55458) 2024-06-06T08:40:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe44759e700 1 -- 172.21.15.27:0/2997338130 learned_addr learned my addr 172.21.15.27:0/2997338130 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:08.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe43e59c700 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 0x563d2dc8aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 0x563d2dc83990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe44759e700 1 -- 172.21.15.27:0/2997338130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 msgr2=0x563d2dc83990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe44759e700 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 0x563d2dc83990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe44759e700 1 -- 172.21.15.27:0/2997338130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 msgr2=0x563d2dc8aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe44759e700 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 0x563d2dc8aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe44759e700 1 -- 172.21.15.27:0/2997338130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2dc7e330 con 0x563d2dc89f10 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.468+0000 7fe446d9d700 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 0x563d2dc83990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.469+0000 7fe44759e700 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc89f10 0x563d2dca7c80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe440003be0 tx=0x7fe440009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53d65284e86c4e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.469+0000 7fe43cd99700 1 -- 172.21.15.27:0/2997338130 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44001a070 con 0x563d2dc89f10 2024-06-06T08:40:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.469+0000 7fe43cd99700 1 -- 172.21.15.27:0/2997338130 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe44000e9d0 con 0x563d2dc89f10 2024-06-06T08:40:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.469+0000 7fe43cd99700 1 -- 172.21.15.27:0/2997338130 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440007960 con 0x563d2dc89f10 2024-06-06T08:40:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.469+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2997338130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc89f10 msgr2=0x563d2dca7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.469+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc89f10 0x563d2dca7c80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe440003be0 tx=0x7fe440009920 comp rx=0 tx=0).stop 2024-06-06T08:40:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2997338130 shutdown_connections 2024-06-06T08:40:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc89f10 0x563d2dca7c80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc83ed0 0x563d2dc83990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/2997338130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc895f0 0x563d2dc8aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2997338130 >> 172.21.15.27:0/2997338130 conn(0x563d2dbaec70 msgr2=0x563d2dc8a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2997338130 shutdown_connections 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 -- 172.21.15.27:0/2997338130 wait complete. 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 -- start start 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 Processor -- start 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dca80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc895f0 0x563d2dca1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 0x563d2dca1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563d2dc9e6a0 con 0x563d2dc83ed0 2024-06-06T08:40:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563d2dc9dfd0 con 0x563d2dc89f10 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563d2dc9ed70 con 0x563d2dc895f0 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe4578f4b80 10 client.0 _ll_register_callbacks cb 0x563d2db6f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe446d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dca80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe446d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dca80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55462/0 (socket says 172.21.15.27:55462) 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.470+0000 7fe446d9d700 1 -- 172.21.15.27:0/385027546 learned_addr learned my addr 172.21.15.27:0/385027546 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe43e59c700 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc895f0 0x563d2dca1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe44759e700 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 0x563d2dca1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe446d9d700 1 -- 172.21.15.27:0/385027546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc895f0 msgr2=0x563d2dca1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe446d9d700 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc895f0 0x563d2dca1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe446d9d700 1 -- 172.21.15.27:0/385027546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 msgr2=0x563d2dca1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe446d9d700 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 0x563d2dca1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe446d9d700 1 -- 172.21.15.27:0/385027546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2dca1f50 con 0x563d2dc83ed0 2024-06-06T08:40:08.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe446d9d700 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dca80f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe43800b2f0 tx=0x7fe438010a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4472c2f202ef7cc3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe42e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe42e7fc700 1 -- 172.21.15.27:0/385027546 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380b9070 con 0x563d2dc83ed0 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe42e7fc700 1 -- 172.21.15.27:0/385027546 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fe438010710 con 0x563d2dc83ed0 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe4578f4b80 1 -- 172.21.15.27:0/385027546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe44000e9d0 con 0x563d2dc83ed0 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe42e7fc700 1 -- 172.21.15.27:0/385027546 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe4380b3030 con 0x563d2dc83ed0 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe4578f4b80 10 client.15177 fetch_fsmap learned FSMap version 24 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe4578f4b80 1 -- 172.21.15.27:0/385027546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563d2dca6950 con 0x563d2dc83ed0 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.471+0000 7fe42e7fc700 1 -- 172.21.15.27:0/385027546 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe44000e9d0 con 0x563d2dc83ed0 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe42e7fc700 1 -- 172.21.15.27:0/385027546 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe438084390 con 0x563d2dc83ed0 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe4578f4b80 10 client.15177 fetch_fsmap finished waiting for FSMap version 24 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe4578f4b80 10 client.15177 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe4578f4b80 20 client.15177 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563d2dc9d3b0 0x563d2dd0c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:08.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe4578f4b80 4 client.15177 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe4578f4b80 1 -- 172.21.15.27:0/385027546 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "status"}) -- 0x563d2dbdd240 con 0x563d2dc9d3b0 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe43e59c700 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563d2dc9d3b0 0x563d2dd0c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe43e59c700 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563d2dc9d3b0 0x563d2dd0c030 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514f8047afa1052b server_cookie=c614c9aee539f39c in_seq=0 out_seq=0 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.472+0000 7fe42e7fc700 10 client.15177 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe42e7fc700 1 -- 172.21.15.27:0/385027546 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563d2dbdd240 con 0x563d2dc9d3b0 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe42e7fc700 10 client.15177 handle_command_reply: tid=0 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 client.15177 shutdown 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 -- 172.21.15.27:0/385027546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 msgr2=0x563d2dca80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dca80f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe43800b2f0 tx=0x7fe438010a30 comp rx=0 tx=0).stop 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 -- 172.21.15.27:0/385027546 shutdown_connections 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563d2dc89f10 0x563d2dca1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563d2dc9d3b0 0x563d2dd0c030 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563d2dc83ed0 0x563d2dca80f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 --2- 172.21.15.27:0/385027546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563d2dc895f0 0x563d2dca1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 -- 172.21.15.27:0/385027546 >> 172.21.15.27:0/385027546 conn(0x563d2dbaec70 msgr2=0x563d2dc77db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe42e7fc700 0 client.15177 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 -- 172.21.15.27:0/385027546 shutdown_connections 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 1 -- 172.21.15.27:0/385027546 wait complete. 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe4578f4b80 20 client.15177 trim_cache size 0 max 16384 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24482, 2024-06-06T08:40:08.478 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:40:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:40:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:40:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 224.57391159700001, 2024-06-06T08:40:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 24, 2024-06-06T08:40:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 99, 2024-06-06T08:40:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 97, 2024-06-06T08:40:08.479 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 243.176835915 2024-06-06T08:40:08.479 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:40:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe44d071700 1 -- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4340a2ca0 msgr2=0x7fe4340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe44d071700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4340a2ca0 0x7fe4340a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe43000b2e0 tx=0x7fe430005960 comp rx=0 tx=0).stop 2024-06-06T08:40:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe44d071700 1 -- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 msgr2=0x7fe448101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.473+0000 7fe44d071700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 0x7fe448101490 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe43800d950 tx=0x7fe43800dc60 comp rx=0 tx=0).stop 2024-06-06T08:40:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.474+0000 7fe44d071700 1 -- 172.21.15.27:0/2304687952 shutdown_connections 2024-06-06T08:40:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.474+0000 7fe44d071700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fe4340a2ca0 0x7fe4340a5160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.474+0000 7fe44d071700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe448108500 0x7fe448101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.474+0000 7fe44d071700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44806abc0 0x7fe448101490 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.474+0000 7fe44d071700 1 --2- 172.21.15.27:0/2304687952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe44806b580 0x7fe4481019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:08.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.474+0000 7fe44d071700 1 -- 172.21.15.27:0/2304687952 >> 172.21.15.27:0/2304687952 conn(0x7fe448078340 msgr2=0x7fe4481000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:08.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.474+0000 7fe44d071700 1 -- 172.21.15.27:0/2304687952 shutdown_connections 2024-06-06T08:40:08.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:08.474+0000 7fe44d071700 1 -- 172.21.15.27:0/2304687952 wait complete. 2024-06-06T08:40:09.187 INFO:teuthology.run_tasks:Running task sequential... 2024-06-06T08:40:09.196 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-06T08:40:09.197 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-06T08:40:09.197 INFO:teuthology.task.print:Enabling mgr modules 2024-06-06T08:40:09.197 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-06-06T08:40:09.198 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T08:40:09.198 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:40:09.198 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-06-06T08:40:09.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.308+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/2645994736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e8102c20 msgr2=0x7fb0e81030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/2645994736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e8102c20 0x7fb0e81030a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0dc02a410 comp rx=0 tx=0).stop 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/2645994736 shutdown_connections 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/2645994736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0e81035e0 0x7fb0e8109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/2645994736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e8102c20 0x7fb0e81030a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/2645994736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e8101a30 0x7fb0e8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/2645994736 >> 172.21.15.27:0/2645994736 conn(0x7fb0e80fcfc0 msgr2=0x7fb0e80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/2645994736 shutdown_connections 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/2645994736 wait complete. 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 Processor -- start 2024-06-06T08:40:09.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.309+0000 7fb0ee1f9640 1 -- start start 2024-06-06T08:40:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0ee1f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e8101a30 0x7fb0e8197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0ee1f9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0e8102c20 0x7fb0e8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0ee1f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e81035e0 0x7fb0e819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0ee1f9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810d7a0 con 0x7fb0e81035e0 2024-06-06T08:40:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0ee1f9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810d620 con 0x7fb0e8102c20 2024-06-06T08:40:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0ee1f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810d920 con 0x7fb0e8101a30 2024-06-06T08:40:09.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e81035e0 0x7fb0e819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e81035e0 0x7fb0e819f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55466/0 (socket says 172.21.15.27:55466) 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e7fff640 1 -- 172.21.15.27:0/999844045 learned_addr learned my addr 172.21.15.27:0/999844045 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e77fe640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e8101a30 0x7fb0e8197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e6ffd640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0e8102c20 0x7fb0e8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e7fff640 1 -- 172.21.15.27:0/999844045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e8101a30 msgr2=0x7fb0e8197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e7fff640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e8101a30 0x7fb0e8197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e7fff640 1 -- 172.21.15.27:0/999844045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0e8102c20 msgr2=0x7fb0e8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e7fff640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0e8102c20 0x7fb0e8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e7fff640 1 -- 172.21.15.27:0/999844045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e819fa10 con 0x7fb0e81035e0 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e7fff640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e81035e0 0x7fb0e819f3a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000b970 tx=0x7fb0e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e4ff9640 1 -- 172.21.15.27:0/999844045 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0019070 con 0x7fb0e81035e0 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e4ff9640 1 -- 172.21.15.27:0/999844045 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e000cb40 con 0x7fb0e81035e0 2024-06-06T08:40:09.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e819fd00 con 0x7fb0e81035e0 2024-06-06T08:40:09.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0e4ff9640 1 -- 172.21.15.27:0/999844045 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0007650 con 0x7fb0e81035e0 2024-06-06T08:40:09.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.310+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81a01d0 con 0x7fb0e81035e0 2024-06-06T08:40:09.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.312+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e8108d50 con 0x7fb0e81035e0 2024-06-06T08:40:09.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.312+0000 7fb0e4ff9640 1 -- 172.21.15.27:0/999844045 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0e001c070 con 0x7fb0e81035e0 2024-06-06T08:40:09.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.313+0000 7fb0e4ff9640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0d00a2c30 0x7fb0d00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:09.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.313+0000 7fb0e77fe640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0d00a2c30 0x7fb0d00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:09.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.313+0000 7fb0e4ff9640 1 -- 172.21.15.27:0/999844045 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fb0e00ceaa0 con 0x7fb0e81035e0 2024-06-06T08:40:09.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.314+0000 7fb0e77fe640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0d00a2c30 0x7fb0d00a50f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8102a80 tx=0x7fb0d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:09.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.318+0000 7fb0e4ff9640 1 -- 172.21.15.27:0/999844045 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb0e009b630 con 0x7fb0e81035e0 2024-06-06T08:40:09.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.558+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fb0e80625a0 con 0x7fb0e81035e0 2024-06-06T08:40:09.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.880+0000 7fb0e4ff9640 1 -- 172.21.15.27:0/999844045 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fb0e009af30 con 0x7fb0e81035e0 2024-06-06T08:40:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0d00a2c30 msgr2=0x7fb0d00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0d00a2c30 0x7fb0d00a50f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8102a80 tx=0x7fb0d8008040 comp rx=0 tx=0).stop 2024-06-06T08:40:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e81035e0 msgr2=0x7fb0e819f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e81035e0 0x7fb0e819f3a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000b970 tx=0x7fb0e000be30 comp rx=0 tx=0).stop 2024-06-06T08:40:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 shutdown_connections 2024-06-06T08:40:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fb0d00a2c30 0x7fb0d00a50f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0e81035e0 0x7fb0e819f3a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0e8102c20 0x7fb0e8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 --2- 172.21.15.27:0/999844045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e8101a30 0x7fb0e8197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 >> 172.21.15.27:0/999844045 conn(0x7fb0e80fcfc0 msgr2=0x7fb0e8105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.882+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 shutdown_connections 2024-06-06T08:40:09.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:09.883+0000 7fb0ee1f9640 1 -- 172.21.15.27:0/999844045 wait complete. 2024-06-06T08:40:09.899 INFO:teuthology.run_tasks:Running task kclient... 2024-06-06T08:40:09.914 INFO:tasks.kclient:Mounting kernel clients... 2024-06-06T08:40:09.914 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-06T08:40:09.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:09.914 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-06-06T08:40:09.932 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-06T08:40:09.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:09.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T08:40:10.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:10.001 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-06-06T08:40:10.110 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-06-06T08:40:10.110 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-06T08:40:10.110 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T08:40:10.110 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-06T08:40:10.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:09 smithi027 ceph-mon[32335]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:40:10.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:09 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/999844045' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T08:40:10.206 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:09 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: ignoring --setuser ceph since I am not root 2024-06-06T08:40:10.206 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:09 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: ignoring --setgroup ceph since I am not root 2024-06-06T08:40:10.207 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:10 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:10.079+0000 7f911141a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T08:40:10.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:09 smithi203 ceph-mon[36579]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:40:10.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:09 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/999844045' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T08:40:10.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:09 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: ignoring --setuser ceph since I am not root 2024-06-06T08:40:10.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:09 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: ignoring --setgroup ceph since I am not root 2024-06-06T08:40:10.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:09 smithi203 ceph-mgr[38421]: -- 172.21.15.203:0/33212695 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b3caf25a0 con 0x564b3cac7000 2024-06-06T08:40:10.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:10.064+0000 7fd872ea3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T08:40:10.248 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:09 smithi176 ceph-mon[36786]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:40:10.248 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:09 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/999844045' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T08:40:10.248 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:09 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: ignoring --setuser ceph since I am not root 2024-06-06T08:40:10.248 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:09 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: ignoring --setgroup ceph since I am not root 2024-06-06T08:40:10.248 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:09 smithi176 ceph-mgr[38160]: -- 172.21.15.176:0/1355487039 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559c8c1d45a0 con 0x559c8c1a8800 2024-06-06T08:40:10.248 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:10.065+0000 7fe85a33e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T08:40:10.248 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:10.249+0000 7fe85a33e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:40:10.369 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:10.506 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:10.353+0000 7fe85a33e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T08:40:10.661 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:10.249+0000 7fd872ea3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:40:10.661 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:10.351+0000 7fd872ea3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T08:40:10.664 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:10 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:10.280+0000 7f911141a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:40:10.664 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:10 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:10.404+0000 7f911141a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T08:40:10.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:10 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/999844045' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T08:40:10.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:10 smithi027 ceph-mon[32335]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T08:40:10.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:10 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:10.735+0000 7f911141a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T08:40:10.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:10 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:10.828+0000 7f911141a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T08:40:10.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:10 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/999844045' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T08:40:10.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:10 smithi203 ceph-mon[36579]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T08:40:10.990 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:10.661+0000 7fd872ea3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T08:40:10.990 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:10.747+0000 7fd872ea3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T08:40:11.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:10 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/999844045' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T08:40:11.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:10 smithi176 ceph-mon[36786]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T08:40:11.007 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:10.660+0000 7fe85a33e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T08:40:11.007 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:10.746+0000 7fe85a33e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T08:40:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.130+0000 7f8500a7f700 1 -- 172.21.15.27:0/2199115880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84fc06a170 msgr2=0x7f84fc06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.130+0000 7f8500a7f700 1 --2- 172.21.15.27:0/2199115880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84fc06a170 0x7f84fc06a5f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009990 tx=0x7f84ec009ca0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.130+0000 7f8500a7f700 1 -- 172.21.15.27:0/2199115880 shutdown_connections 2024-06-06T08:40:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.130+0000 7f8500a7f700 1 --2- 172.21.15.27:0/2199115880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84fc06ab30 0x7f84fc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.130+0000 7f8500a7f700 1 --2- 172.21.15.27:0/2199115880 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84fc108500 0x7f84fc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.130+0000 7f8500a7f700 1 --2- 172.21.15.27:0/2199115880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84fc06a170 0x7f84fc06a5f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.130+0000 7f8500a7f700 1 -- 172.21.15.27:0/2199115880 >> 172.21.15.27:0/2199115880 conn(0x7f84fc078340 msgr2=0x7f84fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.130+0000 7f8500a7f700 1 -- 172.21.15.27:0/2199115880 shutdown_connections 2024-06-06T08:40:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.130+0000 7f8500a7f700 1 -- 172.21.15.27:0/2199115880 wait complete. 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.131+0000 7f8500a7f700 1 Processor -- start 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.131+0000 7f8500a7f700 1 -- start start 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f8500a7f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84fc06a170 0x7f84fc101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f8500a7f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84fc06ab30 0x7f84fc1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f8500a7f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84fc108500 0x7f84fc101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f8500a7f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc1127b0 con 0x7f84fc06ab30 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f8500a7f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc112630 con 0x7f84fc06a170 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f8500a7f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc112930 con 0x7f84fc108500 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84f9d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84fc06ab30 0x7f84fc1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84fa59c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84fc06a170 0x7f84fc101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84f9d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84fc06ab30 0x7f84fc1019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35202/0 (socket says 172.21.15.27:35202) 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84f9d9b700 1 -- 172.21.15.27:0/875015361 learned_addr learned my addr 172.21.15.27:0/875015361 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84fad9d700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84fc108500 0x7f84fc101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84fa59c700 1 -- 172.21.15.27:0/875015361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84fc108500 msgr2=0x7f84fc101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84fa59c700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84fc108500 0x7f84fc101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84fa59c700 1 -- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84fc06ab30 msgr2=0x7f84fc1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84fa59c700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84fc06ab30 0x7f84fc1019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.132+0000 7f84fa59c700 1 -- 172.21.15.27:0/875015361 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc1a7190 con 0x7f84fc06a170 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.133+0000 7f84fa59c700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84fc06a170 0x7f84fc101460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f84e400d950 tx=0x7f84e400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.133+0000 7f84f37fe700 1 -- 172.21.15.27:0/875015361 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4016070 con 0x7f84fc06a170 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.133+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84fc1a7480 con 0x7f84fc06a170 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.133+0000 7f84f37fe700 1 -- 172.21.15.27:0/875015361 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84e400fd70 con 0x7f84fc06a170 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.133+0000 7f84f37fe700 1 -- 172.21.15.27:0/875015361 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e400e3b0 con 0x7f84fc06a170 2024-06-06T08:40:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.133+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84fc1a7a10 con 0x7f84fc06a170 2024-06-06T08:40:12.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.134+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84fc069180 con 0x7f84fc06a170 2024-06-06T08:40:12.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.135+0000 7f84f37fe700 1 -- 172.21.15.27:0/875015361 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f84e4019070 con 0x7f84fc06a170 2024-06-06T08:40:12.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.136+0000 7f84f37fe700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f84e80a2c70 0x7f84e80a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:12.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.136+0000 7f84f9d9b700 1 -- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f84e80a2c70 msgr2=0x7f84e80a5130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3763593724 2024-06-06T08:40:12.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.136+0000 7f84f37fe700 1 -- 172.21.15.27:0/875015361 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f84e40cf060 con 0x7f84fc06a170 2024-06-06T08:40:12.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.136+0000 7f84f9d9b700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f84e80a2c70 0x7f84e80a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T08:40:12.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.139+0000 7f84f37fe700 1 -- 172.21.15.27:0/875015361 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f84e4010460 con 0x7f84fc06a170 2024-06-06T08:40:12.138 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:12 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:12.115+0000 7f911141a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T08:40:12.239 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:11.933+0000 7fd872ea3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T08:40:12.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:12.104+0000 7fd872ea3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T08:40:12.256 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:11.930+0000 7fe85a33e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T08:40:12.256 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:12.102+0000 7fe85a33e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T08:40:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.337+0000 7f84f9d9b700 1 -- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f84e80a2c70 msgr2=0x7f84e80a5130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3763593724 2024-06-06T08:40:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.337+0000 7f84f9d9b700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f84e80a2c70 0x7f84e80a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T08:40:12.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.394+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f84fc102990 con 0x7f84fc06a170 2024-06-06T08:40:12.393 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:12 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:12.299+0000 7f911141a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T08:40:12.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.394+0000 7f84f37fe700 1 -- 172.21.15.27:0/875015361 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f84e409bbf0 con 0x7f84fc06a170 2024-06-06T08:40:12.393 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f84e80a2c70 msgr2=0x7f84e80a5130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f84e80a2c70 0x7f84e80a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84fc06a170 msgr2=0x7f84fc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84fc06a170 0x7f84fc101460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f84e400d950 tx=0x7f84e400dc60 comp rx=0 tx=0).stop 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 shutdown_connections 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f84e80a2c70 0x7f84e80a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84fc108500 0x7f84fc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84fc06a170 0x7f84fc101460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 --2- 172.21.15.27:0/875015361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84fc06ab30 0x7f84fc1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 >> 172.21.15.27:0/875015361 conn(0x7f84fc078340 msgr2=0x7f84fc0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 shutdown_connections 2024-06-06T08:40:12.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:12.398+0000 7f8500a7f700 1 -- 172.21.15.27:0/875015361 wait complete. 2024-06-06T08:40:12.561 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:12.274+0000 7fe85a33e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T08:40:12.561 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:12 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/875015361' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T08:40:12.564 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:12 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/875015361' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T08:40:12.565 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:12.275+0000 7fd872ea3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T08:40:12.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:12 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/875015361' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T08:40:12.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:12 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:12.489+0000 7f911141a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T08:40:12.856 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:12.563+0000 7fe85a33e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T08:40:12.856 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:12.655+0000 7fe85a33e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T08:40:12.861 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:12.566+0000 7fd872ea3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T08:40:12.861 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:12.659+0000 7fd872ea3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T08:40:12.861 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:12.863+0000 7fd872ea3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T08:40:13.112 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:12 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:12.799+0000 7f911141a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T08:40:13.113 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:12 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:12.897+0000 7f911141a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T08:40:13.113 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:13 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:13.114+0000 7f911141a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T08:40:13.238 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-06T08:40:13.239 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-06T08:40:13.239 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-06-06T08:40:13.239 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-06T08:40:13.239 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T08:40:13.239 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-06T08:40:13.239 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-06T08:40:13.239 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-06T08:40:13.239 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-06T08:40:13.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:13.240 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-06-06T08:40:13.240 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:13.060+0000 7fd872ea3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:40:13.256 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:12.858+0000 7fe85a33e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T08:40:13.256 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:13.054+0000 7fe85a33e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:40:13.257 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-06T08:40:13.257 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-06T08:40:13.257 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-06-06T08:40:13.257 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-06-06T08:40:13.257 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-06-06T08:40:13.257 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-06-06T08:40:13.257 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T08:40:13.257 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42018sec preferred_lft 42018sec 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-06-06T08:40:13.258 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-06-06T08:40:13.259 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T08:40:13.259 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-06-06T08:40:13.259 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-06-06T08:40:13.259 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-06T08:40:13.259 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:40:13.259 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:40:13.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-06-06T08:40:13.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-06-06T08:40:13.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-06-06T08:40:13.260 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-06T08:40:13.260 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:40:13.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:40:13.363 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:13 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:13.323+0000 7f911141a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T08:40:13.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:40:13.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:13.407 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-06T08:40:13.433 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-06-06T08:40:13.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:13.435 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-06-06T08:40:13.492 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-06-06T08:40:13.492 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-06T08:40:13.492 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-06T08:40:13.493 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-06T08:40:13.493 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-06T08:40:13.493 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:40:13.493 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:40:13.493 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-06T08:40:13.493 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-06T08:40:13.493 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-06T08:40:13.493 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:40:13.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:40:13.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:40:13.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:13.625 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-06-06T08:40:13.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:13.642 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-06-06T08:40:13.697 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:40:13.697 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:40:13.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T08:40:13.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-06T08:40:13.698 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:40:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:40:13.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:40:13.820 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-06T08:40:13.821 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:40:13.821 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:40:13.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-06T08:40:13.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-06T08:40:13.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-06T08:40:13.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-06T08:40:13.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-06T08:40:13.821 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:40:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:40:13.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:40:13.945 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T08:40:13.946 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-06-06T08:40:13.946 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-06-06T08:40:13.946 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-06-06T08:40:13.946 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-06-06T08:40:13.989 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:13.665+0000 7fd872ea3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T08:40:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:40:14.006 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:13.677+0000 7fe85a33e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T08:40:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:40:14.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T08:40:14.028 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-06T08:40:14.044 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-06T08:40:14.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T08:40:14.044 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-06T08:40:14.100 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-06T08:40:14.100 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-06T08:40:14.100 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86 2024-06-06T08:40:14.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:14.101 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-06-06T08:40:14.162 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:13 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:13.989+0000 7f911141a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T08:40:14.185 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-06-06T08:40:14.186 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-06-06T08:40:14.369 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-06-06T08:40:14.369 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-06-06T08:40:14.369 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.176:3300,172.21.15.203:3300" 2024-06-06T08:40:14.369 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@8da40ada-23de-11ef-bc9b-c7b262605968.cephfs=/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86 2024-06-06T08:40:14.369 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.27:3300/172.21.15.176:3300/172.21.15.203:3300" will pass to kernel 2024-06-06T08:40:14.370 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-06T08:40:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-06-06T08:40:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-06T08:40:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-06-06T08:40:14.370 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-06T08:40:14.372 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-06T08:40:14.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:40:14.372 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-06T08:40:14.386 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:14.088+0000 7fd872ea3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T08:40:14.386 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:14.186+0000 7fd872ea3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T08:40:14.386 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:14.388+0000 7fd872ea3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T08:40:14.424 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:14.118+0000 7fe85a33e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T08:40:14.425 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:14.218+0000 7fe85a33e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T08:40:14.440 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-06T08:40:14.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T08:40:14.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:40:14.540 INFO:teuthology.orchestra.run.smithi027.stdout:client24892 2024-06-06T08:40:14.540 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:40:14.540 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T08:40:14.557 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-06T08:40:14.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T08:40:14.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:40:14.656 INFO:teuthology.orchestra.run.smithi027.stdout:client24892 2024-06-06T08:40:14.656 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-06T08:40:14.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:14.453+0000 7f911141a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T08:40:14.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:14.560+0000 7f911141a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T08:40:14.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T08:40:14.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:40:14.715 INFO:teuthology.orchestra.run.smithi027.stdout:client24892 2024-06-06T08:40:14.715 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8da40ada-23de-11ef-bc9b-c7b262605968.client24892/status') 2024-06-06T08:40:14.756 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:14.427+0000 7fe85a33e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T08:40:14.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T08:40:14.776 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.24892 (3)172.21.15.27:0/3687836827 2024-06-06T08:40:14.776 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2024-06-06T08:40:14.776 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2024-06-06T08:40:14.776 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2024-06-06T08:40:14.776 INFO:teuthology.orchestra.run.smithi027.stderr:71 bytes copied, 2.4458e-05 s, 2.9 MB/s 2024-06-06T08:40:14.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T08:40:14.784 INFO:teuthology.run_tasks:Running task exec... 2024-06-06T08:40:14.794 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T08:40:14.794 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:40:14.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-06-06T08:40:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.903+0000 7fd2c1c95640 1 -- 172.21.15.27:0/656424536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc1024b0 msgr2=0x7fd2bc108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.903+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/656424536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc1024b0 0x7fd2bc108d40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009ca0 tx=0x7fd2b402a8c0 comp rx=0 tx=0).stop 2024-06-06T08:40:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.903+0000 7fd2c1c95640 1 -- 172.21.15.27:0/656424536 shutdown_connections 2024-06-06T08:40:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.903+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/656424536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc1024b0 0x7fd2bc108d40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.903+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/656424536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2bc101af0 0x7fd2bc101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.903+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/656424536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2bc100900 0x7fd2bc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:14.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.903+0000 7fd2c1c95640 1 -- 172.21.15.27:0/656424536 >> 172.21.15.27:0/656424536 conn(0x7fd2bc0fbe70 msgr2=0x7fd2bc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:14.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.903+0000 7fd2c1c95640 1 -- 172.21.15.27:0/656424536 shutdown_connections 2024-06-06T08:40:14.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.903+0000 7fd2c1c95640 1 -- 172.21.15.27:0/656424536 wait complete. 2024-06-06T08:40:14.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2c1c95640 1 Processor -- start 2024-06-06T08:40:14.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2c1c95640 1 -- start start 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2c1c95640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2bc100900 0x7fd2bc197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2c1c95640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc101af0 0x7fd2bc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2c1c95640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2bc1024b0 0x7fd2bc19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2c1c95640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc10d660 con 0x7fd2bc101af0 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2c1c95640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc10d4e0 con 0x7fd2bc1024b0 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2c1c95640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc10d7e0 con 0x7fd2bc100900 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2baffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc101af0 0x7fd2bc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2baffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc101af0 0x7fd2bc198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35210/0 (socket says 172.21.15.27:35210) 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2baffd640 1 -- 172.21.15.27:0/950327395 learned_addr learned my addr 172.21.15.27:0/950327395 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2bb7fe640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2bc100900 0x7fd2bc197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:14.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2bbfff640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2bc1024b0 0x7fd2bc19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2baffd640 1 -- 172.21.15.27:0/950327395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2bc100900 msgr2=0x7fd2bc197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2baffd640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2bc100900 0x7fd2bc197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2baffd640 1 -- 172.21.15.27:0/950327395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2bc1024b0 msgr2=0x7fd2bc19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2baffd640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2bc1024b0 0x7fd2bc19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.904+0000 7fd2baffd640 1 -- 172.21.15.27:0/950327395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc19f850 con 0x7fd2bc101af0 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.905+0000 7fd2baffd640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc101af0 0x7fd2bc198180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000bea0 tx=0x7fd2b000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.905+0000 7fd2b8ff9640 1 -- 172.21.15.27:0/950327395 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001c070 con 0x7fd2bc101af0 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.905+0000 7fd2b8ff9640 1 -- 172.21.15.27:0/950327395 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2b0009d00 con 0x7fd2bc101af0 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.905+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2bc109760 con 0x7fd2bc101af0 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.905+0000 7fd2b8ff9640 1 -- 172.21.15.27:0/950327395 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001fd70 con 0x7fd2bc101af0 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.905+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2bc109cb0 con 0x7fd2bc101af0 2024-06-06T08:40:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.906+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2bc107c20 con 0x7fd2bc101af0 2024-06-06T08:40:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.907+0000 7fd2b8ff9640 1 -- 172.21.15.27:0/950327395 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fd2b0009820 con 0x7fd2bc101af0 2024-06-06T08:40:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.908+0000 7fd2b8ff9640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2a40a2c80 0x7fd2a40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.908+0000 7fd2bb7fe640 1 -- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2a40a2c80 msgr2=0x7fd2a40a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3763593724 2024-06-06T08:40:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.908+0000 7fd2b8ff9640 1 -- 172.21.15.27:0/950327395 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7fd2b00d77f0 con 0x7fd2bc101af0 2024-06-06T08:40:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.908+0000 7fd2bb7fe640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2a40a2c80 0x7fd2a40a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T08:40:14.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:14.912+0000 7fd2b8ff9640 1 -- 172.21.15.27:0/950327395 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd2b00a4380 con 0x7fd2bc101af0 2024-06-06T08:40:15.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.109+0000 7fd2bb7fe640 1 -- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2a40a2c80 msgr2=0x7fd2a40a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3763593724 2024-06-06T08:40:15.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.109+0000 7fd2bb7fe640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2a40a2c80 0x7fd2a40a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T08:40:15.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.138+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fd2bc10b0f0 con 0x7fd2bc101af0 2024-06-06T08:40:15.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.190+0000 7fd2b8ff9640 1 -- 172.21.15.27:0/950327395 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7fd2b00a3c80 con 0x7fd2bc101af0 2024-06-06T08:40:15.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2a40a2c80 msgr2=0x7fd2a40a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:40:15.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2a40a2c80 0x7fd2a40a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc101af0 msgr2=0x7fd2bc198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc101af0 0x7fd2bc198180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000bea0 tx=0x7fd2b000bf90 comp rx=0 tx=0).stop 2024-06-06T08:40:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 shutdown_connections 2024-06-06T08:40:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7fd2a40a2c80 0x7fd2a40a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2bc1024b0 0x7fd2bc19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2bc101af0 0x7fd2bc198180 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 --2- 172.21.15.27:0/950327395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2bc100900 0x7fd2bc197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 >> 172.21.15.27:0/950327395 conn(0x7fd2bc0fbe70 msgr2=0x7fd2bc104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 shutdown_connections 2024-06-06T08:40:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.193+0000 7fd2c1c95640 1 -- 172.21.15.27:0/950327395 wait complete. 2024-06-06T08:40:15.192 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:14.917+0000 7fd872ea3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T08:40:15.193 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:15.010+0000 7fd872ea3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T08:40:15.193 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:14.781+0000 7f911141a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T08:40:15.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-06-06T08:40:15.256 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:14.955+0000 7fe85a33e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T08:40:15.256 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:15.049+0000 7fe85a33e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T08:40:15.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:15 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/950327395' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-06T08:40:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.313+0000 7f86386bd640 1 -- 172.21.15.27:0/1017498419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8630100900 msgr2=0x7f8630100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.313+0000 7f86386bd640 1 --2- 172.21.15.27:0/1017498419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8630100900 0x7f8630100d20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f862c009900 tx=0x7f862c02a410 comp rx=0 tx=0).stop 2024-06-06T08:40:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.313+0000 7f86386bd640 1 -- 172.21.15.27:0/1017498419 shutdown_connections 2024-06-06T08:40:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.313+0000 7f86386bd640 1 --2- 172.21.15.27:0/1017498419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86301024b0 0x7f8630108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.313+0000 7f86386bd640 1 --2- 172.21.15.27:0/1017498419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8630101af0 0x7f8630101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.313+0000 7f86386bd640 1 --2- 172.21.15.27:0/1017498419 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8630100900 0x7f8630100d20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.313+0000 7f86386bd640 1 -- 172.21.15.27:0/1017498419 >> 172.21.15.27:0/1017498419 conn(0x7f86300fbe70 msgr2=0x7f86300fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.313+0000 7f86386bd640 1 -- 172.21.15.27:0/1017498419 shutdown_connections 2024-06-06T08:40:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.313+0000 7f86386bd640 1 -- 172.21.15.27:0/1017498419 wait complete. 2024-06-06T08:40:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f86386bd640 1 Processor -- start 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f86386bd640 1 -- start start 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f86386bd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8630100900 0x7f8630197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f86386bd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630101af0 0x7f8630198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f86386bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86301024b0 0x7f863019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f86386bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f863010d660 con 0x7f86301024b0 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f86386bd640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f863010d4e0 con 0x7f8630100900 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f86386bd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f863010d7e0 con 0x7f8630101af0 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f8636c5a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86301024b0 0x7f863019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f8636c5a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86301024b0 0x7f863019f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35224/0 (socket says 172.21.15.27:35224) 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f8636459640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8630100900 0x7f8630197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f8636c5a640 1 -- 172.21.15.27:0/403297407 learned_addr learned my addr 172.21.15.27:0/403297407 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.314+0000 7f8635c58640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630101af0 0x7f8630198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:15.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f8636c5a640 1 -- 172.21.15.27:0/403297407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630101af0 msgr2=0x7f8630198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f8636c5a640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8630101af0 0x7f8630198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f8636c5a640 1 -- 172.21.15.27:0/403297407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8630100900 msgr2=0x7f8630197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f8636c5a640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8630100900 0x7f8630197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f8636c5a640 1 -- 172.21.15.27:0/403297407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863019f910 con 0x7f86301024b0 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f8636c5a640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86301024b0 0x7f863019f270 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f862800efd0 tx=0x7f862800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628003b70 con 0x7f86301024b0 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8628003d10 con 0x7f86301024b0 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86280109f0 con 0x7f86301024b0 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863019fc00 con 0x7f86301024b0 2024-06-06T08:40:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.315+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86301a00d0 con 0x7f86301024b0 2024-06-06T08:40:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.316+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8630107c20 con 0x7f86301024b0 2024-06-06T08:40:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.317+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f8628004020 con 0x7f86301024b0 2024-06-06T08:40:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.318+0000 7f86277fe640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 0x7f861c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.318+0000 7f8636459640 1 -- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 msgr2=0x7f861c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3763593724 2024-06-06T08:40:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.318+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9216+0+0 (secure 0 0 0) 0x7f86280cf940 con 0x7f86301024b0 2024-06-06T08:40:15.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.318+0000 7f8636459640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 0x7f861c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T08:40:15.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.322+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f862809c4d0 con 0x7f86301024b0 2024-06-06T08:40:15.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:15 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/950327395' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-06T08:40:15.449 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:15 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:15.351+0000 7f911141a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T08:40:15.485 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:15 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/950327395' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-06T08:40:15.485 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:15.256+0000 7fd872ea3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T08:40:15.485 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:15.390+0000 7fd872ea3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T08:40:15.485 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:15.486+0000 7fd872ea3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T08:40:15.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.519+0000 7f8636459640 1 -- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 msgr2=0x7f861c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3763593724 2024-06-06T08:40:15.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.519+0000 7f8636459640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 0x7f861c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T08:40:15.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.535+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 --> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f8630061e80 con 0x7f861c0a2c80 2024-06-06T08:40:15.641 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:15.297+0000 7fe85a33e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T08:40:15.641 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:15.431+0000 7fe85a33e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T08:40:15.641 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:15.526+0000 7fe85a33e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T08:40:15.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:15 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:15.451+0000 7f911141a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T08:40:15.739 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:15.603+0000 7fd872ea3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T08:40:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.920+0000 7f8636459640 1 -- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 msgr2=0x7f861c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3763593724 2024-06-06T08:40:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:15.920+0000 7f8636459640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 0x7f861c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T08:40:15.943 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:15.642+0000 7fe85a33e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T08:40:16.083 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:15 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:15.713+0000 7f911141a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T08:40:16.084 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:15 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:15.856+0000 7f911141a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T08:40:16.084 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:15 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:15.960+0000 7f911141a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T08:40:16.084 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:16 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:16.085+0000 7f911141a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T08:40:16.194 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:15.944+0000 7fe85a33e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T08:40:16.194 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:15.905+0000 7fd872ea3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T08:40:16.397 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:16 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/950327395' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-06T08:40:16.398 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:16 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:40:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:16 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/950327395' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-06T08:40:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:16 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:40:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:16 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/950327395' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-06T08:40:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:16 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:40:16.706 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:16 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:16.398+0000 7f911141a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T08:40:16.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:16.720+0000 7f8636459640 1 -- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 msgr2=0x7f861c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3763593724 2024-06-06T08:40:16.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:16.720+0000 7f8636459640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 0x7f861c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T08:40:16.990 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:16 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:16.510+0000 7fd872ea3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T08:40:16.990 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:40:16 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z[38417]: 2024-06-06T08:40:16.727+0000 7fd872ea3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T08:40:17.006 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:16 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:16.541+0000 7fe85a33e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T08:40:17.006 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:40:16 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y[38155]: 2024-06-06T08:40:16.755+0000 7fe85a33e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T08:40:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:17.207+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f8628004310 con 0x7f86301024b0 2024-06-06T08:40:17.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:17.292+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f86280cd0f0 con 0x7f86301024b0 2024-06-06T08:40:17.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:17.293+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 msgr2=0x7f861c0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:40:17.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:17.293+0000 7f86277fe640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/3763593724,v1:172.21.15.27:6801/3763593724] conn(0x7f861c0a2c80 0x7f861c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:17.457 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.041+0000 7f911141a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T08:40:17.457 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.277+0000 7f911141a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T08:40:17.458 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: Standby manager daemon z restarted 2024-06-06T08:40:17.458 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: Standby manager daemon z started 2024-06-06T08:40:17.458 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T08:40:17.458 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:40:17.458 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T08:40:17.458 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:40:17.458 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: Standby manager daemon y restarted 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: Standby manager daemon y started 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:17 smithi176 ceph-mon[36786]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: Standby manager daemon z restarted 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: Standby manager daemon z started 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: Standby manager daemon y restarted 2024-06-06T08:40:17.459 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: Standby manager daemon y started 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:17 smithi203 ceph-mon[36579]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: Standby manager daemon z restarted 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: Standby manager daemon z started 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.203:0/3377580403' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: Standby manager daemon y restarted 2024-06-06T08:40:17.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: Standby manager daemon y started 2024-06-06T08:40:17.461 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T08:40:17.461 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T08:40:17.461 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T08:40:17.461 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-mon[32335]: from='mgr.? 172.21.15.176:0/2317524092' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T08:40:17.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f906f48a700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:17.957 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f906f48a700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:17.957 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f906f48a700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:17.957 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f906f48a700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:17.957 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f906f48a700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:17.957 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f9072c91700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:17.957 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f9072c91700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:17.957 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f9072c91700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:17.957 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f9072c91700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:17.957 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:17.595+0000 7f9072c91700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.296+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mon.0 v2:172.21.15.27:3300/0 9 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f8628078360 con 0x7f86301024b0 2024-06-06T08:40:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.297+0000 7f86277fe640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f861c035390 0x7f861c035770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.297+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f86280bfb20 con 0x7f861c035390 2024-06-06T08:40:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.299+0000 7f8636459640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f861c035390 0x7f861c035770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:18.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.303+0000 7f8636459640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f861c035390 0x7f861c035770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f862c005ab0 tx=0x7f862c000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:18.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.321+0000 7f86277fe640 1 -- 172.21.15.27:0/403297407 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f86280bfb20 con 0x7f861c035390 2024-06-06T08:40:18.320 INFO:teuthology.orchestra.run.smithi027.stdout:{} 2024-06-06T08:40:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.323+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f861c035390 msgr2=0x7f861c035770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.323+0000 7f86386bd640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f861c035390 0x7f861c035770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f862c005ab0 tx=0x7f862c000bc0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.323+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86301024b0 msgr2=0x7f863019f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.323+0000 7f86386bd640 1 --2- 172.21.15.27:0/403297407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86301024b0 0x7f863019f270 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f862800efd0 tx=0x7f862800c6a0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.323+0000 7f8636c5a640 1 -- 172.21.15.27:0/403297407 reap_dead start 2024-06-06T08:40:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.323+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 shutdown_connections 2024-06-06T08:40:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.323+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 >> 172.21.15.27:0/403297407 conn(0x7f86300fbe70 msgr2=0x7f8630104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.323+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 shutdown_connections 2024-06-06T08:40:18.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.323+0000 7f86386bd640 1 -- 172.21.15.27:0/403297407 wait complete. 2024-06-06T08:40:18.336 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-06T08:40:18.349 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-06T08:40:18.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-06T08:40:18.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:18.310+0000 7f906dc87700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:18.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:18.310+0000 7f906dc87700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:18.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:18.310+0000 7f906dc87700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:18.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:18.310+0000 7f906dc87700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:18.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:40:18.310+0000 7f906dc87700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: Active manager daemon x restarted 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: Activating manager daemon x 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: mgrmap e21: x(active, starting, since 0.015545s), standbys: z, y 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T08:40:18.458 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:40:18.459 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:40:18.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T08:40:18.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T08:40:18.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T08:40:18.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: Manager daemon x is now available 2024-06-06T08:40:18.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:18.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T08:40:18.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T08:40:18.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: [06/Jun/2024:08:40:17] ENGINE Bus STARTING 2024-06-06T08:40:18.460 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:18 smithi027 ceph-mon[32335]: [06/Jun/2024:08:40:17] ENGINE Serving on https://172.21.15.27:7150 2024-06-06T08:40:18.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: Active manager daemon x restarted 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: Activating manager daemon x 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: mgrmap e21: x(active, starting, since 0.015545s), standbys: z, y 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T08:40:18.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:40:18.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T08:40:18.493 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: Manager daemon x is now available 2024-06-06T08:40:18.494 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:18.494 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T08:40:18.494 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T08:40:18.494 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: [06/Jun/2024:08:40:17] ENGINE Bus STARTING 2024-06-06T08:40:18.494 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:18 smithi203 ceph-mon[36579]: [06/Jun/2024:08:40:17] ENGINE Serving on https://172.21.15.27:7150 2024-06-06T08:40:18.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.492+0000 7f29e8c76640 1 -- 172.21.15.27:0/1585973545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101e90 msgr2=0x7f29e41022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.492+0000 7f29e8c76640 1 --2- 172.21.15.27:0/1585973545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101e90 0x7f29e41022b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009900 tx=0x7f29cc02a410 comp rx=0 tx=0).stop 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.492+0000 7f29e8c76640 1 -- 172.21.15.27:0/1585973545 shutdown_connections 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.492+0000 7f29e8c76640 1 --2- 172.21.15.27:0/1585973545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e4101580 0x7f29e4108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.492+0000 7f29e8c76640 1 --2- 172.21.15.27:0/1585973545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e4100bc0 0x7f29e4101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.492+0000 7f29e8c76640 1 --2- 172.21.15.27:0/1585973545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101e90 0x7f29e41022b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.492+0000 7f29e8c76640 1 -- 172.21.15.27:0/1585973545 >> 172.21.15.27:0/1585973545 conn(0x7f29e40fb5d0 msgr2=0x7f29e40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.492+0000 7f29e8c76640 1 -- 172.21.15.27:0/1585973545 shutdown_connections 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.493+0000 7f29e8c76640 1 -- 172.21.15.27:0/1585973545 wait complete. 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.493+0000 7f29e8c76640 1 Processor -- start 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.493+0000 7f29e8c76640 1 -- start start 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.493+0000 7f29e8c76640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e4100bc0 0x7f29e4196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.493+0000 7f29e8c76640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101580 0x7f29e4197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:18.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.493+0000 7f29e8c76640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e4101e90 0x7f29e419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.493+0000 7f29e8c76640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f29e410c560 con 0x7f29e4101580 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.493+0000 7f29e8c76640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f29e410c3e0 con 0x7f29e4100bc0 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.493+0000 7f29e8c76640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29e410c6e0 con 0x7f29e4101e90 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101580 0x7f29e4197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101580 0x7f29e4197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35388/0 (socket says 172.21.15.27:35388) 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e1d9b640 1 -- 172.21.15.27:0/4094114896 learned_addr learned my addr 172.21.15.27:0/4094114896 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e259c640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e4100bc0 0x7f29e4196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e2d9d640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e4101e90 0x7f29e419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e1d9b640 1 -- 172.21.15.27:0/4094114896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e4101e90 msgr2=0x7f29e419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e1d9b640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e4101e90 0x7f29e419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e1d9b640 1 -- 172.21.15.27:0/4094114896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e4100bc0 msgr2=0x7f29e4196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e1d9b640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e4100bc0 0x7f29e4196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e1d9b640 1 -- 172.21.15.27:0/4094114896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e419e810 con 0x7f29e4101580 2024-06-06T08:40:18.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e1d9b640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101580 0x7f29e4197130 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d950 tx=0x7f29d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:18.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29db7fe640 1 -- 172.21.15.27:0/4094114896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4017070 con 0x7f29e4101580 2024-06-06T08:40:18.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29db7fe640 1 -- 172.21.15.27:0/4094114896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d4009c30 con 0x7f29e4101580 2024-06-06T08:40:18.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e419eb00 con 0x7f29e4101580 2024-06-06T08:40:18.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29db7fe640 1 -- 172.21.15.27:0/4094114896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d400f6b0 con 0x7f29e4101580 2024-06-06T08:40:18.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.494+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e419f050 con 0x7f29e4101580 2024-06-06T08:40:18.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.495+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e4107da0 con 0x7f29e4101580 2024-06-06T08:40:18.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.496+0000 7f29db7fe640 1 -- 172.21.15.27:0/4094114896 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f29d40109d0 con 0x7f29e4101580 2024-06-06T08:40:18.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.498+0000 7f29db7fe640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29d00a2990 0x7f29d00a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:18.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.498+0000 7f29e259c640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29d00a2990 0x7f29d00a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:18.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.498+0000 7f29db7fe640 1 -- 172.21.15.27:0/4094114896 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f29d40cecc0 con 0x7f29e4101580 2024-06-06T08:40:18.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.498+0000 7f29e259c640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29d00a2990 0x7f29d00a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29cc005ab0 tx=0x7f29cc000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:18.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.502+0000 7f29db7fe640 1 -- 172.21.15.27:0/4094114896 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f29d4009de0 con 0x7f29e4101580 2024-06-06T08:40:18.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: Active manager daemon x restarted 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: Activating manager daemon x 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: mgrmap e21: x(active, starting, since 0.015545s), standbys: z, y 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T08:40:18.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T08:40:18.508 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: Manager daemon x is now available 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T08:40:18.509 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T08:40:18.510 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: [06/Jun/2024:08:40:17] ENGINE Bus STARTING 2024-06-06T08:40:18.510 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:18 smithi176 ceph-mon[36786]: [06/Jun/2024:08:40:17] ENGINE Serving on https://172.21.15.27:7150 2024-06-06T08:40:18.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.693+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29e40625d0 con 0x7f29e4101580 2024-06-06T08:40:18.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.694+0000 7f29db7fe640 1 -- 172.21.15.27:0/4094114896 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25213 (secure 0 0 0) 0x7f29d409b3e0 con 0x7f29e4101580 2024-06-06T08:40:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:18.694 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:40:17.280912+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/876705499":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/550980984":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/4174349888":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3053481437":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:40:18.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29d00a2990 msgr2=0x7f29d00a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29d00a2990 0x7f29d00a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29cc005ab0 tx=0x7f29cc000bc0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101580 msgr2=0x7f29e4197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101580 0x7f29e4197130 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d950 tx=0x7f29d400de10 comp rx=0 tx=0).stop 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 shutdown_connections 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29d00a2990 0x7f29d00a4e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e4101e90 0x7f29e419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29e4101580 0x7f29e4197130 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 --2- 172.21.15.27:0/4094114896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29e4100bc0 0x7f29e4196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 >> 172.21.15.27:0/4094114896 conn(0x7f29e40fb5d0 msgr2=0x7f29e4104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 shutdown_connections 2024-06-06T08:40:18.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.698+0000 7f29e8c76640 1 -- 172.21.15.27:0/4094114896 wait complete. 2024-06-06T08:40:18.713 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T08:31:22.877337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-06T08:36:16.098725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-06T08:36:19.818146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-06T08:36:30.341458+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-06T08:40:18.714 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-06T08:40:18.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 -- 172.21.15.27:0/363451962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c102490 msgr2=0x7f4b7c108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/363451962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c102490 0x7f4b7c108d20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009ca0 tx=0x7f4b7402a8c0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 -- 172.21.15.27:0/363451962 shutdown_connections 2024-06-06T08:40:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/363451962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c102490 0x7f4b7c108d20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/363451962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c101ad0 0x7f4b7c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/363451962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b7c1008e0 0x7f4b7c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 -- 172.21.15.27:0/363451962 >> 172.21.15.27:0/363451962 conn(0x7f4b7c0fbe50 msgr2=0x7f4b7c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:18.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 -- 172.21.15.27:0/363451962 shutdown_connections 2024-06-06T08:40:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 -- 172.21.15.27:0/363451962 wait complete. 2024-06-06T08:40:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.824+0000 7f4b81aa5640 1 Processor -- start 2024-06-06T08:40:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b81aa5640 1 -- start start 2024-06-06T08:40:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b81aa5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b7c1008e0 0x7f4b7c197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b81aa5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c101ad0 0x7f4b7c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b81aa5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c102490 0x7f4b7c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b81aa5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10c540 con 0x7f4b7c101ad0 2024-06-06T08:40:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b81aa5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10c3c0 con 0x7f4b7c1008e0 2024-06-06T08:40:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b81aa5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c10c6c0 con 0x7f4b7c102490 2024-06-06T08:40:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c101ad0 0x7f4b7c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c101ad0 0x7f4b7c1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35414/0 (socket says 172.21.15.27:35414) 2024-06-06T08:40:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7affd640 1 -- 172.21.15.27:0/153331120 learned_addr learned my addr 172.21.15.27:0/153331120 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7b7fe640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b7c1008e0 0x7f4b7c197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7bfff640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c102490 0x7f4b7c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7affd640 1 -- 172.21.15.27:0/153331120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c102490 msgr2=0x7f4b7c19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7affd640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c102490 0x7f4b7c19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7affd640 1 -- 172.21.15.27:0/153331120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b7c1008e0 msgr2=0x7f4b7c197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7affd640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b7c1008e0 0x7f4b7c197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:18.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7affd640 1 -- 172.21.15.27:0/153331120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7c19f940 con 0x7f4b7c101ad0 2024-06-06T08:40:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b7affd640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c101ad0 0x7f4b7c1982c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000d950 tx=0x7f4b7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b78ff9640 1 -- 172.21.15.27:0/153331120 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70017030 con 0x7f4b7c101ad0 2024-06-06T08:40:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b78ff9640 1 -- 172.21.15.27:0/153331120 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b70004020 con 0x7f4b7c101ad0 2024-06-06T08:40:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7c19fc30 con 0x7f4b7c101ad0 2024-06-06T08:40:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.825+0000 7f4b78ff9640 1 -- 172.21.15.27:0/153331120 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700047a0 con 0x7f4b7c101ad0 2024-06-06T08:40:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.826+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7c1a0180 con 0x7f4b7c101ad0 2024-06-06T08:40:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.827+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b7c1a03b0 con 0x7f4b7c101ad0 2024-06-06T08:40:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.828+0000 7f4b78ff9640 1 -- 172.21.15.27:0/153331120 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f4b700041d0 con 0x7f4b7c101ad0 2024-06-06T08:40:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.828+0000 7f4b78ff9640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4b640a2990 0x7f4b640a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.829+0000 7f4b78ff9640 1 -- 172.21.15.27:0/153331120 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f4b700ceb90 con 0x7f4b7c101ad0 2024-06-06T08:40:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.829+0000 7f4b7b7fe640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4b640a2990 0x7f4b640a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.829+0000 7f4b7b7fe640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4b640a2990 0x7f4b640a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c199200 tx=0x7f4b6c005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:18.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:18.833+0000 7f4b78ff9640 1 -- 172.21.15.27:0/153331120 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4b7009b900 con 0x7f4b7c101ad0 2024-06-06T08:40:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.020+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b7c1a0580 con 0x7f4b7c101ad0 2024-06-06T08:40:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.021+0000 7f4b78ff9640 1 -- 172.21.15.27:0/153331120 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f4b7009b200 con 0x7f4b7c101ad0 2024-06-06T08:40:19.020 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-06-06T08:40:19.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4b640a2990 msgr2=0x7f4b640a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4b640a2990 0x7f4b640a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c199200 tx=0x7f4b6c005f90 comp rx=0 tx=0).stop 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c101ad0 msgr2=0x7f4b7c1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c101ad0 0x7f4b7c1982c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000d950 tx=0x7f4b7000de10 comp rx=0 tx=0).stop 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 shutdown_connections 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4b640a2990 0x7f4b640a4e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b7c102490 0x7f4b7c19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b7c101ad0 0x7f4b7c1982c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 --2- 172.21.15.27:0/153331120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b7c1008e0 0x7f4b7c197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 >> 172.21.15.27:0/153331120 conn(0x7f4b7c0fbe50 msgr2=0x7f4b7c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 shutdown_connections 2024-06-06T08:40:19.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.023+0000 7f4b81aa5640 1 -- 172.21.15.27:0/153331120 wait complete. 2024-06-06T08:40:19.034 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-06T08:40:19.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.150+0000 7f1b789f9640 1 -- 172.21.15.27:0/2373781869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b74100900 msgr2=0x7f1b74100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.150+0000 7f1b789f9640 1 --2- 172.21.15.27:0/2373781869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b74100900 0x7f1b74100d20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009860 tx=0x7f1b6402a340 comp rx=0 tx=0).stop 2024-06-06T08:40:19.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.151+0000 7f1b789f9640 1 -- 172.21.15.27:0/2373781869 shutdown_connections 2024-06-06T08:40:19.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.151+0000 7f1b789f9640 1 --2- 172.21.15.27:0/2373781869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b741024b0 0x7f1b74108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.151+0000 7f1b789f9640 1 --2- 172.21.15.27:0/2373781869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b74101af0 0x7f1b74101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.151+0000 7f1b789f9640 1 --2- 172.21.15.27:0/2373781869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b74100900 0x7f1b74100d20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.151+0000 7f1b789f9640 1 -- 172.21.15.27:0/2373781869 >> 172.21.15.27:0/2373781869 conn(0x7f1b740fbe90 msgr2=0x7f1b740fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:19.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.151+0000 7f1b789f9640 1 -- 172.21.15.27:0/2373781869 shutdown_connections 2024-06-06T08:40:19.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.151+0000 7f1b789f9640 1 -- 172.21.15.27:0/2373781869 wait complete. 2024-06-06T08:40:19.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.151+0000 7f1b789f9640 1 Processor -- start 2024-06-06T08:40:19.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.151+0000 7f1b789f9640 1 -- start start 2024-06-06T08:40:19.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b789f9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b74100900 0x7f1b74197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b789f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b74101af0 0x7f1b74198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b789f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b741024b0 0x7f1b7419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b789f9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7410c550 con 0x7f1b74101af0 2024-06-06T08:40:19.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b789f9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7410c3d0 con 0x7f1b74100900 2024-06-06T08:40:19.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b789f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7410c6d0 con 0x7f1b741024b0 2024-06-06T08:40:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b6bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b74101af0 0x7f1b74198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b6bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b74101af0 0x7f1b74198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35430/0 (socket says 172.21.15.27:35430) 2024-06-06T08:40:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b6bfff640 1 -- 172.21.15.27:0/1954041218 learned_addr learned my addr 172.21.15.27:0/1954041218 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b7259c640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b74100900 0x7f1b74197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b72d9d640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b741024b0 0x7f1b7419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b7259c640 1 -- 172.21.15.27:0/1954041218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b741024b0 msgr2=0x7f1b7419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b7259c640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b741024b0 0x7f1b7419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b7259c640 1 -- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b74101af0 msgr2=0x7f1b74198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b7259c640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b74101af0 0x7f1b74198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b7259c640 1 -- 172.21.15.27:0/1954041218 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7419f9f0 con 0x7f1b74100900 2024-06-06T08:40:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b6bfff640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b74101af0 0x7f1b74198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b7259c640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b74100900 0x7f1b74197dd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b6402a860 tx=0x7f1b64033ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b6b7fe640 1 -- 172.21.15.27:0/1954041218 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6403f020 con 0x7f1b74100900 2024-06-06T08:40:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.152+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7419fc80 con 0x7f1b74100900 2024-06-06T08:40:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.153+0000 7f1b6b7fe640 1 -- 172.21.15.27:0/1954041218 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64004040 con 0x7f1b74100900 2024-06-06T08:40:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.153+0000 7f1b6b7fe640 1 -- 172.21.15.27:0/1954041218 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64037040 con 0x7f1b74100900 2024-06-06T08:40:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.153+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741a01c0 con 0x7f1b74100900 2024-06-06T08:40:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.154+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b741a03f0 con 0x7f1b74100900 2024-06-06T08:40:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.155+0000 7f1b6b7fe640 1 -- 172.21.15.27:0/1954041218 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f1b64042020 con 0x7f1b74100900 2024-06-06T08:40:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.156+0000 7f1b6b7fe640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1b600a2990 0x7f1b600a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.156+0000 7f1b6bfff640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1b600a2990 0x7f1b600a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.156+0000 7f1b6b7fe640 1 -- 172.21.15.27:0/1954041218 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f1b640ef220 con 0x7f1b74100900 2024-06-06T08:40:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.156+0000 7f1b6bfff640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1b600a2990 0x7f1b600a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b741992e0 tx=0x7f1b5c005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.160+0000 7f1b6b7fe640 1 -- 172.21.15.27:0/1954041218 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1b64049020 con 0x7f1b74100900 2024-06-06T08:40:19.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.304+0000 7f1b6b7fe640 1 -- 172.21.15.27:0/1954041218 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7f1b640b7ec0 con 0x7f1b74100900 2024-06-06T08:40:19.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.358+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b741a0780 con 0x7f1b74100900 2024-06-06T08:40:19.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.359+0000 7f1b6b7fe640 1 -- 172.21.15.27:0/1954041218 <== mon.2 v2:172.21.15.176:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f1b640bbf50 con 0x7f1b74100900 2024-06-06T08:40:19.359 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1b600a2990 msgr2=0x7f1b600a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1b600a2990 0x7f1b600a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b741992e0 tx=0x7f1b5c005f90 comp rx=0 tx=0).stop 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b74100900 msgr2=0x7f1b74197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b74100900 0x7f1b74197dd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b6402a860 tx=0x7f1b64033ae0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 shutdown_connections 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1b600a2990 0x7f1b600a4e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b741024b0 0x7f1b7419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b74101af0 0x7f1b74198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 --2- 172.21.15.27:0/1954041218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b74100900 0x7f1b74197dd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 >> 172.21.15.27:0/1954041218 conn(0x7f1b740fbe90 msgr2=0x7f1b74069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 shutdown_connections 2024-06-06T08:40:19.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.361+0000 7f1b789f9640 1 -- 172.21.15.27:0/1954041218 wait complete. 2024-06-06T08:40:19.373 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-06T08:40:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.506+0000 7fa05945d640 1 -- 172.21.15.27:0/2284480050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0541019f0 msgr2=0x7fa054101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.506+0000 7fa05945d640 1 --2- 172.21.15.27:0/2284480050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0541019f0 0x7fa054101e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa044009ca0 tx=0x7fa04402a8c0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.506+0000 7fa05945d640 1 -- 172.21.15.27:0/2284480050 shutdown_connections 2024-06-06T08:40:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.506+0000 7fa05945d640 1 --2- 172.21.15.27:0/2284480050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0541035a0 0x7fa054109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.506+0000 7fa05945d640 1 --2- 172.21.15.27:0/2284480050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa054102be0 0x7fa054103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.506+0000 7fa05945d640 1 --2- 172.21.15.27:0/2284480050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0541019f0 0x7fa054101e10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.507+0000 7fa05945d640 1 -- 172.21.15.27:0/2284480050 >> 172.21.15.27:0/2284480050 conn(0x7fa0540fcf80 msgr2=0x7fa0540ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.507+0000 7fa05945d640 1 -- 172.21.15.27:0/2284480050 shutdown_connections 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.507+0000 7fa05945d640 1 -- 172.21.15.27:0/2284480050 wait complete. 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.507+0000 7fa05945d640 1 Processor -- start 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.507+0000 7fa05945d640 1 -- start start 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.507+0000 7fa05945d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0541019f0 0x7fa054075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.507+0000 7fa05945d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054102be0 0x7fa054073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa05945d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0541035a0 0x7fa0540742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa05945d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410d6c0 con 0x7fa054102be0 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa05945d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410d540 con 0x7fa0541019f0 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa05945d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa05410d840 con 0x7fa0541035a0 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0527fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054102be0 0x7fa054073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0537fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0541035a0 0x7fa0540742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0527fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054102be0 0x7fa054073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35474/0 (socket says 172.21.15.27:35474) 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0527fc640 1 -- 172.21.15.27:0/2512997501 learned_addr learned my addr 172.21.15.27:0/2512997501 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa052ffd640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0541019f0 0x7fa054075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0527fc640 1 -- 172.21.15.27:0/2512997501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0541035a0 msgr2=0x7fa0540742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0527fc640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0541035a0 0x7fa0540742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0527fc640 1 -- 172.21.15.27:0/2512997501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0541019f0 msgr2=0x7fa054075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0527fc640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0541019f0 0x7fa054075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0527fc640 1 -- 172.21.15.27:0/2512997501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa054074b60 con 0x7fa054102be0 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa052ffd640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0541019f0 0x7fa054075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa0527fc640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054102be0 0x7fa054073d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00e9f0 tx=0x7fa03c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa04bfff640 1 -- 172.21.15.27:0/2512997501 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00cd80 con 0x7fa054102be0 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa04bfff640 1 -- 172.21.15.27:0/2512997501 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa03c010910 con 0x7fa054102be0 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa04bfff640 1 -- 172.21.15.27:0/2512997501 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c018990 con 0x7fa054102be0 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.508+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0541a1e20 con 0x7fa054102be0 2024-06-06T08:40:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.509+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0541a2230 con 0x7fa054102be0 2024-06-06T08:40:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.510+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa054108d10 con 0x7fa054102be0 2024-06-06T08:40:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.511+0000 7fa04bfff640 1 -- 172.21.15.27:0/2512997501 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7fa03c010ac0 con 0x7fa054102be0 2024-06-06T08:40:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.512+0000 7fa04bfff640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa0400a2d90 0x7fa0400a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.512+0000 7fa052ffd640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa0400a2d90 0x7fa0400a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.512+0000 7fa04bfff640 1 -- 172.21.15.27:0/2512997501 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fa03c0cfc40 con 0x7fa054102be0 2024-06-06T08:40:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.512+0000 7fa052ffd640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa0400a2d90 0x7fa0400a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa044000f80 tx=0x7fa044004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:19.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.516+0000 7fa04bfff640 1 -- 172.21.15.27:0/2512997501 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa03c09bce0 con 0x7fa054102be0 2024-06-06T08:40:19.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:19 smithi027 ceph-mon[32335]: [06/Jun/2024:08:40:17] ENGINE Serving on http://172.21.15.27:8765 2024-06-06T08:40:19.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:19 smithi027 ceph-mon[32335]: [06/Jun/2024:08:40:17] ENGINE Bus STARTED 2024-06-06T08:40:19.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:19 smithi027 ceph-mon[32335]: mgrmap e22: x(active, since 1.01952s), standbys: z, y 2024-06-06T08:40:19.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:19 smithi027 ceph-mon[32335]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:19.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:19 smithi027 ceph-mon[32335]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:40:19.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:19 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4094114896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:19.680 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:19 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/153331120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:40:19.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.709+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa05402d530 con 0x7fa054102be0 2024-06-06T08:40:19.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.710+0000 7fa04bfff640 1 -- 172.21.15.27:0/2512997501 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fa03c09ffe0 con 0x7fa054102be0 2024-06-06T08:40:19.709 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-06-06T08:40:19.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa0400a2d90 msgr2=0x7fa0400a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa0400a2d90 0x7fa0400a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa044000f80 tx=0x7fa044004a40 comp rx=0 tx=0).stop 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054102be0 msgr2=0x7fa054073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054102be0 0x7fa054073d90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00e9f0 tx=0x7fa03c00eeb0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 shutdown_connections 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa0400a2d90 0x7fa0400a5250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0541035a0 0x7fa0540742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa054102be0 0x7fa054073d90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 --2- 172.21.15.27:0/2512997501 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0541019f0 0x7fa054075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 >> 172.21.15.27:0/2512997501 conn(0x7fa0540fcf80 msgr2=0x7fa054105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 shutdown_connections 2024-06-06T08:40:19.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.712+0000 7fa05945d640 1 -- 172.21.15.27:0/2512997501 wait complete. 2024-06-06T08:40:19.723 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-06T08:40:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:19 smithi203 ceph-mon[36579]: [06/Jun/2024:08:40:17] ENGINE Serving on http://172.21.15.27:8765 2024-06-06T08:40:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:19 smithi203 ceph-mon[36579]: [06/Jun/2024:08:40:17] ENGINE Bus STARTED 2024-06-06T08:40:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:19 smithi203 ceph-mon[36579]: mgrmap e22: x(active, since 1.01952s), standbys: z, y 2024-06-06T08:40:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:19 smithi203 ceph-mon[36579]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:19 smithi203 ceph-mon[36579]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:40:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:19 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4094114896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:19 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/153331120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:40:19.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:19 smithi176 ceph-mon[36786]: [06/Jun/2024:08:40:17] ENGINE Serving on http://172.21.15.27:8765 2024-06-06T08:40:19.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:19 smithi176 ceph-mon[36786]: [06/Jun/2024:08:40:17] ENGINE Bus STARTED 2024-06-06T08:40:19.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:19 smithi176 ceph-mon[36786]: mgrmap e22: x(active, since 1.01952s), standbys: z, y 2024-06-06T08:40:19.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:19 smithi176 ceph-mon[36786]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:19.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:19 smithi176 ceph-mon[36786]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:40:19.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:19 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4094114896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:19.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:19 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/153331120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:40:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.835+0000 7f661c79e640 1 -- 172.21.15.27:0/2863801312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614100bc0 msgr2=0x7f6614101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.835+0000 7f661c79e640 1 --2- 172.21.15.27:0/2863801312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614100bc0 0x7f6614101040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6604009900 tx=0x7f660402a410 comp rx=0 tx=0).stop 2024-06-06T08:40:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.835+0000 7f661c79e640 1 -- 172.21.15.27:0/2863801312 shutdown_connections 2024-06-06T08:40:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.835+0000 7f661c79e640 1 --2- 172.21.15.27:0/2863801312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6614101580 0x7f6614108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.835+0000 7f661c79e640 1 --2- 172.21.15.27:0/2863801312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614100bc0 0x7f6614101040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.835+0000 7f661c79e640 1 --2- 172.21.15.27:0/2863801312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614101e90 0x7f66141022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.835+0000 7f661c79e640 1 -- 172.21.15.27:0/2863801312 >> 172.21.15.27:0/2863801312 conn(0x7f66140fb5d0 msgr2=0x7f66140fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 -- 172.21.15.27:0/2863801312 shutdown_connections 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 -- 172.21.15.27:0/2863801312 wait complete. 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 Processor -- start 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 -- start start 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6614100bc0 0x7f661410b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614101580 0x7f66141098a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614101e90 0x7f6614109de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f661410d710 con 0x7f6614101580 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f661410d590 con 0x7f6614101e90 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f661c79e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f661410d890 con 0x7f6614100bc0 2024-06-06T08:40:19.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.836+0000 7f6619d39640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614101580 0x7f66141098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f6619d39640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614101580 0x7f66141098a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58834/0 (socket says 172.21.15.27:58834) 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f6619d39640 1 -- 172.21.15.27:0/1074312322 learned_addr learned my addr 172.21.15.27:0/1074312322 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f661a53a640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6614100bc0 0x7f661410b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f661ad3b640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614101e90 0x7f6614109de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f6619d39640 1 -- 172.21.15.27:0/1074312322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6614100bc0 msgr2=0x7f661410b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f6619d39640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6614100bc0 0x7f661410b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f6619d39640 1 -- 172.21.15.27:0/1074312322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614101e90 msgr2=0x7f6614109de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f6619d39640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614101e90 0x7f6614109de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f6619d39640 1 -- 172.21.15.27:0/1074312322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661410a670 con 0x7f6614101580 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f661ad3b640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614101e90 0x7f6614109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f6619d39640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614101580 0x7f66141098a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6604031440 tx=0x7f6604032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f660b7fe640 1 -- 172.21.15.27:0/1074312322 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660403f030 con 0x7f6614101580 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66141a1de0 con 0x7f6614101580 2024-06-06T08:40:19.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f660b7fe640 1 -- 172.21.15.27:0/1074312322 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6604003ce0 con 0x7f6614101580 2024-06-06T08:40:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f660b7fe640 1 -- 172.21.15.27:0/1074312322 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604035070 con 0x7f6614101580 2024-06-06T08:40:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.837+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66141a2250 con 0x7f6614101580 2024-06-06T08:40:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.839+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6614107d00 con 0x7f6614101580 2024-06-06T08:40:19.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.841+0000 7f660b7fe640 1 -- 172.21.15.27:0/1074312322 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7f6604042030 con 0x7f6614101580 2024-06-06T08:40:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.841+0000 7f660b7fe640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f66000a2d90 0x7f66000a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.841+0000 7f660b7fe640 1 -- 172.21.15.27:0/1074312322 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f66040f01b0 con 0x7f6614101580 2024-06-06T08:40:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.842+0000 7f661a53a640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f66000a2d90 0x7f66000a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:19.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.842+0000 7f661a53a640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f66000a2d90 0x7f66000a5250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6614069d20 tx=0x7f6610005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:19.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:19.845+0000 7f660b7fe640 1 -- 172.21.15.27:0/1074312322 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f66040bc250 con 0x7f6614101580 2024-06-06T08:40:20.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.064+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f661402d4e0 con 0x7f6614101580 2024-06-06T08:40:20.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.064+0000 7f660b7fe640 1 -- 172.21.15.27:0/1074312322 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7f66040c0550 con 0x7f6614101580 2024-06-06T08:40:20.064 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-06-06T08:40:20.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.066+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f66000a2d90 msgr2=0x7f66000a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:20.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.066+0000 7f661c79e640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f66000a2d90 0x7f66000a5250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6614069d20 tx=0x7f6610005f90 comp rx=0 tx=0).stop 2024-06-06T08:40:20.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614101580 msgr2=0x7f66141098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:20.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614101580 0x7f66141098a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6604031440 tx=0x7f6604032ce0 comp rx=0 tx=0).stop 2024-06-06T08:40:20.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 shutdown_connections 2024-06-06T08:40:20.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f66000a2d90 0x7f66000a5250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:20.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6614101e90 0x7f6614109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:20.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6614101580 0x7f66141098a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:20.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 --2- 172.21.15.27:0/1074312322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6614100bc0 0x7f661410b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:20.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 >> 172.21.15.27:0/1074312322 conn(0x7f66140fb5d0 msgr2=0x7f6614104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:20.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 shutdown_connections 2024-06-06T08:40:20.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:20.067+0000 7f661c79e640 1 -- 172.21.15.27:0/1074312322 wait complete. 2024-06-06T08:40:20.078 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-06T08:40:20.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:40:20.351 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:20.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:40:20.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-06-06T08:40:20.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1954041218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T08:40:20.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2512997501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T08:40:20.457 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:20 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1074312322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1954041218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2512997501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T08:40:20.741 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:20 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1074312322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T08:40:20.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:40:20.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-06-06T08:40:20.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1954041218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T08:40:20.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:20.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2512997501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T08:40:20.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:20 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1074312322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T08:40:21.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: Unable to set osd_memory_target on smithi203 to 184758067: error parsing value: Value '184758067' is below minimum 939524096 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi176 to 176.2M 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: Unable to set osd_memory_target on smithi176 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:21.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:21 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:21.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-06-06T08:40:21.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: Unable to set osd_memory_target on smithi203 to 184758067: error parsing value: Value '184758067' is below minimum 939524096 2024-06-06T08:40:21.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi176 to 176.2M 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: Unable to set osd_memory_target on smithi176 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:21.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:21.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:21 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:22.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: Unable to set osd_memory_target on smithi203 to 184758067: error parsing value: Value '184758067' is below minimum 939524096 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi176 to 176.2M 2024-06-06T08:40:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: Unable to set osd_memory_target on smithi176 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-06-06T08:40:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:40:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:22.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:21 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.347+0000 7fa8a5692700 1 -- 172.21.15.27:0/289657755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0107400 msgr2=0x7fa8a01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.347+0000 7fa8a5692700 1 --2- 172.21.15.27:0/289657755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0107400 0x7fa8a01077e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa890009990 tx=0x7fa890009ca0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.347+0000 7fa8a5692700 1 -- 172.21.15.27:0/289657755 shutdown_connections 2024-06-06T08:40:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.347+0000 7fa8a5692700 1 --2- 172.21.15.27:0/289657755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8a0101d50 0x7fa8a010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.347+0000 7fa8a5692700 1 --2- 172.21.15.27:0/289657755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0107400 0x7fa8a01077e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.347+0000 7fa8a5692700 1 --2- 172.21.15.27:0/289657755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8a0101390 0x7fa8a0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.347+0000 7fa8a5692700 1 -- 172.21.15.27:0/289657755 >> 172.21.15.27:0/289657755 conn(0x7fa8a0078340 msgr2=0x7fa8a0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:22.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.347+0000 7fa8a5692700 1 -- 172.21.15.27:0/289657755 shutdown_connections 2024-06-06T08:40:22.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.347+0000 7fa8a5692700 1 -- 172.21.15.27:0/289657755 wait complete. 2024-06-06T08:40:22.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.348+0000 7fa8a5692700 1 Processor -- start 2024-06-06T08:40:22.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.348+0000 7fa8a5692700 1 -- start start 2024-06-06T08:40:22.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa8a5692700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0101390 0x7fa8a0103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa8a5692700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8a0101d50 0x7fa8a01035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa8a5692700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8a0107400 0x7fa8a0103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa8a5692700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a01116a0 con 0x7fa8a0101390 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa8a5692700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a0111520 con 0x7fa8a0107400 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa8a5692700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a0111820 con 0x7fa8a0101d50 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0101390 0x7fa8a0103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0101390 0x7fa8a0103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58850/0 (socket says 172.21.15.27:58850) 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89effd700 1 -- 172.21.15.27:0/1285673932 learned_addr learned my addr 172.21.15.27:0/1285673932 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89f7fe700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8a0107400 0x7fa8a0103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa8967fc700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8a0101d50 0x7fa8a01035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89effd700 1 -- 172.21.15.27:0/1285673932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8a0101d50 msgr2=0x7fa8a01035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89effd700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8a0101d50 0x7fa8a01035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89effd700 1 -- 172.21.15.27:0/1285673932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8a0107400 msgr2=0x7fa8a0103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89effd700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8a0107400 0x7fa8a0103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89effd700 1 -- 172.21.15.27:0/1285673932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a01a7170 con 0x7fa8a0101390 2024-06-06T08:40:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89f7fe700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8a0107400 0x7fa8a0103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89effd700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0101390 0x7fa8a0103070 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa89000bae0 tx=0x7fa89000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89cff9700 1 -- 172.21.15.27:0/1285673932 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890026070 con 0x7fa8a0101390 2024-06-06T08:40:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89cff9700 1 -- 172.21.15.27:0/1285673932 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa890003d70 con 0x7fa8a0101390 2024-06-06T08:40:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a01a7400 con 0x7fa8a0101390 2024-06-06T08:40:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.349+0000 7fa89cff9700 1 -- 172.21.15.27:0/1285673932 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890021410 con 0x7fa8a0101390 2024-06-06T08:40:22.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.350+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a01a7930 con 0x7fa8a0101390 2024-06-06T08:40:22.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.350+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a0069180 con 0x7fa8a0101390 2024-06-06T08:40:22.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.353+0000 7fa89cff9700 1 -- 172.21.15.27:0/1285673932 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fa890029070 con 0x7fa8a0101390 2024-06-06T08:40:22.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.353+0000 7fa89cff9700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa88c0a2f30 0x7fa88c0a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:22.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.353+0000 7fa89cff9700 1 -- 172.21.15.27:0/1285673932 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fa8900d0c90 con 0x7fa8a0101390 2024-06-06T08:40:22.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.353+0000 7fa8967fc700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa88c0a2f30 0x7fa88c0a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:22.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.354+0000 7fa8967fc700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa88c0a2f30 0x7fa88c0a53f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0078940 tx=0x7fa888009260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:22.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.359+0000 7fa89cff9700 1 -- 172.21.15.27:0/1285673932 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa890030080 con 0x7fa8a0101390 2024-06-06T08:40:22.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.617+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8a0104550 con 0x7fa8a0101390 2024-06-06T08:40:22.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.617+0000 7fa89cff9700 1 -- 172.21.15.27:0/1285673932 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7fa8a0104550 con 0x7fa8a0101390 2024-06-06T08:40:22.618 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:22.618 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:40:22.618 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-06-06T08:40:22.618 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Unable to set osd_memory_target on smithi027 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T08:40:22.618 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Updating smithi027:/etc/ceph/ceph.conf 2024-06-06T08:40:22.618 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Updating smithi176:/etc/ceph/ceph.conf 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Updating smithi203:/etc/ceph/ceph.conf 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:22.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:22.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:22.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:22 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa88c0a2f30 msgr2=0x7fa88c0a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:22.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa88c0a2f30 0x7fa88c0a53f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0078940 tx=0x7fa888009260 comp rx=0 tx=0).stop 2024-06-06T08:40:22.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0101390 msgr2=0x7fa8a0103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:22.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0101390 0x7fa8a0103070 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa89000bae0 tx=0x7fa89000bb10 comp rx=0 tx=0).stop 2024-06-06T08:40:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 shutdown_connections 2024-06-06T08:40:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa88c0a2f30 0x7fa88c0a53f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8a0107400 0x7fa8a0103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8a0101390 0x7fa8a0103070 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 --2- 172.21.15.27:0/1285673932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8a0101d50 0x7fa8a01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 >> 172.21.15.27:0/1285673932 conn(0x7fa8a0078340 msgr2=0x7fa8a01006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 shutdown_connections 2024-06-06T08:40:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:22.621+0000 7fa8a5692700 1 -- 172.21.15.27:0/1285673932 wait complete. 2024-06-06T08:40:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:40:22.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-06-06T08:40:22.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Unable to set osd_memory_target on smithi027 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T08:40:22.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Updating smithi027:/etc/ceph/ceph.conf 2024-06-06T08:40:22.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Updating smithi176:/etc/ceph/ceph.conf 2024-06-06T08:40:22.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Updating smithi203:/etc/ceph/ceph.conf 2024-06-06T08:40:22.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:40:22.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:40:22.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:40:22.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:22.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:22 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:23.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-06-06T08:40:23.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Unable to set osd_memory_target on smithi027 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T08:40:23.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Updating smithi027:/etc/ceph/ceph.conf 2024-06-06T08:40:23.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Updating smithi176:/etc/ceph/ceph.conf 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Updating smithi203:/etc/ceph/ceph.conf 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:23.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:23.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:23.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:23.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:23.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:23.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:23.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:23.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:23.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:23.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:22 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:23.393 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-06T08:40:23.393 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-06T08:40:23.393 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T08:40:23.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 -- 172.21.15.27:0/2240387857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681079b0 msgr2=0x7fef68107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:23.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 --2- 172.21.15.27:0/2240387857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681079b0 0x7fef68107d90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fef58009900 tx=0x7fef5802a480 comp rx=0 tx=0).stop 2024-06-06T08:40:23.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 -- 172.21.15.27:0/2240387857 shutdown_connections 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 --2- 172.21.15.27:0/2240387857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef681023c0 0x7fef6810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 --2- 172.21.15.27:0/2240387857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef68101a20 0x7fef68101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 --2- 172.21.15.27:0/2240387857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681079b0 0x7fef68107d90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 -- 172.21.15.27:0/2240387857 >> 172.21.15.27:0/2240387857 conn(0x7fef68078b30 msgr2=0x7fef68078f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 -- 172.21.15.27:0/2240387857 shutdown_connections 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 -- 172.21.15.27:0/2240387857 wait complete. 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 Processor -- start 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.506+0000 7fef6daa3640 1 -- start start 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef6daa3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef68101a20 0x7fef68198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef6daa3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681023c0 0x7fef68198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef6daa3640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef681079b0 0x7fef6819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef6daa3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef68111d70 con 0x7fef681023c0 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef6daa3640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fef68111bf0 con 0x7fef681079b0 2024-06-06T08:40:23.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef6daa3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef68111ef0 con 0x7fef68101a20 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef66ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681023c0 0x7fef68198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef66ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681023c0 0x7fef68198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58870/0 (socket says 172.21.15.27:58870) 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef66ffd640 1 -- 172.21.15.27:0/41047832 learned_addr learned my addr 172.21.15.27:0/41047832 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef67fff640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef681079b0 0x7fef6819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef677fe640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef68101a20 0x7fef68198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef66ffd640 1 -- 172.21.15.27:0/41047832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef68101a20 msgr2=0x7fef68198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef66ffd640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef68101a20 0x7fef68198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef66ffd640 1 -- 172.21.15.27:0/41047832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef681079b0 msgr2=0x7fef6819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef66ffd640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef681079b0 0x7fef6819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef66ffd640 1 -- 172.21.15.27:0/41047832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6819d220 con 0x7fef681023c0 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef67fff640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef681079b0 0x7fef6819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:23.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef66ffd640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681023c0 0x7fef68198770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00d950 tx=0x7fef5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:23.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.507+0000 7fef64ff9640 1 -- 172.21.15.27:0/41047832 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c017070 con 0x7fef681023c0 2024-06-06T08:40:23.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.508+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6819d510 con 0x7fef681023c0 2024-06-06T08:40:23.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.508+0000 7fef64ff9640 1 -- 172.21.15.27:0/41047832 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef5c009c30 con 0x7fef681023c0 2024-06-06T08:40:23.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.508+0000 7fef64ff9640 1 -- 172.21.15.27:0/41047832 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00f6b0 con 0x7fef681023c0 2024-06-06T08:40:23.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.508+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef681a4da0 con 0x7fef681023c0 2024-06-06T08:40:23.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.509+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef68068c10 con 0x7fef681023c0 2024-06-06T08:40:23.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.510+0000 7fef64ff9640 1 -- 172.21.15.27:0/41047832 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fef5c0109d0 con 0x7fef681023c0 2024-06-06T08:40:23.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.511+0000 7fef64ff9640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fef500a2ea0 0x7fef500a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:23.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.511+0000 7fef677fe640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fef500a2ea0 0x7fef500a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:23.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.511+0000 7fef64ff9640 1 -- 172.21.15.27:0/41047832 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fef5c0cf7e0 con 0x7fef681023c0 2024-06-06T08:40:23.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.511+0000 7fef677fe640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fef500a2ea0 0x7fef500a5360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef58005ab0 tx=0x7fef58005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:23.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.516+0000 7fef64ff9640 1 -- 172.21.15.27:0/41047832 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef5c009de0 con 0x7fef681023c0 2024-06-06T08:40:23.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.696+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef68061de0 con 0x7fef500a2ea0 2024-06-06T08:40:23.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:23 smithi027 ceph-mon[32335]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:40:23.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:23 smithi027 ceph-mon[32335]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:40:23.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:23 smithi027 ceph-mon[32335]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:40:23.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:23 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1285673932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.706+0000 7fef64ff9640 1 -- 172.21.15.27:0/41047832 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339820 (secure 0 0 0) 0x7fef68061de0 con 0x7fef500a2ea0 2024-06-06T08:40:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:23.713 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:40:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fef500a2ea0 msgr2=0x7fef500a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fef500a2ea0 0x7fef500a5360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef58005ab0 tx=0x7fef58005a20 comp rx=0 tx=0).stop 2024-06-06T08:40:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681023c0 msgr2=0x7fef68198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681023c0 0x7fef68198770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00d950 tx=0x7fef5c00de10 comp rx=0 tx=0).stop 2024-06-06T08:40:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 shutdown_connections 2024-06-06T08:40:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fef500a2ea0 0x7fef500a5360 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef681079b0 0x7fef6819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef681023c0 0x7fef68198770 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 --2- 172.21.15.27:0/41047832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef68101a20 0x7fef68198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 >> 172.21.15.27:0/41047832 conn(0x7fef68078b30 msgr2=0x7fef68100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 shutdown_connections 2024-06-06T08:40:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.710+0000 7fef6daa3640 1 -- 172.21.15.27:0/41047832 wait complete. 2024-06-06T08:40:23.721 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-06-06T08:40:23.305590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192337,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1331,"num_read_kb":837,"num_write":487,"num_write_kb":4391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":877,"ondisk_log_size":877,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358620,"kb_used_data":13596,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583204,"statfs":{"total":1151940427776,"available":1151573200896,"internally_reserved":0,"allocated":13922304,"data_stored":10495425,"data_compressed":604544,"data_compressed_allocated":819200,"data_compressed_original":4489216,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2910,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":19,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.994628"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543717+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:40:17.543717+0000","last_peered":"2024-06-06T08:40:17.543717+0000","last_clean":"2024-06-06T08:40:17.543717+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:40:17.543717+0000","last_undegraded":"2024-06-06T08:40:17.543717+0000","last_fullsized":"2024-06-06T08:40:17.543717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603636+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:40:17.603636+0000","last_peered":"2024-06-06T08:40:17.603636+0000","last_clean":"2024-06-06T08:40:17.603636+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:40:17.603636+0000","last_undegraded":"2024-06-06T08:40:17.603636+0000","last_fullsized":"2024-06-06T08:40:17.603636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603315+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:40:17.603315+0000","last_peered":"2024-06-06T08:40:17.603315+0000","last_clean":"2024-06-06T08:40:17.603315+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:40:17.603315+0000","last_undegraded":"2024-06-06T08:40:17.603315+0000","last_fullsized":"2024-06-06T08:40:17.603315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602639+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:40:17.602639+0000","last_peered":"2024-06-06T08:40:17.602639+0000","last_clean":"2024-06-06T08:40:17.602639+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:40:17.602639+0000","last_undegraded":"2024-06-06T08:40:17.602639+0000","last_fullsized":"2024-06-06T08:40:17.602639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397130+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:40:17.397130+0000","last_peered":"2024-06-06T08:40:17.397130+0000","last_clean":"2024-06-06T08:40:17.397130+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:40:17.397130+0000","last_undegraded":"2024-06-06T08:40:17.397130+0000","last_fullsized":"2024-06-06T08:40:17.397130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406459+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:40:17.406459+0000","last_peered":"2024-06-06T08:40:17.406459+0000","last_clean":"2024-06-06T08:40:17.406459+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:40:17.406459+0000","last_undegraded":"2024-06-06T08:40:17.406459+0000","last_fullsized":"2024-06-06T08:40:17.406459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290504+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:40:17.290504+0000","last_peered":"2024-06-06T08:40:17.290504+0000","last_clean":"2024-06-06T08:40:17.290504+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:40:17.290504+0000","last_undegraded":"2024-06-06T08:40:17.290504+0000","last_fullsized":"2024-06-06T08:40:17.290504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394037+0000","last_change":"2024-06-06T08:38:01.335419+0000","last_active":"2024-06-06T08:40:17.394037+0000","last_peered":"2024-06-06T08:40:17.394037+0000","last_clean":"2024-06-06T08:40:17.394037+0000","last_became_active":"2024-06-06T08:38:01.335191+0000","last_became_peered":"2024-06-06T08:38:01.335191+0000","last_unstale":"2024-06-06T08:40:17.394037+0000","last_undegraded":"2024-06-06T08:40:17.394037+0000","last_fullsized":"2024-06-06T08:40:17.394037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:13.253838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619376+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:40:17.619376+0000","last_peered":"2024-06-06T08:40:17.619376+0000","last_clean":"2024-06-06T08:40:17.619376+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:40:17.619376+0000","last_undegraded":"2024-06-06T08:40:17.619376+0000","last_fullsized":"2024-06-06T08:40:17.619376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406777+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:40:17.406777+0000","last_peered":"2024-06-06T08:40:17.406777+0000","last_clean":"2024-06-06T08:40:17.406777+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:40:17.406777+0000","last_undegraded":"2024-06-06T08:40:17.406777+0000","last_fullsized":"2024-06-06T08:40:17.406777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290994+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:40:17.290994+0000","last_peered":"2024-06-06T08:40:17.290994+0000","last_clean":"2024-06-06T08:40:17.290994+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:40:17.290994+0000","last_undegraded":"2024-06-06T08:40:17.290994+0000","last_fullsized":"2024-06-06T08:40:17.290994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602644+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:40:17.602644+0000","last_peered":"2024-06-06T08:40:17.602644+0000","last_clean":"2024-06-06T08:40:17.602644+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:40:17.602644+0000","last_undegraded":"2024-06-06T08:40:17.602644+0000","last_fullsized":"2024-06-06T08:40:17.602644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289360+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:40:17.289360+0000","last_peered":"2024-06-06T08:40:17.289360+0000","last_clean":"2024-06-06T08:40:17.289360+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:40:17.289360+0000","last_undegraded":"2024-06-06T08:40:17.289360+0000","last_fullsized":"2024-06-06T08:40:17.289360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602964+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:40:17.602964+0000","last_peered":"2024-06-06T08:40:17.602964+0000","last_clean":"2024-06-06T08:40:17.602964+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:40:17.602964+0000","last_undegraded":"2024-06-06T08:40:17.602964+0000","last_fullsized":"2024-06-06T08:40:17.602964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603050+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:40:17.603050+0000","last_peered":"2024-06-06T08:40:17.603050+0000","last_clean":"2024-06-06T08:40:17.603050+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:40:17.603050+0000","last_undegraded":"2024-06-06T08:40:17.603050+0000","last_fullsized":"2024-06-06T08:40:17.603050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618538+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:40:17.618538+0000","last_peered":"2024-06-06T08:40:17.618538+0000","last_clean":"2024-06-06T08:40:17.618538+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:40:17.618538+0000","last_undegraded":"2024-06-06T08:40:17.618538+0000","last_fullsized":"2024-06-06T08:40:17.618538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543492+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:40:17.543492+0000","last_peered":"2024-06-06T08:40:17.543492+0000","last_clean":"2024-06-06T08:40:17.543492+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:40:17.543492+0000","last_undegraded":"2024-06-06T08:40:17.543492+0000","last_fullsized":"2024-06-06T08:40:17.543492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394947+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:40:17.394947+0000","last_peered":"2024-06-06T08:40:17.394947+0000","last_clean":"2024-06-06T08:40:17.394947+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:40:17.394947+0000","last_undegraded":"2024-06-06T08:40:17.394947+0000","last_fullsized":"2024-06-06T08:40:17.394947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291235+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:40:17.291235+0000","last_peered":"2024-06-06T08:40:17.291235+0000","last_clean":"2024-06-06T08:40:17.291235+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:40:17.291235+0000","last_undegraded":"2024-06-06T08:40:17.291235+0000","last_fullsized":"2024-06-06T08:40:17.291235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291190+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:40:17.291190+0000","last_peered":"2024-06-06T08:40:17.291190+0000","last_clean":"2024-06-06T08:40:17.291190+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:40:17.291190+0000","last_undegraded":"2024-06-06T08:40:17.291190+0000","last_fullsized":"2024-06-06T08:40:17.291190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.395159+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:40:17.395159+0000","last_peered":"2024-06-06T08:40:17.395159+0000","last_clean":"2024-06-06T08:40:17.395159+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:40:17.395159+0000","last_undegraded":"2024-06-06T08:40:17.395159+0000","last_fullsized":"2024-06-06T08:40:17.395159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397547+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:40:17.397547+0000","last_peered":"2024-06-06T08:40:17.397547+0000","last_clean":"2024-06-06T08:40:17.397547+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:40:17.397547+0000","last_undegraded":"2024-06-06T08:40:17.397547+0000","last_fullsized":"2024-06-06T08:40:17.397547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602515+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:40:17.602515+0000","last_peered":"2024-06-06T08:40:17.602515+0000","last_clean":"2024-06-06T08:40:17.602515+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:40:17.602515+0000","last_undegraded":"2024-06-06T08:40:17.602515+0000","last_fullsized":"2024-06-06T08:40:17.602515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397886+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:40:17.397886+0000","last_peered":"2024-06-06T08:40:17.397886+0000","last_clean":"2024-06-06T08:40:17.397886+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:40:17.397886+0000","last_undegraded":"2024-06-06T08:40:17.397886+0000","last_fullsized":"2024-06-06T08:40:17.397886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603637+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:40:17.603637+0000","last_peered":"2024-06-06T08:40:17.603637+0000","last_clean":"2024-06-06T08:40:17.603637+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:40:17.603637+0000","last_undegraded":"2024-06-06T08:40:17.603637+0000","last_fullsized":"2024-06-06T08:40:17.603637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602634+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:40:17.602634+0000","last_peered":"2024-06-06T08:40:17.602634+0000","last_clean":"2024-06-06T08:40:17.602634+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:40:17.602634+0000","last_undegraded":"2024-06-06T08:40:17.602634+0000","last_fullsized":"2024-06-06T08:40:17.602634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397399+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:40:17.397399+0000","last_peered":"2024-06-06T08:40:17.397399+0000","last_clean":"2024-06-06T08:40:17.397399+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:40:17.397399+0000","last_undegraded":"2024-06-06T08:40:17.397399+0000","last_fullsized":"2024-06-06T08:40:17.397399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618791+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:40:17.618791+0000","last_peered":"2024-06-06T08:40:17.618791+0000","last_clean":"2024-06-06T08:40:17.618791+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:40:17.618791+0000","last_undegraded":"2024-06-06T08:40:17.618791+0000","last_fullsized":"2024-06-06T08:40:17.618791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602522+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:40:17.602522+0000","last_peered":"2024-06-06T08:40:17.602522+0000","last_clean":"2024-06-06T08:40:17.602522+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:40:17.602522+0000","last_undegraded":"2024-06-06T08:40:17.602522+0000","last_fullsized":"2024-06-06T08:40:17.602522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397350+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:40:17.397350+0000","last_peered":"2024-06-06T08:40:17.397350+0000","last_clean":"2024-06-06T08:40:17.397350+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:40:17.397350+0000","last_undegraded":"2024-06-06T08:40:17.397350+0000","last_fullsized":"2024-06-06T08:40:17.397350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394926+0000","last_change":"2024-06-06T08:38:01.728398+0000","last_active":"2024-06-06T08:40:17.394926+0000","last_peered":"2024-06-06T08:40:17.394926+0000","last_clean":"2024-06-06T08:40:17.394926+0000","last_became_active":"2024-06-06T08:38:01.728004+0000","last_became_peered":"2024-06-06T08:38:01.728004+0000","last_unstale":"2024-06-06T08:40:17.394926+0000","last_undegraded":"2024-06-06T08:40:17.394926+0000","last_fullsized":"2024-06-06T08:40:17.394926+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:02:03.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291009+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:40:17.291009+0000","last_peered":"2024-06-06T08:40:17.291009+0000","last_clean":"2024-06-06T08:40:17.291009+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:40:17.291009+0000","last_undegraded":"2024-06-06T08:40:17.291009+0000","last_fullsized":"2024-06-06T08:40:17.291009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":232,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:18.086232+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:40:18.086232+0000","last_peered":"2024-06-06T08:40:18.086232+0000","last_clean":"2024-06-06T08:40:18.086232+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:40:18.086232+0000","last_undegraded":"2024-06-06T08:40:18.086232+0000","last_fullsized":"2024-06-06T08:40:18.086232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1320,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396976+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:40:17.396976+0000","last_peered":"2024-06-06T08:40:17.396976+0000","last_clean":"2024-06-06T08:40:17.396976+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:40:17.396976+0000","last_undegraded":"2024-06-06T08:40:17.396976+0000","last_fullsized":"2024-06-06T08:40:17.396976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290894+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:40:17.290894+0000","last_peered":"2024-06-06T08:40:17.290894+0000","last_clean":"2024-06-06T08:40:17.290894+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:40:17.290894+0000","last_undegraded":"2024-06-06T08:40:17.290894+0000","last_fullsized":"2024-06-06T08:40:17.290894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406405+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:40:17.406405+0000","last_peered":"2024-06-06T08:40:17.406405+0000","last_clean":"2024-06-06T08:40:17.406405+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:40:17.406405+0000","last_undegraded":"2024-06-06T08:40:17.406405+0000","last_fullsized":"2024-06-06T08:40:17.406405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394458+0000","last_change":"2024-06-06T08:38:01.728238+0000","last_active":"2024-06-06T08:40:17.394458+0000","last_peered":"2024-06-06T08:40:17.394458+0000","last_clean":"2024-06-06T08:40:17.394458+0000","last_became_active":"2024-06-06T08:38:01.727844+0000","last_became_peered":"2024-06-06T08:38:01.727844+0000","last_unstale":"2024-06-06T08:40:17.394458+0000","last_undegraded":"2024-06-06T08:40:17.394458+0000","last_fullsized":"2024-06-06T08:40:17.394458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:19:46.825321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543511+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:40:17.543511+0000","last_peered":"2024-06-06T08:40:17.543511+0000","last_clean":"2024-06-06T08:40:17.543511+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:40:17.543511+0000","last_undegraded":"2024-06-06T08:40:17.543511+0000","last_fullsized":"2024-06-06T08:40:17.543511+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288975+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:40:17.288975+0000","last_peered":"2024-06-06T08:40:17.288975+0000","last_clean":"2024-06-06T08:40:17.288975+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:40:17.288975+0000","last_undegraded":"2024-06-06T08:40:17.288975+0000","last_fullsized":"2024-06-06T08:40:17.288975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396897+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:40:17.396897+0000","last_peered":"2024-06-06T08:40:17.396897+0000","last_clean":"2024-06-06T08:40:17.396897+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:40:17.396897+0000","last_undegraded":"2024-06-06T08:40:17.396897+0000","last_fullsized":"2024-06-06T08:40:17.396897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603585+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:40:17.603585+0000","last_peered":"2024-06-06T08:40:17.603585+0000","last_clean":"2024-06-06T08:40:17.603585+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:40:17.603585+0000","last_undegraded":"2024-06-06T08:40:17.603585+0000","last_fullsized":"2024-06-06T08:40:17.603585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403412+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:40:17.403412+0000","last_peered":"2024-06-06T08:40:17.403412+0000","last_clean":"2024-06-06T08:40:17.403412+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:40:17.403412+0000","last_undegraded":"2024-06-06T08:40:17.403412+0000","last_fullsized":"2024-06-06T08:40:17.403412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394236+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:40:17.394236+0000","last_peered":"2024-06-06T08:40:17.394236+0000","last_clean":"2024-06-06T08:40:17.394236+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:40:17.394236+0000","last_undegraded":"2024-06-06T08:40:17.394236+0000","last_fullsized":"2024-06-06T08:40:17.394236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":230,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:22.091225+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:40:22.091225+0000","last_peered":"2024-06-06T08:40:22.091225+0000","last_clean":"2024-06-06T08:40:22.091225+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:40:22.091225+0000","last_undegraded":"2024-06-06T08:40:22.091225+0000","last_fullsized":"2024-06-06T08:40:22.091225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619308+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:40:17.619308+0000","last_peered":"2024-06-06T08:40:17.619308+0000","last_clean":"2024-06-06T08:40:17.619308+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:40:17.619308+0000","last_undegraded":"2024-06-06T08:40:17.619308+0000","last_fullsized":"2024-06-06T08:40:17.619308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394167+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:40:17.394167+0000","last_peered":"2024-06-06T08:40:17.394167+0000","last_clean":"2024-06-06T08:40:17.394167+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:40:17.394167+0000","last_undegraded":"2024-06-06T08:40:17.394167+0000","last_fullsized":"2024-06-06T08:40:17.394167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290830+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:40:17.290830+0000","last_peered":"2024-06-06T08:40:17.290830+0000","last_clean":"2024-06-06T08:40:17.290830+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:40:17.290830+0000","last_undegraded":"2024-06-06T08:40:17.290830+0000","last_fullsized":"2024-06-06T08:40:17.290830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603715+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:40:17.603715+0000","last_peered":"2024-06-06T08:40:17.603715+0000","last_clean":"2024-06-06T08:40:17.603715+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:40:17.603715+0000","last_undegraded":"2024-06-06T08:40:17.603715+0000","last_fullsized":"2024-06-06T08:40:17.603715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602684+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:40:17.602684+0000","last_peered":"2024-06-06T08:40:17.602684+0000","last_clean":"2024-06-06T08:40:17.602684+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:40:17.602684+0000","last_undegraded":"2024-06-06T08:40:17.602684+0000","last_fullsized":"2024-06-06T08:40:17.602684+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403562+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:40:17.403562+0000","last_peered":"2024-06-06T08:40:17.403562+0000","last_clean":"2024-06-06T08:40:17.403562+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:40:17.403562+0000","last_undegraded":"2024-06-06T08:40:17.403562+0000","last_fullsized":"2024-06-06T08:40:17.403562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618947+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:40:17.618947+0000","last_peered":"2024-06-06T08:40:17.618947+0000","last_clean":"2024-06-06T08:40:17.618947+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:40:17.618947+0000","last_undegraded":"2024-06-06T08:40:17.618947+0000","last_fullsized":"2024-06-06T08:40:17.618947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406287+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:40:17.406287+0000","last_peered":"2024-06-06T08:40:17.406287+0000","last_clean":"2024-06-06T08:40:17.406287+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:40:17.406287+0000","last_undegraded":"2024-06-06T08:40:17.406287+0000","last_fullsized":"2024-06-06T08:40:17.406287+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'43","reported_seq":480,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:23.092616+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:40:23.092616+0000","last_peered":"2024-06-06T08:40:23.092616+0000","last_clean":"2024-06-06T08:40:23.092616+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:40:23.092616+0000","last_undegraded":"2024-06-06T08:40:23.092616+0000","last_fullsized":"2024-06-06T08:40:23.092616+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71690,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":374,"num_read_kb":258,"num_write":44,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397172+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:40:17.397172+0000","last_peered":"2024-06-06T08:40:17.397172+0000","last_clean":"2024-06-06T08:40:17.397172+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:40:17.397172+0000","last_undegraded":"2024-06-06T08:40:17.397172+0000","last_fullsized":"2024-06-06T08:40:17.397172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291329+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:40:17.291329+0000","last_peered":"2024-06-06T08:40:17.291329+0000","last_clean":"2024-06-06T08:40:17.291329+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:40:17.291329+0000","last_undegraded":"2024-06-06T08:40:17.291329+0000","last_fullsized":"2024-06-06T08:40:17.291329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289082+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:40:17.289082+0000","last_peered":"2024-06-06T08:40:17.289082+0000","last_clean":"2024-06-06T08:40:17.289082+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:40:17.289082+0000","last_undegraded":"2024-06-06T08:40:17.289082+0000","last_fullsized":"2024-06-06T08:40:17.289082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543337+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:40:17.543337+0000","last_peered":"2024-06-06T08:40:17.543337+0000","last_clean":"2024-06-06T08:40:17.543337+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:40:17.543337+0000","last_undegraded":"2024-06-06T08:40:17.543337+0000","last_fullsized":"2024-06-06T08:40:17.543337+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403613+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:40:17.403613+0000","last_peered":"2024-06-06T08:40:17.403613+0000","last_clean":"2024-06-06T08:40:17.403613+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:40:17.403613+0000","last_undegraded":"2024-06-06T08:40:17.403613+0000","last_fullsized":"2024-06-06T08:40:17.403613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290581+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:40:17.290581+0000","last_peered":"2024-06-06T08:40:17.290581+0000","last_clean":"2024-06-06T08:40:17.290581+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:40:17.290581+0000","last_undegraded":"2024-06-06T08:40:17.290581+0000","last_fullsized":"2024-06-06T08:40:17.290581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394932+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:40:17.394932+0000","last_peered":"2024-06-06T08:40:17.394932+0000","last_clean":"2024-06-06T08:40:17.394932+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:40:17.394932+0000","last_undegraded":"2024-06-06T08:40:17.394932+0000","last_fullsized":"2024-06-06T08:40:17.394932+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'6","reported_seq":232,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:18.085943+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:40:18.085943+0000","last_peered":"2024-06-06T08:40:18.085943+0000","last_clean":"2024-06-06T08:40:18.085943+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:40:18.085943+0000","last_undegraded":"2024-06-06T08:40:18.085943+0000","last_fullsized":"2024-06-06T08:40:18.085943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290588+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:40:17.290588+0000","last_peered":"2024-06-06T08:40:17.290588+0000","last_clean":"2024-06-06T08:40:17.290588+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:40:17.290588+0000","last_undegraded":"2024-06-06T08:40:17.290588+0000","last_fullsized":"2024-06-06T08:40:17.290588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289726+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:40:17.289726+0000","last_peered":"2024-06-06T08:40:17.289726+0000","last_clean":"2024-06-06T08:40:17.289726+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:40:17.289726+0000","last_undegraded":"2024-06-06T08:40:17.289726+0000","last_fullsized":"2024-06-06T08:40:17.289726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396877+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:40:17.396877+0000","last_peered":"2024-06-06T08:40:17.396877+0000","last_clean":"2024-06-06T08:40:17.396877+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:40:17.396877+0000","last_undegraded":"2024-06-06T08:40:17.396877+0000","last_fullsized":"2024-06-06T08:40:17.396877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403466+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:40:17.403466+0000","last_peered":"2024-06-06T08:40:17.403466+0000","last_clean":"2024-06-06T08:40:17.403466+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:40:17.403466+0000","last_undegraded":"2024-06-06T08:40:17.403466+0000","last_fullsized":"2024-06-06T08:40:17.403466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403457+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:40:17.403457+0000","last_peered":"2024-06-06T08:40:17.403457+0000","last_clean":"2024-06-06T08:40:17.403457+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:40:17.403457+0000","last_undegraded":"2024-06-06T08:40:17.403457+0000","last_fullsized":"2024-06-06T08:40:17.403457+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"100'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397313+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:40:17.397313+0000","last_peered":"2024-06-06T08:40:17.397313+0000","last_clean":"2024-06-06T08:40:17.397313+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:40:17.397313+0000","last_undegraded":"2024-06-06T08:40:17.397313+0000","last_fullsized":"2024-06-06T08:40:17.397313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543570+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:40:17.543570+0000","last_peered":"2024-06-06T08:40:17.543570+0000","last_clean":"2024-06-06T08:40:17.543570+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:40:17.543570+0000","last_undegraded":"2024-06-06T08:40:17.543570+0000","last_fullsized":"2024-06-06T08:40:17.543570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394732+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:40:17.394732+0000","last_peered":"2024-06-06T08:40:17.394732+0000","last_clean":"2024-06-06T08:40:17.394732+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:40:17.394732+0000","last_undegraded":"2024-06-06T08:40:17.394732+0000","last_fullsized":"2024-06-06T08:40:17.394732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406540+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:40:17.406540+0000","last_peered":"2024-06-06T08:40:17.406540+0000","last_clean":"2024-06-06T08:40:17.406540+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:40:17.406540+0000","last_undegraded":"2024-06-06T08:40:17.406540+0000","last_fullsized":"2024-06-06T08:40:17.406540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289289+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:40:17.289289+0000","last_peered":"2024-06-06T08:40:17.289289+0000","last_clean":"2024-06-06T08:40:17.289289+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:40:17.289289+0000","last_undegraded":"2024-06-06T08:40:17.289289+0000","last_fullsized":"2024-06-06T08:40:17.289289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290584+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:40:17.290584+0000","last_peered":"2024-06-06T08:40:17.290584+0000","last_clean":"2024-06-06T08:40:17.290584+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:40:17.290584+0000","last_undegraded":"2024-06-06T08:40:17.290584+0000","last_fullsized":"2024-06-06T08:40:17.290584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406420+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:40:17.406420+0000","last_peered":"2024-06-06T08:40:17.406420+0000","last_clean":"2024-06-06T08:40:17.406420+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:40:17.406420+0000","last_undegraded":"2024-06-06T08:40:17.406420+0000","last_fullsized":"2024-06-06T08:40:17.406420+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289488+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:40:17.289488+0000","last_peered":"2024-06-06T08:40:17.289488+0000","last_clean":"2024-06-06T08:40:17.289488+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:40:17.289488+0000","last_undegraded":"2024-06-06T08:40:17.289488+0000","last_fullsized":"2024-06-06T08:40:17.289488+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403779+0000","last_change":"2024-06-06T08:38:01.545781+0000","last_active":"2024-06-06T08:40:17.403779+0000","last_peered":"2024-06-06T08:40:17.403779+0000","last_clean":"2024-06-06T08:40:17.403779+0000","last_became_active":"2024-06-06T08:38:01.545462+0000","last_became_peered":"2024-06-06T08:38:01.545462+0000","last_unstale":"2024-06-06T08:40:17.403779+0000","last_undegraded":"2024-06-06T08:40:17.403779+0000","last_fullsized":"2024-06-06T08:40:17.403779+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:14:36.722960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":232,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:22.091205+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:40:22.091205+0000","last_peered":"2024-06-06T08:40:22.091205+0000","last_clean":"2024-06-06T08:40:22.091205+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:40:22.091205+0000","last_undegraded":"2024-06-06T08:40:22.091205+0000","last_fullsized":"2024-06-06T08:40:22.091205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290959+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:40:17.290959+0000","last_peered":"2024-06-06T08:40:17.290959+0000","last_clean":"2024-06-06T08:40:17.290959+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:40:17.290959+0000","last_undegraded":"2024-06-06T08:40:17.290959+0000","last_fullsized":"2024-06-06T08:40:17.290959+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397121+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:40:17.397121+0000","last_peered":"2024-06-06T08:40:17.397121+0000","last_clean":"2024-06-06T08:40:17.397121+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:40:17.397121+0000","last_undegraded":"2024-06-06T08:40:17.397121+0000","last_fullsized":"2024-06-06T08:40:17.397121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602368+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:40:17.602368+0000","last_peered":"2024-06-06T08:40:17.602368+0000","last_clean":"2024-06-06T08:40:17.602368+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:40:17.602368+0000","last_undegraded":"2024-06-06T08:40:17.602368+0000","last_fullsized":"2024-06-06T08:40:17.602368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":378,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290379+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:40:17.290379+0000","last_peered":"2024-06-06T08:40:17.290379+0000","last_clean":"2024-06-06T08:40:17.290379+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:40:17.290379+0000","last_undegraded":"2024-06-06T08:40:17.290379+0000","last_fullsized":"2024-06-06T08:40:17.290379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288818+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:40:17.288818+0000","last_peered":"2024-06-06T08:40:17.288818+0000","last_clean":"2024-06-06T08:40:17.288818+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:40:17.288818+0000","last_undegraded":"2024-06-06T08:40:17.288818+0000","last_fullsized":"2024-06-06T08:40:17.288818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543213+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:40:17.543213+0000","last_peered":"2024-06-06T08:40:17.543213+0000","last_clean":"2024-06-06T08:40:17.543213+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:40:17.543213+0000","last_undegraded":"2024-06-06T08:40:17.543213+0000","last_fullsized":"2024-06-06T08:40:17.543213+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543287+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:40:17.543287+0000","last_peered":"2024-06-06T08:40:17.543287+0000","last_clean":"2024-06-06T08:40:17.543287+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:40:17.543287+0000","last_undegraded":"2024-06-06T08:40:17.543287+0000","last_fullsized":"2024-06-06T08:40:17.543287+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543496+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:40:17.543496+0000","last_peered":"2024-06-06T08:40:17.543496+0000","last_clean":"2024-06-06T08:40:17.543496+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:40:17.543496+0000","last_undegraded":"2024-06-06T08:40:17.543496+0000","last_fullsized":"2024-06-06T08:40:17.543496+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289481+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:40:17.289481+0000","last_peered":"2024-06-06T08:40:17.289481+0000","last_clean":"2024-06-06T08:40:17.289481+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:40:17.289481+0000","last_undegraded":"2024-06-06T08:40:17.289481+0000","last_fullsized":"2024-06-06T08:40:17.289481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397142+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:40:17.397142+0000","last_peered":"2024-06-06T08:40:17.397142+0000","last_clean":"2024-06-06T08:40:17.397142+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:40:17.397142+0000","last_undegraded":"2024-06-06T08:40:17.397142+0000","last_fullsized":"2024-06-06T08:40:17.397142+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'49","reported_seq":119,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.518606+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:40:17.518606+0000","last_peered":"2024-06-06T08:40:17.518606+0000","last_clean":"2024-06-06T08:40:17.518606+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:40:17.518606+0000","last_undegraded":"2024-06-06T08:40:17.518606+0000","last_fullsized":"2024-06-06T08:40:17.518606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397610+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:40:17.397610+0000","last_peered":"2024-06-06T08:40:17.397610+0000","last_clean":"2024-06-06T08:40:17.397610+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:40:17.397610+0000","last_undegraded":"2024-06-06T08:40:17.397610+0000","last_fullsized":"2024-06-06T08:40:17.397610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290756+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:40:17.290756+0000","last_peered":"2024-06-06T08:40:17.290756+0000","last_clean":"2024-06-06T08:40:17.290756+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:40:17.290756+0000","last_undegraded":"2024-06-06T08:40:17.290756+0000","last_fullsized":"2024-06-06T08:40:17.290756+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603067+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:40:17.603067+0000","last_peered":"2024-06-06T08:40:17.603067+0000","last_clean":"2024-06-06T08:40:17.603067+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:40:17.603067+0000","last_undegraded":"2024-06-06T08:40:17.603067+0000","last_fullsized":"2024-06-06T08:40:17.603067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603700+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:40:17.603700+0000","last_peered":"2024-06-06T08:40:17.603700+0000","last_clean":"2024-06-06T08:40:17.603700+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:40:17.603700+0000","last_undegraded":"2024-06-06T08:40:17.603700+0000","last_fullsized":"2024-06-06T08:40:17.603700+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602722+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:40:17.602722+0000","last_peered":"2024-06-06T08:40:17.602722+0000","last_clean":"2024-06-06T08:40:17.602722+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:40:17.602722+0000","last_undegraded":"2024-06-06T08:40:17.602722+0000","last_fullsized":"2024-06-06T08:40:17.602722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618369+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:40:17.618369+0000","last_peered":"2024-06-06T08:40:17.618369+0000","last_clean":"2024-06-06T08:40:17.618369+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:40:17.618369+0000","last_undegraded":"2024-06-06T08:40:17.618369+0000","last_fullsized":"2024-06-06T08:40:17.618369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396892+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:40:17.396892+0000","last_peered":"2024-06-06T08:40:17.396892+0000","last_clean":"2024-06-06T08:40:17.396892+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:40:17.396892+0000","last_undegraded":"2024-06-06T08:40:17.396892+0000","last_fullsized":"2024-06-06T08:40:17.396892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.395030+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:40:17.395030+0000","last_peered":"2024-06-06T08:40:17.395030+0000","last_clean":"2024-06-06T08:40:17.395030+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:40:17.395030+0000","last_undegraded":"2024-06-06T08:40:17.395030+0000","last_fullsized":"2024-06-06T08:40:17.395030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397132+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:40:17.397132+0000","last_peered":"2024-06-06T08:40:17.397132+0000","last_clean":"2024-06-06T08:40:17.397132+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:40:17.397132+0000","last_undegraded":"2024-06-06T08:40:17.397132+0000","last_fullsized":"2024-06-06T08:40:17.397132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543302+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:40:17.543302+0000","last_peered":"2024-06-06T08:40:17.543302+0000","last_clean":"2024-06-06T08:40:17.543302+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:40:17.543302+0000","last_undegraded":"2024-06-06T08:40:17.543302+0000","last_fullsized":"2024-06-06T08:40:17.543302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602805+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:40:17.602805+0000","last_peered":"2024-06-06T08:40:17.602805+0000","last_clean":"2024-06-06T08:40:17.602805+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:40:17.602805+0000","last_undegraded":"2024-06-06T08:40:17.602805+0000","last_fullsized":"2024-06-06T08:40:17.602805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291108+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:40:17.291108+0000","last_peered":"2024-06-06T08:40:17.291108+0000","last_clean":"2024-06-06T08:40:17.291108+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:40:17.291108+0000","last_undegraded":"2024-06-06T08:40:17.291108+0000","last_fullsized":"2024-06-06T08:40:17.291108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396937+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:40:17.396937+0000","last_peered":"2024-06-06T08:40:17.396937+0000","last_clean":"2024-06-06T08:40:17.396937+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:40:17.396937+0000","last_undegraded":"2024-06-06T08:40:17.396937+0000","last_fullsized":"2024-06-06T08:40:17.396937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602730+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:40:17.602730+0000","last_peered":"2024-06-06T08:40:17.602730+0000","last_clean":"2024-06-06T08:40:17.602730+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:40:17.602730+0000","last_undegraded":"2024-06-06T08:40:17.602730+0000","last_fullsized":"2024-06-06T08:40:17.602730+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396987+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:40:17.396987+0000","last_peered":"2024-06-06T08:40:17.396987+0000","last_clean":"2024-06-06T08:40:17.396987+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:40:17.396987+0000","last_undegraded":"2024-06-06T08:40:17.396987+0000","last_fullsized":"2024-06-06T08:40:17.396987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290632+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:40:17.290632+0000","last_peered":"2024-06-06T08:40:17.290632+0000","last_clean":"2024-06-06T08:40:17.290632+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:40:17.290632+0000","last_undegraded":"2024-06-06T08:40:17.290632+0000","last_fullsized":"2024-06-06T08:40:17.290632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618976+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:40:17.618976+0000","last_peered":"2024-06-06T08:40:17.618976+0000","last_clean":"2024-06-06T08:40:17.618976+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:40:17.618976+0000","last_undegraded":"2024-06-06T08:40:17.618976+0000","last_fullsized":"2024-06-06T08:40:17.618976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291175+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:40:17.291175+0000","last_peered":"2024-06-06T08:40:17.291175+0000","last_clean":"2024-06-06T08:40:17.291175+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:40:17.291175+0000","last_undegraded":"2024-06-06T08:40:17.291175+0000","last_fullsized":"2024-06-06T08:40:17.291175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291116+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:40:17.291116+0000","last_peered":"2024-06-06T08:40:17.291116+0000","last_clean":"2024-06-06T08:40:17.291116+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:40:17.291116+0000","last_undegraded":"2024-06-06T08:40:17.291116+0000","last_fullsized":"2024-06-06T08:40:17.291116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603292+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:40:17.603292+0000","last_peered":"2024-06-06T08:40:17.603292+0000","last_clean":"2024-06-06T08:40:17.603292+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:40:17.603292+0000","last_undegraded":"2024-06-06T08:40:17.603292+0000","last_fullsized":"2024-06-06T08:40:17.603292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'39","reported_seq":94,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.513053+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:40:17.513053+0000","last_peered":"2024-06-06T08:40:17.513053+0000","last_clean":"2024-06-06T08:40:17.513053+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:40:17.513053+0000","last_undegraded":"2024-06-06T08:40:17.513053+0000","last_fullsized":"2024-06-06T08:40:17.513053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290659+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:40:17.290659+0000","last_peered":"2024-06-06T08:40:17.290659+0000","last_clean":"2024-06-06T08:40:17.290659+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:40:17.290659+0000","last_undegraded":"2024-06-06T08:40:17.290659+0000","last_fullsized":"2024-06-06T08:40:17.290659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403261+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:40:17.403261+0000","last_peered":"2024-06-06T08:40:17.403261+0000","last_clean":"2024-06-06T08:40:17.403261+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:40:17.403261+0000","last_undegraded":"2024-06-06T08:40:17.403261+0000","last_fullsized":"2024-06-06T08:40:17.403261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289754+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:40:17.289754+0000","last_peered":"2024-06-06T08:40:17.289754+0000","last_clean":"2024-06-06T08:40:17.289754+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:40:17.289754+0000","last_undegraded":"2024-06-06T08:40:17.289754+0000","last_fullsized":"2024-06-06T08:40:17.289754+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291007+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:40:17.291007+0000","last_peered":"2024-06-06T08:40:17.291007+0000","last_clean":"2024-06-06T08:40:17.291007+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:40:17.291007+0000","last_undegraded":"2024-06-06T08:40:17.291007+0000","last_fullsized":"2024-06-06T08:40:17.291007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543466+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:40:17.543466+0000","last_peered":"2024-06-06T08:40:17.543466+0000","last_clean":"2024-06-06T08:40:17.543466+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:40:17.543466+0000","last_undegraded":"2024-06-06T08:40:17.543466+0000","last_fullsized":"2024-06-06T08:40:17.543466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291000+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:40:17.291000+0000","last_peered":"2024-06-06T08:40:17.291000+0000","last_clean":"2024-06-06T08:40:17.291000+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:40:17.291000+0000","last_undegraded":"2024-06-06T08:40:17.291000+0000","last_fullsized":"2024-06-06T08:40:17.291000+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290895+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:40:17.290895+0000","last_peered":"2024-06-06T08:40:17.290895+0000","last_clean":"2024-06-06T08:40:17.290895+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:40:17.290895+0000","last_undegraded":"2024-06-06T08:40:17.290895+0000","last_fullsized":"2024-06-06T08:40:17.290895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603342+0000","last_change":"2024-06-06T08:38:01.151003+0000","last_active":"2024-06-06T08:40:17.603342+0000","last_peered":"2024-06-06T08:40:17.603342+0000","last_clean":"2024-06-06T08:40:17.603342+0000","last_became_active":"2024-06-06T08:38:01.150557+0000","last_became_peered":"2024-06-06T08:38:01.150557+0000","last_unstale":"2024-06-06T08:40:17.603342+0000","last_undegraded":"2024-06-06T08:40:17.603342+0000","last_fullsized":"2024-06-06T08:40:17.603342+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:29:44.629894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289585+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:40:17.289585+0000","last_peered":"2024-06-06T08:40:17.289585+0000","last_clean":"2024-06-06T08:40:17.289585+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:40:17.289585+0000","last_undegraded":"2024-06-06T08:40:17.289585+0000","last_fullsized":"2024-06-06T08:40:17.289585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543470+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:40:17.543470+0000","last_peered":"2024-06-06T08:40:17.543470+0000","last_clean":"2024-06-06T08:40:17.543470+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:40:17.543470+0000","last_undegraded":"2024-06-06T08:40:17.543470+0000","last_fullsized":"2024-06-06T08:40:17.543470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291317+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:40:17.291317+0000","last_peered":"2024-06-06T08:40:17.291317+0000","last_clean":"2024-06-06T08:40:17.291317+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:40:17.291317+0000","last_undegraded":"2024-06-06T08:40:17.291317+0000","last_fullsized":"2024-06-06T08:40:17.291317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397744+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:40:17.397744+0000","last_peered":"2024-06-06T08:40:17.397744+0000","last_clean":"2024-06-06T08:40:17.397744+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:40:17.397744+0000","last_undegraded":"2024-06-06T08:40:17.397744+0000","last_fullsized":"2024-06-06T08:40:17.397744+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403480+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:40:17.403480+0000","last_peered":"2024-06-06T08:40:17.403480+0000","last_clean":"2024-06-06T08:40:17.403480+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:40:17.403480+0000","last_undegraded":"2024-06-06T08:40:17.403480+0000","last_fullsized":"2024-06-06T08:40:17.403480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397272+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:40:17.397272+0000","last_peered":"2024-06-06T08:40:17.397272+0000","last_clean":"2024-06-06T08:40:17.397272+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:40:17.397272+0000","last_undegraded":"2024-06-06T08:40:17.397272+0000","last_fullsized":"2024-06-06T08:40:17.397272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618845+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:40:17.618845+0000","last_peered":"2024-06-06T08:40:17.618845+0000","last_clean":"2024-06-06T08:40:17.618845+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:40:17.618845+0000","last_undegraded":"2024-06-06T08:40:17.618845+0000","last_fullsized":"2024-06-06T08:40:17.618845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289213+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:40:17.289213+0000","last_peered":"2024-06-06T08:40:17.289213+0000","last_clean":"2024-06-06T08:40:17.289213+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:40:17.289213+0000","last_undegraded":"2024-06-06T08:40:17.289213+0000","last_fullsized":"2024-06-06T08:40:17.289213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543444+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:40:17.543444+0000","last_peered":"2024-06-06T08:40:17.543444+0000","last_clean":"2024-06-06T08:40:17.543444+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:40:17.543444+0000","last_undegraded":"2024-06-06T08:40:17.543444+0000","last_fullsized":"2024-06-06T08:40:17.543444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107163,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601735,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1195,"num_read_kb":718,"num_write":304,"num_write_kb":1451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":1799775,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383743,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":569943,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645382,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32428,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712724,"statfs":{"total":95995035648,"available":95961829376,"internally_reserved":0,"allocated":1282048,"data_stored":1371722,"data_compressed":77234,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1150976,"data_stored":977401,"data_compressed":54503,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070543,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1142784,"data_stored":973203,"data_compressed":54503,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332180,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1171456,"data_stored":1177146,"data_compressed":71078,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495707,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":575265,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1175552,"data_stored":643955,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":576653,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577223,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1100275,"data_compressed":60659,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920868,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1191936,"data_stored":649616,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757342,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":1303023,"data_compressed":77395,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133774,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":206179,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:40:23.727 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T08:40:23.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.866+0000 7f29bb491640 1 -- 172.21.15.27:0/3091390655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 msgr2=0x7f29b410e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.866+0000 7f29bb491640 1 --2- 172.21.15.27:0/3091390655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 0x7f29b410e410 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009ca0 tx=0x7f29ac02a9e0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 -- 172.21.15.27:0/3091390655 shutdown_connections 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 --2- 172.21.15.27:0/3091390655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 0x7f29b410e410 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 --2- 172.21.15.27:0/3091390655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29b4103550 0x7f29b410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 --2- 172.21.15.27:0/3091390655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b4102c30 0x7f29b4103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 -- 172.21.15.27:0/3091390655 >> 172.21.15.27:0/3091390655 conn(0x7f29b40779a0 msgr2=0x7f29b4077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 -- 172.21.15.27:0/3091390655 shutdown_connections 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 -- 172.21.15.27:0/3091390655 wait complete. 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 Processor -- start 2024-06-06T08:40:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 -- start start 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b4102c30 0x7f29b4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29b4103550 0x7f29b4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 0x7f29b419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f29b4111d70 con 0x7f29b410bf70 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f29b4111bf0 con 0x7f29b4103550 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.867+0000 7f29bb491640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29b4111ef0 con 0x7f29b4102c30 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b9a2e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 0x7f29b419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b9a2e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 0x7f29b419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58898/0 (socket says 172.21.15.27:58898) 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b9a2e640 1 -- 172.21.15.27:0/1219577391 learned_addr learned my addr 172.21.15.27:0/1219577391 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b922d640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b4102c30 0x7f29b4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b8a2c640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29b4103550 0x7f29b4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b9a2e640 1 -- 172.21.15.27:0/1219577391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b4102c30 msgr2=0x7f29b4198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b9a2e640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b4102c30 0x7f29b4198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b9a2e640 1 -- 172.21.15.27:0/1219577391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29b4103550 msgr2=0x7f29b4198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b9a2e640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29b4103550 0x7f29b4198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b9a2e640 1 -- 172.21.15.27:0/1219577391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b419d1d0 con 0x7f29b410bf70 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29b9a2e640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 0x7f29b419caa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f29ac006010 tx=0x7f29ac003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29aa7fc640 1 -- 172.21.15.27:0/1219577391 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac047070 con 0x7f29b410bf70 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29aa7fc640 1 -- 172.21.15.27:0/1219577391 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29ac003e50 con 0x7f29b410bf70 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b419d460 con 0x7f29b410bf70 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29aa7fc640 1 -- 172.21.15.27:0/1219577391 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac033050 con 0x7f29b410bf70 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.868+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b41a4d50 con 0x7f29b410bf70 2024-06-06T08:40:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.869+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b40687c0 con 0x7f29b410bf70 2024-06-06T08:40:23.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.871+0000 7f29aa7fc640 1 -- 172.21.15.27:0/1219577391 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f29ac04a070 con 0x7f29b410bf70 2024-06-06T08:40:23.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.872+0000 7f29aa7fc640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29a00a2ea0 0x7f29a00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:23.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.872+0000 7f29aa7fc640 1 -- 172.21.15.27:0/1219577391 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f29ac0f0ad0 con 0x7f29b410bf70 2024-06-06T08:40:23.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.872+0000 7f29b922d640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29a00a2ea0 0x7f29a00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:23.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.872+0000 7f29b922d640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29a00a2ea0 0x7f29a00a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29b000bbe0 tx=0x7f29b000b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:23.875+0000 7f29aa7fc640 1 -- 172.21.15.27:0/1219577391 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f29ac051080 con 0x7f29b410bf70 2024-06-06T08:40:23.984 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:23 smithi176 ceph-mon[36786]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:40:23.984 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:23 smithi176 ceph-mon[36786]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:40:23.984 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:23 smithi176 ceph-mon[36786]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:40:23.984 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:23 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1285673932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:23.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:23 smithi203 ceph-mon[36579]: Updating smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:40:23.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:23 smithi203 ceph-mon[36579]: Updating smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:40:23.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:23 smithi203 ceph-mon[36579]: Updating smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T08:40:23.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:23 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1285673932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.060+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29b4061de0 con 0x7f29a00a2ea0 2024-06-06T08:40:24.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.070+0000 7f29aa7fc640 1 -- 172.21.15.27:0/1219577391 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339820 (secure 0 0 0) 0x7f29b4061de0 con 0x7f29a00a2ea0 2024-06-06T08:40:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:24.074 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:40:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.073+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29a00a2ea0 msgr2=0x7f29a00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.073+0000 7f29bb491640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29a00a2ea0 0x7f29a00a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29b000bbe0 tx=0x7f29b000b390 comp rx=0 tx=0).stop 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.073+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 msgr2=0x7f29b419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.073+0000 7f29bb491640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 0x7f29b419caa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f29ac006010 tx=0x7f29ac003ce0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.073+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 shutdown_connections 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.073+0000 7f29bb491640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f29a00a2ea0 0x7f29a00a5360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.073+0000 7f29bb491640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29b410bf70 0x7f29b419caa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.073+0000 7f29bb491640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f29b4103550 0x7f29b4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.074+0000 7f29bb491640 1 --2- 172.21.15.27:0/1219577391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29b4102c30 0x7f29b4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.074+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 >> 172.21.15.27:0/1219577391 conn(0x7f29b40779a0 msgr2=0x7f29b40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.074+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 shutdown_connections 2024-06-06T08:40:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.074+0000 7f29bb491640 1 -- 172.21.15.27:0/1219577391 wait complete. 2024-06-06T08:40:24.083 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-06-06T08:40:23.305590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192337,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1331,"num_read_kb":837,"num_write":487,"num_write_kb":4391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":877,"ondisk_log_size":877,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358620,"kb_used_data":13596,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583204,"statfs":{"total":1151940427776,"available":1151573200896,"internally_reserved":0,"allocated":13922304,"data_stored":10495425,"data_compressed":604544,"data_compressed_allocated":819200,"data_compressed_original":4489216,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2910,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":19,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.994628"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543717+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:40:17.543717+0000","last_peered":"2024-06-06T08:40:17.543717+0000","last_clean":"2024-06-06T08:40:17.543717+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:40:17.543717+0000","last_undegraded":"2024-06-06T08:40:17.543717+0000","last_fullsized":"2024-06-06T08:40:17.543717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603636+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:40:17.603636+0000","last_peered":"2024-06-06T08:40:17.603636+0000","last_clean":"2024-06-06T08:40:17.603636+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:40:17.603636+0000","last_undegraded":"2024-06-06T08:40:17.603636+0000","last_fullsized":"2024-06-06T08:40:17.603636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603315+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:40:17.603315+0000","last_peered":"2024-06-06T08:40:17.603315+0000","last_clean":"2024-06-06T08:40:17.603315+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:40:17.603315+0000","last_undegraded":"2024-06-06T08:40:17.603315+0000","last_fullsized":"2024-06-06T08:40:17.603315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602639+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:40:17.602639+0000","last_peered":"2024-06-06T08:40:17.602639+0000","last_clean":"2024-06-06T08:40:17.602639+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:40:17.602639+0000","last_undegraded":"2024-06-06T08:40:17.602639+0000","last_fullsized":"2024-06-06T08:40:17.602639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397130+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:40:17.397130+0000","last_peered":"2024-06-06T08:40:17.397130+0000","last_clean":"2024-06-06T08:40:17.397130+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:40:17.397130+0000","last_undegraded":"2024-06-06T08:40:17.397130+0000","last_fullsized":"2024-06-06T08:40:17.397130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406459+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:40:17.406459+0000","last_peered":"2024-06-06T08:40:17.406459+0000","last_clean":"2024-06-06T08:40:17.406459+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:40:17.406459+0000","last_undegraded":"2024-06-06T08:40:17.406459+0000","last_fullsized":"2024-06-06T08:40:17.406459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290504+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:40:17.290504+0000","last_peered":"2024-06-06T08:40:17.290504+0000","last_clean":"2024-06-06T08:40:17.290504+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:40:17.290504+0000","last_undegraded":"2024-06-06T08:40:17.290504+0000","last_fullsized":"2024-06-06T08:40:17.290504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394037+0000","last_change":"2024-06-06T08:38:01.335419+0000","last_active":"2024-06-06T08:40:17.394037+0000","last_peered":"2024-06-06T08:40:17.394037+0000","last_clean":"2024-06-06T08:40:17.394037+0000","last_became_active":"2024-06-06T08:38:01.335191+0000","last_became_peered":"2024-06-06T08:38:01.335191+0000","last_unstale":"2024-06-06T08:40:17.394037+0000","last_undegraded":"2024-06-06T08:40:17.394037+0000","last_fullsized":"2024-06-06T08:40:17.394037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:13.253838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619376+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:40:17.619376+0000","last_peered":"2024-06-06T08:40:17.619376+0000","last_clean":"2024-06-06T08:40:17.619376+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:40:17.619376+0000","last_undegraded":"2024-06-06T08:40:17.619376+0000","last_fullsized":"2024-06-06T08:40:17.619376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406777+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:40:17.406777+0000","last_peered":"2024-06-06T08:40:17.406777+0000","last_clean":"2024-06-06T08:40:17.406777+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:40:17.406777+0000","last_undegraded":"2024-06-06T08:40:17.406777+0000","last_fullsized":"2024-06-06T08:40:17.406777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290994+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:40:17.290994+0000","last_peered":"2024-06-06T08:40:17.290994+0000","last_clean":"2024-06-06T08:40:17.290994+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:40:17.290994+0000","last_undegraded":"2024-06-06T08:40:17.290994+0000","last_fullsized":"2024-06-06T08:40:17.290994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602644+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:40:17.602644+0000","last_peered":"2024-06-06T08:40:17.602644+0000","last_clean":"2024-06-06T08:40:17.602644+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:40:17.602644+0000","last_undegraded":"2024-06-06T08:40:17.602644+0000","last_fullsized":"2024-06-06T08:40:17.602644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289360+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:40:17.289360+0000","last_peered":"2024-06-06T08:40:17.289360+0000","last_clean":"2024-06-06T08:40:17.289360+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:40:17.289360+0000","last_undegraded":"2024-06-06T08:40:17.289360+0000","last_fullsized":"2024-06-06T08:40:17.289360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602964+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:40:17.602964+0000","last_peered":"2024-06-06T08:40:17.602964+0000","last_clean":"2024-06-06T08:40:17.602964+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:40:17.602964+0000","last_undegraded":"2024-06-06T08:40:17.602964+0000","last_fullsized":"2024-06-06T08:40:17.602964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603050+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:40:17.603050+0000","last_peered":"2024-06-06T08:40:17.603050+0000","last_clean":"2024-06-06T08:40:17.603050+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:40:17.603050+0000","last_undegraded":"2024-06-06T08:40:17.603050+0000","last_fullsized":"2024-06-06T08:40:17.603050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618538+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:40:17.618538+0000","last_peered":"2024-06-06T08:40:17.618538+0000","last_clean":"2024-06-06T08:40:17.618538+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:40:17.618538+0000","last_undegraded":"2024-06-06T08:40:17.618538+0000","last_fullsized":"2024-06-06T08:40:17.618538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543492+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:40:17.543492+0000","last_peered":"2024-06-06T08:40:17.543492+0000","last_clean":"2024-06-06T08:40:17.543492+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:40:17.543492+0000","last_undegraded":"2024-06-06T08:40:17.543492+0000","last_fullsized":"2024-06-06T08:40:17.543492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394947+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:40:17.394947+0000","last_peered":"2024-06-06T08:40:17.394947+0000","last_clean":"2024-06-06T08:40:17.394947+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:40:17.394947+0000","last_undegraded":"2024-06-06T08:40:17.394947+0000","last_fullsized":"2024-06-06T08:40:17.394947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291235+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:40:17.291235+0000","last_peered":"2024-06-06T08:40:17.291235+0000","last_clean":"2024-06-06T08:40:17.291235+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:40:17.291235+0000","last_undegraded":"2024-06-06T08:40:17.291235+0000","last_fullsized":"2024-06-06T08:40:17.291235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291190+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:40:17.291190+0000","last_peered":"2024-06-06T08:40:17.291190+0000","last_clean":"2024-06-06T08:40:17.291190+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:40:17.291190+0000","last_undegraded":"2024-06-06T08:40:17.291190+0000","last_fullsized":"2024-06-06T08:40:17.291190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.395159+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:40:17.395159+0000","last_peered":"2024-06-06T08:40:17.395159+0000","last_clean":"2024-06-06T08:40:17.395159+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:40:17.395159+0000","last_undegraded":"2024-06-06T08:40:17.395159+0000","last_fullsized":"2024-06-06T08:40:17.395159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397547+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:40:17.397547+0000","last_peered":"2024-06-06T08:40:17.397547+0000","last_clean":"2024-06-06T08:40:17.397547+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:40:17.397547+0000","last_undegraded":"2024-06-06T08:40:17.397547+0000","last_fullsized":"2024-06-06T08:40:17.397547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602515+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:40:17.602515+0000","last_peered":"2024-06-06T08:40:17.602515+0000","last_clean":"2024-06-06T08:40:17.602515+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:40:17.602515+0000","last_undegraded":"2024-06-06T08:40:17.602515+0000","last_fullsized":"2024-06-06T08:40:17.602515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397886+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:40:17.397886+0000","last_peered":"2024-06-06T08:40:17.397886+0000","last_clean":"2024-06-06T08:40:17.397886+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:40:17.397886+0000","last_undegraded":"2024-06-06T08:40:17.397886+0000","last_fullsized":"2024-06-06T08:40:17.397886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603637+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:40:17.603637+0000","last_peered":"2024-06-06T08:40:17.603637+0000","last_clean":"2024-06-06T08:40:17.603637+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:40:17.603637+0000","last_undegraded":"2024-06-06T08:40:17.603637+0000","last_fullsized":"2024-06-06T08:40:17.603637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602634+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:40:17.602634+0000","last_peered":"2024-06-06T08:40:17.602634+0000","last_clean":"2024-06-06T08:40:17.602634+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:40:17.602634+0000","last_undegraded":"2024-06-06T08:40:17.602634+0000","last_fullsized":"2024-06-06T08:40:17.602634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397399+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:40:17.397399+0000","last_peered":"2024-06-06T08:40:17.397399+0000","last_clean":"2024-06-06T08:40:17.397399+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:40:17.397399+0000","last_undegraded":"2024-06-06T08:40:17.397399+0000","last_fullsized":"2024-06-06T08:40:17.397399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618791+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:40:17.618791+0000","last_peered":"2024-06-06T08:40:17.618791+0000","last_clean":"2024-06-06T08:40:17.618791+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:40:17.618791+0000","last_undegraded":"2024-06-06T08:40:17.618791+0000","last_fullsized":"2024-06-06T08:40:17.618791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602522+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:40:17.602522+0000","last_peered":"2024-06-06T08:40:17.602522+0000","last_clean":"2024-06-06T08:40:17.602522+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:40:17.602522+0000","last_undegraded":"2024-06-06T08:40:17.602522+0000","last_fullsized":"2024-06-06T08:40:17.602522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397350+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:40:17.397350+0000","last_peered":"2024-06-06T08:40:17.397350+0000","last_clean":"2024-06-06T08:40:17.397350+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:40:17.397350+0000","last_undegraded":"2024-06-06T08:40:17.397350+0000","last_fullsized":"2024-06-06T08:40:17.397350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394926+0000","last_change":"2024-06-06T08:38:01.728398+0000","last_active":"2024-06-06T08:40:17.394926+0000","last_peered":"2024-06-06T08:40:17.394926+0000","last_clean":"2024-06-06T08:40:17.394926+0000","last_became_active":"2024-06-06T08:38:01.728004+0000","last_became_peered":"2024-06-06T08:38:01.728004+0000","last_unstale":"2024-06-06T08:40:17.394926+0000","last_undegraded":"2024-06-06T08:40:17.394926+0000","last_fullsized":"2024-06-06T08:40:17.394926+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:02:03.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291009+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:40:17.291009+0000","last_peered":"2024-06-06T08:40:17.291009+0000","last_clean":"2024-06-06T08:40:17.291009+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:40:17.291009+0000","last_undegraded":"2024-06-06T08:40:17.291009+0000","last_fullsized":"2024-06-06T08:40:17.291009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":232,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:18.086232+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:40:18.086232+0000","last_peered":"2024-06-06T08:40:18.086232+0000","last_clean":"2024-06-06T08:40:18.086232+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:40:18.086232+0000","last_undegraded":"2024-06-06T08:40:18.086232+0000","last_fullsized":"2024-06-06T08:40:18.086232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1320,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396976+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:40:17.396976+0000","last_peered":"2024-06-06T08:40:17.396976+0000","last_clean":"2024-06-06T08:40:17.396976+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:40:17.396976+0000","last_undegraded":"2024-06-06T08:40:17.396976+0000","last_fullsized":"2024-06-06T08:40:17.396976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290894+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:40:17.290894+0000","last_peered":"2024-06-06T08:40:17.290894+0000","last_clean":"2024-06-06T08:40:17.290894+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:40:17.290894+0000","last_undegraded":"2024-06-06T08:40:17.290894+0000","last_fullsized":"2024-06-06T08:40:17.290894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406405+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:40:17.406405+0000","last_peered":"2024-06-06T08:40:17.406405+0000","last_clean":"2024-06-06T08:40:17.406405+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:40:17.406405+0000","last_undegraded":"2024-06-06T08:40:17.406405+0000","last_fullsized":"2024-06-06T08:40:17.406405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394458+0000","last_change":"2024-06-06T08:38:01.728238+0000","last_active":"2024-06-06T08:40:17.394458+0000","last_peered":"2024-06-06T08:40:17.394458+0000","last_clean":"2024-06-06T08:40:17.394458+0000","last_became_active":"2024-06-06T08:38:01.727844+0000","last_became_peered":"2024-06-06T08:38:01.727844+0000","last_unstale":"2024-06-06T08:40:17.394458+0000","last_undegraded":"2024-06-06T08:40:17.394458+0000","last_fullsized":"2024-06-06T08:40:17.394458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:19:46.825321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543511+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:40:17.543511+0000","last_peered":"2024-06-06T08:40:17.543511+0000","last_clean":"2024-06-06T08:40:17.543511+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:40:17.543511+0000","last_undegraded":"2024-06-06T08:40:17.543511+0000","last_fullsized":"2024-06-06T08:40:17.543511+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288975+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:40:17.288975+0000","last_peered":"2024-06-06T08:40:17.288975+0000","last_clean":"2024-06-06T08:40:17.288975+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:40:17.288975+0000","last_undegraded":"2024-06-06T08:40:17.288975+0000","last_fullsized":"2024-06-06T08:40:17.288975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396897+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:40:17.396897+0000","last_peered":"2024-06-06T08:40:17.396897+0000","last_clean":"2024-06-06T08:40:17.396897+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:40:17.396897+0000","last_undegraded":"2024-06-06T08:40:17.396897+0000","last_fullsized":"2024-06-06T08:40:17.396897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603585+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:40:17.603585+0000","last_peered":"2024-06-06T08:40:17.603585+0000","last_clean":"2024-06-06T08:40:17.603585+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:40:17.603585+0000","last_undegraded":"2024-06-06T08:40:17.603585+0000","last_fullsized":"2024-06-06T08:40:17.603585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403412+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:40:17.403412+0000","last_peered":"2024-06-06T08:40:17.403412+0000","last_clean":"2024-06-06T08:40:17.403412+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:40:17.403412+0000","last_undegraded":"2024-06-06T08:40:17.403412+0000","last_fullsized":"2024-06-06T08:40:17.403412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394236+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:40:17.394236+0000","last_peered":"2024-06-06T08:40:17.394236+0000","last_clean":"2024-06-06T08:40:17.394236+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:40:17.394236+0000","last_undegraded":"2024-06-06T08:40:17.394236+0000","last_fullsized":"2024-06-06T08:40:17.394236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":230,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:22.091225+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:40:22.091225+0000","last_peered":"2024-06-06T08:40:22.091225+0000","last_clean":"2024-06-06T08:40:22.091225+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:40:22.091225+0000","last_undegraded":"2024-06-06T08:40:22.091225+0000","last_fullsized":"2024-06-06T08:40:22.091225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619308+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:40:17.619308+0000","last_peered":"2024-06-06T08:40:17.619308+0000","last_clean":"2024-06-06T08:40:17.619308+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:40:17.619308+0000","last_undegraded":"2024-06-06T08:40:17.619308+0000","last_fullsized":"2024-06-06T08:40:17.619308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394167+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:40:17.394167+0000","last_peered":"2024-06-06T08:40:17.394167+0000","last_clean":"2024-06-06T08:40:17.394167+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:40:17.394167+0000","last_undegraded":"2024-06-06T08:40:17.394167+0000","last_fullsized":"2024-06-06T08:40:17.394167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290830+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:40:17.290830+0000","last_peered":"2024-06-06T08:40:17.290830+0000","last_clean":"2024-06-06T08:40:17.290830+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:40:17.290830+0000","last_undegraded":"2024-06-06T08:40:17.290830+0000","last_fullsized":"2024-06-06T08:40:17.290830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603715+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:40:17.603715+0000","last_peered":"2024-06-06T08:40:17.603715+0000","last_clean":"2024-06-06T08:40:17.603715+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:40:17.603715+0000","last_undegraded":"2024-06-06T08:40:17.603715+0000","last_fullsized":"2024-06-06T08:40:17.603715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602684+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:40:17.602684+0000","last_peered":"2024-06-06T08:40:17.602684+0000","last_clean":"2024-06-06T08:40:17.602684+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:40:17.602684+0000","last_undegraded":"2024-06-06T08:40:17.602684+0000","last_fullsized":"2024-06-06T08:40:17.602684+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403562+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:40:17.403562+0000","last_peered":"2024-06-06T08:40:17.403562+0000","last_clean":"2024-06-06T08:40:17.403562+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:40:17.403562+0000","last_undegraded":"2024-06-06T08:40:17.403562+0000","last_fullsized":"2024-06-06T08:40:17.403562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618947+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:40:17.618947+0000","last_peered":"2024-06-06T08:40:17.618947+0000","last_clean":"2024-06-06T08:40:17.618947+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:40:17.618947+0000","last_undegraded":"2024-06-06T08:40:17.618947+0000","last_fullsized":"2024-06-06T08:40:17.618947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406287+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:40:17.406287+0000","last_peered":"2024-06-06T08:40:17.406287+0000","last_clean":"2024-06-06T08:40:17.406287+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:40:17.406287+0000","last_undegraded":"2024-06-06T08:40:17.406287+0000","last_fullsized":"2024-06-06T08:40:17.406287+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'43","reported_seq":480,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:23.092616+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:40:23.092616+0000","last_peered":"2024-06-06T08:40:23.092616+0000","last_clean":"2024-06-06T08:40:23.092616+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:40:23.092616+0000","last_undegraded":"2024-06-06T08:40:23.092616+0000","last_fullsized":"2024-06-06T08:40:23.092616+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71690,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":374,"num_read_kb":258,"num_write":44,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397172+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:40:17.397172+0000","last_peered":"2024-06-06T08:40:17.397172+0000","last_clean":"2024-06-06T08:40:17.397172+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:40:17.397172+0000","last_undegraded":"2024-06-06T08:40:17.397172+0000","last_fullsized":"2024-06-06T08:40:17.397172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291329+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:40:17.291329+0000","last_peered":"2024-06-06T08:40:17.291329+0000","last_clean":"2024-06-06T08:40:17.291329+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:40:17.291329+0000","last_undegraded":"2024-06-06T08:40:17.291329+0000","last_fullsized":"2024-06-06T08:40:17.291329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289082+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:40:17.289082+0000","last_peered":"2024-06-06T08:40:17.289082+0000","last_clean":"2024-06-06T08:40:17.289082+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:40:17.289082+0000","last_undegraded":"2024-06-06T08:40:17.289082+0000","last_fullsized":"2024-06-06T08:40:17.289082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543337+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:40:17.543337+0000","last_peered":"2024-06-06T08:40:17.543337+0000","last_clean":"2024-06-06T08:40:17.543337+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:40:17.543337+0000","last_undegraded":"2024-06-06T08:40:17.543337+0000","last_fullsized":"2024-06-06T08:40:17.543337+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403613+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:40:17.403613+0000","last_peered":"2024-06-06T08:40:17.403613+0000","last_clean":"2024-06-06T08:40:17.403613+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:40:17.403613+0000","last_undegraded":"2024-06-06T08:40:17.403613+0000","last_fullsized":"2024-06-06T08:40:17.403613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290581+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:40:17.290581+0000","last_peered":"2024-06-06T08:40:17.290581+0000","last_clean":"2024-06-06T08:40:17.290581+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:40:17.290581+0000","last_undegraded":"2024-06-06T08:40:17.290581+0000","last_fullsized":"2024-06-06T08:40:17.290581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394932+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:40:17.394932+0000","last_peered":"2024-06-06T08:40:17.394932+0000","last_clean":"2024-06-06T08:40:17.394932+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:40:17.394932+0000","last_undegraded":"2024-06-06T08:40:17.394932+0000","last_fullsized":"2024-06-06T08:40:17.394932+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'6","reported_seq":232,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:18.085943+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:40:18.085943+0000","last_peered":"2024-06-06T08:40:18.085943+0000","last_clean":"2024-06-06T08:40:18.085943+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:40:18.085943+0000","last_undegraded":"2024-06-06T08:40:18.085943+0000","last_fullsized":"2024-06-06T08:40:18.085943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290588+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:40:17.290588+0000","last_peered":"2024-06-06T08:40:17.290588+0000","last_clean":"2024-06-06T08:40:17.290588+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:40:17.290588+0000","last_undegraded":"2024-06-06T08:40:17.290588+0000","last_fullsized":"2024-06-06T08:40:17.290588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289726+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:40:17.289726+0000","last_peered":"2024-06-06T08:40:17.289726+0000","last_clean":"2024-06-06T08:40:17.289726+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:40:17.289726+0000","last_undegraded":"2024-06-06T08:40:17.289726+0000","last_fullsized":"2024-06-06T08:40:17.289726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396877+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:40:17.396877+0000","last_peered":"2024-06-06T08:40:17.396877+0000","last_clean":"2024-06-06T08:40:17.396877+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:40:17.396877+0000","last_undegraded":"2024-06-06T08:40:17.396877+0000","last_fullsized":"2024-06-06T08:40:17.396877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403466+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:40:17.403466+0000","last_peered":"2024-06-06T08:40:17.403466+0000","last_clean":"2024-06-06T08:40:17.403466+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:40:17.403466+0000","last_undegraded":"2024-06-06T08:40:17.403466+0000","last_fullsized":"2024-06-06T08:40:17.403466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403457+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:40:17.403457+0000","last_peered":"2024-06-06T08:40:17.403457+0000","last_clean":"2024-06-06T08:40:17.403457+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:40:17.403457+0000","last_undegraded":"2024-06-06T08:40:17.403457+0000","last_fullsized":"2024-06-06T08:40:17.403457+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"100'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397313+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:40:17.397313+0000","last_peered":"2024-06-06T08:40:17.397313+0000","last_clean":"2024-06-06T08:40:17.397313+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:40:17.397313+0000","last_undegraded":"2024-06-06T08:40:17.397313+0000","last_fullsized":"2024-06-06T08:40:17.397313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543570+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:40:17.543570+0000","last_peered":"2024-06-06T08:40:17.543570+0000","last_clean":"2024-06-06T08:40:17.543570+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:40:17.543570+0000","last_undegraded":"2024-06-06T08:40:17.543570+0000","last_fullsized":"2024-06-06T08:40:17.543570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394732+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:40:17.394732+0000","last_peered":"2024-06-06T08:40:17.394732+0000","last_clean":"2024-06-06T08:40:17.394732+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:40:17.394732+0000","last_undegraded":"2024-06-06T08:40:17.394732+0000","last_fullsized":"2024-06-06T08:40:17.394732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406540+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:40:17.406540+0000","last_peered":"2024-06-06T08:40:17.406540+0000","last_clean":"2024-06-06T08:40:17.406540+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:40:17.406540+0000","last_undegraded":"2024-06-06T08:40:17.406540+0000","last_fullsized":"2024-06-06T08:40:17.406540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289289+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:40:17.289289+0000","last_peered":"2024-06-06T08:40:17.289289+0000","last_clean":"2024-06-06T08:40:17.289289+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:40:17.289289+0000","last_undegraded":"2024-06-06T08:40:17.289289+0000","last_fullsized":"2024-06-06T08:40:17.289289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290584+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:40:17.290584+0000","last_peered":"2024-06-06T08:40:17.290584+0000","last_clean":"2024-06-06T08:40:17.290584+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:40:17.290584+0000","last_undegraded":"2024-06-06T08:40:17.290584+0000","last_fullsized":"2024-06-06T08:40:17.290584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406420+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:40:17.406420+0000","last_peered":"2024-06-06T08:40:17.406420+0000","last_clean":"2024-06-06T08:40:17.406420+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:40:17.406420+0000","last_undegraded":"2024-06-06T08:40:17.406420+0000","last_fullsized":"2024-06-06T08:40:17.406420+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289488+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:40:17.289488+0000","last_peered":"2024-06-06T08:40:17.289488+0000","last_clean":"2024-06-06T08:40:17.289488+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:40:17.289488+0000","last_undegraded":"2024-06-06T08:40:17.289488+0000","last_fullsized":"2024-06-06T08:40:17.289488+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403779+0000","last_change":"2024-06-06T08:38:01.545781+0000","last_active":"2024-06-06T08:40:17.403779+0000","last_peered":"2024-06-06T08:40:17.403779+0000","last_clean":"2024-06-06T08:40:17.403779+0000","last_became_active":"2024-06-06T08:38:01.545462+0000","last_became_peered":"2024-06-06T08:38:01.545462+0000","last_unstale":"2024-06-06T08:40:17.403779+0000","last_undegraded":"2024-06-06T08:40:17.403779+0000","last_fullsized":"2024-06-06T08:40:17.403779+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:14:36.722960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":232,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:22.091205+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:40:22.091205+0000","last_peered":"2024-06-06T08:40:22.091205+0000","last_clean":"2024-06-06T08:40:22.091205+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:40:22.091205+0000","last_undegraded":"2024-06-06T08:40:22.091205+0000","last_fullsized":"2024-06-06T08:40:22.091205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290959+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:40:17.290959+0000","last_peered":"2024-06-06T08:40:17.290959+0000","last_clean":"2024-06-06T08:40:17.290959+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:40:17.290959+0000","last_undegraded":"2024-06-06T08:40:17.290959+0000","last_fullsized":"2024-06-06T08:40:17.290959+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397121+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:40:17.397121+0000","last_peered":"2024-06-06T08:40:17.397121+0000","last_clean":"2024-06-06T08:40:17.397121+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:40:17.397121+0000","last_undegraded":"2024-06-06T08:40:17.397121+0000","last_fullsized":"2024-06-06T08:40:17.397121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602368+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:40:17.602368+0000","last_peered":"2024-06-06T08:40:17.602368+0000","last_clean":"2024-06-06T08:40:17.602368+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:40:17.602368+0000","last_undegraded":"2024-06-06T08:40:17.602368+0000","last_fullsized":"2024-06-06T08:40:17.602368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":378,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290379+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:40:17.290379+0000","last_peered":"2024-06-06T08:40:17.290379+0000","last_clean":"2024-06-06T08:40:17.290379+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:40:17.290379+0000","last_undegraded":"2024-06-06T08:40:17.290379+0000","last_fullsized":"2024-06-06T08:40:17.290379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288818+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:40:17.288818+0000","last_peered":"2024-06-06T08:40:17.288818+0000","last_clean":"2024-06-06T08:40:17.288818+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:40:17.288818+0000","last_undegraded":"2024-06-06T08:40:17.288818+0000","last_fullsized":"2024-06-06T08:40:17.288818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543213+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:40:17.543213+0000","last_peered":"2024-06-06T08:40:17.543213+0000","last_clean":"2024-06-06T08:40:17.543213+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:40:17.543213+0000","last_undegraded":"2024-06-06T08:40:17.543213+0000","last_fullsized":"2024-06-06T08:40:17.543213+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543287+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:40:17.543287+0000","last_peered":"2024-06-06T08:40:17.543287+0000","last_clean":"2024-06-06T08:40:17.543287+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:40:17.543287+0000","last_undegraded":"2024-06-06T08:40:17.543287+0000","last_fullsized":"2024-06-06T08:40:17.543287+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543496+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:40:17.543496+0000","last_peered":"2024-06-06T08:40:17.543496+0000","last_clean":"2024-06-06T08:40:17.543496+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:40:17.543496+0000","last_undegraded":"2024-06-06T08:40:17.543496+0000","last_fullsized":"2024-06-06T08:40:17.543496+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289481+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:40:17.289481+0000","last_peered":"2024-06-06T08:40:17.289481+0000","last_clean":"2024-06-06T08:40:17.289481+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:40:17.289481+0000","last_undegraded":"2024-06-06T08:40:17.289481+0000","last_fullsized":"2024-06-06T08:40:17.289481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397142+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:40:17.397142+0000","last_peered":"2024-06-06T08:40:17.397142+0000","last_clean":"2024-06-06T08:40:17.397142+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:40:17.397142+0000","last_undegraded":"2024-06-06T08:40:17.397142+0000","last_fullsized":"2024-06-06T08:40:17.397142+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'49","reported_seq":119,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.518606+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:40:17.518606+0000","last_peered":"2024-06-06T08:40:17.518606+0000","last_clean":"2024-06-06T08:40:17.518606+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:40:17.518606+0000","last_undegraded":"2024-06-06T08:40:17.518606+0000","last_fullsized":"2024-06-06T08:40:17.518606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397610+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:40:17.397610+0000","last_peered":"2024-06-06T08:40:17.397610+0000","last_clean":"2024-06-06T08:40:17.397610+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:40:17.397610+0000","last_undegraded":"2024-06-06T08:40:17.397610+0000","last_fullsized":"2024-06-06T08:40:17.397610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290756+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:40:17.290756+0000","last_peered":"2024-06-06T08:40:17.290756+0000","last_clean":"2024-06-06T08:40:17.290756+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:40:17.290756+0000","last_undegraded":"2024-06-06T08:40:17.290756+0000","last_fullsized":"2024-06-06T08:40:17.290756+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603067+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:40:17.603067+0000","last_peered":"2024-06-06T08:40:17.603067+0000","last_clean":"2024-06-06T08:40:17.603067+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:40:17.603067+0000","last_undegraded":"2024-06-06T08:40:17.603067+0000","last_fullsized":"2024-06-06T08:40:17.603067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603700+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:40:17.603700+0000","last_peered":"2024-06-06T08:40:17.603700+0000","last_clean":"2024-06-06T08:40:17.603700+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:40:17.603700+0000","last_undegraded":"2024-06-06T08:40:17.603700+0000","last_fullsized":"2024-06-06T08:40:17.603700+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602722+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:40:17.602722+0000","last_peered":"2024-06-06T08:40:17.602722+0000","last_clean":"2024-06-06T08:40:17.602722+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:40:17.602722+0000","last_undegraded":"2024-06-06T08:40:17.602722+0000","last_fullsized":"2024-06-06T08:40:17.602722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618369+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:40:17.618369+0000","last_peered":"2024-06-06T08:40:17.618369+0000","last_clean":"2024-06-06T08:40:17.618369+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:40:17.618369+0000","last_undegraded":"2024-06-06T08:40:17.618369+0000","last_fullsized":"2024-06-06T08:40:17.618369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396892+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:40:17.396892+0000","last_peered":"2024-06-06T08:40:17.396892+0000","last_clean":"2024-06-06T08:40:17.396892+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:40:17.396892+0000","last_undegraded":"2024-06-06T08:40:17.396892+0000","last_fullsized":"2024-06-06T08:40:17.396892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.395030+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:40:17.395030+0000","last_peered":"2024-06-06T08:40:17.395030+0000","last_clean":"2024-06-06T08:40:17.395030+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:40:17.395030+0000","last_undegraded":"2024-06-06T08:40:17.395030+0000","last_fullsized":"2024-06-06T08:40:17.395030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397132+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:40:17.397132+0000","last_peered":"2024-06-06T08:40:17.397132+0000","last_clean":"2024-06-06T08:40:17.397132+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:40:17.397132+0000","last_undegraded":"2024-06-06T08:40:17.397132+0000","last_fullsized":"2024-06-06T08:40:17.397132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543302+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:40:17.543302+0000","last_peered":"2024-06-06T08:40:17.543302+0000","last_clean":"2024-06-06T08:40:17.543302+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:40:17.543302+0000","last_undegraded":"2024-06-06T08:40:17.543302+0000","last_fullsized":"2024-06-06T08:40:17.543302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602805+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:40:17.602805+0000","last_peered":"2024-06-06T08:40:17.602805+0000","last_clean":"2024-06-06T08:40:17.602805+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:40:17.602805+0000","last_undegraded":"2024-06-06T08:40:17.602805+0000","last_fullsized":"2024-06-06T08:40:17.602805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291108+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:40:17.291108+0000","last_peered":"2024-06-06T08:40:17.291108+0000","last_clean":"2024-06-06T08:40:17.291108+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:40:17.291108+0000","last_undegraded":"2024-06-06T08:40:17.291108+0000","last_fullsized":"2024-06-06T08:40:17.291108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396937+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:40:17.396937+0000","last_peered":"2024-06-06T08:40:17.396937+0000","last_clean":"2024-06-06T08:40:17.396937+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:40:17.396937+0000","last_undegraded":"2024-06-06T08:40:17.396937+0000","last_fullsized":"2024-06-06T08:40:17.396937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602730+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:40:17.602730+0000","last_peered":"2024-06-06T08:40:17.602730+0000","last_clean":"2024-06-06T08:40:17.602730+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:40:17.602730+0000","last_undegraded":"2024-06-06T08:40:17.602730+0000","last_fullsized":"2024-06-06T08:40:17.602730+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396987+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:40:17.396987+0000","last_peered":"2024-06-06T08:40:17.396987+0000","last_clean":"2024-06-06T08:40:17.396987+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:40:17.396987+0000","last_undegraded":"2024-06-06T08:40:17.396987+0000","last_fullsized":"2024-06-06T08:40:17.396987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290632+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:40:17.290632+0000","last_peered":"2024-06-06T08:40:17.290632+0000","last_clean":"2024-06-06T08:40:17.290632+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:40:17.290632+0000","last_undegraded":"2024-06-06T08:40:17.290632+0000","last_fullsized":"2024-06-06T08:40:17.290632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618976+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:40:17.618976+0000","last_peered":"2024-06-06T08:40:17.618976+0000","last_clean":"2024-06-06T08:40:17.618976+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:40:17.618976+0000","last_undegraded":"2024-06-06T08:40:17.618976+0000","last_fullsized":"2024-06-06T08:40:17.618976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291175+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:40:17.291175+0000","last_peered":"2024-06-06T08:40:17.291175+0000","last_clean":"2024-06-06T08:40:17.291175+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:40:17.291175+0000","last_undegraded":"2024-06-06T08:40:17.291175+0000","last_fullsized":"2024-06-06T08:40:17.291175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291116+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:40:17.291116+0000","last_peered":"2024-06-06T08:40:17.291116+0000","last_clean":"2024-06-06T08:40:17.291116+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:40:17.291116+0000","last_undegraded":"2024-06-06T08:40:17.291116+0000","last_fullsized":"2024-06-06T08:40:17.291116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603292+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:40:17.603292+0000","last_peered":"2024-06-06T08:40:17.603292+0000","last_clean":"2024-06-06T08:40:17.603292+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:40:17.603292+0000","last_undegraded":"2024-06-06T08:40:17.603292+0000","last_fullsized":"2024-06-06T08:40:17.603292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'39","reported_seq":94,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.513053+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:40:17.513053+0000","last_peered":"2024-06-06T08:40:17.513053+0000","last_clean":"2024-06-06T08:40:17.513053+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:40:17.513053+0000","last_undegraded":"2024-06-06T08:40:17.513053+0000","last_fullsized":"2024-06-06T08:40:17.513053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290659+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:40:17.290659+0000","last_peered":"2024-06-06T08:40:17.290659+0000","last_clean":"2024-06-06T08:40:17.290659+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:40:17.290659+0000","last_undegraded":"2024-06-06T08:40:17.290659+0000","last_fullsized":"2024-06-06T08:40:17.290659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403261+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:40:17.403261+0000","last_peered":"2024-06-06T08:40:17.403261+0000","last_clean":"2024-06-06T08:40:17.403261+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:40:17.403261+0000","last_undegraded":"2024-06-06T08:40:17.403261+0000","last_fullsized":"2024-06-06T08:40:17.403261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289754+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:40:17.289754+0000","last_peered":"2024-06-06T08:40:17.289754+0000","last_clean":"2024-06-06T08:40:17.289754+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:40:17.289754+0000","last_undegraded":"2024-06-06T08:40:17.289754+0000","last_fullsized":"2024-06-06T08:40:17.289754+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291007+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:40:17.291007+0000","last_peered":"2024-06-06T08:40:17.291007+0000","last_clean":"2024-06-06T08:40:17.291007+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:40:17.291007+0000","last_undegraded":"2024-06-06T08:40:17.291007+0000","last_fullsized":"2024-06-06T08:40:17.291007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543466+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:40:17.543466+0000","last_peered":"2024-06-06T08:40:17.543466+0000","last_clean":"2024-06-06T08:40:17.543466+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:40:17.543466+0000","last_undegraded":"2024-06-06T08:40:17.543466+0000","last_fullsized":"2024-06-06T08:40:17.543466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291000+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:40:17.291000+0000","last_peered":"2024-06-06T08:40:17.291000+0000","last_clean":"2024-06-06T08:40:17.291000+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:40:17.291000+0000","last_undegraded":"2024-06-06T08:40:17.291000+0000","last_fullsized":"2024-06-06T08:40:17.291000+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290895+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:40:17.290895+0000","last_peered":"2024-06-06T08:40:17.290895+0000","last_clean":"2024-06-06T08:40:17.290895+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:40:17.290895+0000","last_undegraded":"2024-06-06T08:40:17.290895+0000","last_fullsized":"2024-06-06T08:40:17.290895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603342+0000","last_change":"2024-06-06T08:38:01.151003+0000","last_active":"2024-06-06T08:40:17.603342+0000","last_peered":"2024-06-06T08:40:17.603342+0000","last_clean":"2024-06-06T08:40:17.603342+0000","last_became_active":"2024-06-06T08:38:01.150557+0000","last_became_peered":"2024-06-06T08:38:01.150557+0000","last_unstale":"2024-06-06T08:40:17.603342+0000","last_undegraded":"2024-06-06T08:40:17.603342+0000","last_fullsized":"2024-06-06T08:40:17.603342+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:29:44.629894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289585+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:40:17.289585+0000","last_peered":"2024-06-06T08:40:17.289585+0000","last_clean":"2024-06-06T08:40:17.289585+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:40:17.289585+0000","last_undegraded":"2024-06-06T08:40:17.289585+0000","last_fullsized":"2024-06-06T08:40:17.289585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543470+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:40:17.543470+0000","last_peered":"2024-06-06T08:40:17.543470+0000","last_clean":"2024-06-06T08:40:17.543470+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:40:17.543470+0000","last_undegraded":"2024-06-06T08:40:17.543470+0000","last_fullsized":"2024-06-06T08:40:17.543470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291317+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:40:17.291317+0000","last_peered":"2024-06-06T08:40:17.291317+0000","last_clean":"2024-06-06T08:40:17.291317+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:40:17.291317+0000","last_undegraded":"2024-06-06T08:40:17.291317+0000","last_fullsized":"2024-06-06T08:40:17.291317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397744+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:40:17.397744+0000","last_peered":"2024-06-06T08:40:17.397744+0000","last_clean":"2024-06-06T08:40:17.397744+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:40:17.397744+0000","last_undegraded":"2024-06-06T08:40:17.397744+0000","last_fullsized":"2024-06-06T08:40:17.397744+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403480+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:40:17.403480+0000","last_peered":"2024-06-06T08:40:17.403480+0000","last_clean":"2024-06-06T08:40:17.403480+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:40:17.403480+0000","last_undegraded":"2024-06-06T08:40:17.403480+0000","last_fullsized":"2024-06-06T08:40:17.403480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397272+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:40:17.397272+0000","last_peered":"2024-06-06T08:40:17.397272+0000","last_clean":"2024-06-06T08:40:17.397272+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:40:17.397272+0000","last_undegraded":"2024-06-06T08:40:17.397272+0000","last_fullsized":"2024-06-06T08:40:17.397272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618845+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:40:17.618845+0000","last_peered":"2024-06-06T08:40:17.618845+0000","last_clean":"2024-06-06T08:40:17.618845+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:40:17.618845+0000","last_undegraded":"2024-06-06T08:40:17.618845+0000","last_fullsized":"2024-06-06T08:40:17.618845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289213+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:40:17.289213+0000","last_peered":"2024-06-06T08:40:17.289213+0000","last_clean":"2024-06-06T08:40:17.289213+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:40:17.289213+0000","last_undegraded":"2024-06-06T08:40:17.289213+0000","last_fullsized":"2024-06-06T08:40:17.289213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543444+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:40:17.543444+0000","last_peered":"2024-06-06T08:40:17.543444+0000","last_clean":"2024-06-06T08:40:17.543444+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:40:17.543444+0000","last_undegraded":"2024-06-06T08:40:17.543444+0000","last_fullsized":"2024-06-06T08:40:17.543444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107163,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601735,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1195,"num_read_kb":718,"num_write":304,"num_write_kb":1451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":1799775,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383743,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":569943,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645382,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32428,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712724,"statfs":{"total":95995035648,"available":95961829376,"internally_reserved":0,"allocated":1282048,"data_stored":1371722,"data_compressed":77234,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1150976,"data_stored":977401,"data_compressed":54503,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070543,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1142784,"data_stored":973203,"data_compressed":54503,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332180,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1171456,"data_stored":1177146,"data_compressed":71078,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495707,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":575265,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1175552,"data_stored":643955,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":576653,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577223,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1100275,"data_compressed":60659,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920868,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1191936,"data_stored":649616,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757342,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":1303023,"data_compressed":77395,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133774,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":206179,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:40:24.088 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-06T08:40:24.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T08:40:24.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 -- 172.21.15.27:0/3535753392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 msgr2=0x7f5c48100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 --2- 172.21.15.27:0/3535753392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 0x7f5c48100cf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009930 tx=0x7f5c3c02a420 comp rx=0 tx=0).stop 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 -- 172.21.15.27:0/3535753392 shutdown_connections 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 --2- 172.21.15.27:0/3535753392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c48101230 0x7f5c4810d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 --2- 172.21.15.27:0/3535753392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 0x7f5c48100cf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 --2- 172.21.15.27:0/3535753392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c48106880 0x7f5c48106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 -- 172.21.15.27:0/3535753392 >> 172.21.15.27:0/3535753392 conn(0x7f5c48076c20 msgr2=0x7f5c48077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 -- 172.21.15.27:0/3535753392 shutdown_connections 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 -- 172.21.15.27:0/3535753392 wait complete. 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 Processor -- start 2024-06-06T08:40:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.230+0000 7f5c506c1640 1 -- start start 2024-06-06T08:40:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c506c1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 0x7f5c481971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c506c1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c48101230 0x7f5c481976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c506c1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c48106880 0x7f5c4819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c506c1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48110c90 con 0x7f5c48100870 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c506c1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48110b10 con 0x7f5c48101230 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c506c1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48110e10 con 0x7f5c48106880 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4e45d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 0x7f5c481971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4e45d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 0x7f5c481971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58914/0 (socket says 172.21.15.27:58914) 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4e45d640 1 -- 172.21.15.27:0/1401626231 learned_addr learned my addr 172.21.15.27:0/1401626231 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4dc5c640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c48101230 0x7f5c481976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4e45d640 1 -- 172.21.15.27:0/1401626231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c48106880 msgr2=0x7f5c4819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4e45d640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c48106880 0x7f5c4819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4e45d640 1 -- 172.21.15.27:0/1401626231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c48101230 msgr2=0x7f5c481976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4e45d640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c48101230 0x7f5c481976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4e45d640 1 -- 172.21.15.27:0/1401626231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4819c1a0 con 0x7f5c48100870 2024-06-06T08:40:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c4e45d640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 0x7f5c481971b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800eb90 tx=0x7f5c3800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c477fe640 1 -- 172.21.15.27:0/1401626231 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38004d20 con 0x7f5c48100870 2024-06-06T08:40:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c477fe640 1 -- 172.21.15.27:0/1401626231 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c3800ee90 con 0x7f5c48100870 2024-06-06T08:40:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4819c490 con 0x7f5c48100870 2024-06-06T08:40:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c477fe640 1 -- 172.21.15.27:0/1401626231 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38010830 con 0x7f5c48100870 2024-06-06T08:40:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.231+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c481a3d20 con 0x7f5c48100870 2024-06-06T08:40:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.233+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c48101fb0 con 0x7f5c48100870 2024-06-06T08:40:24.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.234+0000 7f5c477fe640 1 -- 172.21.15.27:0/1401626231 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f5c38005020 con 0x7f5c48100870 2024-06-06T08:40:24.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.235+0000 7f5c477fe640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c300a2ea0 0x7f5c300a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:24.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.235+0000 7f5c4dc5c640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c300a2ea0 0x7f5c300a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:24.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.235+0000 7f5c477fe640 1 -- 172.21.15.27:0/1401626231 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f5c380d0170 con 0x7f5c48100870 2024-06-06T08:40:24.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.235+0000 7f5c4dc5c640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c300a2ea0 0x7f5c300a5360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0096c0 tx=0x7f5c3c000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:24.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.239+0000 7f5c477fe640 1 -- 172.21.15.27:0/1401626231 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c3809c780 con 0x7f5c48100870 2024-06-06T08:40:24.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.421+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c48061de0 con 0x7f5c300a2ea0 2024-06-06T08:40:24.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.431+0000 7f5c477fe640 1 -- 172.21.15.27:0/1401626231 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339820 (secure 0 0 0) 0x7f5c48061de0 con 0x7f5c300a2ea0 2024-06-06T08:40:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:24.434 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:40:24.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c300a2ea0 msgr2=0x7f5c300a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:24.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c300a2ea0 0x7f5c300a5360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0096c0 tx=0x7f5c3c000bc0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 msgr2=0x7f5c481971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:24.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 0x7f5c481971b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800eb90 tx=0x7f5c3800c6a0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 shutdown_connections 2024-06-06T08:40:24.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c300a2ea0 0x7f5c300a5360 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c48106880 0x7f5c4819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c48101230 0x7f5c481976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 --2- 172.21.15.27:0/1401626231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c48100870 0x7f5c481971b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:24.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 >> 172.21.15.27:0/1401626231 conn(0x7f5c48076c20 msgr2=0x7f5c480fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:24.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 shutdown_connections 2024-06-06T08:40:24.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:24.434+0000 7f5c506c1640 1 -- 172.21.15.27:0/1401626231 wait complete. 2024-06-06T08:40:24.443 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-06-06T08:40:23.305590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192337,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1331,"num_read_kb":837,"num_write":487,"num_write_kb":4391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":877,"ondisk_log_size":877,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358620,"kb_used_data":13596,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583204,"statfs":{"total":1151940427776,"available":1151573200896,"internally_reserved":0,"allocated":13922304,"data_stored":10495425,"data_compressed":604544,"data_compressed_allocated":819200,"data_compressed_original":4489216,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2910,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":19,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.994628"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543717+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:40:17.543717+0000","last_peered":"2024-06-06T08:40:17.543717+0000","last_clean":"2024-06-06T08:40:17.543717+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:40:17.543717+0000","last_undegraded":"2024-06-06T08:40:17.543717+0000","last_fullsized":"2024-06-06T08:40:17.543717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603636+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:40:17.603636+0000","last_peered":"2024-06-06T08:40:17.603636+0000","last_clean":"2024-06-06T08:40:17.603636+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:40:17.603636+0000","last_undegraded":"2024-06-06T08:40:17.603636+0000","last_fullsized":"2024-06-06T08:40:17.603636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603315+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:40:17.603315+0000","last_peered":"2024-06-06T08:40:17.603315+0000","last_clean":"2024-06-06T08:40:17.603315+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:40:17.603315+0000","last_undegraded":"2024-06-06T08:40:17.603315+0000","last_fullsized":"2024-06-06T08:40:17.603315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602639+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:40:17.602639+0000","last_peered":"2024-06-06T08:40:17.602639+0000","last_clean":"2024-06-06T08:40:17.602639+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:40:17.602639+0000","last_undegraded":"2024-06-06T08:40:17.602639+0000","last_fullsized":"2024-06-06T08:40:17.602639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397130+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:40:17.397130+0000","last_peered":"2024-06-06T08:40:17.397130+0000","last_clean":"2024-06-06T08:40:17.397130+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:40:17.397130+0000","last_undegraded":"2024-06-06T08:40:17.397130+0000","last_fullsized":"2024-06-06T08:40:17.397130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406459+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:40:17.406459+0000","last_peered":"2024-06-06T08:40:17.406459+0000","last_clean":"2024-06-06T08:40:17.406459+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:40:17.406459+0000","last_undegraded":"2024-06-06T08:40:17.406459+0000","last_fullsized":"2024-06-06T08:40:17.406459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290504+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:40:17.290504+0000","last_peered":"2024-06-06T08:40:17.290504+0000","last_clean":"2024-06-06T08:40:17.290504+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:40:17.290504+0000","last_undegraded":"2024-06-06T08:40:17.290504+0000","last_fullsized":"2024-06-06T08:40:17.290504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394037+0000","last_change":"2024-06-06T08:38:01.335419+0000","last_active":"2024-06-06T08:40:17.394037+0000","last_peered":"2024-06-06T08:40:17.394037+0000","last_clean":"2024-06-06T08:40:17.394037+0000","last_became_active":"2024-06-06T08:38:01.335191+0000","last_became_peered":"2024-06-06T08:38:01.335191+0000","last_unstale":"2024-06-06T08:40:17.394037+0000","last_undegraded":"2024-06-06T08:40:17.394037+0000","last_fullsized":"2024-06-06T08:40:17.394037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:13.253838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619376+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:40:17.619376+0000","last_peered":"2024-06-06T08:40:17.619376+0000","last_clean":"2024-06-06T08:40:17.619376+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:40:17.619376+0000","last_undegraded":"2024-06-06T08:40:17.619376+0000","last_fullsized":"2024-06-06T08:40:17.619376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406777+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:40:17.406777+0000","last_peered":"2024-06-06T08:40:17.406777+0000","last_clean":"2024-06-06T08:40:17.406777+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:40:17.406777+0000","last_undegraded":"2024-06-06T08:40:17.406777+0000","last_fullsized":"2024-06-06T08:40:17.406777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290994+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:40:17.290994+0000","last_peered":"2024-06-06T08:40:17.290994+0000","last_clean":"2024-06-06T08:40:17.290994+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:40:17.290994+0000","last_undegraded":"2024-06-06T08:40:17.290994+0000","last_fullsized":"2024-06-06T08:40:17.290994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602644+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:40:17.602644+0000","last_peered":"2024-06-06T08:40:17.602644+0000","last_clean":"2024-06-06T08:40:17.602644+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:40:17.602644+0000","last_undegraded":"2024-06-06T08:40:17.602644+0000","last_fullsized":"2024-06-06T08:40:17.602644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289360+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:40:17.289360+0000","last_peered":"2024-06-06T08:40:17.289360+0000","last_clean":"2024-06-06T08:40:17.289360+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:40:17.289360+0000","last_undegraded":"2024-06-06T08:40:17.289360+0000","last_fullsized":"2024-06-06T08:40:17.289360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602964+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:40:17.602964+0000","last_peered":"2024-06-06T08:40:17.602964+0000","last_clean":"2024-06-06T08:40:17.602964+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:40:17.602964+0000","last_undegraded":"2024-06-06T08:40:17.602964+0000","last_fullsized":"2024-06-06T08:40:17.602964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603050+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:40:17.603050+0000","last_peered":"2024-06-06T08:40:17.603050+0000","last_clean":"2024-06-06T08:40:17.603050+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:40:17.603050+0000","last_undegraded":"2024-06-06T08:40:17.603050+0000","last_fullsized":"2024-06-06T08:40:17.603050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618538+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:40:17.618538+0000","last_peered":"2024-06-06T08:40:17.618538+0000","last_clean":"2024-06-06T08:40:17.618538+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:40:17.618538+0000","last_undegraded":"2024-06-06T08:40:17.618538+0000","last_fullsized":"2024-06-06T08:40:17.618538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543492+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:40:17.543492+0000","last_peered":"2024-06-06T08:40:17.543492+0000","last_clean":"2024-06-06T08:40:17.543492+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:40:17.543492+0000","last_undegraded":"2024-06-06T08:40:17.543492+0000","last_fullsized":"2024-06-06T08:40:17.543492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394947+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:40:17.394947+0000","last_peered":"2024-06-06T08:40:17.394947+0000","last_clean":"2024-06-06T08:40:17.394947+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:40:17.394947+0000","last_undegraded":"2024-06-06T08:40:17.394947+0000","last_fullsized":"2024-06-06T08:40:17.394947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291235+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:40:17.291235+0000","last_peered":"2024-06-06T08:40:17.291235+0000","last_clean":"2024-06-06T08:40:17.291235+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:40:17.291235+0000","last_undegraded":"2024-06-06T08:40:17.291235+0000","last_fullsized":"2024-06-06T08:40:17.291235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291190+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:40:17.291190+0000","last_peered":"2024-06-06T08:40:17.291190+0000","last_clean":"2024-06-06T08:40:17.291190+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:40:17.291190+0000","last_undegraded":"2024-06-06T08:40:17.291190+0000","last_fullsized":"2024-06-06T08:40:17.291190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.395159+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:40:17.395159+0000","last_peered":"2024-06-06T08:40:17.395159+0000","last_clean":"2024-06-06T08:40:17.395159+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:40:17.395159+0000","last_undegraded":"2024-06-06T08:40:17.395159+0000","last_fullsized":"2024-06-06T08:40:17.395159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397547+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:40:17.397547+0000","last_peered":"2024-06-06T08:40:17.397547+0000","last_clean":"2024-06-06T08:40:17.397547+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:40:17.397547+0000","last_undegraded":"2024-06-06T08:40:17.397547+0000","last_fullsized":"2024-06-06T08:40:17.397547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602515+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:40:17.602515+0000","last_peered":"2024-06-06T08:40:17.602515+0000","last_clean":"2024-06-06T08:40:17.602515+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:40:17.602515+0000","last_undegraded":"2024-06-06T08:40:17.602515+0000","last_fullsized":"2024-06-06T08:40:17.602515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397886+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:40:17.397886+0000","last_peered":"2024-06-06T08:40:17.397886+0000","last_clean":"2024-06-06T08:40:17.397886+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:40:17.397886+0000","last_undegraded":"2024-06-06T08:40:17.397886+0000","last_fullsized":"2024-06-06T08:40:17.397886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603637+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:40:17.603637+0000","last_peered":"2024-06-06T08:40:17.603637+0000","last_clean":"2024-06-06T08:40:17.603637+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:40:17.603637+0000","last_undegraded":"2024-06-06T08:40:17.603637+0000","last_fullsized":"2024-06-06T08:40:17.603637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602634+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:40:17.602634+0000","last_peered":"2024-06-06T08:40:17.602634+0000","last_clean":"2024-06-06T08:40:17.602634+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:40:17.602634+0000","last_undegraded":"2024-06-06T08:40:17.602634+0000","last_fullsized":"2024-06-06T08:40:17.602634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397399+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:40:17.397399+0000","last_peered":"2024-06-06T08:40:17.397399+0000","last_clean":"2024-06-06T08:40:17.397399+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:40:17.397399+0000","last_undegraded":"2024-06-06T08:40:17.397399+0000","last_fullsized":"2024-06-06T08:40:17.397399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618791+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:40:17.618791+0000","last_peered":"2024-06-06T08:40:17.618791+0000","last_clean":"2024-06-06T08:40:17.618791+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:40:17.618791+0000","last_undegraded":"2024-06-06T08:40:17.618791+0000","last_fullsized":"2024-06-06T08:40:17.618791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602522+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:40:17.602522+0000","last_peered":"2024-06-06T08:40:17.602522+0000","last_clean":"2024-06-06T08:40:17.602522+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:40:17.602522+0000","last_undegraded":"2024-06-06T08:40:17.602522+0000","last_fullsized":"2024-06-06T08:40:17.602522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397350+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:40:17.397350+0000","last_peered":"2024-06-06T08:40:17.397350+0000","last_clean":"2024-06-06T08:40:17.397350+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:40:17.397350+0000","last_undegraded":"2024-06-06T08:40:17.397350+0000","last_fullsized":"2024-06-06T08:40:17.397350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394926+0000","last_change":"2024-06-06T08:38:01.728398+0000","last_active":"2024-06-06T08:40:17.394926+0000","last_peered":"2024-06-06T08:40:17.394926+0000","last_clean":"2024-06-06T08:40:17.394926+0000","last_became_active":"2024-06-06T08:38:01.728004+0000","last_became_peered":"2024-06-06T08:38:01.728004+0000","last_unstale":"2024-06-06T08:40:17.394926+0000","last_undegraded":"2024-06-06T08:40:17.394926+0000","last_fullsized":"2024-06-06T08:40:17.394926+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:02:03.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291009+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:40:17.291009+0000","last_peered":"2024-06-06T08:40:17.291009+0000","last_clean":"2024-06-06T08:40:17.291009+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:40:17.291009+0000","last_undegraded":"2024-06-06T08:40:17.291009+0000","last_fullsized":"2024-06-06T08:40:17.291009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":232,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:18.086232+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:40:18.086232+0000","last_peered":"2024-06-06T08:40:18.086232+0000","last_clean":"2024-06-06T08:40:18.086232+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:40:18.086232+0000","last_undegraded":"2024-06-06T08:40:18.086232+0000","last_fullsized":"2024-06-06T08:40:18.086232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1320,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396976+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:40:17.396976+0000","last_peered":"2024-06-06T08:40:17.396976+0000","last_clean":"2024-06-06T08:40:17.396976+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:40:17.396976+0000","last_undegraded":"2024-06-06T08:40:17.396976+0000","last_fullsized":"2024-06-06T08:40:17.396976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290894+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:40:17.290894+0000","last_peered":"2024-06-06T08:40:17.290894+0000","last_clean":"2024-06-06T08:40:17.290894+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:40:17.290894+0000","last_undegraded":"2024-06-06T08:40:17.290894+0000","last_fullsized":"2024-06-06T08:40:17.290894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406405+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:40:17.406405+0000","last_peered":"2024-06-06T08:40:17.406405+0000","last_clean":"2024-06-06T08:40:17.406405+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:40:17.406405+0000","last_undegraded":"2024-06-06T08:40:17.406405+0000","last_fullsized":"2024-06-06T08:40:17.406405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394458+0000","last_change":"2024-06-06T08:38:01.728238+0000","last_active":"2024-06-06T08:40:17.394458+0000","last_peered":"2024-06-06T08:40:17.394458+0000","last_clean":"2024-06-06T08:40:17.394458+0000","last_became_active":"2024-06-06T08:38:01.727844+0000","last_became_peered":"2024-06-06T08:38:01.727844+0000","last_unstale":"2024-06-06T08:40:17.394458+0000","last_undegraded":"2024-06-06T08:40:17.394458+0000","last_fullsized":"2024-06-06T08:40:17.394458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:19:46.825321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543511+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:40:17.543511+0000","last_peered":"2024-06-06T08:40:17.543511+0000","last_clean":"2024-06-06T08:40:17.543511+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:40:17.543511+0000","last_undegraded":"2024-06-06T08:40:17.543511+0000","last_fullsized":"2024-06-06T08:40:17.543511+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288975+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:40:17.288975+0000","last_peered":"2024-06-06T08:40:17.288975+0000","last_clean":"2024-06-06T08:40:17.288975+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:40:17.288975+0000","last_undegraded":"2024-06-06T08:40:17.288975+0000","last_fullsized":"2024-06-06T08:40:17.288975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396897+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:40:17.396897+0000","last_peered":"2024-06-06T08:40:17.396897+0000","last_clean":"2024-06-06T08:40:17.396897+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:40:17.396897+0000","last_undegraded":"2024-06-06T08:40:17.396897+0000","last_fullsized":"2024-06-06T08:40:17.396897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603585+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:40:17.603585+0000","last_peered":"2024-06-06T08:40:17.603585+0000","last_clean":"2024-06-06T08:40:17.603585+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:40:17.603585+0000","last_undegraded":"2024-06-06T08:40:17.603585+0000","last_fullsized":"2024-06-06T08:40:17.603585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403412+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:40:17.403412+0000","last_peered":"2024-06-06T08:40:17.403412+0000","last_clean":"2024-06-06T08:40:17.403412+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:40:17.403412+0000","last_undegraded":"2024-06-06T08:40:17.403412+0000","last_fullsized":"2024-06-06T08:40:17.403412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394236+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:40:17.394236+0000","last_peered":"2024-06-06T08:40:17.394236+0000","last_clean":"2024-06-06T08:40:17.394236+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:40:17.394236+0000","last_undegraded":"2024-06-06T08:40:17.394236+0000","last_fullsized":"2024-06-06T08:40:17.394236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":230,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:22.091225+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:40:22.091225+0000","last_peered":"2024-06-06T08:40:22.091225+0000","last_clean":"2024-06-06T08:40:22.091225+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:40:22.091225+0000","last_undegraded":"2024-06-06T08:40:22.091225+0000","last_fullsized":"2024-06-06T08:40:22.091225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619308+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:40:17.619308+0000","last_peered":"2024-06-06T08:40:17.619308+0000","last_clean":"2024-06-06T08:40:17.619308+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:40:17.619308+0000","last_undegraded":"2024-06-06T08:40:17.619308+0000","last_fullsized":"2024-06-06T08:40:17.619308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394167+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:40:17.394167+0000","last_peered":"2024-06-06T08:40:17.394167+0000","last_clean":"2024-06-06T08:40:17.394167+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:40:17.394167+0000","last_undegraded":"2024-06-06T08:40:17.394167+0000","last_fullsized":"2024-06-06T08:40:17.394167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290830+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:40:17.290830+0000","last_peered":"2024-06-06T08:40:17.290830+0000","last_clean":"2024-06-06T08:40:17.290830+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:40:17.290830+0000","last_undegraded":"2024-06-06T08:40:17.290830+0000","last_fullsized":"2024-06-06T08:40:17.290830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603715+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:40:17.603715+0000","last_peered":"2024-06-06T08:40:17.603715+0000","last_clean":"2024-06-06T08:40:17.603715+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:40:17.603715+0000","last_undegraded":"2024-06-06T08:40:17.603715+0000","last_fullsized":"2024-06-06T08:40:17.603715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602684+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:40:17.602684+0000","last_peered":"2024-06-06T08:40:17.602684+0000","last_clean":"2024-06-06T08:40:17.602684+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:40:17.602684+0000","last_undegraded":"2024-06-06T08:40:17.602684+0000","last_fullsized":"2024-06-06T08:40:17.602684+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403562+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:40:17.403562+0000","last_peered":"2024-06-06T08:40:17.403562+0000","last_clean":"2024-06-06T08:40:17.403562+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:40:17.403562+0000","last_undegraded":"2024-06-06T08:40:17.403562+0000","last_fullsized":"2024-06-06T08:40:17.403562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618947+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:40:17.618947+0000","last_peered":"2024-06-06T08:40:17.618947+0000","last_clean":"2024-06-06T08:40:17.618947+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:40:17.618947+0000","last_undegraded":"2024-06-06T08:40:17.618947+0000","last_fullsized":"2024-06-06T08:40:17.618947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406287+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:40:17.406287+0000","last_peered":"2024-06-06T08:40:17.406287+0000","last_clean":"2024-06-06T08:40:17.406287+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:40:17.406287+0000","last_undegraded":"2024-06-06T08:40:17.406287+0000","last_fullsized":"2024-06-06T08:40:17.406287+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'43","reported_seq":480,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:23.092616+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:40:23.092616+0000","last_peered":"2024-06-06T08:40:23.092616+0000","last_clean":"2024-06-06T08:40:23.092616+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:40:23.092616+0000","last_undegraded":"2024-06-06T08:40:23.092616+0000","last_fullsized":"2024-06-06T08:40:23.092616+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71690,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":374,"num_read_kb":258,"num_write":44,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397172+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:40:17.397172+0000","last_peered":"2024-06-06T08:40:17.397172+0000","last_clean":"2024-06-06T08:40:17.397172+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:40:17.397172+0000","last_undegraded":"2024-06-06T08:40:17.397172+0000","last_fullsized":"2024-06-06T08:40:17.397172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291329+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:40:17.291329+0000","last_peered":"2024-06-06T08:40:17.291329+0000","last_clean":"2024-06-06T08:40:17.291329+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:40:17.291329+0000","last_undegraded":"2024-06-06T08:40:17.291329+0000","last_fullsized":"2024-06-06T08:40:17.291329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289082+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:40:17.289082+0000","last_peered":"2024-06-06T08:40:17.289082+0000","last_clean":"2024-06-06T08:40:17.289082+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:40:17.289082+0000","last_undegraded":"2024-06-06T08:40:17.289082+0000","last_fullsized":"2024-06-06T08:40:17.289082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543337+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:40:17.543337+0000","last_peered":"2024-06-06T08:40:17.543337+0000","last_clean":"2024-06-06T08:40:17.543337+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:40:17.543337+0000","last_undegraded":"2024-06-06T08:40:17.543337+0000","last_fullsized":"2024-06-06T08:40:17.543337+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403613+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:40:17.403613+0000","last_peered":"2024-06-06T08:40:17.403613+0000","last_clean":"2024-06-06T08:40:17.403613+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:40:17.403613+0000","last_undegraded":"2024-06-06T08:40:17.403613+0000","last_fullsized":"2024-06-06T08:40:17.403613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290581+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:40:17.290581+0000","last_peered":"2024-06-06T08:40:17.290581+0000","last_clean":"2024-06-06T08:40:17.290581+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:40:17.290581+0000","last_undegraded":"2024-06-06T08:40:17.290581+0000","last_fullsized":"2024-06-06T08:40:17.290581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394932+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:40:17.394932+0000","last_peered":"2024-06-06T08:40:17.394932+0000","last_clean":"2024-06-06T08:40:17.394932+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:40:17.394932+0000","last_undegraded":"2024-06-06T08:40:17.394932+0000","last_fullsized":"2024-06-06T08:40:17.394932+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'6","reported_seq":232,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:18.085943+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:40:18.085943+0000","last_peered":"2024-06-06T08:40:18.085943+0000","last_clean":"2024-06-06T08:40:18.085943+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:40:18.085943+0000","last_undegraded":"2024-06-06T08:40:18.085943+0000","last_fullsized":"2024-06-06T08:40:18.085943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290588+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:40:17.290588+0000","last_peered":"2024-06-06T08:40:17.290588+0000","last_clean":"2024-06-06T08:40:17.290588+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:40:17.290588+0000","last_undegraded":"2024-06-06T08:40:17.290588+0000","last_fullsized":"2024-06-06T08:40:17.290588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289726+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:40:17.289726+0000","last_peered":"2024-06-06T08:40:17.289726+0000","last_clean":"2024-06-06T08:40:17.289726+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:40:17.289726+0000","last_undegraded":"2024-06-06T08:40:17.289726+0000","last_fullsized":"2024-06-06T08:40:17.289726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396877+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:40:17.396877+0000","last_peered":"2024-06-06T08:40:17.396877+0000","last_clean":"2024-06-06T08:40:17.396877+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:40:17.396877+0000","last_undegraded":"2024-06-06T08:40:17.396877+0000","last_fullsized":"2024-06-06T08:40:17.396877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403466+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:40:17.403466+0000","last_peered":"2024-06-06T08:40:17.403466+0000","last_clean":"2024-06-06T08:40:17.403466+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:40:17.403466+0000","last_undegraded":"2024-06-06T08:40:17.403466+0000","last_fullsized":"2024-06-06T08:40:17.403466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403457+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:40:17.403457+0000","last_peered":"2024-06-06T08:40:17.403457+0000","last_clean":"2024-06-06T08:40:17.403457+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:40:17.403457+0000","last_undegraded":"2024-06-06T08:40:17.403457+0000","last_fullsized":"2024-06-06T08:40:17.403457+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"100'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397313+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:40:17.397313+0000","last_peered":"2024-06-06T08:40:17.397313+0000","last_clean":"2024-06-06T08:40:17.397313+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:40:17.397313+0000","last_undegraded":"2024-06-06T08:40:17.397313+0000","last_fullsized":"2024-06-06T08:40:17.397313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543570+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:40:17.543570+0000","last_peered":"2024-06-06T08:40:17.543570+0000","last_clean":"2024-06-06T08:40:17.543570+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:40:17.543570+0000","last_undegraded":"2024-06-06T08:40:17.543570+0000","last_fullsized":"2024-06-06T08:40:17.543570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394732+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:40:17.394732+0000","last_peered":"2024-06-06T08:40:17.394732+0000","last_clean":"2024-06-06T08:40:17.394732+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:40:17.394732+0000","last_undegraded":"2024-06-06T08:40:17.394732+0000","last_fullsized":"2024-06-06T08:40:17.394732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406540+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:40:17.406540+0000","last_peered":"2024-06-06T08:40:17.406540+0000","last_clean":"2024-06-06T08:40:17.406540+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:40:17.406540+0000","last_undegraded":"2024-06-06T08:40:17.406540+0000","last_fullsized":"2024-06-06T08:40:17.406540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289289+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:40:17.289289+0000","last_peered":"2024-06-06T08:40:17.289289+0000","last_clean":"2024-06-06T08:40:17.289289+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:40:17.289289+0000","last_undegraded":"2024-06-06T08:40:17.289289+0000","last_fullsized":"2024-06-06T08:40:17.289289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290584+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:40:17.290584+0000","last_peered":"2024-06-06T08:40:17.290584+0000","last_clean":"2024-06-06T08:40:17.290584+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:40:17.290584+0000","last_undegraded":"2024-06-06T08:40:17.290584+0000","last_fullsized":"2024-06-06T08:40:17.290584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406420+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:40:17.406420+0000","last_peered":"2024-06-06T08:40:17.406420+0000","last_clean":"2024-06-06T08:40:17.406420+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:40:17.406420+0000","last_undegraded":"2024-06-06T08:40:17.406420+0000","last_fullsized":"2024-06-06T08:40:17.406420+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289488+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:40:17.289488+0000","last_peered":"2024-06-06T08:40:17.289488+0000","last_clean":"2024-06-06T08:40:17.289488+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:40:17.289488+0000","last_undegraded":"2024-06-06T08:40:17.289488+0000","last_fullsized":"2024-06-06T08:40:17.289488+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403779+0000","last_change":"2024-06-06T08:38:01.545781+0000","last_active":"2024-06-06T08:40:17.403779+0000","last_peered":"2024-06-06T08:40:17.403779+0000","last_clean":"2024-06-06T08:40:17.403779+0000","last_became_active":"2024-06-06T08:38:01.545462+0000","last_became_peered":"2024-06-06T08:38:01.545462+0000","last_unstale":"2024-06-06T08:40:17.403779+0000","last_undegraded":"2024-06-06T08:40:17.403779+0000","last_fullsized":"2024-06-06T08:40:17.403779+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:14:36.722960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":232,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:22.091205+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:40:22.091205+0000","last_peered":"2024-06-06T08:40:22.091205+0000","last_clean":"2024-06-06T08:40:22.091205+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:40:22.091205+0000","last_undegraded":"2024-06-06T08:40:22.091205+0000","last_fullsized":"2024-06-06T08:40:22.091205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290959+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:40:17.290959+0000","last_peered":"2024-06-06T08:40:17.290959+0000","last_clean":"2024-06-06T08:40:17.290959+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:40:17.290959+0000","last_undegraded":"2024-06-06T08:40:17.290959+0000","last_fullsized":"2024-06-06T08:40:17.290959+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397121+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:40:17.397121+0000","last_peered":"2024-06-06T08:40:17.397121+0000","last_clean":"2024-06-06T08:40:17.397121+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:40:17.397121+0000","last_undegraded":"2024-06-06T08:40:17.397121+0000","last_fullsized":"2024-06-06T08:40:17.397121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602368+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:40:17.602368+0000","last_peered":"2024-06-06T08:40:17.602368+0000","last_clean":"2024-06-06T08:40:17.602368+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:40:17.602368+0000","last_undegraded":"2024-06-06T08:40:17.602368+0000","last_fullsized":"2024-06-06T08:40:17.602368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":378,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290379+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:40:17.290379+0000","last_peered":"2024-06-06T08:40:17.290379+0000","last_clean":"2024-06-06T08:40:17.290379+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:40:17.290379+0000","last_undegraded":"2024-06-06T08:40:17.290379+0000","last_fullsized":"2024-06-06T08:40:17.290379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288818+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:40:17.288818+0000","last_peered":"2024-06-06T08:40:17.288818+0000","last_clean":"2024-06-06T08:40:17.288818+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:40:17.288818+0000","last_undegraded":"2024-06-06T08:40:17.288818+0000","last_fullsized":"2024-06-06T08:40:17.288818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543213+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:40:17.543213+0000","last_peered":"2024-06-06T08:40:17.543213+0000","last_clean":"2024-06-06T08:40:17.543213+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:40:17.543213+0000","last_undegraded":"2024-06-06T08:40:17.543213+0000","last_fullsized":"2024-06-06T08:40:17.543213+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543287+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:40:17.543287+0000","last_peered":"2024-06-06T08:40:17.543287+0000","last_clean":"2024-06-06T08:40:17.543287+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:40:17.543287+0000","last_undegraded":"2024-06-06T08:40:17.543287+0000","last_fullsized":"2024-06-06T08:40:17.543287+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543496+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:40:17.543496+0000","last_peered":"2024-06-06T08:40:17.543496+0000","last_clean":"2024-06-06T08:40:17.543496+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:40:17.543496+0000","last_undegraded":"2024-06-06T08:40:17.543496+0000","last_fullsized":"2024-06-06T08:40:17.543496+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289481+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:40:17.289481+0000","last_peered":"2024-06-06T08:40:17.289481+0000","last_clean":"2024-06-06T08:40:17.289481+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:40:17.289481+0000","last_undegraded":"2024-06-06T08:40:17.289481+0000","last_fullsized":"2024-06-06T08:40:17.289481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397142+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:40:17.397142+0000","last_peered":"2024-06-06T08:40:17.397142+0000","last_clean":"2024-06-06T08:40:17.397142+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:40:17.397142+0000","last_undegraded":"2024-06-06T08:40:17.397142+0000","last_fullsized":"2024-06-06T08:40:17.397142+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'49","reported_seq":119,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.518606+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:40:17.518606+0000","last_peered":"2024-06-06T08:40:17.518606+0000","last_clean":"2024-06-06T08:40:17.518606+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:40:17.518606+0000","last_undegraded":"2024-06-06T08:40:17.518606+0000","last_fullsized":"2024-06-06T08:40:17.518606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397610+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:40:17.397610+0000","last_peered":"2024-06-06T08:40:17.397610+0000","last_clean":"2024-06-06T08:40:17.397610+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:40:17.397610+0000","last_undegraded":"2024-06-06T08:40:17.397610+0000","last_fullsized":"2024-06-06T08:40:17.397610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290756+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:40:17.290756+0000","last_peered":"2024-06-06T08:40:17.290756+0000","last_clean":"2024-06-06T08:40:17.290756+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:40:17.290756+0000","last_undegraded":"2024-06-06T08:40:17.290756+0000","last_fullsized":"2024-06-06T08:40:17.290756+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603067+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:40:17.603067+0000","last_peered":"2024-06-06T08:40:17.603067+0000","last_clean":"2024-06-06T08:40:17.603067+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:40:17.603067+0000","last_undegraded":"2024-06-06T08:40:17.603067+0000","last_fullsized":"2024-06-06T08:40:17.603067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603700+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:40:17.603700+0000","last_peered":"2024-06-06T08:40:17.603700+0000","last_clean":"2024-06-06T08:40:17.603700+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:40:17.603700+0000","last_undegraded":"2024-06-06T08:40:17.603700+0000","last_fullsized":"2024-06-06T08:40:17.603700+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602722+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:40:17.602722+0000","last_peered":"2024-06-06T08:40:17.602722+0000","last_clean":"2024-06-06T08:40:17.602722+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:40:17.602722+0000","last_undegraded":"2024-06-06T08:40:17.602722+0000","last_fullsized":"2024-06-06T08:40:17.602722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618369+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:40:17.618369+0000","last_peered":"2024-06-06T08:40:17.618369+0000","last_clean":"2024-06-06T08:40:17.618369+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:40:17.618369+0000","last_undegraded":"2024-06-06T08:40:17.618369+0000","last_fullsized":"2024-06-06T08:40:17.618369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396892+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:40:17.396892+0000","last_peered":"2024-06-06T08:40:17.396892+0000","last_clean":"2024-06-06T08:40:17.396892+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:40:17.396892+0000","last_undegraded":"2024-06-06T08:40:17.396892+0000","last_fullsized":"2024-06-06T08:40:17.396892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.395030+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:40:17.395030+0000","last_peered":"2024-06-06T08:40:17.395030+0000","last_clean":"2024-06-06T08:40:17.395030+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:40:17.395030+0000","last_undegraded":"2024-06-06T08:40:17.395030+0000","last_fullsized":"2024-06-06T08:40:17.395030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397132+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:40:17.397132+0000","last_peered":"2024-06-06T08:40:17.397132+0000","last_clean":"2024-06-06T08:40:17.397132+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:40:17.397132+0000","last_undegraded":"2024-06-06T08:40:17.397132+0000","last_fullsized":"2024-06-06T08:40:17.397132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543302+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:40:17.543302+0000","last_peered":"2024-06-06T08:40:17.543302+0000","last_clean":"2024-06-06T08:40:17.543302+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:40:17.543302+0000","last_undegraded":"2024-06-06T08:40:17.543302+0000","last_fullsized":"2024-06-06T08:40:17.543302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602805+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:40:17.602805+0000","last_peered":"2024-06-06T08:40:17.602805+0000","last_clean":"2024-06-06T08:40:17.602805+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:40:17.602805+0000","last_undegraded":"2024-06-06T08:40:17.602805+0000","last_fullsized":"2024-06-06T08:40:17.602805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291108+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:40:17.291108+0000","last_peered":"2024-06-06T08:40:17.291108+0000","last_clean":"2024-06-06T08:40:17.291108+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:40:17.291108+0000","last_undegraded":"2024-06-06T08:40:17.291108+0000","last_fullsized":"2024-06-06T08:40:17.291108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396937+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:40:17.396937+0000","last_peered":"2024-06-06T08:40:17.396937+0000","last_clean":"2024-06-06T08:40:17.396937+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:40:17.396937+0000","last_undegraded":"2024-06-06T08:40:17.396937+0000","last_fullsized":"2024-06-06T08:40:17.396937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602730+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:40:17.602730+0000","last_peered":"2024-06-06T08:40:17.602730+0000","last_clean":"2024-06-06T08:40:17.602730+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:40:17.602730+0000","last_undegraded":"2024-06-06T08:40:17.602730+0000","last_fullsized":"2024-06-06T08:40:17.602730+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396987+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:40:17.396987+0000","last_peered":"2024-06-06T08:40:17.396987+0000","last_clean":"2024-06-06T08:40:17.396987+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:40:17.396987+0000","last_undegraded":"2024-06-06T08:40:17.396987+0000","last_fullsized":"2024-06-06T08:40:17.396987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290632+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:40:17.290632+0000","last_peered":"2024-06-06T08:40:17.290632+0000","last_clean":"2024-06-06T08:40:17.290632+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:40:17.290632+0000","last_undegraded":"2024-06-06T08:40:17.290632+0000","last_fullsized":"2024-06-06T08:40:17.290632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618976+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:40:17.618976+0000","last_peered":"2024-06-06T08:40:17.618976+0000","last_clean":"2024-06-06T08:40:17.618976+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:40:17.618976+0000","last_undegraded":"2024-06-06T08:40:17.618976+0000","last_fullsized":"2024-06-06T08:40:17.618976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291175+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:40:17.291175+0000","last_peered":"2024-06-06T08:40:17.291175+0000","last_clean":"2024-06-06T08:40:17.291175+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:40:17.291175+0000","last_undegraded":"2024-06-06T08:40:17.291175+0000","last_fullsized":"2024-06-06T08:40:17.291175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291116+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:40:17.291116+0000","last_peered":"2024-06-06T08:40:17.291116+0000","last_clean":"2024-06-06T08:40:17.291116+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:40:17.291116+0000","last_undegraded":"2024-06-06T08:40:17.291116+0000","last_fullsized":"2024-06-06T08:40:17.291116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603292+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:40:17.603292+0000","last_peered":"2024-06-06T08:40:17.603292+0000","last_clean":"2024-06-06T08:40:17.603292+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:40:17.603292+0000","last_undegraded":"2024-06-06T08:40:17.603292+0000","last_fullsized":"2024-06-06T08:40:17.603292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'39","reported_seq":94,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.513053+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:40:17.513053+0000","last_peered":"2024-06-06T08:40:17.513053+0000","last_clean":"2024-06-06T08:40:17.513053+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:40:17.513053+0000","last_undegraded":"2024-06-06T08:40:17.513053+0000","last_fullsized":"2024-06-06T08:40:17.513053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290659+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:40:17.290659+0000","last_peered":"2024-06-06T08:40:17.290659+0000","last_clean":"2024-06-06T08:40:17.290659+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:40:17.290659+0000","last_undegraded":"2024-06-06T08:40:17.290659+0000","last_fullsized":"2024-06-06T08:40:17.290659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403261+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:40:17.403261+0000","last_peered":"2024-06-06T08:40:17.403261+0000","last_clean":"2024-06-06T08:40:17.403261+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:40:17.403261+0000","last_undegraded":"2024-06-06T08:40:17.403261+0000","last_fullsized":"2024-06-06T08:40:17.403261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289754+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:40:17.289754+0000","last_peered":"2024-06-06T08:40:17.289754+0000","last_clean":"2024-06-06T08:40:17.289754+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:40:17.289754+0000","last_undegraded":"2024-06-06T08:40:17.289754+0000","last_fullsized":"2024-06-06T08:40:17.289754+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291007+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:40:17.291007+0000","last_peered":"2024-06-06T08:40:17.291007+0000","last_clean":"2024-06-06T08:40:17.291007+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:40:17.291007+0000","last_undegraded":"2024-06-06T08:40:17.291007+0000","last_fullsized":"2024-06-06T08:40:17.291007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543466+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:40:17.543466+0000","last_peered":"2024-06-06T08:40:17.543466+0000","last_clean":"2024-06-06T08:40:17.543466+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:40:17.543466+0000","last_undegraded":"2024-06-06T08:40:17.543466+0000","last_fullsized":"2024-06-06T08:40:17.543466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291000+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:40:17.291000+0000","last_peered":"2024-06-06T08:40:17.291000+0000","last_clean":"2024-06-06T08:40:17.291000+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:40:17.291000+0000","last_undegraded":"2024-06-06T08:40:17.291000+0000","last_fullsized":"2024-06-06T08:40:17.291000+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290895+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:40:17.290895+0000","last_peered":"2024-06-06T08:40:17.290895+0000","last_clean":"2024-06-06T08:40:17.290895+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:40:17.290895+0000","last_undegraded":"2024-06-06T08:40:17.290895+0000","last_fullsized":"2024-06-06T08:40:17.290895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603342+0000","last_change":"2024-06-06T08:38:01.151003+0000","last_active":"2024-06-06T08:40:17.603342+0000","last_peered":"2024-06-06T08:40:17.603342+0000","last_clean":"2024-06-06T08:40:17.603342+0000","last_became_active":"2024-06-06T08:38:01.150557+0000","last_became_peered":"2024-06-06T08:38:01.150557+0000","last_unstale":"2024-06-06T08:40:17.603342+0000","last_undegraded":"2024-06-06T08:40:17.603342+0000","last_fullsized":"2024-06-06T08:40:17.603342+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:29:44.629894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289585+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:40:17.289585+0000","last_peered":"2024-06-06T08:40:17.289585+0000","last_clean":"2024-06-06T08:40:17.289585+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:40:17.289585+0000","last_undegraded":"2024-06-06T08:40:17.289585+0000","last_fullsized":"2024-06-06T08:40:17.289585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543470+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:40:17.543470+0000","last_peered":"2024-06-06T08:40:17.543470+0000","last_clean":"2024-06-06T08:40:17.543470+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:40:17.543470+0000","last_undegraded":"2024-06-06T08:40:17.543470+0000","last_fullsized":"2024-06-06T08:40:17.543470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291317+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:40:17.291317+0000","last_peered":"2024-06-06T08:40:17.291317+0000","last_clean":"2024-06-06T08:40:17.291317+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:40:17.291317+0000","last_undegraded":"2024-06-06T08:40:17.291317+0000","last_fullsized":"2024-06-06T08:40:17.291317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397744+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:40:17.397744+0000","last_peered":"2024-06-06T08:40:17.397744+0000","last_clean":"2024-06-06T08:40:17.397744+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:40:17.397744+0000","last_undegraded":"2024-06-06T08:40:17.397744+0000","last_fullsized":"2024-06-06T08:40:17.397744+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403480+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:40:17.403480+0000","last_peered":"2024-06-06T08:40:17.403480+0000","last_clean":"2024-06-06T08:40:17.403480+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:40:17.403480+0000","last_undegraded":"2024-06-06T08:40:17.403480+0000","last_fullsized":"2024-06-06T08:40:17.403480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397272+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:40:17.397272+0000","last_peered":"2024-06-06T08:40:17.397272+0000","last_clean":"2024-06-06T08:40:17.397272+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:40:17.397272+0000","last_undegraded":"2024-06-06T08:40:17.397272+0000","last_fullsized":"2024-06-06T08:40:17.397272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618845+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:40:17.618845+0000","last_peered":"2024-06-06T08:40:17.618845+0000","last_clean":"2024-06-06T08:40:17.618845+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:40:17.618845+0000","last_undegraded":"2024-06-06T08:40:17.618845+0000","last_fullsized":"2024-06-06T08:40:17.618845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289213+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:40:17.289213+0000","last_peered":"2024-06-06T08:40:17.289213+0000","last_clean":"2024-06-06T08:40:17.289213+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:40:17.289213+0000","last_undegraded":"2024-06-06T08:40:17.289213+0000","last_fullsized":"2024-06-06T08:40:17.289213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543444+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:40:17.543444+0000","last_peered":"2024-06-06T08:40:17.543444+0000","last_clean":"2024-06-06T08:40:17.543444+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:40:17.543444+0000","last_undegraded":"2024-06-06T08:40:17.543444+0000","last_fullsized":"2024-06-06T08:40:17.543444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107163,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601735,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1195,"num_read_kb":718,"num_write":304,"num_write_kb":1451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":1799775,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383743,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":569943,"data_compressed":32387,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645382,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32428,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712724,"statfs":{"total":95995035648,"available":95961829376,"internally_reserved":0,"allocated":1282048,"data_stored":1371722,"data_compressed":77234,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1150976,"data_stored":977401,"data_compressed":54503,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070543,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1142784,"data_stored":973203,"data_compressed":54503,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332180,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1171456,"data_stored":1177146,"data_compressed":71078,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495707,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":575265,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1175552,"data_stored":643955,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":576653,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577223,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1100275,"data_compressed":60659,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920868,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1191936,"data_stored":649616,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757342,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":1303023,"data_compressed":77395,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133774,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":206179,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:40:24.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:40:24.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:24 smithi027 ceph-mon[32335]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.0 KiB/s wr, 5 op/s 2024-06-06T08:40:24.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:24 smithi027 ceph-mon[32335]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:24.731 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:24.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:24 smithi203 ceph-mon[36579]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.0 KiB/s wr, 5 op/s 2024-06-06T08:40:24.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:24 smithi203 ceph-mon[36579]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:25.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:24 smithi176 ceph-mon[36786]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.0 KiB/s wr, 5 op/s 2024-06-06T08:40:25.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:24 smithi176 ceph-mon[36786]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:25.865 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:25 smithi027 ceph-mon[32335]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:25.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:25 smithi027 ceph-mon[32335]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:25.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:25 smithi203 ceph-mon[36579]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:25.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:25 smithi203 ceph-mon[36579]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:26.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:25 smithi176 ceph-mon[36786]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:26.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:25 smithi176 ceph-mon[36786]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:26.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.538+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2242414074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 msgr2=0x7ff7a41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:26.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.538+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2242414074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 0x7ff7a41077e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff794009990 tx=0x7ff794009ca0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.539+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2242414074 shutdown_connections 2024-06-06T08:40:26.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.539+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2242414074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101d50 0x7ff7a410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.539+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2242414074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 0x7ff7a41077e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.539+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2242414074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a4101390 0x7ff7a4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.539+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2242414074 >> 172.21.15.27:0/2242414074 conn(0x7ff7a4078340 msgr2=0x7ff7a4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:26.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.539+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2242414074 shutdown_connections 2024-06-06T08:40:26.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.539+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2242414074 wait complete. 2024-06-06T08:40:26.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a8ddf700 1 Processor -- start 2024-06-06T08:40:26.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a8ddf700 1 -- start start 2024-06-06T08:40:26.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a8ddf700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a4101390 0x7ff7a4199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a8ddf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101d50 0x7ff7a419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a8ddf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 0x7ff7a419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a8ddf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41116a0 con 0x7ff7a4107400 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a8ddf700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111520 con 0x7ff7a4101390 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a8ddf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111820 con 0x7ff7a4101d50 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 0x7ff7a419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 0x7ff7a419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58928/0 (socket says 172.21.15.27:58928) 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a2d9d700 1 -- 172.21.15.27:0/2793817327 learned_addr learned my addr 172.21.15.27:0/2793817327 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a1d9b700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101d50 0x7ff7a419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:26.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a259c700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a4101390 0x7ff7a4199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a2d9d700 1 -- 172.21.15.27:0/2793817327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101d50 msgr2=0x7ff7a419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.540+0000 7ff7a2d9d700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101d50 0x7ff7a419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.541+0000 7ff7a2d9d700 1 -- 172.21.15.27:0/2793817327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a4101390 msgr2=0x7ff7a4199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.541+0000 7ff7a2d9d700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a4101390 0x7ff7a4199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.541+0000 7ff7a2d9d700 1 -- 172.21.15.27:0/2793817327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a41a4f90 con 0x7ff7a4107400 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.541+0000 7ff7a2d9d700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 0x7ff7a419a620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00ef30 tx=0x7ff79c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.541+0000 7ff7937fe700 1 -- 172.21.15.27:0/2793817327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0092e0 con 0x7ff7a4107400 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.541+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a41a5220 con 0x7ff7a4107400 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.541+0000 7ff7937fe700 1 -- 172.21.15.27:0/2793817327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c004bb0 con 0x7ff7a4107400 2024-06-06T08:40:26.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.541+0000 7ff7937fe700 1 -- 172.21.15.27:0/2793817327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c004280 con 0x7ff7a4107400 2024-06-06T08:40:26.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.541+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a41a57b0 con 0x7ff7a4107400 2024-06-06T08:40:26.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.542+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a4069180 con 0x7ff7a4107400 2024-06-06T08:40:26.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.544+0000 7ff7937fe700 1 -- 172.21.15.27:0/2793817327 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7ff79c01b030 con 0x7ff7a4107400 2024-06-06T08:40:26.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.545+0000 7ff7937fe700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff78c0a2f30 0x7ff78c0a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:26.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.545+0000 7ff7a259c700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff78c0a2f30 0x7ff78c0a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:26.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.545+0000 7ff7937fe700 1 -- 172.21.15.27:0/2793817327 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7ff79c0cfdf0 con 0x7ff7a4107400 2024-06-06T08:40:26.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.545+0000 7ff7a259c700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff78c0a2f30 0x7ff78c0a53f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff794006010 tx=0x7ff79400b280 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:26.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.548+0000 7ff7937fe700 1 -- 172.21.15.27:0/2793817327 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff79c022020 con 0x7ff7a4107400 2024-06-06T08:40:26.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:26 smithi027 ceph-mon[32335]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1 KiB/s wr, 23 op/s 2024-06-06T08:40:26.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.799+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7a406a710 con 0x7ff7a4107400 2024-06-06T08:40:26.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.800+0000 7ff7937fe700 1 -- 172.21.15.27:0/2793817327 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7ff79c010040 con 0x7ff7a4107400 2024-06-06T08:40:26.798 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:26.798 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:40:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff78c0a2f30 msgr2=0x7ff78c0a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff78c0a2f30 0x7ff78c0a53f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff794006010 tx=0x7ff79400b280 comp rx=0 tx=0).stop 2024-06-06T08:40:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 msgr2=0x7ff7a419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 0x7ff7a419a620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00ef30 tx=0x7ff79c00c3a0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 shutdown_connections 2024-06-06T08:40:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7a4107400 0x7ff7a419a620 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff78c0a2f30 0x7ff78c0a53f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff7a4101390 0x7ff7a4199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 --2- 172.21.15.27:0/2793817327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4101d50 0x7ff7a419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 >> 172.21.15.27:0/2793817327 conn(0x7ff7a4078340 msgr2=0x7ff7a40fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 shutdown_connections 2024-06-06T08:40:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:26.802+0000 7ff7a8ddf700 1 -- 172.21.15.27:0/2793817327 wait complete. 2024-06-06T08:40:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:40:26.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:26 smithi203 ceph-mon[36579]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1 KiB/s wr, 23 op/s 2024-06-06T08:40:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:26 smithi176 ceph-mon[36786]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1 KiB/s wr, 23 op/s 2024-06-06T08:40:27.485 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:40:27.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:27 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2793817327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:27.739 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:27.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:27 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2793817327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:28.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:27 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2793817327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:28.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:28 smithi027 ceph-mon[32335]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 796 B/s wr, 17 op/s 2024-06-06T08:40:28.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:28 smithi203 ceph-mon[36579]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 796 B/s wr, 17 op/s 2024-06-06T08:40:29.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:28 smithi176 ceph-mon[36786]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 796 B/s wr, 17 op/s 2024-06-06T08:40:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.459+0000 7fc538131700 1 -- 172.21.15.27:0/2464248213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 msgr2=0x7fc5301089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.459+0000 7fc538131700 1 --2- 172.21.15.27:0/2464248213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 0x7fc5301089d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009990 tx=0x7fc52c009ca0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.459+0000 7fc538131700 1 -- 172.21.15.27:0/2464248213 shutdown_connections 2024-06-06T08:40:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.459+0000 7fc538131700 1 --2- 172.21.15.27:0/2464248213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc530102ea0 0x7fc53010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.459+0000 7fc538131700 1 --2- 172.21.15.27:0/2464248213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 0x7fc5301089d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.459+0000 7fc538131700 1 --2- 172.21.15.27:0/2464248213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5301024e0 0x7fc530102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.459+0000 7fc538131700 1 -- 172.21.15.27:0/2464248213 >> 172.21.15.27:0/2464248213 conn(0x7fc530079640 msgr2=0x7fc530079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.459+0000 7fc538131700 1 -- 172.21.15.27:0/2464248213 shutdown_connections 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.459+0000 7fc538131700 1 -- 172.21.15.27:0/2464248213 wait complete. 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc538131700 1 Processor -- start 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc538131700 1 -- start start 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc538131700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5301024e0 0x7fc53019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc538131700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc530102ea0 0x7fc53019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc538131700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 0x7fc530198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc538131700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc530112790 con 0x7fc5301085f0 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc538131700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc530112610 con 0x7fc530102ea0 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc538131700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc530112910 con 0x7fc5301024e0 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc5366ce700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 0x7fc530198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc5366ce700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 0x7fc530198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58942/0 (socket says 172.21.15.27:58942) 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc5366ce700 1 -- 172.21.15.27:0/1375981520 learned_addr learned my addr 172.21.15.27:0/1375981520 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc535ecd700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5301024e0 0x7fc53019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:29.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc5356cc700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc530102ea0 0x7fc53019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:29.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc5366ce700 1 -- 172.21.15.27:0/1375981520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5301024e0 msgr2=0x7fc53019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:29.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc5366ce700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5301024e0 0x7fc53019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc5366ce700 1 -- 172.21.15.27:0/1375981520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc530102ea0 msgr2=0x7fc53019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:29.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc5366ce700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc530102ea0 0x7fc53019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.460+0000 7fc5366ce700 1 -- 172.21.15.27:0/1375981520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc530199170 con 0x7fc5301085f0 2024-06-06T08:40:29.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.461+0000 7fc5366ce700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 0x7fc530198a10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc52800cca0 tx=0x7fc52800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:29.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.461+0000 7fc526ffd700 1 -- 172.21.15.27:0/1375981520 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528016020 con 0x7fc5301085f0 2024-06-06T08:40:29.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.461+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc530199460 con 0x7fc5301085f0 2024-06-06T08:40:29.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.461+0000 7fc526ffd700 1 -- 172.21.15.27:0/1375981520 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc528012420 con 0x7fc5301085f0 2024-06-06T08:40:29.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.461+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5301a5720 con 0x7fc5301085f0 2024-06-06T08:40:29.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.461+0000 7fc526ffd700 1 -- 172.21.15.27:0/1375981520 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5280115e0 con 0x7fc5301085f0 2024-06-06T08:40:29.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.462+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc514005330 con 0x7fc5301085f0 2024-06-06T08:40:29.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.464+0000 7fc526ffd700 1 -- 172.21.15.27:0/1375981520 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fc528019020 con 0x7fc5301085f0 2024-06-06T08:40:29.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.464+0000 7fc526ffd700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc51c0a2eb0 0x7fc51c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:29.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.465+0000 7fc535ecd700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc51c0a2eb0 0x7fc51c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:29.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.465+0000 7fc526ffd700 1 -- 172.21.15.27:0/1375981520 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fc5280cfa20 con 0x7fc5301085f0 2024-06-06T08:40:29.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.465+0000 7fc535ecd700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc51c0a2eb0 0x7fc51c0a5370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b5c0 tx=0x7fc52c005960 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:29.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.468+0000 7fc526ffd700 1 -- 172.21.15.27:0/1375981520 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc528020020 con 0x7fc5301085f0 2024-06-06T08:40:29.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.674+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5140058f0 con 0x7fc5301085f0 2024-06-06T08:40:29.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.676+0000 7fc526ffd700 1 -- 172.21.15.27:0/1375981520 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25213 (secure 0 0 0) 0x7fc52809c140 con 0x7fc5301085f0 2024-06-06T08:40:29.676 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:29.676 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:40:17.280912+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/876705499":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/550980984":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/4174349888":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3053481437":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:40:29.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc51c0a2eb0 msgr2=0x7fc51c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:29.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc51c0a2eb0 0x7fc51c0a5370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b5c0 tx=0x7fc52c005960 comp rx=0 tx=0).stop 2024-06-06T08:40:29.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 msgr2=0x7fc530198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:29.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 0x7fc530198a10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc52800cca0 tx=0x7fc52800a060 comp rx=0 tx=0).stop 2024-06-06T08:40:29.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 shutdown_connections 2024-06-06T08:40:29.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5301085f0 0x7fc530198a10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5301024e0 0x7fc53019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc51c0a2eb0 0x7fc51c0a5370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 --2- 172.21.15.27:0/1375981520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc530102ea0 0x7fc53019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:29.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 >> 172.21.15.27:0/1375981520 conn(0x7fc530079640 msgr2=0x7fc530100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:29.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 shutdown_connections 2024-06-06T08:40:29.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:29.679+0000 7fc538131700 1 -- 172.21.15.27:0/1375981520 wait complete. 2024-06-06T08:40:30.470 DEBUG:tasks.fwd_scrub:Yielding 2024-06-06T08:40:30.470 INFO:teuthology.run_tasks:Running task exec... 2024-06-06T08:40:30.479 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T08:40:30.479 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:40:30.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-06-06T08:40:30.480 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-06T08:40:30.480 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T08:40:30.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T08:40:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.594+0000 7fcd83d89640 1 -- 172.21.15.27:0/1523285098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c102370 msgr2=0x7fcd7c10e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.594+0000 7fcd83d89640 1 --2- 172.21.15.27:0/1523285098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c102370 0x7fcd7c10e880 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009cd0 tx=0x7fcd7402a8e0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.594+0000 7fcd83d89640 1 -- 172.21.15.27:0/1523285098 shutdown_connections 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.594+0000 7fcd83d89640 1 --2- 172.21.15.27:0/1523285098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c102370 0x7fcd7c10e880 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.594+0000 7fcd83d89640 1 --2- 172.21.15.27:0/1523285098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd7c1019b0 0x7fcd7c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.594+0000 7fcd83d89640 1 --2- 172.21.15.27:0/1523285098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd7c1079c0 0x7fcd7c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.594+0000 7fcd83d89640 1 -- 172.21.15.27:0/1523285098 >> 172.21.15.27:0/1523285098 conn(0x7fcd7c0fd4a0 msgr2=0x7fcd7c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 -- 172.21.15.27:0/1523285098 shutdown_connections 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 -- 172.21.15.27:0/1523285098 wait complete. 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 Processor -- start 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 -- start start 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd7c1019b0 0x7fcd7c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:30.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd7c102370 0x7fcd7c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:30.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c1079c0 0x7fcd7c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:30.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c111dd0 con 0x7fcd7c1079c0 2024-06-06T08:40:30.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c111c50 con 0x7fcd7c102370 2024-06-06T08:40:30.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.595+0000 7fcd83d89640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c111f50 con 0x7fcd7c1019b0 2024-06-06T08:40:30.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd82326640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c1079c0 0x7fcd7c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:30.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd82326640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c1079c0 0x7fcd7c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33970/0 (socket says 172.21.15.27:33970) 2024-06-06T08:40:30.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd82326640 1 -- 172.21.15.27:0/4251900607 learned_addr learned my addr 172.21.15.27:0/4251900607 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:30.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd81b25640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd7c1019b0 0x7fcd7c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:30.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd82326640 1 -- 172.21.15.27:0/4251900607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd7c1019b0 msgr2=0x7fcd7c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:30.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd81324640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd7c102370 0x7fcd7c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:30.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd82326640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd7c1019b0 0x7fcd7c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd82326640 1 -- 172.21.15.27:0/4251900607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd7c102370 msgr2=0x7fcd7c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:30.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd82326640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd7c102370 0x7fcd7c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd82326640 1 -- 172.21.15.27:0/4251900607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c19d1b0 con 0x7fcd7c1079c0 2024-06-06T08:40:30.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd82326640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c1079c0 0x7fcd7c19ca80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcd74005cc0 tx=0x7fcd74032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:30.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd72ffd640 1 -- 172.21.15.27:0/4251900607 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74004040 con 0x7fcd7c1079c0 2024-06-06T08:40:30.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd72ffd640 1 -- 172.21.15.27:0/4251900607 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd740046c0 con 0x7fcd7c1079c0 2024-06-06T08:40:30.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c19d440 con 0x7fcd7c1079c0 2024-06-06T08:40:30.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd72ffd640 1 -- 172.21.15.27:0/4251900607 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74030e00 con 0x7fcd7c1079c0 2024-06-06T08:40:30.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.596+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c1a4d30 con 0x7fcd7c1079c0 2024-06-06T08:40:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.597+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7c04f760 con 0x7fcd7c1079c0 2024-06-06T08:40:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.599+0000 7fcd72ffd640 1 -- 172.21.15.27:0/4251900607 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fcd740305a0 con 0x7fcd7c1079c0 2024-06-06T08:40:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.599+0000 7fcd72ffd640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcd680a2f40 0x7fcd680a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.600+0000 7fcd81b25640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcd680a2f40 0x7fcd680a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.600+0000 7fcd72ffd640 1 -- 172.21.15.27:0/4251900607 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fcd7402e030 con 0x7fcd7c1079c0 2024-06-06T08:40:30.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.600+0000 7fcd81b25640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcd680a2f40 0x7fcd680a5400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005e00 tx=0x7fcd78005c50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.604+0000 7fcd72ffd640 1 -- 172.21.15.27:0/4251900607 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcd7404b060 con 0x7fcd7c1079c0 2024-06-06T08:40:30.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:30 smithi027 ceph-mon[32335]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 651 B/s wr, 17 op/s 2024-06-06T08:40:30.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:30 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1375981520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:30.745 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:30.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.805+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fcd7c1a5040 con 0x7fcd7c1079c0 2024-06-06T08:40:30.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.808+0000 7fcd72ffd640 1 -- 172.21.15.27:0/4251900607 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7fcd740bda20 con 0x7fcd7c1079c0 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcd680a2f40 msgr2=0x7fcd680a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcd680a2f40 0x7fcd680a5400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005e00 tx=0x7fcd78005c50 comp rx=0 tx=0).stop 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c1079c0 msgr2=0x7fcd7c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c1079c0 0x7fcd7c19ca80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcd74005cc0 tx=0x7fcd74032f40 comp rx=0 tx=0).stop 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 shutdown_connections 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcd680a2f40 0x7fcd680a5400 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd7c1079c0 0x7fcd7c19ca80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd7c102370 0x7fcd7c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 --2- 172.21.15.27:0/4251900607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd7c1019b0 0x7fcd7c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 >> 172.21.15.27:0/4251900607 conn(0x7fcd7c0fd4a0 msgr2=0x7fcd7c0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 shutdown_connections 2024-06-06T08:40:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.811+0000 7fcd83d89640 1 -- 172.21.15.27:0/4251900607 wait complete. 2024-06-06T08:40:30.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-06-06T08:40:30.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.936+0000 7f8f2d39a640 1 -- 172.21.15.27:0/701003378 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2002c040 con 0x7f8f2810bdd0 2024-06-06T08:40:30.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.937+0000 7f8f2f39e640 1 -- 172.21.15.27:0/701003378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 msgr2=0x7f8f2810e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:30.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.937+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/701003378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 0x7f8f2810e220 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009ca0 tx=0x7f8f2002a950 comp rx=0 tx=0).stop 2024-06-06T08:40:30.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.937+0000 7f8f2f39e640 1 -- 172.21.15.27:0/701003378 shutdown_connections 2024-06-06T08:40:30.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.937+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/701003378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 0x7f8f2810e220 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.937+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/701003378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f28103320 0x7f8f2810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.937+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/701003378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f28102a00 0x7f8f28102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.937+0000 7f8f2f39e640 1 -- 172.21.15.27:0/701003378 >> 172.21.15.27:0/701003378 conn(0x7f8f280fc1c0 msgr2=0x7f8f280fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:30.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.937+0000 7f8f2f39e640 1 -- 172.21.15.27:0/701003378 shutdown_connections 2024-06-06T08:40:30.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.937+0000 7f8f2f39e640 1 -- 172.21.15.27:0/701003378 wait complete. 2024-06-06T08:40:30.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2f39e640 1 Processor -- start 2024-06-06T08:40:30.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2f39e640 1 -- start start 2024-06-06T08:40:30.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2f39e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f28102a00 0x7f8f28198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:30.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2f39e640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f28103320 0x7f8f281985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:30.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2f39e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 0x7f8f2819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:30.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2f39e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28111b70 con 0x7f8f2810bdd0 2024-06-06T08:40:30.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2f39e640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281119f0 con 0x7f8f28103320 2024-06-06T08:40:30.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2f39e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28111cf0 con 0x7f8f28102a00 2024-06-06T08:40:30.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2eb9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 0x7f8f2819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2eb9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 0x7f8f2819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33990/0 (socket says 172.21.15.27:33990) 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2eb9d640 1 -- 172.21.15.27:0/1687892581 learned_addr learned my addr 172.21.15.27:0/1687892581 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2db9b640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f28103320 0x7f8f281985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2e39c640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f28102a00 0x7f8f28198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2eb9d640 1 -- 172.21.15.27:0/1687892581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f28102a00 msgr2=0x7f8f28198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2eb9d640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f28102a00 0x7f8f28198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2eb9d640 1 -- 172.21.15.27:0/1687892581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f28103320 msgr2=0x7f8f281985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2eb9d640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f28103320 0x7f8f281985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.938+0000 7f8f2eb9d640 1 -- 172.21.15.27:0/1687892581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2819d030 con 0x7f8f2810bdd0 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.939+0000 7f8f2eb9d640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 0x7f8f2819c900 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8f2002ae70 tx=0x7f8f2002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.939+0000 7f8f1f7fe640 1 -- 172.21.15.27:0/1687892581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20034040 con 0x7f8f2810bdd0 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.939+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2819d2c0 con 0x7f8f2810bdd0 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.939+0000 7f8f1f7fe640 1 -- 172.21.15.27:0/1687892581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f20030920 con 0x7f8f2810bdd0 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.939+0000 7f8f1f7fe640 1 -- 172.21.15.27:0/1687892581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20045480 con 0x7f8f2810bdd0 2024-06-06T08:40:30.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.939+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281a4bb0 con 0x7f8f2810bdd0 2024-06-06T08:40:30.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.940+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2804ffb0 con 0x7f8f2810bdd0 2024-06-06T08:40:30.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.941+0000 7f8f1f7fe640 1 -- 172.21.15.27:0/1687892581 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f8f20037020 con 0x7f8f2810bdd0 2024-06-06T08:40:30.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.942+0000 7f8f1f7fe640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f8f140a2e10 0x7f8f140a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:30.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.942+0000 7f8f2e39c640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f8f140a2e10 0x7f8f140a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:30.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.942+0000 7f8f1f7fe640 1 -- 172.21.15.27:0/1687892581 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f8f200f08e0 con 0x7f8f2810bdd0 2024-06-06T08:40:30.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.942+0000 7f8f2e39c640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f8f140a2e10 0x7f8f140a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f24005fd0 tx=0x7f8f24005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:30.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:30.946+0000 7f8f1f7fe640 1 -- 172.21.15.27:0/1687892581 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f200bd000 con 0x7f8f2810bdd0 2024-06-06T08:40:30.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:30 smithi203 ceph-mon[36579]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 651 B/s wr, 17 op/s 2024-06-06T08:40:30.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:30 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1375981520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:31.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:30 smithi176 ceph-mon[36786]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 651 B/s wr, 17 op/s 2024-06-06T08:40:31.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:30 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1375981520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:31.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.142+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f8f28106ec0 con 0x7f8f2810bdd0 2024-06-06T08:40:31.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.146+0000 7f8f1f7fe640 1 -- 172.21.15.27:0/1687892581 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f8f200bc900 con 0x7f8f2810bdd0 2024-06-06T08:40:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f8f140a2e10 msgr2=0x7f8f140a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f8f140a2e10 0x7f8f140a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f24005fd0 tx=0x7f8f24005850 comp rx=0 tx=0).stop 2024-06-06T08:40:31.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 msgr2=0x7f8f2819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 0x7f8f2819c900 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8f2002ae70 tx=0x7f8f2002f5b0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 shutdown_connections 2024-06-06T08:40:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f8f140a2e10 0x7f8f140a52d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2810bdd0 0x7f8f2819c900 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f28103320 0x7f8f281985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 --2- 172.21.15.27:0/1687892581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f28102a00 0x7f8f28198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 >> 172.21.15.27:0/1687892581 conn(0x7f8f280fc1c0 msgr2=0x7f8f280fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 shutdown_connections 2024-06-06T08:40:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.149+0000 7f8f2f39e640 1 -- 172.21.15.27:0/1687892581 wait complete. 2024-06-06T08:40:31.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-06-06T08:40:31.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.271+0000 7fdce9d63640 1 -- 172.21.15.27:0/2023420487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 msgr2=0x7fdce4107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.271+0000 7fdce9d63640 1 --2- 172.21.15.27:0/2023420487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 0x7fdce4107f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009900 tx=0x7fdcd402a480 comp rx=0 tx=0).stop 2024-06-06T08:40:31.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.271+0000 7fdce9d63640 1 -- 172.21.15.27:0/2023420487 shutdown_connections 2024-06-06T08:40:31.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.271+0000 7fdce9d63640 1 --2- 172.21.15.27:0/2023420487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce406ab90 0x7fdce410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.271+0000 7fdce9d63640 1 --2- 172.21.15.27:0/2023420487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdce406a1d0 0x7fdce406a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.271+0000 7fdce9d63640 1 --2- 172.21.15.27:0/2023420487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 0x7fdce4107f80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.271+0000 7fdce9d63640 1 -- 172.21.15.27:0/2023420487 >> 172.21.15.27:0/2023420487 conn(0x7fdce40766a0 msgr2=0x7fdce4076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:31.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.271+0000 7fdce9d63640 1 -- 172.21.15.27:0/2023420487 shutdown_connections 2024-06-06T08:40:31.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.271+0000 7fdce9d63640 1 -- 172.21.15.27:0/2023420487 wait complete. 2024-06-06T08:40:31.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce9d63640 1 Processor -- start 2024-06-06T08:40:31.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce9d63640 1 -- start start 2024-06-06T08:40:31.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce9d63640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdce406a1d0 0x7fdce4198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:31.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce9d63640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce406ab90 0x7fdce4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:31.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce9d63640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 0x7fdce419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce9d63640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4111dc0 con 0x7fdce4107ba0 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce9d63640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4111c40 con 0x7fdce406a1d0 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce9d63640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4111f40 con 0x7fdce406ab90 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 0x7fdce419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 0x7fdce419c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34014/0 (socket says 172.21.15.27:34014) 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce3fff640 1 -- 172.21.15.27:0/236888417 learned_addr learned my addr 172.21.15.27:0/236888417 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce37fe640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdce406a1d0 0x7fdce4198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce2ffd640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce406ab90 0x7fdce4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce3fff640 1 -- 172.21.15.27:0/236888417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce406ab90 msgr2=0x7fdce4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce3fff640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce406ab90 0x7fdce4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce3fff640 1 -- 172.21.15.27:0/236888417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdce406a1d0 msgr2=0x7fdce4198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce3fff640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdce406a1d0 0x7fdce4198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.272+0000 7fdce3fff640 1 -- 172.21.15.27:0/236888417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce419d110 con 0x7fdce4107ba0 2024-06-06T08:40:31.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.273+0000 7fdce3fff640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 0x7fdce419c9e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00eaa0 tx=0x7fdcdc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:31.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.273+0000 7fdce0ff9640 1 -- 172.21.15.27:0/236888417 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00cdb0 con 0x7fdce4107ba0 2024-06-06T08:40:31.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.273+0000 7fdce0ff9640 1 -- 172.21.15.27:0/236888417 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcdc005020 con 0x7fdce4107ba0 2024-06-06T08:40:31.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.273+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce419d400 con 0x7fdce4107ba0 2024-06-06T08:40:31.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.273+0000 7fdce0ff9640 1 -- 172.21.15.27:0/236888417 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0116b0 con 0x7fdce4107ba0 2024-06-06T08:40:31.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.273+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41a4c90 con 0x7fdce4107ba0 2024-06-06T08:40:31.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.274+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce41030b0 con 0x7fdce4107ba0 2024-06-06T08:40:31.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.275+0000 7fdce0ff9640 1 -- 172.21.15.27:0/236888417 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fdcdc005560 con 0x7fdce4107ba0 2024-06-06T08:40:31.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.276+0000 7fdce0ff9640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fdcd00a2f40 0x7fdcd00a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:31.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.276+0000 7fdce37fe640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fdcd00a2f40 0x7fdcd00a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:31.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.276+0000 7fdce0ff9640 1 -- 172.21.15.27:0/236888417 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fdcdc0cfbc0 con 0x7fdce4107ba0 2024-06-06T08:40:31.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.276+0000 7fdce37fe640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fdcd00a2f40 0x7fdcd00a5400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009230 tx=0x7fdcd40058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:31.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.280+0000 7fdce0ff9640 1 -- 172.21.15.27:0/236888417 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdcdc016030 con 0x7fdce4107ba0 2024-06-06T08:40:31.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.466+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fdce4199470 con 0x7fdce4107ba0 2024-06-06T08:40:31.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.470+0000 7fdce0ff9640 1 -- 172.21.15.27:0/236888417 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7fdcdc09bb80 con 0x7fdce4107ba0 2024-06-06T08:40:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.472+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fdcd00a2f40 msgr2=0x7fdcd00a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.472+0000 7fdce9d63640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fdcd00a2f40 0x7fdcd00a5400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009230 tx=0x7fdcd40058d0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.472+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 msgr2=0x7fdce419c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.472+0000 7fdce9d63640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 0x7fdce419c9e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00eaa0 tx=0x7fdcdc00ef60 comp rx=0 tx=0).stop 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.473+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 shutdown_connections 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.473+0000 7fdce9d63640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fdcd00a2f40 0x7fdcd00a5400 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.473+0000 7fdce9d63640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4107ba0 0x7fdce419c9e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.473+0000 7fdce9d63640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce406ab90 0x7fdce4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.473+0000 7fdce9d63640 1 --2- 172.21.15.27:0/236888417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdce406a1d0 0x7fdce4198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.473+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 >> 172.21.15.27:0/236888417 conn(0x7fdce40766a0 msgr2=0x7fdce40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.473+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 shutdown_connections 2024-06-06T08:40:31.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.473+0000 7fdce9d63640 1 -- 172.21.15.27:0/236888417 wait complete. 2024-06-06T08:40:31.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-06-06T08:40:31.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.595+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/1198340131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8102c20 msgr2=0x7f5cf8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.595+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/1198340131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8102c20 0x7f5cf8103000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009900 tx=0x7f5cf002a480 comp rx=0 tx=0).stop 2024-06-06T08:40:31.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.596+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/1198340131 shutdown_connections 2024-06-06T08:40:31.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.596+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/1198340131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf810bff0 0x7f5cf810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.596+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/1198340131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cf8103540 0x7f5cf810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.596+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/1198340131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8102c20 0x7f5cf8103000 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.596+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/1198340131 >> 172.21.15.27:0/1198340131 conn(0x7f5cf80779f0 msgr2=0x7f5cf8077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:31.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.596+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/1198340131 shutdown_connections 2024-06-06T08:40:31.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.596+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/1198340131 wait complete. 2024-06-06T08:40:31.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.596+0000 7f5cfd9e7640 1 Processor -- start 2024-06-06T08:40:31.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.596+0000 7f5cfd9e7640 1 -- start start 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cfd9e7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf8102c20 0x7f5cf81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cfd9e7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cf8103540 0x7f5cf8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cfd9e7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf810bff0 0x7f5cf819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cfd9e7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8111d80 con 0x7f5cf810bff0 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cfd9e7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8111c00 con 0x7f5cf8102c20 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cfd9e7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8111f00 con 0x7f5cf8103540 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf810bff0 0x7f5cf819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf810bff0 0x7f5cf819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34030/0 (socket says 172.21.15.27:34030) 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf77fe640 1 -- 172.21.15.27:0/878687565 learned_addr learned my addr 172.21.15.27:0/878687565 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf6ffd640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf8102c20 0x7f5cf81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf77fe640 1 -- 172.21.15.27:0/878687565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cf8103540 msgr2=0x7f5cf8198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf77fe640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cf8103540 0x7f5cf8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf77fe640 1 -- 172.21.15.27:0/878687565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf8102c20 msgr2=0x7f5cf81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf77fe640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf8102c20 0x7f5cf81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf77fe640 1 -- 172.21.15.27:0/878687565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf819d280 con 0x7f5cf810bff0 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf6ffd640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf8102c20 0x7f5cf81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cf77fe640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf810bff0 0x7f5cf819cb50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800eaa0 tx=0x7f5ce800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.597+0000 7f5cfc9e5640 1 -- 172.21.15.27:0/878687565 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800cdb0 con 0x7f5cf810bff0 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.598+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf819d570 con 0x7f5cf810bff0 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.598+0000 7f5cfc9e5640 1 -- 172.21.15.27:0/878687565 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce8005020 con 0x7f5cf810bff0 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.598+0000 7f5cfc9e5640 1 -- 172.21.15.27:0/878687565 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80116b0 con 0x7f5cf810bff0 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.598+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a4e00 con 0x7f5cf810bff0 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.599+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf804f760 con 0x7f5cf810bff0 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.600+0000 7f5cfc9e5640 1 -- 172.21.15.27:0/878687565 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f5ce8005560 con 0x7f5cf810bff0 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.600+0000 7f5cfc9e5640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5ce40a2ec0 0x7f5ce40a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:31.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.600+0000 7f5cf6ffd640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5ce40a2ec0 0x7f5ce40a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.600+0000 7f5cfc9e5640 1 -- 172.21.15.27:0/878687565 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f5ce80cfd50 con 0x7f5cf810bff0 2024-06-06T08:40:31.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.601+0000 7f5cf6ffd640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5ce40a2ec0 0x7f5ce40a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0005ab0 tx=0x7f5cf0005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:31.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.605+0000 7f5cfc9e5640 1 -- 172.21.15.27:0/878687565 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5ce80d3030 con 0x7f5cf810bff0 2024-06-06T08:40:31.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.814+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f5cf8061de0 con 0x7f5ce40a2ec0 2024-06-06T08:40:31.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.878+0000 7f5cfc9e5640 1 -- 172.21.15.27:0/878687565 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5cf8061de0 con 0x7f5ce40a2ec0 2024-06-06T08:40:31.877 INFO:teuthology.orchestra.run.smithi027.stdout:Schedule set for path / 2024-06-06T08:40:31.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5ce40a2ec0 msgr2=0x7f5ce40a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5ce40a2ec0 0x7f5ce40a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0005ab0 tx=0x7f5cf0005a20 comp rx=0 tx=0).stop 2024-06-06T08:40:31.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf810bff0 msgr2=0x7f5cf819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:31.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf810bff0 0x7f5cf819cb50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800eaa0 tx=0x7f5ce800ef60 comp rx=0 tx=0).stop 2024-06-06T08:40:31.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 shutdown_connections 2024-06-06T08:40:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5ce40a2ec0 0x7f5ce40a5380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf810bff0 0x7f5cf819cb50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cf8103540 0x7f5cf8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 --2- 172.21.15.27:0/878687565 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cf8102c20 0x7f5cf81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.880+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 >> 172.21.15.27:0/878687565 conn(0x7f5cf80779f0 msgr2=0x7f5cf80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.881+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 shutdown_connections 2024-06-06T08:40:31.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:31.881+0000 7f5cfd9e7640 1 -- 172.21.15.27:0/878687565 wait complete. 2024-06-06T08:40:31.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-06-06T08:40:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.007+0000 7fed6ce7d640 1 -- 172.21.15.27:0/1652342782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806a1d0 msgr2=0x7fed6806a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.007+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/1652342782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806a1d0 0x7fed6806a650 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fed50009900 tx=0x7fed5002a480 comp rx=0 tx=0).stop 2024-06-06T08:40:32.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.007+0000 7fed6ce7d640 1 -- 172.21.15.27:0/1652342782 shutdown_connections 2024-06-06T08:40:32.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.007+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/1652342782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6806ab90 0x7fed6810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.007+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/1652342782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806a1d0 0x7fed6806a650 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.007+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/1652342782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed68107ba0 0x7fed68107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.007+0000 7fed6ce7d640 1 -- 172.21.15.27:0/1652342782 >> 172.21.15.27:0/1652342782 conn(0x7fed680766a0 msgr2=0x7fed68076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.008+0000 7fed6ce7d640 1 -- 172.21.15.27:0/1652342782 shutdown_connections 2024-06-06T08:40:32.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.008+0000 7fed6ce7d640 1 -- 172.21.15.27:0/1652342782 wait complete. 2024-06-06T08:40:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed6ce7d640 1 Processor -- start 2024-06-06T08:40:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed6ce7d640 1 -- start start 2024-06-06T08:40:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed6ce7d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6806a1d0 0x7fed68198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed6ce7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806ab90 0x7fed68198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed6ce7d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed68107ba0 0x7fed6819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed6ce7d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fed68111dc0 con 0x7fed6806ab90 2024-06-06T08:40:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed6ce7d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fed68111c40 con 0x7fed68107ba0 2024-06-06T08:40:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed6ce7d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fed68111f40 con 0x7fed6806a1d0 2024-06-06T08:40:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806ab90 0x7fed68198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806ab90 0x7fed68198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34048/0 (socket says 172.21.15.27:34048) 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed5ffff640 1 -- 172.21.15.27:0/2892399269 learned_addr learned my addr 172.21.15.27:0/2892399269 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed66d9d640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed68107ba0 0x7fed6819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed6659c640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6806a1d0 0x7fed68198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed5ffff640 1 -- 172.21.15.27:0/2892399269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6806a1d0 msgr2=0x7fed68198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed5ffff640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6806a1d0 0x7fed68198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed5ffff640 1 -- 172.21.15.27:0/2892399269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed68107ba0 msgr2=0x7fed6819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed5ffff640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed68107ba0 0x7fed6819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.009+0000 7fed5ffff640 1 -- 172.21.15.27:0/2892399269 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6819d1a0 con 0x7fed6806ab90 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.010+0000 7fed5ffff640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806ab90 0x7fed68198740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fed50006010 tx=0x7fed50003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.010+0000 7fed5f7fe640 1 -- 172.21.15.27:0/2892399269 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5003f070 con 0x7fed6806ab90 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.010+0000 7fed5f7fe640 1 -- 172.21.15.27:0/2892399269 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed50031c30 con 0x7fed6806ab90 2024-06-06T08:40:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.010+0000 7fed5f7fe640 1 -- 172.21.15.27:0/2892399269 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50035030 con 0x7fed6806ab90 2024-06-06T08:40:32.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.010+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6819d430 con 0x7fed6806ab90 2024-06-06T08:40:32.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.010+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed681a4d20 con 0x7fed6806ab90 2024-06-06T08:40:32.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.011+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed681030b0 con 0x7fed6806ab90 2024-06-06T08:40:32.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.012+0000 7fed5f7fe640 1 -- 172.21.15.27:0/2892399269 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fed50042070 con 0x7fed6806ab90 2024-06-06T08:40:32.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.013+0000 7fed5f7fe640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed540a2ec0 0x7fed540a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.013+0000 7fed5f7fe640 1 -- 172.21.15.27:0/2892399269 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fed500f0100 con 0x7fed6806ab90 2024-06-06T08:40:32.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.013+0000 7fed6659c640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed540a2ec0 0x7fed540a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.014+0000 7fed6659c640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed540a2ec0 0x7fed540a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed58009990 tx=0x7fed58009110 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.018+0000 7fed5f7fe640 1 -- 172.21.15.27:0/2892399269 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed500bc7e0 con 0x7fed6806ab90 2024-06-06T08:40:32.111 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4251900607' entity='client.admin' 2024-06-06T08:40:32.111 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1687892581' entity='client.admin' 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/236888417' entity='client.admin' 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:32.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:32.113 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:32.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.231+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fed68061de0 con 0x7fed540a2ec0 2024-06-06T08:40:32.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4251900607' entity='client.admin' 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1687892581' entity='client.admin' 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/236888417' entity='client.admin' 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:32.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:32.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:32.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4251900607' entity='client.admin' 2024-06-06T08:40:32.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:32.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:32.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1687892581' entity='client.admin' 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/236888417' entity='client.admin' 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:40:32.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:40:32.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.292+0000 7fed5f7fe640 1 -- 172.21.15.27:0/2892399269 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fed68061de0 con 0x7fed540a2ec0 2024-06-06T08:40:32.291 INFO:teuthology.orchestra.run.smithi027.stdout:Retention added to path / 2024-06-06T08:40:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.294+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed540a2ec0 msgr2=0x7fed540a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.294+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed540a2ec0 0x7fed540a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed58009990 tx=0x7fed58009110 comp rx=0 tx=0).stop 2024-06-06T08:40:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.294+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806ab90 msgr2=0x7fed68198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.294+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806ab90 0x7fed68198740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fed50006010 tx=0x7fed50003c10 comp rx=0 tx=0).stop 2024-06-06T08:40:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.295+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 shutdown_connections 2024-06-06T08:40:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.295+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed540a2ec0 0x7fed540a5380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.295+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed68107ba0 0x7fed6819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.295+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed6806ab90 0x7fed68198740 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.295+0000 7fed6ce7d640 1 --2- 172.21.15.27:0/2892399269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6806a1d0 0x7fed68198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.295+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 >> 172.21.15.27:0/2892399269 conn(0x7fed680766a0 msgr2=0x7fed680ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.295+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 shutdown_connections 2024-06-06T08:40:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.295+0000 7fed6ce7d640 1 -- 172.21.15.27:0/2892399269 wait complete. 2024-06-06T08:40:32.306 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.414+0000 7f5195dee640 1 -- 172.21.15.27:0/185695172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901012c0 msgr2=0x7f519010d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.414+0000 7f5195dee640 1 --2- 172.21.15.27:0/185695172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901012c0 0x7f519010d780 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5188009cd0 tx=0x7f518802a8e0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.414+0000 7f5195dee640 1 -- 172.21.15.27:0/185695172 shutdown_connections 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.414+0000 7f5195dee640 1 --2- 172.21.15.27:0/185695172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901012c0 0x7f519010d780 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.414+0000 7f5195dee640 1 --2- 172.21.15.27:0/185695172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5190100900 0x7f5190100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.414+0000 7f5195dee640 1 --2- 172.21.15.27:0/185695172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51901068c0 0x7f5190106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.414+0000 7f5195dee640 1 -- 172.21.15.27:0/185695172 >> 172.21.15.27:0/185695172 conn(0x7f5190077ab0 msgr2=0x7f5190077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.414+0000 7f5195dee640 1 -- 172.21.15.27:0/185695172 shutdown_connections 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.414+0000 7f5195dee640 1 -- 172.21.15.27:0/185695172 wait complete. 2024-06-06T08:40:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f5195dee640 1 Processor -- start 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f5195dee640 1 -- start start 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f5195dee640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5190100900 0x7f5190198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f5195dee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51901012c0 0x7f5190199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f5195dee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901068c0 0x7f51901a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f5195dee640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5190110cd0 con 0x7f51901068c0 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f5195dee640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5190110b50 con 0x7f5190100900 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f5195dee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5190110e50 con 0x7f51901012c0 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f518ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901068c0 0x7f51901a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f518ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901068c0 0x7f51901a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34064/0 (socket says 172.21.15.27:34064) 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f518ffff640 1 -- 172.21.15.27:0/3726936783 learned_addr learned my addr 172.21.15.27:0/3726936783 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.415+0000 7f518effd640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51901012c0 0x7f5190199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518f7fe640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5190100900 0x7f5190198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518ffff640 1 -- 172.21.15.27:0/3726936783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51901012c0 msgr2=0x7f5190199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518ffff640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51901012c0 0x7f5190199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518ffff640 1 -- 172.21.15.27:0/3726936783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5190100900 msgr2=0x7f5190198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518ffff640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5190100900 0x7f5190198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518ffff640 1 -- 172.21.15.27:0/3726936783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51901a4590 con 0x7f51901068c0 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518ffff640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901068c0 0x7f51901a4050 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5188005cc0 tx=0x7f51880093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518cff9640 1 -- 172.21.15.27:0/3726936783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188003e00 con 0x7f51901068c0 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51901a4760 con 0x7f51901068c0 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518cff9640 1 -- 172.21.15.27:0/3726936783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5188004520 con 0x7f51901068c0 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f518cff9640 1 -- 172.21.15.27:0/3726936783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188030d00 con 0x7f51901068c0 2024-06-06T08:40:32.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.416+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51901a4c50 con 0x7f51901068c0 2024-06-06T08:40:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.417+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519004f760 con 0x7f51901068c0 2024-06-06T08:40:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.418+0000 7f518cff9640 1 -- 172.21.15.27:0/3726936783 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f51880046d0 con 0x7f51901068c0 2024-06-06T08:40:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.419+0000 7f518cff9640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f51780a2ec0 0x7f51780a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.419+0000 7f518cff9640 1 -- 172.21.15.27:0/3726936783 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f518802e030 con 0x7f51901068c0 2024-06-06T08:40:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.419+0000 7f518f7fe640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f51780a2ec0 0x7f51780a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.419+0000 7f518f7fe640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f51780a2ec0 0x7f51780a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5180009ae0 tx=0x7f5180005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.423+0000 7f518cff9640 1 -- 172.21.15.27:0/3726936783 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f518804b060 con 0x7f51901068c0 2024-06-06T08:40:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.563+0000 7fc84341b700 1 -- 172.21.15.27:0/2583448814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c102380 msgr2=0x7fc83c105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.563+0000 7fc84341b700 1 --2- 172.21.15.27:0/2583448814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c102380 0x7fc83c105ff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc8300099c0 tx=0x7fc830009cd0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.563+0000 7fc84341b700 1 -- 172.21.15.27:0/2583448814 shutdown_connections 2024-06-06T08:40:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.563+0000 7fc84341b700 1 --2- 172.21.15.27:0/2583448814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc83c106720 0x7fc83c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.563+0000 7fc84341b700 1 --2- 172.21.15.27:0/2583448814 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc83c1019d0 0x7fc83c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.563+0000 7fc84341b700 1 --2- 172.21.15.27:0/2583448814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c102380 0x7fc83c105ff0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.563+0000 7fc84341b700 1 -- 172.21.15.27:0/2583448814 >> 172.21.15.27:0/2583448814 conn(0x7fc83c078340 msgr2=0x7fc83c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.563+0000 7fc84341b700 1 -- 172.21.15.27:0/2583448814 shutdown_connections 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.563+0000 7fc84341b700 1 -- 172.21.15.27:0/2583448814 wait complete. 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc84341b700 1 Processor -- start 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc84341b700 1 -- start start 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc84341b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1019d0 0x7fc83c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc84341b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc83c102380 0x7fc83c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc84341b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc83c106720 0x7fc83c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc84341b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1116c0 con 0x7fc83c1019d0 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc84341b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c111540 con 0x7fc83c102380 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc84341b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c111840 con 0x7fc83c106720 2024-06-06T08:40:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8411b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1019d0 0x7fc83c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8411b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1019d0 0x7fc83c197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34092/0 (socket says 172.21.15.27:34092) 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8411b7700 1 -- 172.21.15.27:0/2073687154 learned_addr learned my addr 172.21.15.27:0/2073687154 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8409b6700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc83c102380 0x7fc83c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8411b7700 1 -- 172.21.15.27:0/2073687154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc83c106720 msgr2=0x7fc83c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8411b7700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc83c106720 0x7fc83c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8411b7700 1 -- 172.21.15.27:0/2073687154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc83c102380 msgr2=0x7fc83c197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8411b7700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc83c102380 0x7fc83c197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8411b7700 1 -- 172.21.15.27:0/2073687154 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c1a7180 con 0x7fc83c1019d0 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.564+0000 7fc8409b6700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc83c102380 0x7fc83c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.565+0000 7fc8411b7700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1019d0 0x7fc83c197910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d950 tx=0x7fc83800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.565+0000 7fc82e7fc700 1 -- 172.21.15.27:0/2073687154 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838016070 con 0x7fc83c1019d0 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.565+0000 7fc82e7fc700 1 -- 172.21.15.27:0/2073687154 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc838009b40 con 0x7fc83c1019d0 2024-06-06T08:40:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.565+0000 7fc84341b700 1 -- 172.21.15.27:0/2073687154 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83c1a7470 con 0x7fc83c1019d0 2024-06-06T08:40:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.565+0000 7fc82e7fc700 1 -- 172.21.15.27:0/2073687154 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83800f620 con 0x7fc83c1019d0 2024-06-06T08:40:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.565+0000 7fc84341b700 1 -- 172.21.15.27:0/2073687154 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83c1a79b0 con 0x7fc83c1019d0 2024-06-06T08:40:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.568+0000 7fc82e7fc700 1 -- 172.21.15.27:0/2073687154 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fc838019070 con 0x7fc83c1019d0 2024-06-06T08:40:32.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc84dc9eb80 1 Processor -- start 2024-06-06T08:40:32.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc84dc9eb80 1 -- start start 2024-06-06T08:40:32.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc82e7fc700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc8280a2eb0 0x7fc8280a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a6ab13b890 con 0x55a6ab1effe0 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a6ab0e37a0 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a6ab137850 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc8419b8700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc8280a2eb0 0x7fc8280a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc8409b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc82e7fc700 1 -- 172.21.15.27:0/2073687154 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fc8380cfb00 con 0x7fc83c1019d0 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc8409b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1e9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34100/0 (socket says 172.21.15.27:34100) 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc8409b6700 1 -- 172.21.15.27:0/362757669 learned_addr learned my addr 172.21.15.27:0/362757669 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.569+0000 7fc8411b7700 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8419b8700 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8419b8700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc8280a2eb0 0x7fc8280a5370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc834007f50 tx=0x7fc83400a7a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8409b6700 1 -- 172.21.15.27:0/362757669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1e9fa0 msgr2=0x55a6ab1ea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8409b6700 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1ea3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8409b6700 1 -- 172.21.15.27:0/362757669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 msgr2=0x55a6ab1efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8409b6700 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1efaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8409b6700 1 -- 172.21.15.27:0/362757669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6ab0d9fd0 con 0x55a6ab1effe0 2024-06-06T08:40:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8411b7700 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8419b8700 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8409b6700 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1e9a60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc830006010 tx=0x7fc830005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c37d5fb5ab151f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8277fe700 1 -- 172.21.15.27:0/362757669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830026070 con 0x55a6ab1effe0 2024-06-06T08:40:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8277fe700 1 -- 172.21.15.27:0/362757669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc830003e40 con 0x55a6ab1effe0 2024-06-06T08:40:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.570+0000 7fc8277fe700 1 -- 172.21.15.27:0/362757669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83000bb00 con 0x55a6ab1effe0 2024-06-06T08:40:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/362757669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 msgr2=0x55a6ab1e9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1e9a60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc830006010 tx=0x7fc830005e80 comp rx=0 tx=0).stop 2024-06-06T08:40:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/362757669 shutdown_connections 2024-06-06T08:40:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/362757669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1e9a60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/362757669 >> 172.21.15.27:0/362757669 conn(0x55a6ab114d40 msgr2=0x55a6ab1e48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/362757669 shutdown_connections 2024-06-06T08:40:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/362757669 wait complete. 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.571+0000 7fc84dc9eb80 1 -- start start 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc84dc9eb80 1 Processor -- start 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a6ab1f69f0 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a6ab0dbc70 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a6ab1f6b70 con 0x55a6ab1effe0 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc84dc9eb80 10 client.0 _ll_register_callbacks cb 0x55a6ab0d59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc8409b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc8409b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34104/0 (socket says 172.21.15.27:34104) 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc8409b6700 1 -- 172.21.15.27:0/3308402646 learned_addr learned my addr 172.21.15.27:0/3308402646 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.572+0000 7fc8411b7700 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc8419b8700 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc8409b6700 1 -- 172.21.15.27:0/3308402646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 msgr2=0x55a6ab1f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc8409b6700 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc8409b6700 1 -- 172.21.15.27:0/3308402646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 msgr2=0x55a6ab1f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc8409b6700 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc8409b6700 1 -- 172.21.15.27:0/3308402646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6ab08a950 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc8419b8700 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc8409b6700 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1f3020 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc830005fe0 tx=0x7fc830009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a56f25b5b4a3eb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc824ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc824ff9700 1 -- 172.21.15.27:0/3308402646 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830026040 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc824ff9700 1 -- 172.21.15.27:0/3308402646 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fc8300053d0 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3308402646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a6ab148d90 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc824ff9700 1 -- 172.21.15.27:0/3308402646 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc830038050 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc84dc9eb80 10 client.15381 fetch_fsmap learned FSMap version 25 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3308402646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6ab201330 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.573+0000 7fc824ff9700 1 -- 172.21.15.27:0/3308402646 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a6ab148d90 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.574+0000 7fc824ff9700 1 -- 172.21.15.27:0/3308402646 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc830029070 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.574+0000 7fc84dc9eb80 10 client.15381 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.574+0000 7fc84dc9eb80 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.574+0000 7fc84dc9eb80 20 client.15381 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.574+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a6ab203800 0x55a6ab205bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.574+0000 7fc84dc9eb80 4 client.15381 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.574+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3308402646 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6ab143310 con 0x55a6ab203800 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.574+0000 7fc8411b7700 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a6ab203800 0x55a6ab205bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.575+0000 7fc8411b7700 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a6ab203800 0x55a6ab205bf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b714b3de841de51 server_cookie=78bb8242d9ddb474 in_seq=0 out_seq=0 2024-06-06T08:40:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.575+0000 7fc824ff9700 10 client.15381 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:32.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.575+0000 7fc824ff9700 1 -- 172.21.15.27:0/3308402646 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a6ab143310 con 0x55a6ab203800 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.575+0000 7fc824ff9700 10 client.15381 handle_command_reply: tid=0 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.575+0000 7fc84dc9eb80 1 client.15381 shutdown 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.575+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3308402646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1ef6c0 msgr2=0x55a6ab1f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.575+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1f3020 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc830005fe0 tx=0x7fc830009390 comp rx=0 tx=0).stop 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3308402646 shutdown_connections 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a6ab203800 0x55a6ab205bf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3308402646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1f3020 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3308402646 >> 172.21.15.27:0/3308402646 conn(0x55a6ab114d40 msgr2=0x55a6ab1e8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc824ff9700 0 client.15381 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3308402646 shutdown_connections 2024-06-06T08:40:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3308402646 wait complete. 2024-06-06T08:40:32.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.576+0000 7fc84dc9eb80 20 client.15381 trim_cache size 0 max 16384 2024-06-06T08:40:32.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc84dc9eb80 1 Processor -- start 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc84dc9eb80 1 -- start start 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1de710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1b30d0 con 0x55a6ab1effe0 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1b0960 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1ae7b0 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8419b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8419b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34116/0 (socket says 172.21.15.27:34116) 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8419b8700 1 -- 172.21.15.27:0/3075744288 learned_addr learned my addr 172.21.15.27:0/3075744288 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8411b7700 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.619 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8409b6700 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1de710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8419b8700 1 -- 172.21.15.27:0/3075744288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1ef6c0 msgr2=0x55a6ab1de710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:32.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8419b8700 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1de710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8419b8700 1 -- 172.21.15.27:0/3075744288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 msgr2=0x55a6ab1e9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:32.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8419b8700 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1e9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.620+0000 7fc8419b8700 1 -- 172.21.15.27:0/3075744288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6ab1f1590 con 0x55a6ab1effe0 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.621+0000 7fc8411b7700 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.621+0000 7fc8419b8700 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f0af0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc834003e30 tx=0x7fc834017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb8af5bc321e77b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.621+0000 7fc826ffd700 1 -- 172.21.15.27:0/3075744288 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834026070 con 0x55a6ab1effe0 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.621+0000 7fc826ffd700 1 -- 172.21.15.27:0/3075744288 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834021900 con 0x55a6ab1effe0 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.621+0000 7fc826ffd700 1 -- 172.21.15.27:0/3075744288 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83401d040 con 0x55a6ab1effe0 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3075744288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 msgr2=0x55a6ab1f0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f0af0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc834003e30 tx=0x7fc834017000 comp rx=0 tx=0).stop 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3075744288 shutdown_connections 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1effe0 0x55a6ab1f0af0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab1e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/3075744288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab1de710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3075744288 >> 172.21.15.27:0/3075744288 conn(0x55a6ab114d40 msgr2=0x55a6ab1f4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3075744288 shutdown_connections 2024-06-06T08:40:32.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/3075744288 wait complete. 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 -- start start 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.622+0000 7fc84dc9eb80 1 Processor -- start 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab207140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab207680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc84dc9eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 0x55a6ab26d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1aee20 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1b0290 con 0x55a6ab1ef6c0 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc84dc9eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1af4f0 con 0x55a6ab1effe0 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8411b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab207140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc84dc9eb80 10 client.0 _ll_register_callbacks cb 0x55a6ab0d59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8411b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab207140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34126/0 (socket says 172.21.15.27:34126) 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8411b7700 1 -- 172.21.15.27:0/1506141488 learned_addr learned my addr 172.21.15.27:0/1506141488 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8409b6700 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab207680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8419b8700 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 0x55a6ab26d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8411b7700 1 -- 172.21.15.27:0/1506141488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 msgr2=0x55a6ab26d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:32.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8411b7700 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 0x55a6ab26d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8411b7700 1 -- 172.21.15.27:0/1506141488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 msgr2=0x55a6ab207680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:32.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8411b7700 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab207680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8411b7700 1 -- 172.21.15.27:0/1506141488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6ab26d680 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc8411b7700 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab207140 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc8380cce20 tx=0x7fc83809b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7bf6e1612b1f87 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.623+0000 7fc824ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc824ff9700 1 -- 172.21.15.27:0/1506141488 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838020040 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc824ff9700 1 -- 172.21.15.27:0/1506141488 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fc838098080 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/1506141488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc834021900 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc824ff9700 1 -- 172.21.15.27:0/1506141488 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc83809fdc0 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc84dc9eb80 10 client.15387 fetch_fsmap learned FSMap version 25 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/1506141488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6ab26e9a0 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc824ff9700 1 -- 172.21.15.27:0/1506141488 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83807f810 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc824ff9700 1 -- 172.21.15.27:0/1506141488 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc8380ccff0 con 0x55a6ab1e9fa0 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc84dc9eb80 10 client.15387 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:40:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc84dc9eb80 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc84dc9eb80 20 client.15387 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a6ab26efb0 0x55a6ab2713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc84dc9eb80 4 client.15387 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.624+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/1506141488 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fc834021900 con 0x55a6ab26efb0 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.625+0000 7fc8409b6700 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a6ab26efb0 0x55a6ab2713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.625+0000 7fc8409b6700 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a6ab26efb0 0x55a6ab2713d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c1fd00ee8ab366e server_cookie=dd607fb45f41bb3e in_seq=0 out_seq=0 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.625+0000 7fc824ff9700 10 client.15387 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.626+0000 7fc824ff9700 1 -- 172.21.15.27:0/1506141488 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fc834021900 con 0x55a6ab26efb0 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.626+0000 7fc824ff9700 10 client.15387 handle_command_reply: tid=0 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.626+0000 7fc84dc9eb80 1 client.15387 shutdown 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/1506141488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1e9fa0 msgr2=0x55a6ab207140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab207140 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc8380cce20 tx=0x7fc83809b910 comp rx=0 tx=0).stop 2024-06-06T08:40:32.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/1506141488 shutdown_connections 2024-06-06T08:40:32.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6ab1effe0 0x55a6ab26d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a6ab26efb0 0x55a6ab2713d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a6ab1e9fa0 0x55a6ab207140 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc824ff9700 0 client.15387 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:32.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 --2- 172.21.15.27:0/1506141488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a6ab1ef6c0 0x55a6ab207680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/1506141488 >> 172.21.15.27:0/1506141488 conn(0x55a6ab114d40 msgr2=0x55a6ab1f2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/1506141488 shutdown_connections 2024-06-06T08:40:32.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 1 -- 172.21.15.27:0/1506141488 wait complete. 2024-06-06T08:40:32.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.627+0000 7fc84dc9eb80 20 client.15387 trim_cache size 0 max 16384 2024-06-06T08:40:32.630 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:40:32.630 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-06-06T08:40:32.630 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "9dae157b-1457-4606-83ea-8502d1035717", 2024-06-06T08:40:32.630 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-06-06T08:40:32.630 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 -- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc8280a2eb0 msgr2=0x7fc8280a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc8280a2eb0 0x7fc8280a5370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc834007f50 tx=0x7fc83400a7a0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 -- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1019d0 msgr2=0x7fc83c197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1019d0 0x7fc83c197910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d950 tx=0x7fc83800dd10 comp rx=0 tx=0).stop 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 -- 172.21.15.27:0/2073687154 shutdown_connections 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc83c106720 0x7fc83c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc83c1019d0 0x7fc83c197910 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc8280a2eb0 0x7fc8280a5370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 --2- 172.21.15.27:0/2073687154 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc83c102380 0x7fc83c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 -- 172.21.15.27:0/2073687154 >> 172.21.15.27:0/2073687154 conn(0x7fc83c078340 msgr2=0x7fc83c107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 -- 172.21.15.27:0/2073687154 shutdown_connections 2024-06-06T08:40:32.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.628+0000 7fc84341b700 1 -- 172.21.15.27:0/2073687154 wait complete. 2024-06-06T08:40:32.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.665+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5190103a90 con 0x7f51780a2ec0 2024-06-06T08:40:32.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.673+0000 7f518cff9640 1 -- 172.21.15.27:0/3726936783 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f5190103a90 con 0x7f51780a2ec0 2024-06-06T08:40:32.671 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-06T00:00:00", "created": "2024-06-06T08:40:31", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-06-06T08:40:32.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f51780a2ec0 msgr2=0x7f51780a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f51780a2ec0 0x7f51780a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5180009ae0 tx=0x7f5180005f90 comp rx=0 tx=0).stop 2024-06-06T08:40:32.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901068c0 msgr2=0x7f51901a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901068c0 0x7f51901a4050 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5188005cc0 tx=0x7f51880093d0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 shutdown_connections 2024-06-06T08:40:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f51780a2ec0 0x7f51780a5380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51901068c0 0x7f51901a4050 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51901012c0 0x7f5190199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 --2- 172.21.15.27:0/3726936783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5190100900 0x7f5190198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 >> 172.21.15.27:0/3726936783 conn(0x7f5190077ab0 msgr2=0x7f5190103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 shutdown_connections 2024-06-06T08:40:32.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.675+0000 7f5195dee640 1 -- 172.21.15.27:0/3726936783 wait complete. 2024-06-06T08:40:32.684 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-06T08:40:32.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.793+0000 7f9f5df4f640 1 -- 172.21.15.27:0/2656672940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 msgr2=0x7f9f58100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.793+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/2656672940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 0x7f9f58100d50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009930 tx=0x7f9f4c02a420 comp rx=0 tx=0).stop 2024-06-06T08:40:32.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.793+0000 7f9f5df4f640 1 -- 172.21.15.27:0/2656672940 shutdown_connections 2024-06-06T08:40:32.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.793+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/2656672940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f58101290 0x7f9f5810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.793+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/2656672940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 0x7f9f58100d50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.793+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/2656672940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f58106890 0x7f9f58106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.793+0000 7f9f5df4f640 1 -- 172.21.15.27:0/2656672940 >> 172.21.15.27:0/2656672940 conn(0x7f9f58077a20 msgr2=0x7f9f58077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:32.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.793+0000 7f9f5df4f640 1 -- 172.21.15.27:0/2656672940 shutdown_connections 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.793+0000 7f9f5df4f640 1 -- 172.21.15.27:0/2656672940 wait complete. 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f5df4f640 1 Processor -- start 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f5df4f640 1 -- start start 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f5df4f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 0x7f9f58197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f5df4f640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f58101290 0x7f9f58197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f5df4f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f58106890 0x7f9f5819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f5df4f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58110cf0 con 0x7f9f581008d0 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f5df4f640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58110b70 con 0x7f9f58101290 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f5df4f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58110e70 con 0x7f9f58106890 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f577fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 0x7f9f58197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f577fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 0x7f9f58197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34134/0 (socket says 172.21.15.27:34134) 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f577fe640 1 -- 172.21.15.27:0/3138432141 learned_addr learned my addr 172.21.15.27:0/3138432141 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:32.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f57fff640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f58106890 0x7f9f5819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f577fe640 1 -- 172.21.15.27:0/3138432141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f58106890 msgr2=0x7f9f5819b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f577fe640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f58106890 0x7f9f5819b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.794+0000 7f9f577fe640 1 -- 172.21.15.27:0/3138432141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f58101290 msgr2=0x7f9f58197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f56ffd640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f58101290 0x7f9f58197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f577fe640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f58101290 0x7f9f58197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f577fe640 1 -- 172.21.15.27:0/3138432141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5819c0b0 con 0x7f9f581008d0 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f57fff640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f58106890 0x7f9f5819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f577fe640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 0x7f9f58197110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800d950 tx=0x7f9f4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f54ff9640 1 -- 172.21.15.27:0/3138432141 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48017070 con 0x7f9f581008d0 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5819c3a0 con 0x7f9f581008d0 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f54ff9640 1 -- 172.21.15.27:0/3138432141 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f48009c30 con 0x7f9f581008d0 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f54ff9640 1 -- 172.21.15.27:0/3138432141 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4800f6b0 con 0x7f9f581008d0 2024-06-06T08:40:32.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.795+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f581a3c30 con 0x7f9f581008d0 2024-06-06T08:40:32.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.796+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5804f760 con 0x7f9f581008d0 2024-06-06T08:40:32.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.797+0000 7f9f54ff9640 1 -- 172.21.15.27:0/3138432141 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f9f480109d0 con 0x7f9f581008d0 2024-06-06T08:40:32.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.798+0000 7f9f54ff9640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f400a2ec0 0x7f9f400a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:32.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.798+0000 7f9f54ff9640 1 -- 172.21.15.27:0/3138432141 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9f480cf720 con 0x7f9f581008d0 2024-06-06T08:40:32.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.798+0000 7f9f56ffd640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f400a2ec0 0x7f9f400a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:32.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.798+0000 7f9f56ffd640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f400a2ec0 0x7f9f400a5380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0096c0 tx=0x7f9f4c005fd0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:32.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:32.801+0000 7f9f54ff9640 1 -- 172.21.15.27:0/3138432141 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9f48009de0 con 0x7f9f581008d0 2024-06-06T08:40:33.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.009+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9f58103a60 con 0x7f9f400a2ec0 2024-06-06T08:40:33.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.016+0000 7f9f54ff9640 1 -- 172.21.15.27:0/3138432141 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9f58103a60 con 0x7f9f400a2ec0 2024-06-06T08:40:33.015 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-06-06T08:40:33.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f400a2ec0 msgr2=0x7f9f400a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:33.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f400a2ec0 0x7f9f400a5380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0096c0 tx=0x7f9f4c005fd0 comp rx=0 tx=0).stop 2024-06-06T08:40:33.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 msgr2=0x7f9f58197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:33.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 0x7f9f58197110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800d950 tx=0x7f9f4800de10 comp rx=0 tx=0).stop 2024-06-06T08:40:33.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 shutdown_connections 2024-06-06T08:40:33.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f400a2ec0 0x7f9f400a5380 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:33.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f58106890 0x7f9f5819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:33.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f58101290 0x7f9f58197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:33.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 --2- 172.21.15.27:0/3138432141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f581008d0 0x7f9f58197110 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:33.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.018+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 >> 172.21.15.27:0/3138432141 conn(0x7f9f58077a20 msgr2=0x7f9f58103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:33.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.019+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 shutdown_connections 2024-06-06T08:40:33.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:33.019+0000 7f9f5df4f640 1 -- 172.21.15.27:0/3138432141 wait complete. 2024-06-06T08:40:33.027 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:32 smithi027 ceph-mon[32335]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-06-06T08:40:33.027 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:32 smithi027 ceph-mon[32335]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:33.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-06-06T08:40:33.054 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-06-06T08:40:33.065 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-06T08:40:33.106 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:40:33.107 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-06T08:40:33.109 INFO:teuthology.run_tasks:Running task quiescer... 2024-06-06T08:40:33.143 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T08:40:33.143 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T08:40:33.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:32 smithi203 ceph-mon[36579]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-06-06T08:40:33.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:32 smithi203 ceph-mon[36579]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:33.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:32 smithi176 ceph-mon[36786]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 597 B/s wr, 15 op/s 2024-06-06T08:40:33.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:32 smithi176 ceph-mon[36786]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:33.431 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T08:40:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:33 smithi027 ceph-mon[32335]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:33 smithi027 ceph-mon[32335]: scrub queued for path: / 2024-06-06T08:40:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:33 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:40:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:33 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:40:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:33 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:40:34.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:33 smithi027 ceph-mon[32335]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:34.225 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:33 smithi203 ceph-mon[36579]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:33 smithi203 ceph-mon[36579]: scrub queued for path: / 2024-06-06T08:40:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:33 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:40:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:33 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:40:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:33 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:40:34.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:33 smithi203 ceph-mon[36579]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:33 smithi176 ceph-mon[36786]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:33 smithi176 ceph-mon[36786]: scrub queued for path: / 2024-06-06T08:40:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:33 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:40:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:33 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:40:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:33 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:40:34.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:33 smithi176 ceph-mon[36786]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:35.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:34 smithi027 ceph-mon[32335]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:35.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:34 smithi027 ceph-mon[32335]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-06-06T08:40:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:34 smithi203 ceph-mon[36579]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:34 smithi203 ceph-mon[36579]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-06-06T08:40:35.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:34 smithi176 ceph-mon[36786]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:40:35.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:34 smithi176 ceph-mon[36786]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-06-06T08:40:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.916+0000 7f75f48c7700 1 -- 172.21.15.27:0/2096469271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0102740 msgr2=0x7f75f0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.916+0000 7f75f48c7700 1 --2- 172.21.15.27:0/2096469271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0102740 0x7f75f0102b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f75e000b0d0 tx=0x7f75e000b3e0 comp rx=0 tx=0).stop 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.916+0000 7f75f48c7700 1 -- 172.21.15.27:0/2096469271 shutdown_connections 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.916+0000 7f75f48c7700 1 --2- 172.21.15.27:0/2096469271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f0107840 0x7f75f010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.916+0000 7f75f48c7700 1 --2- 172.21.15.27:0/2096469271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0102740 0x7f75f0102b20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.916+0000 7f75f48c7700 1 --2- 172.21.15.27:0/2096469271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f01030f0 0x7f75f0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.916+0000 7f75f48c7700 1 -- 172.21.15.27:0/2096469271 >> 172.21.15.27:0/2096469271 conn(0x7f75f00795f0 msgr2=0x7f75f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.916+0000 7f75f48c7700 1 -- 172.21.15.27:0/2096469271 shutdown_connections 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.916+0000 7f75f48c7700 1 -- 172.21.15.27:0/2096469271 wait complete. 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75f48c7700 1 Processor -- start 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75f48c7700 1 -- start start 2024-06-06T08:40:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75f48c7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f0102740 0x7f75f019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75f48c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f01030f0 0x7f75f019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75f48c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0107840 0x7f75f0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75f48c7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0112780 con 0x7f75f0107840 2024-06-06T08:40:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75f48c7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0112600 con 0x7f75f0102740 2024-06-06T08:40:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75f48c7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0112900 con 0x7f75f01030f0 2024-06-06T08:40:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75eed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0107840 0x7f75f0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75eed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0107840 0x7f75f0198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34150/0 (socket says 172.21.15.27:34150) 2024-06-06T08:40:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.917+0000 7f75eed9d700 1 -- 172.21.15.27:0/3691793830 learned_addr learned my addr 172.21.15.27:0/3691793830 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75edd9b700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f01030f0 0x7f75f019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75ee59c700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f0102740 0x7f75f019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75eed9d700 1 -- 172.21.15.27:0/3691793830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f01030f0 msgr2=0x7f75f019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75eed9d700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f01030f0 0x7f75f019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75eed9d700 1 -- 172.21.15.27:0/3691793830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f0102740 msgr2=0x7f75f019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75eed9d700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f0102740 0x7f75f019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75eed9d700 1 -- 172.21.15.27:0/3691793830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f0199160 con 0x7f75f0107840 2024-06-06T08:40:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75eed9d700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0107840 0x7f75f0198a00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f75e800d980 tx=0x7f75e800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75df7fe700 1 -- 172.21.15.27:0/3691793830 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8017070 con 0x7f75f0107840 2024-06-06T08:40:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f0199450 con 0x7f75f0107840 2024-06-06T08:40:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75df7fe700 1 -- 172.21.15.27:0/3691793830 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e8009e20 con 0x7f75f0107840 2024-06-06T08:40:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75df7fe700 1 -- 172.21.15.27:0/3691793830 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8005440 con 0x7f75f0107840 2024-06-06T08:40:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.918+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f01a5710 con 0x7f75f0107840 2024-06-06T08:40:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.919+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f010aa80 con 0x7f75f0107840 2024-06-06T08:40:35.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.920+0000 7f75df7fe700 1 -- 172.21.15.27:0/3691793830 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f75e80055e0 con 0x7f75f0107840 2024-06-06T08:40:35.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.921+0000 7f75df7fe700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75d80a2f30 0x7f75d80a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:35.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.921+0000 7f75ee59c700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75d80a2f30 0x7f75d80a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:35.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.921+0000 7f75df7fe700 1 -- 172.21.15.27:0/3691793830 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f75e80cf9d0 con 0x7f75f0107840 2024-06-06T08:40:35.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.922+0000 7f75ee59c700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75d80a2f30 0x7f75d80a53f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009850 tx=0x7f75e0009780 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:35.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:35.926+0000 7f75df7fe700 1 -- 172.21.15.27:0/3691793830 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f75e8020020 con 0x7f75f0107840 2024-06-06T08:40:36.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.133+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75f00628d0 con 0x7f75d80a2f30 2024-06-06T08:40:36.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.143+0000 7f75df7fe700 1 -- 172.21.15.27:0/3691793830 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339846 (secure 0 0 0) 0x7f75f00628d0 con 0x7f75d80a2f30 2024-06-06T08:40:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.146+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75d80a2f30 msgr2=0x7f75d80a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.146+0000 7f75f48c7700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75d80a2f30 0x7f75d80a53f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009850 tx=0x7f75e0009780 comp rx=0 tx=0).stop 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.146+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0107840 msgr2=0x7f75f0198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.147+0000 7f75f48c7700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0107840 0x7f75f0198a00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f75e800d980 tx=0x7f75e800dd40 comp rx=0 tx=0).stop 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.147+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 shutdown_connections 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.147+0000 7f75f48c7700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75f0107840 0x7f75f0198a00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.147+0000 7f75f48c7700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75d80a2f30 0x7f75d80a53f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.147+0000 7f75f48c7700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75f0102740 0x7f75f019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.147+0000 7f75f48c7700 1 --2- 172.21.15.27:0/3691793830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f01030f0 0x7f75f019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.147+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 >> 172.21.15.27:0/3691793830 conn(0x7f75f00795f0 msgr2=0x7f75f0100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.147+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 shutdown_connections 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:36.147+0000 7f75f48c7700 1 -- 172.21.15.27:0/3691793830 wait complete. 2024-06-06T08:40:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:40:37.096 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:36 smithi027 ceph-mon[32335]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-06-06T08:40:37.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:36 smithi203 ceph-mon[36579]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-06-06T08:40:37.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:36 smithi176 ceph-mon[36786]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-06-06T08:40:37.450 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-06-06T08:40:35.308560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389359,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1758,"num_read_kb":1311,"num_write":636,"num_write_kb":5307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":969,"ondisk_log_size":969,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362876,"kb_used_data":13688,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578948,"statfs":{"total":1151940427776,"available":1151568842752,"internally_reserved":0,"allocated":14016512,"data_stored":11102243,"data_compressed":634920,"data_compressed_allocated":860160,"data_compressed_original":5050368,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197022,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":427,"num_read_kb":474,"num_write":149,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002970"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543767+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:40:17.543767+0000","last_peered":"2024-06-06T08:40:17.543767+0000","last_clean":"2024-06-06T08:40:17.543767+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:40:17.543767+0000","last_undegraded":"2024-06-06T08:40:17.543767+0000","last_fullsized":"2024-06-06T08:40:17.543767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603694+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:40:17.603694+0000","last_peered":"2024-06-06T08:40:17.603694+0000","last_clean":"2024-06-06T08:40:17.603694+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:40:17.603694+0000","last_undegraded":"2024-06-06T08:40:17.603694+0000","last_fullsized":"2024-06-06T08:40:17.603694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603368+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:40:17.603368+0000","last_peered":"2024-06-06T08:40:17.603368+0000","last_clean":"2024-06-06T08:40:17.603368+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:40:17.603368+0000","last_undegraded":"2024-06-06T08:40:17.603368+0000","last_fullsized":"2024-06-06T08:40:17.603368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602693+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:40:17.602693+0000","last_peered":"2024-06-06T08:40:17.602693+0000","last_clean":"2024-06-06T08:40:17.602693+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:40:17.602693+0000","last_undegraded":"2024-06-06T08:40:17.602693+0000","last_fullsized":"2024-06-06T08:40:17.602693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397130+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:40:17.397130+0000","last_peered":"2024-06-06T08:40:17.397130+0000","last_clean":"2024-06-06T08:40:17.397130+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:40:17.397130+0000","last_undegraded":"2024-06-06T08:40:17.397130+0000","last_fullsized":"2024-06-06T08:40:17.397130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406459+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:40:17.406459+0000","last_peered":"2024-06-06T08:40:17.406459+0000","last_clean":"2024-06-06T08:40:17.406459+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:40:17.406459+0000","last_undegraded":"2024-06-06T08:40:17.406459+0000","last_fullsized":"2024-06-06T08:40:17.406459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290504+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:40:17.290504+0000","last_peered":"2024-06-06T08:40:17.290504+0000","last_clean":"2024-06-06T08:40:17.290504+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:40:17.290504+0000","last_undegraded":"2024-06-06T08:40:17.290504+0000","last_fullsized":"2024-06-06T08:40:17.290504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394085+0000","last_change":"2024-06-06T08:38:01.335419+0000","last_active":"2024-06-06T08:40:17.394085+0000","last_peered":"2024-06-06T08:40:17.394085+0000","last_clean":"2024-06-06T08:40:17.394085+0000","last_became_active":"2024-06-06T08:38:01.335191+0000","last_became_peered":"2024-06-06T08:38:01.335191+0000","last_unstale":"2024-06-06T08:40:17.394085+0000","last_undegraded":"2024-06-06T08:40:17.394085+0000","last_fullsized":"2024-06-06T08:40:17.394085+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:13.253838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619376+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:40:17.619376+0000","last_peered":"2024-06-06T08:40:17.619376+0000","last_clean":"2024-06-06T08:40:17.619376+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:40:17.619376+0000","last_undegraded":"2024-06-06T08:40:17.619376+0000","last_fullsized":"2024-06-06T08:40:17.619376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406777+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:40:17.406777+0000","last_peered":"2024-06-06T08:40:17.406777+0000","last_clean":"2024-06-06T08:40:17.406777+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:40:17.406777+0000","last_undegraded":"2024-06-06T08:40:17.406777+0000","last_fullsized":"2024-06-06T08:40:17.406777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290994+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:40:17.290994+0000","last_peered":"2024-06-06T08:40:17.290994+0000","last_clean":"2024-06-06T08:40:17.290994+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:40:17.290994+0000","last_undegraded":"2024-06-06T08:40:17.290994+0000","last_fullsized":"2024-06-06T08:40:17.290994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602644+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:40:17.602644+0000","last_peered":"2024-06-06T08:40:17.602644+0000","last_clean":"2024-06-06T08:40:17.602644+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:40:17.602644+0000","last_undegraded":"2024-06-06T08:40:17.602644+0000","last_fullsized":"2024-06-06T08:40:17.602644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289360+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:40:17.289360+0000","last_peered":"2024-06-06T08:40:17.289360+0000","last_clean":"2024-06-06T08:40:17.289360+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:40:17.289360+0000","last_undegraded":"2024-06-06T08:40:17.289360+0000","last_fullsized":"2024-06-06T08:40:17.289360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602964+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:40:17.602964+0000","last_peered":"2024-06-06T08:40:17.602964+0000","last_clean":"2024-06-06T08:40:17.602964+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:40:17.602964+0000","last_undegraded":"2024-06-06T08:40:17.602964+0000","last_fullsized":"2024-06-06T08:40:17.602964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603050+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:40:17.603050+0000","last_peered":"2024-06-06T08:40:17.603050+0000","last_clean":"2024-06-06T08:40:17.603050+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:40:17.603050+0000","last_undegraded":"2024-06-06T08:40:17.603050+0000","last_fullsized":"2024-06-06T08:40:17.603050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618538+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:40:17.618538+0000","last_peered":"2024-06-06T08:40:17.618538+0000","last_clean":"2024-06-06T08:40:17.618538+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:40:17.618538+0000","last_undegraded":"2024-06-06T08:40:17.618538+0000","last_fullsized":"2024-06-06T08:40:17.618538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543492+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:40:17.543492+0000","last_peered":"2024-06-06T08:40:17.543492+0000","last_clean":"2024-06-06T08:40:17.543492+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:40:17.543492+0000","last_undegraded":"2024-06-06T08:40:17.543492+0000","last_fullsized":"2024-06-06T08:40:17.543492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394947+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:40:17.394947+0000","last_peered":"2024-06-06T08:40:17.394947+0000","last_clean":"2024-06-06T08:40:17.394947+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:40:17.394947+0000","last_undegraded":"2024-06-06T08:40:17.394947+0000","last_fullsized":"2024-06-06T08:40:17.394947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291235+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:40:17.291235+0000","last_peered":"2024-06-06T08:40:17.291235+0000","last_clean":"2024-06-06T08:40:17.291235+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:40:17.291235+0000","last_undegraded":"2024-06-06T08:40:17.291235+0000","last_fullsized":"2024-06-06T08:40:17.291235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291190+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:40:17.291190+0000","last_peered":"2024-06-06T08:40:17.291190+0000","last_clean":"2024-06-06T08:40:17.291190+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:40:17.291190+0000","last_undegraded":"2024-06-06T08:40:17.291190+0000","last_fullsized":"2024-06-06T08:40:17.291190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:23.296771+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:40:23.296771+0000","last_peered":"2024-06-06T08:40:23.296771+0000","last_clean":"2024-06-06T08:40:23.296771+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:40:23.296771+0000","last_undegraded":"2024-06-06T08:40:23.296771+0000","last_fullsized":"2024-06-06T08:40:23.296771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397547+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:40:17.397547+0000","last_peered":"2024-06-06T08:40:17.397547+0000","last_clean":"2024-06-06T08:40:17.397547+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:40:17.397547+0000","last_undegraded":"2024-06-06T08:40:17.397547+0000","last_fullsized":"2024-06-06T08:40:17.397547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602515+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:40:17.602515+0000","last_peered":"2024-06-06T08:40:17.602515+0000","last_clean":"2024-06-06T08:40:17.602515+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:40:17.602515+0000","last_undegraded":"2024-06-06T08:40:17.602515+0000","last_fullsized":"2024-06-06T08:40:17.602515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397886+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:40:17.397886+0000","last_peered":"2024-06-06T08:40:17.397886+0000","last_clean":"2024-06-06T08:40:17.397886+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:40:17.397886+0000","last_undegraded":"2024-06-06T08:40:17.397886+0000","last_fullsized":"2024-06-06T08:40:17.397886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603696+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:40:17.603696+0000","last_peered":"2024-06-06T08:40:17.603696+0000","last_clean":"2024-06-06T08:40:17.603696+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:40:17.603696+0000","last_undegraded":"2024-06-06T08:40:17.603696+0000","last_fullsized":"2024-06-06T08:40:17.603696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602634+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:40:17.602634+0000","last_peered":"2024-06-06T08:40:17.602634+0000","last_clean":"2024-06-06T08:40:17.602634+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:40:17.602634+0000","last_undegraded":"2024-06-06T08:40:17.602634+0000","last_fullsized":"2024-06-06T08:40:17.602634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397399+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:40:17.397399+0000","last_peered":"2024-06-06T08:40:17.397399+0000","last_clean":"2024-06-06T08:40:17.397399+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:40:17.397399+0000","last_undegraded":"2024-06-06T08:40:17.397399+0000","last_fullsized":"2024-06-06T08:40:17.397399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618791+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:40:17.618791+0000","last_peered":"2024-06-06T08:40:17.618791+0000","last_clean":"2024-06-06T08:40:17.618791+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:40:17.618791+0000","last_undegraded":"2024-06-06T08:40:17.618791+0000","last_fullsized":"2024-06-06T08:40:17.618791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602522+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:40:17.602522+0000","last_peered":"2024-06-06T08:40:17.602522+0000","last_clean":"2024-06-06T08:40:17.602522+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:40:17.602522+0000","last_undegraded":"2024-06-06T08:40:17.602522+0000","last_fullsized":"2024-06-06T08:40:17.602522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397350+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:40:17.397350+0000","last_peered":"2024-06-06T08:40:17.397350+0000","last_clean":"2024-06-06T08:40:17.397350+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:40:17.397350+0000","last_undegraded":"2024-06-06T08:40:17.397350+0000","last_fullsized":"2024-06-06T08:40:17.397350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394983+0000","last_change":"2024-06-06T08:38:01.728398+0000","last_active":"2024-06-06T08:40:17.394983+0000","last_peered":"2024-06-06T08:40:17.394983+0000","last_clean":"2024-06-06T08:40:17.394983+0000","last_became_active":"2024-06-06T08:38:01.728004+0000","last_became_peered":"2024-06-06T08:38:01.728004+0000","last_unstale":"2024-06-06T08:40:17.394983+0000","last_undegraded":"2024-06-06T08:40:17.394983+0000","last_fullsized":"2024-06-06T08:40:17.394983+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:02:03.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291009+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:40:17.291009+0000","last_peered":"2024-06-06T08:40:17.291009+0000","last_clean":"2024-06-06T08:40:17.291009+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:40:17.291009+0000","last_undegraded":"2024-06-06T08:40:17.291009+0000","last_fullsized":"2024-06-06T08:40:17.291009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":247,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:32.106940+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:40:32.106940+0000","last_peered":"2024-06-06T08:40:32.106940+0000","last_clean":"2024-06-06T08:40:32.106940+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:40:32.106940+0000","last_undegraded":"2024-06-06T08:40:32.106940+0000","last_fullsized":"2024-06-06T08:40:32.106940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1320,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":193,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396976+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:40:17.396976+0000","last_peered":"2024-06-06T08:40:17.396976+0000","last_clean":"2024-06-06T08:40:17.396976+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:40:17.396976+0000","last_undegraded":"2024-06-06T08:40:17.396976+0000","last_fullsized":"2024-06-06T08:40:17.396976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290894+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:40:17.290894+0000","last_peered":"2024-06-06T08:40:17.290894+0000","last_clean":"2024-06-06T08:40:17.290894+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:40:17.290894+0000","last_undegraded":"2024-06-06T08:40:17.290894+0000","last_fullsized":"2024-06-06T08:40:17.290894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406405+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:40:17.406405+0000","last_peered":"2024-06-06T08:40:17.406405+0000","last_clean":"2024-06-06T08:40:17.406405+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:40:17.406405+0000","last_undegraded":"2024-06-06T08:40:17.406405+0000","last_fullsized":"2024-06-06T08:40:17.406405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394510+0000","last_change":"2024-06-06T08:38:01.728238+0000","last_active":"2024-06-06T08:40:17.394510+0000","last_peered":"2024-06-06T08:40:17.394510+0000","last_clean":"2024-06-06T08:40:17.394510+0000","last_became_active":"2024-06-06T08:38:01.727844+0000","last_became_peered":"2024-06-06T08:38:01.727844+0000","last_unstale":"2024-06-06T08:40:17.394510+0000","last_undegraded":"2024-06-06T08:40:17.394510+0000","last_fullsized":"2024-06-06T08:40:17.394510+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:19:46.825321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543511+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:40:17.543511+0000","last_peered":"2024-06-06T08:40:17.543511+0000","last_clean":"2024-06-06T08:40:17.543511+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:40:17.543511+0000","last_undegraded":"2024-06-06T08:40:17.543511+0000","last_fullsized":"2024-06-06T08:40:17.543511+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288975+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:40:17.288975+0000","last_peered":"2024-06-06T08:40:17.288975+0000","last_clean":"2024-06-06T08:40:17.288975+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:40:17.288975+0000","last_undegraded":"2024-06-06T08:40:17.288975+0000","last_fullsized":"2024-06-06T08:40:17.288975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396897+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:40:17.396897+0000","last_peered":"2024-06-06T08:40:17.396897+0000","last_clean":"2024-06-06T08:40:17.396897+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:40:17.396897+0000","last_undegraded":"2024-06-06T08:40:17.396897+0000","last_fullsized":"2024-06-06T08:40:17.396897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603585+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:40:17.603585+0000","last_peered":"2024-06-06T08:40:17.603585+0000","last_clean":"2024-06-06T08:40:17.603585+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:40:17.603585+0000","last_undegraded":"2024-06-06T08:40:17.603585+0000","last_fullsized":"2024-06-06T08:40:17.603585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403412+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:40:17.403412+0000","last_peered":"2024-06-06T08:40:17.403412+0000","last_clean":"2024-06-06T08:40:17.403412+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:40:17.403412+0000","last_undegraded":"2024-06-06T08:40:17.403412+0000","last_fullsized":"2024-06-06T08:40:17.403412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394236+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:40:17.394236+0000","last_peered":"2024-06-06T08:40:17.394236+0000","last_clean":"2024-06-06T08:40:17.394236+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:40:17.394236+0000","last_undegraded":"2024-06-06T08:40:17.394236+0000","last_fullsized":"2024-06-06T08:40:17.394236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":240,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:32.106475+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:40:32.106475+0000","last_peered":"2024-06-06T08:40:32.106475+0000","last_clean":"2024-06-06T08:40:32.106475+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:40:32.106475+0000","last_undegraded":"2024-06-06T08:40:32.106475+0000","last_fullsized":"2024-06-06T08:40:32.106475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619308+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:40:17.619308+0000","last_peered":"2024-06-06T08:40:17.619308+0000","last_clean":"2024-06-06T08:40:17.619308+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:40:17.619308+0000","last_undegraded":"2024-06-06T08:40:17.619308+0000","last_fullsized":"2024-06-06T08:40:17.619308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394167+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:40:17.394167+0000","last_peered":"2024-06-06T08:40:17.394167+0000","last_clean":"2024-06-06T08:40:17.394167+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:40:17.394167+0000","last_undegraded":"2024-06-06T08:40:17.394167+0000","last_fullsized":"2024-06-06T08:40:17.394167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290830+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:40:17.290830+0000","last_peered":"2024-06-06T08:40:17.290830+0000","last_clean":"2024-06-06T08:40:17.290830+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:40:17.290830+0000","last_undegraded":"2024-06-06T08:40:17.290830+0000","last_fullsized":"2024-06-06T08:40:17.290830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603715+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:40:17.603715+0000","last_peered":"2024-06-06T08:40:17.603715+0000","last_clean":"2024-06-06T08:40:17.603715+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:40:17.603715+0000","last_undegraded":"2024-06-06T08:40:17.603715+0000","last_fullsized":"2024-06-06T08:40:17.603715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602684+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:40:17.602684+0000","last_peered":"2024-06-06T08:40:17.602684+0000","last_clean":"2024-06-06T08:40:17.602684+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:40:17.602684+0000","last_undegraded":"2024-06-06T08:40:17.602684+0000","last_fullsized":"2024-06-06T08:40:17.602684+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403562+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:40:17.403562+0000","last_peered":"2024-06-06T08:40:17.403562+0000","last_clean":"2024-06-06T08:40:17.403562+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:40:17.403562+0000","last_undegraded":"2024-06-06T08:40:17.403562+0000","last_fullsized":"2024-06-06T08:40:17.403562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618947+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:40:17.618947+0000","last_peered":"2024-06-06T08:40:17.618947+0000","last_clean":"2024-06-06T08:40:17.618947+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:40:17.618947+0000","last_undegraded":"2024-06-06T08:40:17.618947+0000","last_fullsized":"2024-06-06T08:40:17.618947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406287+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:40:17.406287+0000","last_peered":"2024-06-06T08:40:17.406287+0000","last_clean":"2024-06-06T08:40:17.406287+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:40:17.406287+0000","last_undegraded":"2024-06-06T08:40:17.406287+0000","last_fullsized":"2024-06-06T08:40:17.406287+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'43","reported_seq":501,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:33.107991+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:40:33.107991+0000","last_peered":"2024-06-06T08:40:33.107991+0000","last_clean":"2024-06-06T08:40:33.107991+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:40:33.107991+0000","last_undegraded":"2024-06-06T08:40:33.107991+0000","last_fullsized":"2024-06-06T08:40:33.107991+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71690,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":395,"num_read_kb":269,"num_write":44,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397172+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:40:17.397172+0000","last_peered":"2024-06-06T08:40:17.397172+0000","last_clean":"2024-06-06T08:40:17.397172+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:40:17.397172+0000","last_undegraded":"2024-06-06T08:40:17.397172+0000","last_fullsized":"2024-06-06T08:40:17.397172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291329+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:40:17.291329+0000","last_peered":"2024-06-06T08:40:17.291329+0000","last_clean":"2024-06-06T08:40:17.291329+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:40:17.291329+0000","last_undegraded":"2024-06-06T08:40:17.291329+0000","last_fullsized":"2024-06-06T08:40:17.291329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289082+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:40:17.289082+0000","last_peered":"2024-06-06T08:40:17.289082+0000","last_clean":"2024-06-06T08:40:17.289082+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:40:17.289082+0000","last_undegraded":"2024-06-06T08:40:17.289082+0000","last_fullsized":"2024-06-06T08:40:17.289082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543337+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:40:17.543337+0000","last_peered":"2024-06-06T08:40:17.543337+0000","last_clean":"2024-06-06T08:40:17.543337+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:40:17.543337+0000","last_undegraded":"2024-06-06T08:40:17.543337+0000","last_fullsized":"2024-06-06T08:40:17.543337+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403613+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:40:17.403613+0000","last_peered":"2024-06-06T08:40:17.403613+0000","last_clean":"2024-06-06T08:40:17.403613+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:40:17.403613+0000","last_undegraded":"2024-06-06T08:40:17.403613+0000","last_fullsized":"2024-06-06T08:40:17.403613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290581+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:40:17.290581+0000","last_peered":"2024-06-06T08:40:17.290581+0000","last_clean":"2024-06-06T08:40:17.290581+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:40:17.290581+0000","last_undegraded":"2024-06-06T08:40:17.290581+0000","last_fullsized":"2024-06-06T08:40:17.290581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394932+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:40:17.394932+0000","last_peered":"2024-06-06T08:40:17.394932+0000","last_clean":"2024-06-06T08:40:17.394932+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:40:17.394932+0000","last_undegraded":"2024-06-06T08:40:17.394932+0000","last_fullsized":"2024-06-06T08:40:17.394932+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":249,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:32.107236+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:40:32.107236+0000","last_peered":"2024-06-06T08:40:32.107236+0000","last_clean":"2024-06-06T08:40:32.107236+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:40:32.107236+0000","last_undegraded":"2024-06-06T08:40:32.107236+0000","last_fullsized":"2024-06-06T08:40:32.107236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290588+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:40:17.290588+0000","last_peered":"2024-06-06T08:40:17.290588+0000","last_clean":"2024-06-06T08:40:17.290588+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:40:17.290588+0000","last_undegraded":"2024-06-06T08:40:17.290588+0000","last_fullsized":"2024-06-06T08:40:17.290588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289726+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:40:17.289726+0000","last_peered":"2024-06-06T08:40:17.289726+0000","last_clean":"2024-06-06T08:40:17.289726+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:40:17.289726+0000","last_undegraded":"2024-06-06T08:40:17.289726+0000","last_fullsized":"2024-06-06T08:40:17.289726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396877+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:40:17.396877+0000","last_peered":"2024-06-06T08:40:17.396877+0000","last_clean":"2024-06-06T08:40:17.396877+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:40:17.396877+0000","last_undegraded":"2024-06-06T08:40:17.396877+0000","last_fullsized":"2024-06-06T08:40:17.396877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403466+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:40:17.403466+0000","last_peered":"2024-06-06T08:40:17.403466+0000","last_clean":"2024-06-06T08:40:17.403466+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:40:17.403466+0000","last_undegraded":"2024-06-06T08:40:17.403466+0000","last_fullsized":"2024-06-06T08:40:17.403466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403457+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:40:17.403457+0000","last_peered":"2024-06-06T08:40:17.403457+0000","last_clean":"2024-06-06T08:40:17.403457+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:40:17.403457+0000","last_undegraded":"2024-06-06T08:40:17.403457+0000","last_fullsized":"2024-06-06T08:40:17.403457+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"100'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.933761+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:40:17.933761+0000","last_peered":"2024-06-06T08:40:17.933761+0000","last_clean":"2024-06-06T08:40:17.933761+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:40:17.933761+0000","last_undegraded":"2024-06-06T08:40:17.933761+0000","last_fullsized":"2024-06-06T08:40:17.933761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543570+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:40:17.543570+0000","last_peered":"2024-06-06T08:40:17.543570+0000","last_clean":"2024-06-06T08:40:17.543570+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:40:17.543570+0000","last_undegraded":"2024-06-06T08:40:17.543570+0000","last_fullsized":"2024-06-06T08:40:17.543570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394760+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:40:17.394760+0000","last_peered":"2024-06-06T08:40:17.394760+0000","last_clean":"2024-06-06T08:40:17.394760+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:40:17.394760+0000","last_undegraded":"2024-06-06T08:40:17.394760+0000","last_fullsized":"2024-06-06T08:40:17.394760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406540+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:40:17.406540+0000","last_peered":"2024-06-06T08:40:17.406540+0000","last_clean":"2024-06-06T08:40:17.406540+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:40:17.406540+0000","last_undegraded":"2024-06-06T08:40:17.406540+0000","last_fullsized":"2024-06-06T08:40:17.406540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289289+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:40:17.289289+0000","last_peered":"2024-06-06T08:40:17.289289+0000","last_clean":"2024-06-06T08:40:17.289289+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:40:17.289289+0000","last_undegraded":"2024-06-06T08:40:17.289289+0000","last_fullsized":"2024-06-06T08:40:17.289289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290643+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:40:17.290643+0000","last_peered":"2024-06-06T08:40:17.290643+0000","last_clean":"2024-06-06T08:40:17.290643+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:40:17.290643+0000","last_undegraded":"2024-06-06T08:40:17.290643+0000","last_fullsized":"2024-06-06T08:40:17.290643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406476+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:40:17.406476+0000","last_peered":"2024-06-06T08:40:17.406476+0000","last_clean":"2024-06-06T08:40:17.406476+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:40:17.406476+0000","last_undegraded":"2024-06-06T08:40:17.406476+0000","last_fullsized":"2024-06-06T08:40:17.406476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289488+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:40:17.289488+0000","last_peered":"2024-06-06T08:40:17.289488+0000","last_clean":"2024-06-06T08:40:17.289488+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:40:17.289488+0000","last_undegraded":"2024-06-06T08:40:17.289488+0000","last_fullsized":"2024-06-06T08:40:17.289488+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403823+0000","last_change":"2024-06-06T08:38:01.545781+0000","last_active":"2024-06-06T08:40:17.403823+0000","last_peered":"2024-06-06T08:40:17.403823+0000","last_clean":"2024-06-06T08:40:17.403823+0000","last_became_active":"2024-06-06T08:38:01.545462+0000","last_became_peered":"2024-06-06T08:38:01.545462+0000","last_unstale":"2024-06-06T08:40:17.403823+0000","last_undegraded":"2024-06-06T08:40:17.403823+0000","last_fullsized":"2024-06-06T08:40:17.403823+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:14:36.722960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":243,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:32.106560+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:40:32.106560+0000","last_peered":"2024-06-06T08:40:32.106560+0000","last_clean":"2024-06-06T08:40:32.106560+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:40:32.106560+0000","last_undegraded":"2024-06-06T08:40:32.106560+0000","last_fullsized":"2024-06-06T08:40:32.106560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290959+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:40:17.290959+0000","last_peered":"2024-06-06T08:40:17.290959+0000","last_clean":"2024-06-06T08:40:17.290959+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:40:17.290959+0000","last_undegraded":"2024-06-06T08:40:17.290959+0000","last_fullsized":"2024-06-06T08:40:17.290959+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397178+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:40:17.397178+0000","last_peered":"2024-06-06T08:40:17.397178+0000","last_clean":"2024-06-06T08:40:17.397178+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:40:17.397178+0000","last_undegraded":"2024-06-06T08:40:17.397178+0000","last_fullsized":"2024-06-06T08:40:17.397178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602424+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:40:17.602424+0000","last_peered":"2024-06-06T08:40:17.602424+0000","last_clean":"2024-06-06T08:40:17.602424+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:40:17.602424+0000","last_undegraded":"2024-06-06T08:40:17.602424+0000","last_fullsized":"2024-06-06T08:40:17.602424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"100'148","reported_seq":422,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:19.422865+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:40:19.422865+0000","last_peered":"2024-06-06T08:40:19.422865+0000","last_clean":"2024-06-06T08:40:19.422865+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:40:19.422865+0000","last_undegraded":"2024-06-06T08:40:19.422865+0000","last_fullsized":"2024-06-06T08:40:19.422865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288880+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:40:17.288880+0000","last_peered":"2024-06-06T08:40:17.288880+0000","last_clean":"2024-06-06T08:40:17.288880+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:40:17.288880+0000","last_undegraded":"2024-06-06T08:40:17.288880+0000","last_fullsized":"2024-06-06T08:40:17.288880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543213+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:40:17.543213+0000","last_peered":"2024-06-06T08:40:17.543213+0000","last_clean":"2024-06-06T08:40:17.543213+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:40:17.543213+0000","last_undegraded":"2024-06-06T08:40:17.543213+0000","last_fullsized":"2024-06-06T08:40:17.543213+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543287+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:40:17.543287+0000","last_peered":"2024-06-06T08:40:17.543287+0000","last_clean":"2024-06-06T08:40:17.543287+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:40:17.543287+0000","last_undegraded":"2024-06-06T08:40:17.543287+0000","last_fullsized":"2024-06-06T08:40:17.543287+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543496+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:40:17.543496+0000","last_peered":"2024-06-06T08:40:17.543496+0000","last_clean":"2024-06-06T08:40:17.543496+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:40:17.543496+0000","last_undegraded":"2024-06-06T08:40:17.543496+0000","last_fullsized":"2024-06-06T08:40:17.543496+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289533+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:40:17.289533+0000","last_peered":"2024-06-06T08:40:17.289533+0000","last_clean":"2024-06-06T08:40:17.289533+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:40:17.289533+0000","last_undegraded":"2024-06-06T08:40:17.289533+0000","last_fullsized":"2024-06-06T08:40:17.289533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397196+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:40:17.397196+0000","last_peered":"2024-06-06T08:40:17.397196+0000","last_clean":"2024-06-06T08:40:17.397196+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:40:17.397196+0000","last_undegraded":"2024-06-06T08:40:17.397196+0000","last_fullsized":"2024-06-06T08:40:17.397196+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'102","reported_seq":241,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:33.015049+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:40:33.015049+0000","last_peered":"2024-06-06T08:40:33.015049+0000","last_clean":"2024-06-06T08:40:33.015049+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:40:33.015049+0000","last_undegraded":"2024-06-06T08:40:33.015049+0000","last_fullsized":"2024-06-06T08:40:33.015049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":187,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397610+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:40:17.397610+0000","last_peered":"2024-06-06T08:40:17.397610+0000","last_clean":"2024-06-06T08:40:17.397610+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:40:17.397610+0000","last_undegraded":"2024-06-06T08:40:17.397610+0000","last_fullsized":"2024-06-06T08:40:17.397610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290756+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:40:17.290756+0000","last_peered":"2024-06-06T08:40:17.290756+0000","last_clean":"2024-06-06T08:40:17.290756+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:40:17.290756+0000","last_undegraded":"2024-06-06T08:40:17.290756+0000","last_fullsized":"2024-06-06T08:40:17.290756+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603067+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:40:17.603067+0000","last_peered":"2024-06-06T08:40:17.603067+0000","last_clean":"2024-06-06T08:40:17.603067+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:40:17.603067+0000","last_undegraded":"2024-06-06T08:40:17.603067+0000","last_fullsized":"2024-06-06T08:40:17.603067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603700+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:40:17.603700+0000","last_peered":"2024-06-06T08:40:17.603700+0000","last_clean":"2024-06-06T08:40:17.603700+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:40:17.603700+0000","last_undegraded":"2024-06-06T08:40:17.603700+0000","last_fullsized":"2024-06-06T08:40:17.603700+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602722+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:40:17.602722+0000","last_peered":"2024-06-06T08:40:17.602722+0000","last_clean":"2024-06-06T08:40:17.602722+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:40:17.602722+0000","last_undegraded":"2024-06-06T08:40:17.602722+0000","last_fullsized":"2024-06-06T08:40:17.602722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618369+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:40:17.618369+0000","last_peered":"2024-06-06T08:40:17.618369+0000","last_clean":"2024-06-06T08:40:17.618369+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:40:17.618369+0000","last_undegraded":"2024-06-06T08:40:17.618369+0000","last_fullsized":"2024-06-06T08:40:17.618369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396892+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:40:17.396892+0000","last_peered":"2024-06-06T08:40:17.396892+0000","last_clean":"2024-06-06T08:40:17.396892+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:40:17.396892+0000","last_undegraded":"2024-06-06T08:40:17.396892+0000","last_fullsized":"2024-06-06T08:40:17.396892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.395030+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:40:17.395030+0000","last_peered":"2024-06-06T08:40:17.395030+0000","last_clean":"2024-06-06T08:40:17.395030+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:40:17.395030+0000","last_undegraded":"2024-06-06T08:40:17.395030+0000","last_fullsized":"2024-06-06T08:40:17.395030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397132+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:40:17.397132+0000","last_peered":"2024-06-06T08:40:17.397132+0000","last_clean":"2024-06-06T08:40:17.397132+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:40:17.397132+0000","last_undegraded":"2024-06-06T08:40:17.397132+0000","last_fullsized":"2024-06-06T08:40:17.397132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543302+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:40:17.543302+0000","last_peered":"2024-06-06T08:40:17.543302+0000","last_clean":"2024-06-06T08:40:17.543302+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:40:17.543302+0000","last_undegraded":"2024-06-06T08:40:17.543302+0000","last_fullsized":"2024-06-06T08:40:17.543302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602805+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:40:17.602805+0000","last_peered":"2024-06-06T08:40:17.602805+0000","last_clean":"2024-06-06T08:40:17.602805+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:40:17.602805+0000","last_undegraded":"2024-06-06T08:40:17.602805+0000","last_fullsized":"2024-06-06T08:40:17.602805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291108+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:40:17.291108+0000","last_peered":"2024-06-06T08:40:17.291108+0000","last_clean":"2024-06-06T08:40:17.291108+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:40:17.291108+0000","last_undegraded":"2024-06-06T08:40:17.291108+0000","last_fullsized":"2024-06-06T08:40:17.291108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396937+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:40:17.396937+0000","last_peered":"2024-06-06T08:40:17.396937+0000","last_clean":"2024-06-06T08:40:17.396937+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:40:17.396937+0000","last_undegraded":"2024-06-06T08:40:17.396937+0000","last_fullsized":"2024-06-06T08:40:17.396937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602730+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:40:17.602730+0000","last_peered":"2024-06-06T08:40:17.602730+0000","last_clean":"2024-06-06T08:40:17.602730+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:40:17.602730+0000","last_undegraded":"2024-06-06T08:40:17.602730+0000","last_fullsized":"2024-06-06T08:40:17.602730+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396987+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:40:17.396987+0000","last_peered":"2024-06-06T08:40:17.396987+0000","last_clean":"2024-06-06T08:40:17.396987+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:40:17.396987+0000","last_undegraded":"2024-06-06T08:40:17.396987+0000","last_fullsized":"2024-06-06T08:40:17.396987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290632+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:40:17.290632+0000","last_peered":"2024-06-06T08:40:17.290632+0000","last_clean":"2024-06-06T08:40:17.290632+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:40:17.290632+0000","last_undegraded":"2024-06-06T08:40:17.290632+0000","last_fullsized":"2024-06-06T08:40:17.290632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618976+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:40:17.618976+0000","last_peered":"2024-06-06T08:40:17.618976+0000","last_clean":"2024-06-06T08:40:17.618976+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:40:17.618976+0000","last_undegraded":"2024-06-06T08:40:17.618976+0000","last_fullsized":"2024-06-06T08:40:17.618976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291175+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:40:17.291175+0000","last_peered":"2024-06-06T08:40:17.291175+0000","last_clean":"2024-06-06T08:40:17.291175+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:40:17.291175+0000","last_undegraded":"2024-06-06T08:40:17.291175+0000","last_fullsized":"2024-06-06T08:40:17.291175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291116+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:40:17.291116+0000","last_peered":"2024-06-06T08:40:17.291116+0000","last_clean":"2024-06-06T08:40:17.291116+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:40:17.291116+0000","last_undegraded":"2024-06-06T08:40:17.291116+0000","last_fullsized":"2024-06-06T08:40:17.291116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603292+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:40:17.603292+0000","last_peered":"2024-06-06T08:40:17.603292+0000","last_clean":"2024-06-06T08:40:17.603292+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:40:17.603292+0000","last_undegraded":"2024-06-06T08:40:17.603292+0000","last_fullsized":"2024-06-06T08:40:17.603292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'65","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:32.670293+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:40:32.670293+0000","last_peered":"2024-06-06T08:40:32.670293+0000","last_clean":"2024-06-06T08:40:32.670293+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:40:32.670293+0000","last_undegraded":"2024-06-06T08:40:32.670293+0000","last_fullsized":"2024-06-06T08:40:32.670293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290659+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:40:17.290659+0000","last_peered":"2024-06-06T08:40:17.290659+0000","last_clean":"2024-06-06T08:40:17.290659+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:40:17.290659+0000","last_undegraded":"2024-06-06T08:40:17.290659+0000","last_fullsized":"2024-06-06T08:40:17.290659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403261+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:40:17.403261+0000","last_peered":"2024-06-06T08:40:17.403261+0000","last_clean":"2024-06-06T08:40:17.403261+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:40:17.403261+0000","last_undegraded":"2024-06-06T08:40:17.403261+0000","last_fullsized":"2024-06-06T08:40:17.403261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289754+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:40:17.289754+0000","last_peered":"2024-06-06T08:40:17.289754+0000","last_clean":"2024-06-06T08:40:17.289754+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:40:17.289754+0000","last_undegraded":"2024-06-06T08:40:17.289754+0000","last_fullsized":"2024-06-06T08:40:17.289754+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291007+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:40:17.291007+0000","last_peered":"2024-06-06T08:40:17.291007+0000","last_clean":"2024-06-06T08:40:17.291007+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:40:17.291007+0000","last_undegraded":"2024-06-06T08:40:17.291007+0000","last_fullsized":"2024-06-06T08:40:17.291007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543518+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:40:17.543518+0000","last_peered":"2024-06-06T08:40:17.543518+0000","last_clean":"2024-06-06T08:40:17.543518+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:40:17.543518+0000","last_undegraded":"2024-06-06T08:40:17.543518+0000","last_fullsized":"2024-06-06T08:40:17.543518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291058+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:40:17.291058+0000","last_peered":"2024-06-06T08:40:17.291058+0000","last_clean":"2024-06-06T08:40:17.291058+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:40:17.291058+0000","last_undegraded":"2024-06-06T08:40:17.291058+0000","last_fullsized":"2024-06-06T08:40:17.291058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290956+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:40:17.290956+0000","last_peered":"2024-06-06T08:40:17.290956+0000","last_clean":"2024-06-06T08:40:17.290956+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:40:17.290956+0000","last_undegraded":"2024-06-06T08:40:17.290956+0000","last_fullsized":"2024-06-06T08:40:17.290956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603342+0000","last_change":"2024-06-06T08:38:01.151003+0000","last_active":"2024-06-06T08:40:17.603342+0000","last_peered":"2024-06-06T08:40:17.603342+0000","last_clean":"2024-06-06T08:40:17.603342+0000","last_became_active":"2024-06-06T08:38:01.150557+0000","last_became_peered":"2024-06-06T08:38:01.150557+0000","last_unstale":"2024-06-06T08:40:17.603342+0000","last_undegraded":"2024-06-06T08:40:17.603342+0000","last_fullsized":"2024-06-06T08:40:17.603342+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:29:44.629894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289585+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:40:17.289585+0000","last_peered":"2024-06-06T08:40:17.289585+0000","last_clean":"2024-06-06T08:40:17.289585+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:40:17.289585+0000","last_undegraded":"2024-06-06T08:40:17.289585+0000","last_fullsized":"2024-06-06T08:40:17.289585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543470+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:40:17.543470+0000","last_peered":"2024-06-06T08:40:17.543470+0000","last_clean":"2024-06-06T08:40:17.543470+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:40:17.543470+0000","last_undegraded":"2024-06-06T08:40:17.543470+0000","last_fullsized":"2024-06-06T08:40:17.543470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291317+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:40:17.291317+0000","last_peered":"2024-06-06T08:40:17.291317+0000","last_clean":"2024-06-06T08:40:17.291317+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:40:17.291317+0000","last_undegraded":"2024-06-06T08:40:17.291317+0000","last_fullsized":"2024-06-06T08:40:17.291317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397744+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:40:17.397744+0000","last_peered":"2024-06-06T08:40:17.397744+0000","last_clean":"2024-06-06T08:40:17.397744+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:40:17.397744+0000","last_undegraded":"2024-06-06T08:40:17.397744+0000","last_fullsized":"2024-06-06T08:40:17.397744+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403480+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:40:17.403480+0000","last_peered":"2024-06-06T08:40:17.403480+0000","last_clean":"2024-06-06T08:40:17.403480+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:40:17.403480+0000","last_undegraded":"2024-06-06T08:40:17.403480+0000","last_fullsized":"2024-06-06T08:40:17.403480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397272+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:40:17.397272+0000","last_peered":"2024-06-06T08:40:17.397272+0000","last_clean":"2024-06-06T08:40:17.397272+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:40:17.397272+0000","last_undegraded":"2024-06-06T08:40:17.397272+0000","last_fullsized":"2024-06-06T08:40:17.397272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618845+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:40:17.618845+0000","last_peered":"2024-06-06T08:40:17.618845+0000","last_clean":"2024-06-06T08:40:17.618845+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:40:17.618845+0000","last_undegraded":"2024-06-06T08:40:17.618845+0000","last_fullsized":"2024-06-06T08:40:17.618845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289213+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:40:17.289213+0000","last_peered":"2024-06-06T08:40:17.289213+0000","last_clean":"2024-06-06T08:40:17.289213+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:40:17.289213+0000","last_undegraded":"2024-06-06T08:40:17.289213+0000","last_fullsized":"2024-06-06T08:40:17.289213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543444+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:40:17.543444+0000","last_peered":"2024-06-06T08:40:17.543444+0000","last_clean":"2024-06-06T08:40:17.543444+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:40:17.543444+0000","last_undegraded":"2024-06-06T08:40:17.543444+0000","last_fullsized":"2024-06-06T08:40:17.543444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107163,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798757,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1521,"num_read_kb":981,"num_write":433,"num_write_kb":2367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":2396271,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":319,"ondisk_log_size":319,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383746,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":579875,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36620,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708532,"statfs":{"total":95995035648,"available":95957536768,"internally_reserved":0,"allocated":1306624,"data_stored":1568354,"data_compressed":86280,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808909,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":977401,"data_compressed":54646,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070546,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":973203,"data_compressed":54646,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332183,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":1177536,"data_compressed":71078,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495709,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":575265,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":646865,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":576653,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577223,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1183744,"data_stored":1296907,"data_compressed":69848,"data_compressed_allocated":94208,"data_compressed_original":798720,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1196032,"data_stored":652916,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1224704,"data_stored":1500045,"data_compressed":86280,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330406,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":402811,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:40:37.455 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 -- 172.21.15.27:0/3828844731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 msgr2=0x7f9e841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 --2- 172.21.15.27:0/3828844731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 0x7f9e841029b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0099c0 tx=0x7f9e6c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 -- 172.21.15.27:0/3828844731 shutdown_connections 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 --2- 172.21.15.27:0/3828844731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e84102ef0 0x7f9e8410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 --2- 172.21.15.27:0/3828844731 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e841085d0 0x7f9e841089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 --2- 172.21.15.27:0/3828844731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 0x7f9e841029b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 -- 172.21.15.27:0/3828844731 >> 172.21.15.27:0/3828844731 conn(0x7f9e84079640 msgr2=0x7f9e84079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 -- 172.21.15.27:0/3828844731 shutdown_connections 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 -- 172.21.15.27:0/3828844731 wait complete. 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.695+0000 7f9e89a16700 1 Processor -- start 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e89a16700 1 -- start start 2024-06-06T08:40:37.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e89a16700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 0x7f9e8410bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e89a16700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e84102ef0 0x7f9e8410c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e89a16700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e841085d0 0x7f9e841aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e89a16700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84075ad0 con 0x7f9e84102530 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e89a16700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84075950 con 0x7f9e84102ef0 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e89a16700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84075c50 con 0x7f9e841085d0 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e82ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 0x7f9e8410bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e82ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 0x7f9e8410bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34170/0 (socket says 172.21.15.27:34170) 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e82ffd700 1 -- 172.21.15.27:0/719824411 learned_addr learned my addr 172.21.15.27:0/719824411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e827fc700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e84102ef0 0x7f9e8410c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e837fe700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e841085d0 0x7f9e841aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e82ffd700 1 -- 172.21.15.27:0/719824411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e841085d0 msgr2=0x7f9e841aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e841085d0 0x7f9e841aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e82ffd700 1 -- 172.21.15.27:0/719824411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e84102ef0 msgr2=0x7f9e8410c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e84102ef0 0x7f9e8410c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.696+0000 7f9e82ffd700 1 -- 172.21.15.27:0/719824411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e841ab4b0 con 0x7f9e84102530 2024-06-06T08:40:37.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.697+0000 7f9e827fc700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e84102ef0 0x7f9e8410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:37.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.697+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 0x7f9e8410bd00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400d980 tx=0x7f9e7400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:37.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.697+0000 7f9e88a14700 1 -- 172.21.15.27:0/719824411 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74016070 con 0x7f9e84102530 2024-06-06T08:40:37.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.697+0000 7f9e89a16700 1 -- 172.21.15.27:0/719824411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e841ab7a0 con 0x7f9e84102530 2024-06-06T08:40:37.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.697+0000 7f9e88a14700 1 -- 172.21.15.27:0/719824411 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e74004500 con 0x7f9e84102530 2024-06-06T08:40:37.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.697+0000 7f9e88a14700 1 -- 172.21.15.27:0/719824411 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74004d90 con 0x7f9e84102530 2024-06-06T08:40:37.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.697+0000 7f9e89a16700 1 -- 172.21.15.27:0/719824411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e841abd00 con 0x7f9e84102530 2024-06-06T08:40:37.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.700+0000 7f9e88a14700 1 -- 172.21.15.27:0/719824411 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f9e74005020 con 0x7f9e84102530 2024-06-06T08:40:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.701+0000 7f9e88a14700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9e700a2eb0 0x7f9e700a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e88a14700 1 -- 172.21.15.27:0/719824411 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9e740cf920 con 0x7f9e84102530 2024-06-06T08:40:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e827fc700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9e700a2eb0 0x7f9e700a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e94299b80 1 Processor -- start 2024-06-06T08:40:37.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e94299b80 1 -- start start 2024-06-06T08:40:37.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d683262340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 0x55d68325c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d68325cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e94299b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d68314ef60 con 0x55d683261f60 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e94299b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d6831ad850 con 0x55d683262880 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e94299b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d683115460 con 0x55d68325c7a0 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e82ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d683262340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e827fc700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9e700a2eb0 0x7f9e700a5370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e8410d3a0 tx=0x7f9e6c005fb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e82ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d683262340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34174/0 (socket says 172.21.15.27:34174) 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.702+0000 7f9e82ffd700 1 -- 172.21.15.27:0/1134655691 learned_addr learned my addr 172.21.15.27:0/1134655691 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e827fc700 1 --2- 172.21.15.27:0/1134655691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 0x55d68325c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e82ffd700 1 -- 172.21.15.27:0/1134655691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 msgr2=0x55d68325cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/1134655691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d68325cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e82ffd700 1 -- 172.21.15.27:0/1134655691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 msgr2=0x55d68325c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/1134655691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 0x55d68325c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e82ffd700 1 -- 172.21.15.27:0/1134655691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6830fc8d0 con 0x55d683261f60 2024-06-06T08:40:37.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/1134655691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d683262340 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9e7409b7d0 tx=0x7f9e7400dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea232f4b2f62cefb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:37.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e79ffb700 1 -- 172.21.15.27:0/1134655691 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74099dd0 con 0x55d683261f60 2024-06-06T08:40:37.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e79ffb700 1 -- 172.21.15.27:0/1134655691 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e7400bbf0 con 0x55d683261f60 2024-06-06T08:40:37.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.703+0000 7f9e79ffb700 1 -- 172.21.15.27:0/1134655691 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e740cd050 con 0x55d683261f60 2024-06-06T08:40:37.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 -- 172.21.15.27:0/1134655691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 msgr2=0x55d683262340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:37.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1134655691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d683262340 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9e7409b7d0 tx=0x7f9e7400dee0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 -- 172.21.15.27:0/1134655691 shutdown_connections 2024-06-06T08:40:37.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1134655691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d68325cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1134655691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d683262340 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1134655691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 0x55d68325c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 -- 172.21.15.27:0/1134655691 >> 172.21.15.27:0/1134655691 conn(0x55d683189080 msgr2=0x55d683257220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 -- 172.21.15.27:0/1134655691 shutdown_connections 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 -- 172.21.15.27:0/1134655691 wait complete. 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.704+0000 7f9e94299b80 1 -- start start 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e94299b80 1 Processor -- start 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d6832686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d6832659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 0x55d683265ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e94299b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d68314dfc0 con 0x55d683261f60 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e94299b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d6831a9810 con 0x55d683262880 2024-06-06T08:40:37.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e94299b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d683269260 con 0x55d68325c7a0 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e94299b80 10 client.0 _ll_register_callbacks cb 0x55d6831435b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e827fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d6832659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e827fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d6832659a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34176/0 (socket says 172.21.15.27:34176) 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e827fc700 1 -- 172.21.15.27:0/1003459889 learned_addr learned my addr 172.21.15.27:0/1003459889 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d6832686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e837fe700 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 0x55d683265ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e827fc700 1 -- 172.21.15.27:0/1003459889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 msgr2=0x55d6832686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e827fc700 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d6832686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e827fc700 1 -- 172.21.15.27:0/1003459889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 msgr2=0x55d683265ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e827fc700 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 0x55d683265ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e827fc700 1 -- 172.21.15.27:0/1003459889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d683173750 con 0x55d683261f60 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.705+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d6832686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.706+0000 7f9e827fc700 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d6832659a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c015000 tx=0x7f9e6c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5edcb1485c68a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:37.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.706+0000 7f9e62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.706+0000 7f9e62ffd700 1 -- 172.21.15.27:0/1003459889 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c00b9a0 con 0x55d683261f60 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.706+0000 7f9e62ffd700 1 -- 172.21.15.27:0/1003459889 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9e6c00bb40 con 0x55d683261f60 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.706+0000 7f9e94299b80 1 -- 172.21.15.27:0/1003459889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e7400bbf0 con 0x55d683261f60 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.706+0000 7f9e62ffd700 1 -- 172.21.15.27:0/1003459889 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e6c039050 con 0x55d683261f60 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.706+0000 7f9e94299b80 10 client.15411 fetch_fsmap learned FSMap version 25 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.706+0000 7f9e94299b80 1 -- 172.21.15.27:0/1003459889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d683273aa0 con 0x55d683261f60 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.706+0000 7f9e62ffd700 1 -- 172.21.15.27:0/1003459889 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e7400bbf0 con 0x55d683261f60 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e62ffd700 1 -- 172.21.15.27:0/1003459889 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9e6c024410 con 0x55d683261f60 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e94299b80 10 client.15411 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e94299b80 10 client.15411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e94299b80 20 client.15411 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6832763a0 0x55d683278790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e94299b80 4 client.15411 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:40:37.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e94299b80 1 -- 172.21.15.27:0/1003459889 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6831bad50 con 0x55d6832763a0 2024-06-06T08:40:37.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6832763a0 0x55d683278790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6832763a0 0x55d683278790 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f130f319c4a1084 server_cookie=54c136dd5eee9210 in_seq=0 out_seq=0 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.707+0000 7f9e62ffd700 10 client.15411 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e62ffd700 1 -- 172.21.15.27:0/1003459889 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d6831bad50 con 0x55d6832763a0 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e62ffd700 10 client.15411 handle_command_reply: tid=0 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e94299b80 1 client.15411 shutdown 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e94299b80 1 -- 172.21.15.27:0/1003459889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 msgr2=0x55d6832659a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d6832659a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c015000 tx=0x7f9e6c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e94299b80 1 -- 172.21.15.27:0/1003459889 shutdown_connections 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683262880 0x55d683265ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d6832686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6832763a0 0x55d683278790 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e94299b80 1 --2- 172.21.15.27:0/1003459889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683261f60 0x55d6832659a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e94299b80 1 -- 172.21.15.27:0/1003459889 >> 172.21.15.27:0/1003459889 conn(0x55d683189080 msgr2=0x55d683256700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:37.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.708+0000 7f9e62ffd700 0 client.15411 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:37.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.709+0000 7f9e94299b80 1 -- 172.21.15.27:0/1003459889 shutdown_connections 2024-06-06T08:40:37.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.709+0000 7f9e94299b80 1 -- 172.21.15.27:0/1003459889 wait complete. 2024-06-06T08:40:37.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.709+0000 7f9e94299b80 20 client.15411 trim_cache size 0 max 16384 2024-06-06T08:40:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e94299b80 1 Processor -- start 2024-06-06T08:40:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e94299b80 1 -- start start 2024-06-06T08:40:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d683189080 0x55d6832634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d68325c7a0 0x55d6832611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 0x55d6832681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e94299b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841b7520 con 0x55d68325c7a0 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e94299b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841b4db0 con 0x55d683261f60 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e94299b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841b2c00 con 0x55d683189080 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e827fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d68325c7a0 0x55d6832611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e827fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d68325c7a0 0x55d6832611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34186/0 (socket says 172.21.15.27:34186) 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e827fc700 1 -- 172.21.15.27:0/3062266027 learned_addr learned my addr 172.21.15.27:0/3062266027 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e837fe700 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 0x55d6832681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d683189080 0x55d6832634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e827fc700 1 -- 172.21.15.27:0/3062266027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d683189080 msgr2=0x55d6832634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e827fc700 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d683189080 0x55d6832634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e827fc700 1 -- 172.21.15.27:0/3062266027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 msgr2=0x55d6832681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e827fc700 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 0x55d6832681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e827fc700 1 -- 172.21.15.27:0/3062266027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d683263f10 con 0x55d68325c7a0 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.741+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d683189080 0x55d6832634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.742+0000 7f9e827fc700 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d68325c7a0 0x55d6832611e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0098e0 tx=0x7f9e6c00b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b92ba220db3921 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.742+0000 7f9e797fa700 1 -- 172.21.15.27:0/3062266027 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c029030 con 0x55d68325c7a0 2024-06-06T08:40:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.742+0000 7f9e797fa700 1 -- 172.21.15.27:0/3062266027 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c00be80 con 0x55d68325c7a0 2024-06-06T08:40:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.742+0000 7f9e797fa700 1 -- 172.21.15.27:0/3062266027 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c030900 con 0x55d68325c7a0 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 -- 172.21.15.27:0/3062266027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d68325c7a0 msgr2=0x55d6832611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d68325c7a0 0x55d6832611e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0098e0 tx=0x7f9e6c00b5c0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 -- 172.21.15.27:0/3062266027 shutdown_connections 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 0x55d6832681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d683189080 0x55d6832634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3062266027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d68325c7a0 0x55d6832611e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 -- 172.21.15.27:0/3062266027 >> 172.21.15.27:0/3062266027 conn(0x55d683260270 msgr2=0x55d683260ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 -- 172.21.15.27:0/3062266027 shutdown_connections 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 -- 172.21.15.27:0/3062266027 wait complete. 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.743+0000 7f9e94299b80 1 -- start start 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e94299b80 1 Processor -- start 2024-06-06T08:40:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683189080 0x55d68326e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d68326ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e82ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683189080 0x55d68326e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e94299b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 0x55d68326b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e82ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683189080 0x55d68326e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34194/0 (socket says 172.21.15.27:34194) 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e82ffd700 1 -- 172.21.15.27:0/3005972047 learned_addr learned my addr 172.21.15.27:0/3005972047 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841b3270 con 0x55d683189080 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841b46e0 con 0x55d683261f60 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841b3940 con 0x55d68325c7a0 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e827fc700 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d68326ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.744+0000 7f9e837fe700 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 0x55d68326b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e82ffd700 1 -- 172.21.15.27:0/3005972047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 msgr2=0x55d68326ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d68326ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e94299b80 10 client.0 _ll_register_callbacks cb 0x55d6831435b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e82ffd700 1 -- 172.21.15.27:0/3005972047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 msgr2=0x55d68326b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 0x55d68326b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e82ffd700 1 -- 172.21.15.27:0/3005972047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d68326f0e0 con 0x55d683189080 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6831b52d0 con 0x55d683189080 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e82ffd700 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683189080 0x55d68326e4f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9e740984a0 tx=0x7f9e74004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c31b111d3fb2cd7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e62ffd700 10 client.15417 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e62ffd700 1 -- 172.21.15.27:0/3005972047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400bbf0 con 0x55d683189080 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e62ffd700 1 -- 172.21.15.27:0/3005972047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9e740a40b0 con 0x55d683189080 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e62ffd700 1 -- 172.21.15.27:0/3005972047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e740cce10 con 0x55d683189080 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e94299b80 10 client.15417 fetch_fsmap learned FSMap version 25 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.745+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55d6832e11e0 con 0x55d683189080 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e62ffd700 1 -- 172.21.15.27:0/3005972047 <== mon.0 v2:172.21.15.27:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9e7409af20 con 0x55d683189080 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e94299b80 10 client.15417 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e94299b80 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e94299b80 20 client.15417 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d683262880 0x55d6832e33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:37.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e94299b80 4 client.15417 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d683168f50 con 0x55d683262880 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e827fc700 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d683262880 0x55d6832e33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e827fc700 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d683262880 0x55d6832e33c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4146898276c28f21 server_cookie=f274b543ce81fc7e in_seq=0 out_seq=0 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.746+0000 7f9e62ffd700 10 client.15417 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e62ffd700 1 -- 172.21.15.27:0/3005972047 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d683168f50 con 0x55d683262880 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e62ffd700 10 client.15417 handle_command_reply: tid=0 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 client.15417 shutdown 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683189080 msgr2=0x55d68326e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683189080 0x55d68326e4f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9e740984a0 tx=0x7f9e74004830 comp rx=0 tx=0).stop 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 shutdown_connections 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d683261f60 0x55d68326b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d683262880 0x55d6832e33c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d683189080 0x55d68326e4f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 --2- 172.21.15.27:0/3005972047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d68325c7a0 0x55d68326ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 >> 172.21.15.27:0/3005972047 conn(0x55d683260270 msgr2=0x55d683260ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e62ffd700 0 client.15417 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 shutdown_connections 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 1 -- 172.21.15.27:0/3005972047 wait complete. 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.747+0000 7f9e94299b80 20 client.15417 trim_cache size 0 max 16384 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stdout: "9dae157b-1457-4606-83ea-8502d1035717": { 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "9dae157b-1457-4606-83ea-8502d1035717", 2024-06-06T08:40:37.750 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-06-06T08:40:37.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:40:37.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:40:37.751 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:40:37.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 -- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9e700a2eb0 msgr2=0x7f9e700a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:37.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9e700a2eb0 0x7f9e700a5370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e8410d3a0 tx=0x7f9e6c005fb0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 -- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 msgr2=0x7f9e8410bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:37.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 0x7f9e8410bd00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400d980 tx=0x7f9e7400dd40 comp rx=0 tx=0).stop 2024-06-06T08:40:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 -- 172.21.15.27:0/719824411 shutdown_connections 2024-06-06T08:40:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e841085d0 0x7f9e841aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84102530 0x7f9e8410bd00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9e700a2eb0 0x7f9e700a5370 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 --2- 172.21.15.27:0/719824411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e84102ef0 0x7f9e8410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 -- 172.21.15.27:0/719824411 >> 172.21.15.27:0/719824411 conn(0x7f9e84079640 msgr2=0x7f9e841060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 -- 172.21.15.27:0/719824411 shutdown_connections 2024-06-06T08:40:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:37.748+0000 7f9e89a16700 1 -- 172.21.15.27:0/719824411 wait complete. 2024-06-06T08:40:37.755 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:37.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:37 smithi027 ceph-mon[32335]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:38.141 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:37 smithi203 ceph-mon[36579]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:38.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:37 smithi176 ceph-mon[36786]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:38.586 INFO:tasks.cephfs.filesystem:scrub status for tag:9dae157b-1457-4606-83ea-8502d1035717 - {'path': '/', 'tag': '9dae157b-1457-4606-83ea-8502d1035717', 'options': 'recursive,force'} 2024-06-06T08:40:39.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:38 smithi027 ceph-mon[32335]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T08:40:39.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:38 smithi203 ceph-mon[36579]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T08:40:39.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:38 smithi176 ceph-mon[36786]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T08:40:40.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 -- 172.21.15.27:0/1195237440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc102740 msgr2=0x7f9acc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:40.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1195237440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc102740 0x7f9acc102b20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0099c0 tx=0x7f9abc009cd0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 -- 172.21.15.27:0/1195237440 shutdown_connections 2024-06-06T08:40:40.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1195237440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9acc107840 0x7f9acc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1195237440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc102740 0x7f9acc102b20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1195237440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9acc1030f0 0x7f9acc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 -- 172.21.15.27:0/1195237440 >> 172.21.15.27:0/1195237440 conn(0x7f9acc0795f0 msgr2=0x7f9acc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 -- 172.21.15.27:0/1195237440 shutdown_connections 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 -- 172.21.15.27:0/1195237440 wait complete. 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 Processor -- start 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.377+0000 7f9ad2990700 1 -- start start 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9ad2990700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9acc102740 0x7f9acc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9ad2990700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc1030f0 0x7f9acc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9ad2990700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9acc107840 0x7f9acc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9ad2990700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc1127e0 con 0x7f9acc1030f0 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9ad2990700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc112660 con 0x7f9acc107840 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9ad2990700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc112960 con 0x7f9acc102740 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc1030f0 0x7f9acc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc1030f0 0x7f9acc19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54726/0 (socket says 172.21.15.27:54726) 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acb7fe700 1 -- 172.21.15.27:0/1420623661 learned_addr learned my addr 172.21.15.27:0/1420623661 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:40.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acbfff700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9acc102740 0x7f9acc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9ad0f2d700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9acc107840 0x7f9acc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acb7fe700 1 -- 172.21.15.27:0/1420623661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9acc102740 msgr2=0x7f9acc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acb7fe700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9acc102740 0x7f9acc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acb7fe700 1 -- 172.21.15.27:0/1420623661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9acc107840 msgr2=0x7f9acc198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acb7fe700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9acc107840 0x7f9acc198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acb7fe700 1 -- 172.21.15.27:0/1420623661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc1991c0 con 0x7f9acc1030f0 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acbfff700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9acc102740 0x7f9acc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9ad0f2d700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9acc107840 0x7f9acc198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9acb7fe700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc1030f0 0x7f9acc19ed90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d950 tx=0x7f9ac000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.378+0000 7f9ac97fa700 1 -- 172.21.15.27:0/1420623661 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0016070 con 0x7f9acc1030f0 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.379+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc1994b0 con 0x7f9acc1030f0 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.379+0000 7f9ac97fa700 1 -- 172.21.15.27:0/1420623661 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac0009b40 con 0x7f9acc1030f0 2024-06-06T08:40:40.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.379+0000 7f9ac97fa700 1 -- 172.21.15.27:0/1420623661 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000f620 con 0x7f9acc1030f0 2024-06-06T08:40:40.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.379+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc1a5720 con 0x7f9acc1030f0 2024-06-06T08:40:40.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.380+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc10aa80 con 0x7f9acc1030f0 2024-06-06T08:40:40.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.381+0000 7f9ac97fa700 1 -- 172.21.15.27:0/1420623661 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f9ac0019070 con 0x7f9acc1030f0 2024-06-06T08:40:40.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.382+0000 7f9ac97fa700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9ab40a2eb0 0x7f9ab40a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:40.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.382+0000 7f9acbfff700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9ab40a2eb0 0x7f9ab40a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:40.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.382+0000 7f9ac97fa700 1 -- 172.21.15.27:0/1420623661 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9ac00cfae0 con 0x7f9acc1030f0 2024-06-06T08:40:40.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.382+0000 7f9acbfff700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9ab40a2eb0 0x7f9ab40a5370 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9abc000c00 tx=0x7f9abc005940 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:40.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.385+0000 7f9ac97fa700 1 -- 172.21.15.27:0/1420623661 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ac009c0f0 con 0x7f9acc1030f0 2024-06-06T08:40:40.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.588+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9acc0628d0 con 0x7f9ab40a2eb0 2024-06-06T08:40:40.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.597+0000 7f9ac97fa700 1 -- 172.21.15.27:0/1420623661 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339846 (secure 0 0 0) 0x7f9acc0628d0 con 0x7f9ab40a2eb0 2024-06-06T08:40:40.599 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9ab40a2eb0 msgr2=0x7f9ab40a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9ab40a2eb0 0x7f9ab40a5370 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9abc000c00 tx=0x7f9abc005940 comp rx=0 tx=0).stop 2024-06-06T08:40:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc1030f0 msgr2=0x7f9acc19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc1030f0 0x7f9acc19ed90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d950 tx=0x7f9ac000dd10 comp rx=0 tx=0).stop 2024-06-06T08:40:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 shutdown_connections 2024-06-06T08:40:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9ab40a2eb0 0x7f9ab40a5370 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9acc107840 0x7f9acc198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9acc102740 0x7f9acc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 --2- 172.21.15.27:0/1420623661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9acc1030f0 0x7f9acc19ed90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:40.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 >> 172.21.15.27:0/1420623661 conn(0x7f9acc0795f0 msgr2=0x7f9acc100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:40.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 shutdown_connections 2024-06-06T08:40:40.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:40.602+0000 7f9ad2990700 1 -- 172.21.15.27:0/1420623661 wait complete. 2024-06-06T08:40:40.603 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:40:40.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:40 smithi027 ceph-mon[32335]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-06-06T08:40:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:40 smithi203 ceph-mon[36579]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-06-06T08:40:41.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:40 smithi176 ceph-mon[36786]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-06-06T08:40:41.389 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-06-06T08:40:39.309319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389359,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1788,"num_read_kb":1326,"num_write":636,"num_write_kb":5307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":969,"ondisk_log_size":969,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362876,"kb_used_data":13688,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578948,"statfs":{"total":1151940427776,"available":1151568842752,"internally_reserved":0,"allocated":14016512,"data_stored":11102243,"data_compressed":634920,"data_compressed_allocated":860160,"data_compressed_original":5050368,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":345,"num_read_kb":277,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002665"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543767+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:40:17.543767+0000","last_peered":"2024-06-06T08:40:17.543767+0000","last_clean":"2024-06-06T08:40:17.543767+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:40:17.543767+0000","last_undegraded":"2024-06-06T08:40:17.543767+0000","last_fullsized":"2024-06-06T08:40:17.543767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603694+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:40:17.603694+0000","last_peered":"2024-06-06T08:40:17.603694+0000","last_clean":"2024-06-06T08:40:17.603694+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:40:17.603694+0000","last_undegraded":"2024-06-06T08:40:17.603694+0000","last_fullsized":"2024-06-06T08:40:17.603694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603368+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:40:17.603368+0000","last_peered":"2024-06-06T08:40:17.603368+0000","last_clean":"2024-06-06T08:40:17.603368+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:40:17.603368+0000","last_undegraded":"2024-06-06T08:40:17.603368+0000","last_fullsized":"2024-06-06T08:40:17.603368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602693+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:40:17.602693+0000","last_peered":"2024-06-06T08:40:17.602693+0000","last_clean":"2024-06-06T08:40:17.602693+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:40:17.602693+0000","last_undegraded":"2024-06-06T08:40:17.602693+0000","last_fullsized":"2024-06-06T08:40:17.602693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397130+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:40:17.397130+0000","last_peered":"2024-06-06T08:40:17.397130+0000","last_clean":"2024-06-06T08:40:17.397130+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:40:17.397130+0000","last_undegraded":"2024-06-06T08:40:17.397130+0000","last_fullsized":"2024-06-06T08:40:17.397130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406459+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:40:17.406459+0000","last_peered":"2024-06-06T08:40:17.406459+0000","last_clean":"2024-06-06T08:40:17.406459+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:40:17.406459+0000","last_undegraded":"2024-06-06T08:40:17.406459+0000","last_fullsized":"2024-06-06T08:40:17.406459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290504+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:40:17.290504+0000","last_peered":"2024-06-06T08:40:17.290504+0000","last_clean":"2024-06-06T08:40:17.290504+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:40:17.290504+0000","last_undegraded":"2024-06-06T08:40:17.290504+0000","last_fullsized":"2024-06-06T08:40:17.290504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394085+0000","last_change":"2024-06-06T08:38:01.335419+0000","last_active":"2024-06-06T08:40:17.394085+0000","last_peered":"2024-06-06T08:40:17.394085+0000","last_clean":"2024-06-06T08:40:17.394085+0000","last_became_active":"2024-06-06T08:38:01.335191+0000","last_became_peered":"2024-06-06T08:38:01.335191+0000","last_unstale":"2024-06-06T08:40:17.394085+0000","last_undegraded":"2024-06-06T08:40:17.394085+0000","last_fullsized":"2024-06-06T08:40:17.394085+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:13.253838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619376+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:40:17.619376+0000","last_peered":"2024-06-06T08:40:17.619376+0000","last_clean":"2024-06-06T08:40:17.619376+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:40:17.619376+0000","last_undegraded":"2024-06-06T08:40:17.619376+0000","last_fullsized":"2024-06-06T08:40:17.619376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406777+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:40:17.406777+0000","last_peered":"2024-06-06T08:40:17.406777+0000","last_clean":"2024-06-06T08:40:17.406777+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:40:17.406777+0000","last_undegraded":"2024-06-06T08:40:17.406777+0000","last_fullsized":"2024-06-06T08:40:17.406777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290994+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:40:17.290994+0000","last_peered":"2024-06-06T08:40:17.290994+0000","last_clean":"2024-06-06T08:40:17.290994+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:40:17.290994+0000","last_undegraded":"2024-06-06T08:40:17.290994+0000","last_fullsized":"2024-06-06T08:40:17.290994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602644+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:40:17.602644+0000","last_peered":"2024-06-06T08:40:17.602644+0000","last_clean":"2024-06-06T08:40:17.602644+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:40:17.602644+0000","last_undegraded":"2024-06-06T08:40:17.602644+0000","last_fullsized":"2024-06-06T08:40:17.602644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289360+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:40:17.289360+0000","last_peered":"2024-06-06T08:40:17.289360+0000","last_clean":"2024-06-06T08:40:17.289360+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:40:17.289360+0000","last_undegraded":"2024-06-06T08:40:17.289360+0000","last_fullsized":"2024-06-06T08:40:17.289360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602964+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:40:17.602964+0000","last_peered":"2024-06-06T08:40:17.602964+0000","last_clean":"2024-06-06T08:40:17.602964+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:40:17.602964+0000","last_undegraded":"2024-06-06T08:40:17.602964+0000","last_fullsized":"2024-06-06T08:40:17.602964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603050+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:40:17.603050+0000","last_peered":"2024-06-06T08:40:17.603050+0000","last_clean":"2024-06-06T08:40:17.603050+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:40:17.603050+0000","last_undegraded":"2024-06-06T08:40:17.603050+0000","last_fullsized":"2024-06-06T08:40:17.603050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618538+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:40:17.618538+0000","last_peered":"2024-06-06T08:40:17.618538+0000","last_clean":"2024-06-06T08:40:17.618538+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:40:17.618538+0000","last_undegraded":"2024-06-06T08:40:17.618538+0000","last_fullsized":"2024-06-06T08:40:17.618538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543492+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:40:17.543492+0000","last_peered":"2024-06-06T08:40:17.543492+0000","last_clean":"2024-06-06T08:40:17.543492+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:40:17.543492+0000","last_undegraded":"2024-06-06T08:40:17.543492+0000","last_fullsized":"2024-06-06T08:40:17.543492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394947+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:40:17.394947+0000","last_peered":"2024-06-06T08:40:17.394947+0000","last_clean":"2024-06-06T08:40:17.394947+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:40:17.394947+0000","last_undegraded":"2024-06-06T08:40:17.394947+0000","last_fullsized":"2024-06-06T08:40:17.394947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291235+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:40:17.291235+0000","last_peered":"2024-06-06T08:40:17.291235+0000","last_clean":"2024-06-06T08:40:17.291235+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:40:17.291235+0000","last_undegraded":"2024-06-06T08:40:17.291235+0000","last_fullsized":"2024-06-06T08:40:17.291235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291190+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:40:17.291190+0000","last_peered":"2024-06-06T08:40:17.291190+0000","last_clean":"2024-06-06T08:40:17.291190+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:40:17.291190+0000","last_undegraded":"2024-06-06T08:40:17.291190+0000","last_fullsized":"2024-06-06T08:40:17.291190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:23.296771+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:40:23.296771+0000","last_peered":"2024-06-06T08:40:23.296771+0000","last_clean":"2024-06-06T08:40:23.296771+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:40:23.296771+0000","last_undegraded":"2024-06-06T08:40:23.296771+0000","last_fullsized":"2024-06-06T08:40:23.296771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397547+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:40:17.397547+0000","last_peered":"2024-06-06T08:40:17.397547+0000","last_clean":"2024-06-06T08:40:17.397547+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:40:17.397547+0000","last_undegraded":"2024-06-06T08:40:17.397547+0000","last_fullsized":"2024-06-06T08:40:17.397547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602515+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:40:17.602515+0000","last_peered":"2024-06-06T08:40:17.602515+0000","last_clean":"2024-06-06T08:40:17.602515+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:40:17.602515+0000","last_undegraded":"2024-06-06T08:40:17.602515+0000","last_fullsized":"2024-06-06T08:40:17.602515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397886+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:40:17.397886+0000","last_peered":"2024-06-06T08:40:17.397886+0000","last_clean":"2024-06-06T08:40:17.397886+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:40:17.397886+0000","last_undegraded":"2024-06-06T08:40:17.397886+0000","last_fullsized":"2024-06-06T08:40:17.397886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603696+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:40:17.603696+0000","last_peered":"2024-06-06T08:40:17.603696+0000","last_clean":"2024-06-06T08:40:17.603696+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:40:17.603696+0000","last_undegraded":"2024-06-06T08:40:17.603696+0000","last_fullsized":"2024-06-06T08:40:17.603696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602634+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:40:17.602634+0000","last_peered":"2024-06-06T08:40:17.602634+0000","last_clean":"2024-06-06T08:40:17.602634+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:40:17.602634+0000","last_undegraded":"2024-06-06T08:40:17.602634+0000","last_fullsized":"2024-06-06T08:40:17.602634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397399+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:40:17.397399+0000","last_peered":"2024-06-06T08:40:17.397399+0000","last_clean":"2024-06-06T08:40:17.397399+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:40:17.397399+0000","last_undegraded":"2024-06-06T08:40:17.397399+0000","last_fullsized":"2024-06-06T08:40:17.397399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618791+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:40:17.618791+0000","last_peered":"2024-06-06T08:40:17.618791+0000","last_clean":"2024-06-06T08:40:17.618791+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:40:17.618791+0000","last_undegraded":"2024-06-06T08:40:17.618791+0000","last_fullsized":"2024-06-06T08:40:17.618791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602522+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:40:17.602522+0000","last_peered":"2024-06-06T08:40:17.602522+0000","last_clean":"2024-06-06T08:40:17.602522+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:40:17.602522+0000","last_undegraded":"2024-06-06T08:40:17.602522+0000","last_fullsized":"2024-06-06T08:40:17.602522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397350+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:40:17.397350+0000","last_peered":"2024-06-06T08:40:17.397350+0000","last_clean":"2024-06-06T08:40:17.397350+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:40:17.397350+0000","last_undegraded":"2024-06-06T08:40:17.397350+0000","last_fullsized":"2024-06-06T08:40:17.397350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394983+0000","last_change":"2024-06-06T08:38:01.728398+0000","last_active":"2024-06-06T08:40:17.394983+0000","last_peered":"2024-06-06T08:40:17.394983+0000","last_clean":"2024-06-06T08:40:17.394983+0000","last_became_active":"2024-06-06T08:38:01.728004+0000","last_became_peered":"2024-06-06T08:38:01.728004+0000","last_unstale":"2024-06-06T08:40:17.394983+0000","last_undegraded":"2024-06-06T08:40:17.394983+0000","last_fullsized":"2024-06-06T08:40:17.394983+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:02:03.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291009+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:40:17.291009+0000","last_peered":"2024-06-06T08:40:17.291009+0000","last_clean":"2024-06-06T08:40:17.291009+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:40:17.291009+0000","last_undegraded":"2024-06-06T08:40:17.291009+0000","last_fullsized":"2024-06-06T08:40:17.291009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":252,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:37.114832+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:40:37.114832+0000","last_peered":"2024-06-06T08:40:37.114832+0000","last_clean":"2024-06-06T08:40:37.114832+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:40:37.114832+0000","last_undegraded":"2024-06-06T08:40:37.114832+0000","last_fullsized":"2024-06-06T08:40:37.114832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1320,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396976+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:40:17.396976+0000","last_peered":"2024-06-06T08:40:17.396976+0000","last_clean":"2024-06-06T08:40:17.396976+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:40:17.396976+0000","last_undegraded":"2024-06-06T08:40:17.396976+0000","last_fullsized":"2024-06-06T08:40:17.396976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290894+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:40:17.290894+0000","last_peered":"2024-06-06T08:40:17.290894+0000","last_clean":"2024-06-06T08:40:17.290894+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:40:17.290894+0000","last_undegraded":"2024-06-06T08:40:17.290894+0000","last_fullsized":"2024-06-06T08:40:17.290894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406405+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:40:17.406405+0000","last_peered":"2024-06-06T08:40:17.406405+0000","last_clean":"2024-06-06T08:40:17.406405+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:40:17.406405+0000","last_undegraded":"2024-06-06T08:40:17.406405+0000","last_fullsized":"2024-06-06T08:40:17.406405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394510+0000","last_change":"2024-06-06T08:38:01.728238+0000","last_active":"2024-06-06T08:40:17.394510+0000","last_peered":"2024-06-06T08:40:17.394510+0000","last_clean":"2024-06-06T08:40:17.394510+0000","last_became_active":"2024-06-06T08:38:01.727844+0000","last_became_peered":"2024-06-06T08:38:01.727844+0000","last_unstale":"2024-06-06T08:40:17.394510+0000","last_undegraded":"2024-06-06T08:40:17.394510+0000","last_fullsized":"2024-06-06T08:40:17.394510+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:19:46.825321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543511+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:40:17.543511+0000","last_peered":"2024-06-06T08:40:17.543511+0000","last_clean":"2024-06-06T08:40:17.543511+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:40:17.543511+0000","last_undegraded":"2024-06-06T08:40:17.543511+0000","last_fullsized":"2024-06-06T08:40:17.543511+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288975+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:40:17.288975+0000","last_peered":"2024-06-06T08:40:17.288975+0000","last_clean":"2024-06-06T08:40:17.288975+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:40:17.288975+0000","last_undegraded":"2024-06-06T08:40:17.288975+0000","last_fullsized":"2024-06-06T08:40:17.288975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396897+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:40:17.396897+0000","last_peered":"2024-06-06T08:40:17.396897+0000","last_clean":"2024-06-06T08:40:17.396897+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:40:17.396897+0000","last_undegraded":"2024-06-06T08:40:17.396897+0000","last_fullsized":"2024-06-06T08:40:17.396897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603635+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:40:17.603635+0000","last_peered":"2024-06-06T08:40:17.603635+0000","last_clean":"2024-06-06T08:40:17.603635+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:40:17.603635+0000","last_undegraded":"2024-06-06T08:40:17.603635+0000","last_fullsized":"2024-06-06T08:40:17.603635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403412+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:40:17.403412+0000","last_peered":"2024-06-06T08:40:17.403412+0000","last_clean":"2024-06-06T08:40:17.403412+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:40:17.403412+0000","last_undegraded":"2024-06-06T08:40:17.403412+0000","last_fullsized":"2024-06-06T08:40:17.403412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394236+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:40:17.394236+0000","last_peered":"2024-06-06T08:40:17.394236+0000","last_clean":"2024-06-06T08:40:17.394236+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:40:17.394236+0000","last_undegraded":"2024-06-06T08:40:17.394236+0000","last_fullsized":"2024-06-06T08:40:17.394236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":245,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:37.114303+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:40:37.114303+0000","last_peered":"2024-06-06T08:40:37.114303+0000","last_clean":"2024-06-06T08:40:37.114303+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:40:37.114303+0000","last_undegraded":"2024-06-06T08:40:37.114303+0000","last_fullsized":"2024-06-06T08:40:37.114303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619308+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:40:17.619308+0000","last_peered":"2024-06-06T08:40:17.619308+0000","last_clean":"2024-06-06T08:40:17.619308+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:40:17.619308+0000","last_undegraded":"2024-06-06T08:40:17.619308+0000","last_fullsized":"2024-06-06T08:40:17.619308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394167+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:40:17.394167+0000","last_peered":"2024-06-06T08:40:17.394167+0000","last_clean":"2024-06-06T08:40:17.394167+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:40:17.394167+0000","last_undegraded":"2024-06-06T08:40:17.394167+0000","last_fullsized":"2024-06-06T08:40:17.394167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290830+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:40:17.290830+0000","last_peered":"2024-06-06T08:40:17.290830+0000","last_clean":"2024-06-06T08:40:17.290830+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:40:17.290830+0000","last_undegraded":"2024-06-06T08:40:17.290830+0000","last_fullsized":"2024-06-06T08:40:17.290830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603715+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:40:17.603715+0000","last_peered":"2024-06-06T08:40:17.603715+0000","last_clean":"2024-06-06T08:40:17.603715+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:40:17.603715+0000","last_undegraded":"2024-06-06T08:40:17.603715+0000","last_fullsized":"2024-06-06T08:40:17.603715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602684+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:40:17.602684+0000","last_peered":"2024-06-06T08:40:17.602684+0000","last_clean":"2024-06-06T08:40:17.602684+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:40:17.602684+0000","last_undegraded":"2024-06-06T08:40:17.602684+0000","last_fullsized":"2024-06-06T08:40:17.602684+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403562+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:40:17.403562+0000","last_peered":"2024-06-06T08:40:17.403562+0000","last_clean":"2024-06-06T08:40:17.403562+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:40:17.403562+0000","last_undegraded":"2024-06-06T08:40:17.403562+0000","last_fullsized":"2024-06-06T08:40:17.403562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618947+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:40:17.618947+0000","last_peered":"2024-06-06T08:40:17.618947+0000","last_clean":"2024-06-06T08:40:17.618947+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:40:17.618947+0000","last_undegraded":"2024-06-06T08:40:17.618947+0000","last_fullsized":"2024-06-06T08:40:17.618947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406287+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:40:17.406287+0000","last_peered":"2024-06-06T08:40:17.406287+0000","last_clean":"2024-06-06T08:40:17.406287+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:40:17.406287+0000","last_undegraded":"2024-06-06T08:40:17.406287+0000","last_fullsized":"2024-06-06T08:40:17.406287+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'43","reported_seq":511,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:38.115997+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:40:38.115997+0000","last_peered":"2024-06-06T08:40:38.115997+0000","last_clean":"2024-06-06T08:40:38.115997+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:40:38.115997+0000","last_undegraded":"2024-06-06T08:40:38.115997+0000","last_fullsized":"2024-06-06T08:40:38.115997+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71690,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":405,"num_read_kb":274,"num_write":44,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397172+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:40:17.397172+0000","last_peered":"2024-06-06T08:40:17.397172+0000","last_clean":"2024-06-06T08:40:17.397172+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:40:17.397172+0000","last_undegraded":"2024-06-06T08:40:17.397172+0000","last_fullsized":"2024-06-06T08:40:17.397172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291329+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:40:17.291329+0000","last_peered":"2024-06-06T08:40:17.291329+0000","last_clean":"2024-06-06T08:40:17.291329+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:40:17.291329+0000","last_undegraded":"2024-06-06T08:40:17.291329+0000","last_fullsized":"2024-06-06T08:40:17.291329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289082+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:40:17.289082+0000","last_peered":"2024-06-06T08:40:17.289082+0000","last_clean":"2024-06-06T08:40:17.289082+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:40:17.289082+0000","last_undegraded":"2024-06-06T08:40:17.289082+0000","last_fullsized":"2024-06-06T08:40:17.289082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543337+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:40:17.543337+0000","last_peered":"2024-06-06T08:40:17.543337+0000","last_clean":"2024-06-06T08:40:17.543337+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:40:17.543337+0000","last_undegraded":"2024-06-06T08:40:17.543337+0000","last_fullsized":"2024-06-06T08:40:17.543337+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403613+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:40:17.403613+0000","last_peered":"2024-06-06T08:40:17.403613+0000","last_clean":"2024-06-06T08:40:17.403613+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:40:17.403613+0000","last_undegraded":"2024-06-06T08:40:17.403613+0000","last_fullsized":"2024-06-06T08:40:17.403613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290581+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:40:17.290581+0000","last_peered":"2024-06-06T08:40:17.290581+0000","last_clean":"2024-06-06T08:40:17.290581+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:40:17.290581+0000","last_undegraded":"2024-06-06T08:40:17.290581+0000","last_fullsized":"2024-06-06T08:40:17.290581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394932+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:40:17.394932+0000","last_peered":"2024-06-06T08:40:17.394932+0000","last_clean":"2024-06-06T08:40:17.394932+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:40:17.394932+0000","last_undegraded":"2024-06-06T08:40:17.394932+0000","last_fullsized":"2024-06-06T08:40:17.394932+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":254,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:37.115001+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:40:37.115001+0000","last_peered":"2024-06-06T08:40:37.115001+0000","last_clean":"2024-06-06T08:40:37.115001+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:40:37.115001+0000","last_undegraded":"2024-06-06T08:40:37.115001+0000","last_fullsized":"2024-06-06T08:40:37.115001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":201,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290588+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:40:17.290588+0000","last_peered":"2024-06-06T08:40:17.290588+0000","last_clean":"2024-06-06T08:40:17.290588+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:40:17.290588+0000","last_undegraded":"2024-06-06T08:40:17.290588+0000","last_fullsized":"2024-06-06T08:40:17.290588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289726+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:40:17.289726+0000","last_peered":"2024-06-06T08:40:17.289726+0000","last_clean":"2024-06-06T08:40:17.289726+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:40:17.289726+0000","last_undegraded":"2024-06-06T08:40:17.289726+0000","last_fullsized":"2024-06-06T08:40:17.289726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396877+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:40:17.396877+0000","last_peered":"2024-06-06T08:40:17.396877+0000","last_clean":"2024-06-06T08:40:17.396877+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:40:17.396877+0000","last_undegraded":"2024-06-06T08:40:17.396877+0000","last_fullsized":"2024-06-06T08:40:17.396877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403466+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:40:17.403466+0000","last_peered":"2024-06-06T08:40:17.403466+0000","last_clean":"2024-06-06T08:40:17.403466+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:40:17.403466+0000","last_undegraded":"2024-06-06T08:40:17.403466+0000","last_fullsized":"2024-06-06T08:40:17.403466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403457+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:40:17.403457+0000","last_peered":"2024-06-06T08:40:17.403457+0000","last_clean":"2024-06-06T08:40:17.403457+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:40:17.403457+0000","last_undegraded":"2024-06-06T08:40:17.403457+0000","last_fullsized":"2024-06-06T08:40:17.403457+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"100'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.933761+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:40:17.933761+0000","last_peered":"2024-06-06T08:40:17.933761+0000","last_clean":"2024-06-06T08:40:17.933761+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:40:17.933761+0000","last_undegraded":"2024-06-06T08:40:17.933761+0000","last_fullsized":"2024-06-06T08:40:17.933761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543570+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:40:17.543570+0000","last_peered":"2024-06-06T08:40:17.543570+0000","last_clean":"2024-06-06T08:40:17.543570+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:40:17.543570+0000","last_undegraded":"2024-06-06T08:40:17.543570+0000","last_fullsized":"2024-06-06T08:40:17.543570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394760+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:40:17.394760+0000","last_peered":"2024-06-06T08:40:17.394760+0000","last_clean":"2024-06-06T08:40:17.394760+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:40:17.394760+0000","last_undegraded":"2024-06-06T08:40:17.394760+0000","last_fullsized":"2024-06-06T08:40:17.394760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406591+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:40:17.406591+0000","last_peered":"2024-06-06T08:40:17.406591+0000","last_clean":"2024-06-06T08:40:17.406591+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:40:17.406591+0000","last_undegraded":"2024-06-06T08:40:17.406591+0000","last_fullsized":"2024-06-06T08:40:17.406591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289289+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:40:17.289289+0000","last_peered":"2024-06-06T08:40:17.289289+0000","last_clean":"2024-06-06T08:40:17.289289+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:40:17.289289+0000","last_undegraded":"2024-06-06T08:40:17.289289+0000","last_fullsized":"2024-06-06T08:40:17.289289+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290643+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:40:17.290643+0000","last_peered":"2024-06-06T08:40:17.290643+0000","last_clean":"2024-06-06T08:40:17.290643+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:40:17.290643+0000","last_undegraded":"2024-06-06T08:40:17.290643+0000","last_fullsized":"2024-06-06T08:40:17.290643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406476+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:40:17.406476+0000","last_peered":"2024-06-06T08:40:17.406476+0000","last_clean":"2024-06-06T08:40:17.406476+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:40:17.406476+0000","last_undegraded":"2024-06-06T08:40:17.406476+0000","last_fullsized":"2024-06-06T08:40:17.406476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289488+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:40:17.289488+0000","last_peered":"2024-06-06T08:40:17.289488+0000","last_clean":"2024-06-06T08:40:17.289488+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:40:17.289488+0000","last_undegraded":"2024-06-06T08:40:17.289488+0000","last_fullsized":"2024-06-06T08:40:17.289488+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403823+0000","last_change":"2024-06-06T08:38:01.545781+0000","last_active":"2024-06-06T08:40:17.403823+0000","last_peered":"2024-06-06T08:40:17.403823+0000","last_clean":"2024-06-06T08:40:17.403823+0000","last_became_active":"2024-06-06T08:38:01.545462+0000","last_became_peered":"2024-06-06T08:38:01.545462+0000","last_unstale":"2024-06-06T08:40:17.403823+0000","last_undegraded":"2024-06-06T08:40:17.403823+0000","last_fullsized":"2024-06-06T08:40:17.403823+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:14:36.722960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":248,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:37.114351+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:40:37.114351+0000","last_peered":"2024-06-06T08:40:37.114351+0000","last_clean":"2024-06-06T08:40:37.114351+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:40:37.114351+0000","last_undegraded":"2024-06-06T08:40:37.114351+0000","last_fullsized":"2024-06-06T08:40:37.114351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290959+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:40:17.290959+0000","last_peered":"2024-06-06T08:40:17.290959+0000","last_clean":"2024-06-06T08:40:17.290959+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:40:17.290959+0000","last_undegraded":"2024-06-06T08:40:17.290959+0000","last_fullsized":"2024-06-06T08:40:17.290959+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397178+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:40:17.397178+0000","last_peered":"2024-06-06T08:40:17.397178+0000","last_clean":"2024-06-06T08:40:17.397178+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:40:17.397178+0000","last_undegraded":"2024-06-06T08:40:17.397178+0000","last_fullsized":"2024-06-06T08:40:17.397178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602424+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:40:17.602424+0000","last_peered":"2024-06-06T08:40:17.602424+0000","last_clean":"2024-06-06T08:40:17.602424+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:40:17.602424+0000","last_undegraded":"2024-06-06T08:40:17.602424+0000","last_fullsized":"2024-06-06T08:40:17.602424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"100'148","reported_seq":422,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:19.422865+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:40:19.422865+0000","last_peered":"2024-06-06T08:40:19.422865+0000","last_clean":"2024-06-06T08:40:19.422865+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:40:19.422865+0000","last_undegraded":"2024-06-06T08:40:19.422865+0000","last_fullsized":"2024-06-06T08:40:19.422865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288880+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:40:17.288880+0000","last_peered":"2024-06-06T08:40:17.288880+0000","last_clean":"2024-06-06T08:40:17.288880+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:40:17.288880+0000","last_undegraded":"2024-06-06T08:40:17.288880+0000","last_fullsized":"2024-06-06T08:40:17.288880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543213+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:40:17.543213+0000","last_peered":"2024-06-06T08:40:17.543213+0000","last_clean":"2024-06-06T08:40:17.543213+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:40:17.543213+0000","last_undegraded":"2024-06-06T08:40:17.543213+0000","last_fullsized":"2024-06-06T08:40:17.543213+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543287+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:40:17.543287+0000","last_peered":"2024-06-06T08:40:17.543287+0000","last_clean":"2024-06-06T08:40:17.543287+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:40:17.543287+0000","last_undegraded":"2024-06-06T08:40:17.543287+0000","last_fullsized":"2024-06-06T08:40:17.543287+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543496+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:40:17.543496+0000","last_peered":"2024-06-06T08:40:17.543496+0000","last_clean":"2024-06-06T08:40:17.543496+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:40:17.543496+0000","last_undegraded":"2024-06-06T08:40:17.543496+0000","last_fullsized":"2024-06-06T08:40:17.543496+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289533+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:40:17.289533+0000","last_peered":"2024-06-06T08:40:17.289533+0000","last_clean":"2024-06-06T08:40:17.289533+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:40:17.289533+0000","last_undegraded":"2024-06-06T08:40:17.289533+0000","last_fullsized":"2024-06-06T08:40:17.289533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397196+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:40:17.397196+0000","last_peered":"2024-06-06T08:40:17.397196+0000","last_clean":"2024-06-06T08:40:17.397196+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:40:17.397196+0000","last_undegraded":"2024-06-06T08:40:17.397196+0000","last_fullsized":"2024-06-06T08:40:17.397196+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'102","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:33.017271+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:40:33.017271+0000","last_peered":"2024-06-06T08:40:33.017271+0000","last_clean":"2024-06-06T08:40:33.017271+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:40:33.017271+0000","last_undegraded":"2024-06-06T08:40:33.017271+0000","last_fullsized":"2024-06-06T08:40:33.017271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":187,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397610+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:40:17.397610+0000","last_peered":"2024-06-06T08:40:17.397610+0000","last_clean":"2024-06-06T08:40:17.397610+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:40:17.397610+0000","last_undegraded":"2024-06-06T08:40:17.397610+0000","last_fullsized":"2024-06-06T08:40:17.397610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290819+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:40:17.290819+0000","last_peered":"2024-06-06T08:40:17.290819+0000","last_clean":"2024-06-06T08:40:17.290819+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:40:17.290819+0000","last_undegraded":"2024-06-06T08:40:17.290819+0000","last_fullsized":"2024-06-06T08:40:17.290819+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603116+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:40:17.603116+0000","last_peered":"2024-06-06T08:40:17.603116+0000","last_clean":"2024-06-06T08:40:17.603116+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:40:17.603116+0000","last_undegraded":"2024-06-06T08:40:17.603116+0000","last_fullsized":"2024-06-06T08:40:17.603116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603700+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:40:17.603700+0000","last_peered":"2024-06-06T08:40:17.603700+0000","last_clean":"2024-06-06T08:40:17.603700+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:40:17.603700+0000","last_undegraded":"2024-06-06T08:40:17.603700+0000","last_fullsized":"2024-06-06T08:40:17.603700+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602722+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:40:17.602722+0000","last_peered":"2024-06-06T08:40:17.602722+0000","last_clean":"2024-06-06T08:40:17.602722+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:40:17.602722+0000","last_undegraded":"2024-06-06T08:40:17.602722+0000","last_fullsized":"2024-06-06T08:40:17.602722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618429+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:40:17.618429+0000","last_peered":"2024-06-06T08:40:17.618429+0000","last_clean":"2024-06-06T08:40:17.618429+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:40:17.618429+0000","last_undegraded":"2024-06-06T08:40:17.618429+0000","last_fullsized":"2024-06-06T08:40:17.618429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396892+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:40:17.396892+0000","last_peered":"2024-06-06T08:40:17.396892+0000","last_clean":"2024-06-06T08:40:17.396892+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:40:17.396892+0000","last_undegraded":"2024-06-06T08:40:17.396892+0000","last_fullsized":"2024-06-06T08:40:17.396892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.395030+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:40:17.395030+0000","last_peered":"2024-06-06T08:40:17.395030+0000","last_clean":"2024-06-06T08:40:17.395030+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:40:17.395030+0000","last_undegraded":"2024-06-06T08:40:17.395030+0000","last_fullsized":"2024-06-06T08:40:17.395030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397189+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:40:17.397189+0000","last_peered":"2024-06-06T08:40:17.397189+0000","last_clean":"2024-06-06T08:40:17.397189+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:40:17.397189+0000","last_undegraded":"2024-06-06T08:40:17.397189+0000","last_fullsized":"2024-06-06T08:40:17.397189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543302+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:40:17.543302+0000","last_peered":"2024-06-06T08:40:17.543302+0000","last_clean":"2024-06-06T08:40:17.543302+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:40:17.543302+0000","last_undegraded":"2024-06-06T08:40:17.543302+0000","last_fullsized":"2024-06-06T08:40:17.543302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602805+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:40:17.602805+0000","last_peered":"2024-06-06T08:40:17.602805+0000","last_clean":"2024-06-06T08:40:17.602805+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:40:17.602805+0000","last_undegraded":"2024-06-06T08:40:17.602805+0000","last_fullsized":"2024-06-06T08:40:17.602805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.930424+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:40:17.930424+0000","last_peered":"2024-06-06T08:40:17.930424+0000","last_clean":"2024-06-06T08:40:17.930424+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:40:17.930424+0000","last_undegraded":"2024-06-06T08:40:17.930424+0000","last_fullsized":"2024-06-06T08:40:17.930424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396937+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:40:17.396937+0000","last_peered":"2024-06-06T08:40:17.396937+0000","last_clean":"2024-06-06T08:40:17.396937+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:40:17.396937+0000","last_undegraded":"2024-06-06T08:40:17.396937+0000","last_fullsized":"2024-06-06T08:40:17.396937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602730+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:40:17.602730+0000","last_peered":"2024-06-06T08:40:17.602730+0000","last_clean":"2024-06-06T08:40:17.602730+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:40:17.602730+0000","last_undegraded":"2024-06-06T08:40:17.602730+0000","last_fullsized":"2024-06-06T08:40:17.602730+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396987+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:40:17.396987+0000","last_peered":"2024-06-06T08:40:17.396987+0000","last_clean":"2024-06-06T08:40:17.396987+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:40:17.396987+0000","last_undegraded":"2024-06-06T08:40:17.396987+0000","last_fullsized":"2024-06-06T08:40:17.396987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290632+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:40:17.290632+0000","last_peered":"2024-06-06T08:40:17.290632+0000","last_clean":"2024-06-06T08:40:17.290632+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:40:17.290632+0000","last_undegraded":"2024-06-06T08:40:17.290632+0000","last_fullsized":"2024-06-06T08:40:17.290632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618976+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:40:17.618976+0000","last_peered":"2024-06-06T08:40:17.618976+0000","last_clean":"2024-06-06T08:40:17.618976+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:40:17.618976+0000","last_undegraded":"2024-06-06T08:40:17.618976+0000","last_fullsized":"2024-06-06T08:40:17.618976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291175+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:40:17.291175+0000","last_peered":"2024-06-06T08:40:17.291175+0000","last_clean":"2024-06-06T08:40:17.291175+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:40:17.291175+0000","last_undegraded":"2024-06-06T08:40:17.291175+0000","last_fullsized":"2024-06-06T08:40:17.291175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291116+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:40:17.291116+0000","last_peered":"2024-06-06T08:40:17.291116+0000","last_clean":"2024-06-06T08:40:17.291116+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:40:17.291116+0000","last_undegraded":"2024-06-06T08:40:17.291116+0000","last_fullsized":"2024-06-06T08:40:17.291116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603292+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:40:17.603292+0000","last_peered":"2024-06-06T08:40:17.603292+0000","last_clean":"2024-06-06T08:40:17.603292+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:40:17.603292+0000","last_undegraded":"2024-06-06T08:40:17.603292+0000","last_fullsized":"2024-06-06T08:40:17.603292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'65","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:32.670293+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:40:32.670293+0000","last_peered":"2024-06-06T08:40:32.670293+0000","last_clean":"2024-06-06T08:40:32.670293+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:40:32.670293+0000","last_undegraded":"2024-06-06T08:40:32.670293+0000","last_fullsized":"2024-06-06T08:40:32.670293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290659+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:40:17.290659+0000","last_peered":"2024-06-06T08:40:17.290659+0000","last_clean":"2024-06-06T08:40:17.290659+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:40:17.290659+0000","last_undegraded":"2024-06-06T08:40:17.290659+0000","last_fullsized":"2024-06-06T08:40:17.290659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403261+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:40:17.403261+0000","last_peered":"2024-06-06T08:40:17.403261+0000","last_clean":"2024-06-06T08:40:17.403261+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:40:17.403261+0000","last_undegraded":"2024-06-06T08:40:17.403261+0000","last_fullsized":"2024-06-06T08:40:17.403261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289754+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:40:17.289754+0000","last_peered":"2024-06-06T08:40:17.289754+0000","last_clean":"2024-06-06T08:40:17.289754+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:40:17.289754+0000","last_undegraded":"2024-06-06T08:40:17.289754+0000","last_fullsized":"2024-06-06T08:40:17.289754+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291007+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:40:17.291007+0000","last_peered":"2024-06-06T08:40:17.291007+0000","last_clean":"2024-06-06T08:40:17.291007+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:40:17.291007+0000","last_undegraded":"2024-06-06T08:40:17.291007+0000","last_fullsized":"2024-06-06T08:40:17.291007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543518+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:40:17.543518+0000","last_peered":"2024-06-06T08:40:17.543518+0000","last_clean":"2024-06-06T08:40:17.543518+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:40:17.543518+0000","last_undegraded":"2024-06-06T08:40:17.543518+0000","last_fullsized":"2024-06-06T08:40:17.543518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291058+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:40:17.291058+0000","last_peered":"2024-06-06T08:40:17.291058+0000","last_clean":"2024-06-06T08:40:17.291058+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:40:17.291058+0000","last_undegraded":"2024-06-06T08:40:17.291058+0000","last_fullsized":"2024-06-06T08:40:17.291058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290956+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:40:17.290956+0000","last_peered":"2024-06-06T08:40:17.290956+0000","last_clean":"2024-06-06T08:40:17.290956+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:40:17.290956+0000","last_undegraded":"2024-06-06T08:40:17.290956+0000","last_fullsized":"2024-06-06T08:40:17.290956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603418+0000","last_change":"2024-06-06T08:38:01.151003+0000","last_active":"2024-06-06T08:40:17.603418+0000","last_peered":"2024-06-06T08:40:17.603418+0000","last_clean":"2024-06-06T08:40:17.603418+0000","last_became_active":"2024-06-06T08:38:01.150557+0000","last_became_peered":"2024-06-06T08:38:01.150557+0000","last_unstale":"2024-06-06T08:40:17.603418+0000","last_undegraded":"2024-06-06T08:40:17.603418+0000","last_fullsized":"2024-06-06T08:40:17.603418+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:29:44.629894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289585+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:40:17.289585+0000","last_peered":"2024-06-06T08:40:17.289585+0000","last_clean":"2024-06-06T08:40:17.289585+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:40:17.289585+0000","last_undegraded":"2024-06-06T08:40:17.289585+0000","last_fullsized":"2024-06-06T08:40:17.289585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543524+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:40:17.543524+0000","last_peered":"2024-06-06T08:40:17.543524+0000","last_clean":"2024-06-06T08:40:17.543524+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:40:17.543524+0000","last_undegraded":"2024-06-06T08:40:17.543524+0000","last_fullsized":"2024-06-06T08:40:17.543524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291317+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:40:17.291317+0000","last_peered":"2024-06-06T08:40:17.291317+0000","last_clean":"2024-06-06T08:40:17.291317+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:40:17.291317+0000","last_undegraded":"2024-06-06T08:40:17.291317+0000","last_fullsized":"2024-06-06T08:40:17.291317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397744+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:40:17.397744+0000","last_peered":"2024-06-06T08:40:17.397744+0000","last_clean":"2024-06-06T08:40:17.397744+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:40:17.397744+0000","last_undegraded":"2024-06-06T08:40:17.397744+0000","last_fullsized":"2024-06-06T08:40:17.397744+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403480+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:40:17.403480+0000","last_peered":"2024-06-06T08:40:17.403480+0000","last_clean":"2024-06-06T08:40:17.403480+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:40:17.403480+0000","last_undegraded":"2024-06-06T08:40:17.403480+0000","last_fullsized":"2024-06-06T08:40:17.403480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397272+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:40:17.397272+0000","last_peered":"2024-06-06T08:40:17.397272+0000","last_clean":"2024-06-06T08:40:17.397272+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:40:17.397272+0000","last_undegraded":"2024-06-06T08:40:17.397272+0000","last_fullsized":"2024-06-06T08:40:17.397272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618845+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:40:17.618845+0000","last_peered":"2024-06-06T08:40:17.618845+0000","last_clean":"2024-06-06T08:40:17.618845+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:40:17.618845+0000","last_undegraded":"2024-06-06T08:40:17.618845+0000","last_fullsized":"2024-06-06T08:40:17.618845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289213+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:40:17.289213+0000","last_peered":"2024-06-06T08:40:17.289213+0000","last_clean":"2024-06-06T08:40:17.289213+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:40:17.289213+0000","last_undegraded":"2024-06-06T08:40:17.289213+0000","last_fullsized":"2024-06-06T08:40:17.289213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543444+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:40:17.543444+0000","last_peered":"2024-06-06T08:40:17.543444+0000","last_clean":"2024-06-06T08:40:17.543444+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:40:17.543444+0000","last_undegraded":"2024-06-06T08:40:17.543444+0000","last_fullsized":"2024-06-06T08:40:17.543444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107163,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798757,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1551,"num_read_kb":996,"num_write":433,"num_write_kb":2367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":2396271,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":319,"ondisk_log_size":319,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383747,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":579875,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645385,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36620,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708532,"statfs":{"total":95995035648,"available":95957536768,"internally_reserved":0,"allocated":1306624,"data_stored":1568354,"data_compressed":86280,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808910,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":977401,"data_compressed":54646,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070547,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":973203,"data_compressed":54646,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332184,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":1177536,"data_compressed":71078,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495710,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":575265,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":646865,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":576653,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577223,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182509,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1183744,"data_stored":1296907,"data_compressed":69848,"data_compressed_allocated":94208,"data_compressed_original":798720,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920872,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1196032,"data_stored":652916,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757346,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1224704,"data_stored":1500045,"data_compressed":86280,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330406,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":402811,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:40:41.394 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T08:40:41.395 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T08:40:41.648 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:42.021 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:41 smithi027 ceph-mon[32335]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:42.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:41 smithi176 ceph-mon[36786]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:42.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:41 smithi203 ceph-mon[36579]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:43.081 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:42 smithi027 ceph-mon[32335]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T08:40:43.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:42 smithi203 ceph-mon[36579]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T08:40:43.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:42 smithi176 ceph-mon[36786]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-06-06T08:40:43.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.314+0000 7fc0b9616700 1 -- 172.21.15.27:0/3205303067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b41073c0 msgr2=0x7fc0b41077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:43.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.314+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3205303067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b41073c0 0x7fc0b41077a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009990 tx=0x7fc0a4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.315+0000 7fc0b9616700 1 -- 172.21.15.27:0/3205303067 shutdown_connections 2024-06-06T08:40:43.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.315+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3205303067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b4101d40 0x7fc0b410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.315+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3205303067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b41073c0 0x7fc0b41077a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.315+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3205303067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b4101380 0x7fc0b4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.315+0000 7fc0b9616700 1 -- 172.21.15.27:0/3205303067 >> 172.21.15.27:0/3205303067 conn(0x7fc0b4078330 msgr2=0x7fc0b4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.315+0000 7fc0b9616700 1 -- 172.21.15.27:0/3205303067 shutdown_connections 2024-06-06T08:40:43.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.315+0000 7fc0b9616700 1 -- 172.21.15.27:0/3205303067 wait complete. 2024-06-06T08:40:43.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.315+0000 7fc0b9616700 1 Processor -- start 2024-06-06T08:40:43.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.315+0000 7fc0b9616700 1 -- start start 2024-06-06T08:40:43.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b9616700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b4101380 0x7fc0b419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:43.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b9616700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b4101d40 0x7fc0b419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b9616700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b41073c0 0x7fc0b4196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b9616700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b41105a0 con 0x7fc0b4101380 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b9616700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b4110420 con 0x7fc0b4101d40 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b9616700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b4110720 con 0x7fc0b41073c0 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b4101380 0x7fc0b419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b4101380 0x7fc0b419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54738/0 (socket says 172.21.15.27:54738) 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/3469478504 learned_addr learned my addr 172.21.15.27:0/3469478504 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/3469478504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b41073c0 msgr2=0x7fc0b4196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b2ffd700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b41073c0 0x7fc0b4196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/3469478504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b4101d40 msgr2=0x7fc0b419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b2ffd700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b4101d40 0x7fc0b419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b2ffd700 1 -- 172.21.15.27:0/3469478504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b4197080 con 0x7fc0b4101380 2024-06-06T08:40:43.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0b2ffd700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b4101380 0x7fc0b419c760 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40059d0 tx=0x7fc0a4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.316+0000 7fc0abfff700 1 -- 172.21.15.27:0/3469478504 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a401a040 con 0x7fc0b4101380 2024-06-06T08:40:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.317+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b4197310 con 0x7fc0b4101380 2024-06-06T08:40:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.317+0000 7fc0abfff700 1 -- 172.21.15.27:0/3469478504 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a4023070 con 0x7fc0b4101380 2024-06-06T08:40:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.317+0000 7fc0abfff700 1 -- 172.21.15.27:0/3469478504 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4004bb0 con 0x7fc0b4101380 2024-06-06T08:40:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.317+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b41a3630 con 0x7fc0b4101380 2024-06-06T08:40:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.318+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b4102a70 con 0x7fc0b4101380 2024-06-06T08:40:43.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.319+0000 7fc0abfff700 1 -- 172.21.15.27:0/3469478504 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fc0a4009d10 con 0x7fc0b4101380 2024-06-06T08:40:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.320+0000 7fc0abfff700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc0a00a2eb0 0x7fc0a00a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:43.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.320+0000 7fc0b27fc700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc0a00a2eb0 0x7fc0a00a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:43.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.320+0000 7fc0abfff700 1 -- 172.21.15.27:0/3469478504 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fc0a40d2950 con 0x7fc0b4101380 2024-06-06T08:40:43.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.321+0000 7fc0b27fc700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc0a00a2eb0 0x7fc0a00a5370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005950 tx=0x7fc09c0058e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.324+0000 7fc0abfff700 1 -- 172.21.15.27:0/3469478504 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0a409eff0 con 0x7fc0b4101380 2024-06-06T08:40:43.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.526+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0b40628d0 con 0x7fc0a00a2eb0 2024-06-06T08:40:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.535+0000 7fc0abfff700 1 -- 172.21.15.27:0/3469478504 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339847 (secure 0 0 0) 0x7fc0b40628d0 con 0x7fc0a00a2eb0 2024-06-06T08:40:43.538 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc0a00a2eb0 msgr2=0x7fc0a00a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc0a00a2eb0 0x7fc0a00a5370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005950 tx=0x7fc09c0058e0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b4101380 msgr2=0x7fc0b419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b4101380 0x7fc0b419c760 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40059d0 tx=0x7fc0a4004470 comp rx=0 tx=0).stop 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 shutdown_connections 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0b41073c0 0x7fc0b4196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b4101380 0x7fc0b419c760 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc0a00a2eb0 0x7fc0a00a5370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 --2- 172.21.15.27:0/3469478504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b4101d40 0x7fc0b419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 >> 172.21.15.27:0/3469478504 conn(0x7fc0b4078330 msgr2=0x7fc0b4100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 shutdown_connections 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:43.539+0000 7fc0b9616700 1 -- 172.21.15.27:0/3469478504 wait complete. 2024-06-06T08:40:43.541 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-06-06T08:40:44.345 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-06-06T08:40:43.310648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389359,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1813,"num_read_kb":1341,"num_write":636,"num_write_kb":5307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1182348,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":969,"ondisk_log_size":969,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362876,"kb_used_data":13688,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124578948,"statfs":{"total":1151940427776,"available":1151568842752,"internally_reserved":0,"allocated":14016512,"data_stored":11102243,"data_compressed":634920,"data_compressed_allocated":860160,"data_compressed_original":5050368,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":340,"num_read_kb":277,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002905"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543767+0000","last_change":"2024-06-06T08:36:20.901120+0000","last_active":"2024-06-06T08:40:17.543767+0000","last_peered":"2024-06-06T08:40:17.543767+0000","last_clean":"2024-06-06T08:40:17.543767+0000","last_became_active":"2024-06-06T08:36:20.900697+0000","last_became_peered":"2024-06-06T08:36:20.900697+0000","last_unstale":"2024-06-06T08:40:17.543767+0000","last_undegraded":"2024-06-06T08:40:17.543767+0000","last_fullsized":"2024-06-06T08:40:17.543767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:45:07.806094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603694+0000","last_change":"2024-06-06T08:36:20.902043+0000","last_active":"2024-06-06T08:40:17.603694+0000","last_peered":"2024-06-06T08:40:17.603694+0000","last_clean":"2024-06-06T08:40:17.603694+0000","last_became_active":"2024-06-06T08:36:20.901693+0000","last_became_peered":"2024-06-06T08:36:20.901693+0000","last_unstale":"2024-06-06T08:40:17.603694+0000","last_undegraded":"2024-06-06T08:40:17.603694+0000","last_fullsized":"2024-06-06T08:40:17.603694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:14.201216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603368+0000","last_change":"2024-06-06T08:36:20.907128+0000","last_active":"2024-06-06T08:40:17.603368+0000","last_peered":"2024-06-06T08:40:17.603368+0000","last_clean":"2024-06-06T08:40:17.603368+0000","last_became_active":"2024-06-06T08:36:20.906775+0000","last_became_peered":"2024-06-06T08:36:20.906775+0000","last_unstale":"2024-06-06T08:40:17.603368+0000","last_undegraded":"2024-06-06T08:40:17.603368+0000","last_fullsized":"2024-06-06T08:40:17.603368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:06:38.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602693+0000","last_change":"2024-06-06T08:36:20.906347+0000","last_active":"2024-06-06T08:40:17.602693+0000","last_peered":"2024-06-06T08:40:17.602693+0000","last_clean":"2024-06-06T08:40:17.602693+0000","last_became_active":"2024-06-06T08:36:20.906016+0000","last_became_peered":"2024-06-06T08:36:20.906016+0000","last_unstale":"2024-06-06T08:40:17.602693+0000","last_undegraded":"2024-06-06T08:40:17.602693+0000","last_fullsized":"2024-06-06T08:40:17.602693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:52:22.444939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397130+0000","last_change":"2024-06-06T08:36:20.907547+0000","last_active":"2024-06-06T08:40:17.397130+0000","last_peered":"2024-06-06T08:40:17.397130+0000","last_clean":"2024-06-06T08:40:17.397130+0000","last_became_active":"2024-06-06T08:36:20.907224+0000","last_became_peered":"2024-06-06T08:36:20.907224+0000","last_unstale":"2024-06-06T08:40:17.397130+0000","last_undegraded":"2024-06-06T08:40:17.397130+0000","last_fullsized":"2024-06-06T08:40:17.397130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:53:27.317002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406459+0000","last_change":"2024-06-06T08:36:20.903075+0000","last_active":"2024-06-06T08:40:17.406459+0000","last_peered":"2024-06-06T08:40:17.406459+0000","last_clean":"2024-06-06T08:40:17.406459+0000","last_became_active":"2024-06-06T08:36:20.902471+0000","last_became_peered":"2024-06-06T08:36:20.902471+0000","last_unstale":"2024-06-06T08:40:17.406459+0000","last_undegraded":"2024-06-06T08:40:17.406459+0000","last_fullsized":"2024-06-06T08:40:17.406459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:07.680881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290504+0000","last_change":"2024-06-06T08:36:20.889325+0000","last_active":"2024-06-06T08:40:17.290504+0000","last_peered":"2024-06-06T08:40:17.290504+0000","last_clean":"2024-06-06T08:40:17.290504+0000","last_became_active":"2024-06-06T08:36:20.888988+0000","last_became_peered":"2024-06-06T08:36:20.888988+0000","last_unstale":"2024-06-06T08:40:17.290504+0000","last_undegraded":"2024-06-06T08:40:17.290504+0000","last_fullsized":"2024-06-06T08:40:17.290504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:47.063881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394085+0000","last_change":"2024-06-06T08:38:01.335419+0000","last_active":"2024-06-06T08:40:17.394085+0000","last_peered":"2024-06-06T08:40:17.394085+0000","last_clean":"2024-06-06T08:40:17.394085+0000","last_became_active":"2024-06-06T08:38:01.335191+0000","last_became_peered":"2024-06-06T08:38:01.335191+0000","last_unstale":"2024-06-06T08:40:17.394085+0000","last_undegraded":"2024-06-06T08:40:17.394085+0000","last_fullsized":"2024-06-06T08:40:17.394085+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:11:13.253838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619376+0000","last_change":"2024-06-06T08:36:20.896047+0000","last_active":"2024-06-06T08:40:17.619376+0000","last_peered":"2024-06-06T08:40:17.619376+0000","last_clean":"2024-06-06T08:40:17.619376+0000","last_became_active":"2024-06-06T08:36:20.895457+0000","last_became_peered":"2024-06-06T08:36:20.895457+0000","last_unstale":"2024-06-06T08:40:17.619376+0000","last_undegraded":"2024-06-06T08:40:17.619376+0000","last_fullsized":"2024-06-06T08:40:17.619376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:56:20.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406777+0000","last_change":"2024-06-06T08:36:20.904805+0000","last_active":"2024-06-06T08:40:17.406777+0000","last_peered":"2024-06-06T08:40:17.406777+0000","last_clean":"2024-06-06T08:40:17.406777+0000","last_became_active":"2024-06-06T08:36:20.904471+0000","last_became_peered":"2024-06-06T08:36:20.904471+0000","last_unstale":"2024-06-06T08:40:17.406777+0000","last_undegraded":"2024-06-06T08:40:17.406777+0000","last_fullsized":"2024-06-06T08:40:17.406777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:16.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290994+0000","last_change":"2024-06-06T08:36:20.900482+0000","last_active":"2024-06-06T08:40:17.290994+0000","last_peered":"2024-06-06T08:40:17.290994+0000","last_clean":"2024-06-06T08:40:17.290994+0000","last_became_active":"2024-06-06T08:36:20.900169+0000","last_became_peered":"2024-06-06T08:36:20.900169+0000","last_unstale":"2024-06-06T08:40:17.290994+0000","last_undegraded":"2024-06-06T08:40:17.290994+0000","last_fullsized":"2024-06-06T08:40:17.290994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:13.384208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602644+0000","last_change":"2024-06-06T08:36:20.904463+0000","last_active":"2024-06-06T08:40:17.602644+0000","last_peered":"2024-06-06T08:40:17.602644+0000","last_clean":"2024-06-06T08:40:17.602644+0000","last_became_active":"2024-06-06T08:36:20.904134+0000","last_became_peered":"2024-06-06T08:36:20.904134+0000","last_unstale":"2024-06-06T08:40:17.602644+0000","last_undegraded":"2024-06-06T08:40:17.602644+0000","last_fullsized":"2024-06-06T08:40:17.602644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:08:28.121143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289360+0000","last_change":"2024-06-06T08:36:20.894042+0000","last_active":"2024-06-06T08:40:17.289360+0000","last_peered":"2024-06-06T08:40:17.289360+0000","last_clean":"2024-06-06T08:40:17.289360+0000","last_became_active":"2024-06-06T08:36:20.893685+0000","last_became_peered":"2024-06-06T08:36:20.893685+0000","last_unstale":"2024-06-06T08:40:17.289360+0000","last_undegraded":"2024-06-06T08:40:17.289360+0000","last_fullsized":"2024-06-06T08:40:17.289360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:52:06.261842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602964+0000","last_change":"2024-06-06T08:36:20.902320+0000","last_active":"2024-06-06T08:40:17.602964+0000","last_peered":"2024-06-06T08:40:17.602964+0000","last_clean":"2024-06-06T08:40:17.602964+0000","last_became_active":"2024-06-06T08:36:20.901957+0000","last_became_peered":"2024-06-06T08:36:20.901957+0000","last_unstale":"2024-06-06T08:40:17.602964+0000","last_undegraded":"2024-06-06T08:40:17.602964+0000","last_fullsized":"2024-06-06T08:40:17.602964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:31.213810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603050+0000","last_change":"2024-06-06T08:36:20.907755+0000","last_active":"2024-06-06T08:40:17.603050+0000","last_peered":"2024-06-06T08:40:17.603050+0000","last_clean":"2024-06-06T08:40:17.603050+0000","last_became_active":"2024-06-06T08:36:20.907358+0000","last_became_peered":"2024-06-06T08:36:20.907358+0000","last_unstale":"2024-06-06T08:40:17.603050+0000","last_undegraded":"2024-06-06T08:40:17.603050+0000","last_fullsized":"2024-06-06T08:40:17.603050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:28:50.510842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618538+0000","last_change":"2024-06-06T08:36:20.894378+0000","last_active":"2024-06-06T08:40:17.618538+0000","last_peered":"2024-06-06T08:40:17.618538+0000","last_clean":"2024-06-06T08:40:17.618538+0000","last_became_active":"2024-06-06T08:36:20.893443+0000","last_became_peered":"2024-06-06T08:36:20.893443+0000","last_unstale":"2024-06-06T08:40:17.618538+0000","last_undegraded":"2024-06-06T08:40:17.618538+0000","last_fullsized":"2024-06-06T08:40:17.618538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:44:03.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543492+0000","last_change":"2024-06-06T08:36:20.902870+0000","last_active":"2024-06-06T08:40:17.543492+0000","last_peered":"2024-06-06T08:40:17.543492+0000","last_clean":"2024-06-06T08:40:17.543492+0000","last_became_active":"2024-06-06T08:36:20.902529+0000","last_became_peered":"2024-06-06T08:36:20.902529+0000","last_unstale":"2024-06-06T08:40:17.543492+0000","last_undegraded":"2024-06-06T08:40:17.543492+0000","last_fullsized":"2024-06-06T08:40:17.543492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:44:26.746442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394947+0000","last_change":"2024-06-06T08:36:20.897274+0000","last_active":"2024-06-06T08:40:17.394947+0000","last_peered":"2024-06-06T08:40:17.394947+0000","last_clean":"2024-06-06T08:40:17.394947+0000","last_became_active":"2024-06-06T08:36:20.896799+0000","last_became_peered":"2024-06-06T08:36:20.896799+0000","last_unstale":"2024-06-06T08:40:17.394947+0000","last_undegraded":"2024-06-06T08:40:17.394947+0000","last_fullsized":"2024-06-06T08:40:17.394947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:41:07.203568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291235+0000","last_change":"2024-06-06T08:36:20.904152+0000","last_active":"2024-06-06T08:40:17.291235+0000","last_peered":"2024-06-06T08:40:17.291235+0000","last_clean":"2024-06-06T08:40:17.291235+0000","last_became_active":"2024-06-06T08:36:20.903800+0000","last_became_peered":"2024-06-06T08:36:20.903800+0000","last_unstale":"2024-06-06T08:40:17.291235+0000","last_undegraded":"2024-06-06T08:40:17.291235+0000","last_fullsized":"2024-06-06T08:40:17.291235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:34.651737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291190+0000","last_change":"2024-06-06T08:36:31.931667+0000","last_active":"2024-06-06T08:40:17.291190+0000","last_peered":"2024-06-06T08:40:17.291190+0000","last_clean":"2024-06-06T08:40:17.291190+0000","last_became_active":"2024-06-06T08:36:31.931226+0000","last_became_peered":"2024-06-06T08:36:31.931226+0000","last_unstale":"2024-06-06T08:40:17.291190+0000","last_undegraded":"2024-06-06T08:40:17.291190+0000","last_fullsized":"2024-06-06T08:40:17.291190+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:06:20.199033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'9","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:23.296771+0000","last_change":"2024-06-06T08:36:17.431307+0000","last_active":"2024-06-06T08:40:23.296771+0000","last_peered":"2024-06-06T08:40:23.296771+0000","last_clean":"2024-06-06T08:40:23.296771+0000","last_became_active":"2024-06-06T08:36:17.430969+0000","last_became_peered":"2024-06-06T08:36:17.430969+0000","last_unstale":"2024-06-06T08:40:23.296771+0000","last_undegraded":"2024-06-06T08:40:23.296771+0000","last_fullsized":"2024-06-06T08:40:23.296771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:44:03.367697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397547+0000","last_change":"2024-06-06T08:36:20.904856+0000","last_active":"2024-06-06T08:40:17.397547+0000","last_peered":"2024-06-06T08:40:17.397547+0000","last_clean":"2024-06-06T08:40:17.397547+0000","last_became_active":"2024-06-06T08:36:20.904528+0000","last_became_peered":"2024-06-06T08:36:20.904528+0000","last_unstale":"2024-06-06T08:40:17.397547+0000","last_undegraded":"2024-06-06T08:40:17.397547+0000","last_fullsized":"2024-06-06T08:40:17.397547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:01:30.615609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602515+0000","last_change":"2024-06-06T08:36:31.926648+0000","last_active":"2024-06-06T08:40:17.602515+0000","last_peered":"2024-06-06T08:40:17.602515+0000","last_clean":"2024-06-06T08:40:17.602515+0000","last_became_active":"2024-06-06T08:36:31.926249+0000","last_became_peered":"2024-06-06T08:36:31.926249+0000","last_unstale":"2024-06-06T08:40:17.602515+0000","last_undegraded":"2024-06-06T08:40:17.602515+0000","last_fullsized":"2024-06-06T08:40:17.602515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:33:28.129140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397886+0000","last_change":"2024-06-06T08:36:17.430458+0000","last_active":"2024-06-06T08:40:17.397886+0000","last_peered":"2024-06-06T08:40:17.397886+0000","last_clean":"2024-06-06T08:40:17.397886+0000","last_became_active":"2024-06-06T08:36:17.429956+0000","last_became_peered":"2024-06-06T08:36:17.429956+0000","last_unstale":"2024-06-06T08:40:17.397886+0000","last_undegraded":"2024-06-06T08:40:17.397886+0000","last_fullsized":"2024-06-06T08:40:17.397886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:38:15.364240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603696+0000","last_change":"2024-06-06T08:36:20.899311+0000","last_active":"2024-06-06T08:40:17.603696+0000","last_peered":"2024-06-06T08:40:17.603696+0000","last_clean":"2024-06-06T08:40:17.603696+0000","last_became_active":"2024-06-06T08:36:20.899004+0000","last_became_peered":"2024-06-06T08:36:20.899004+0000","last_unstale":"2024-06-06T08:40:17.603696+0000","last_undegraded":"2024-06-06T08:40:17.603696+0000","last_fullsized":"2024-06-06T08:40:17.603696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:39.174383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602634+0000","last_change":"2024-06-06T08:36:31.922771+0000","last_active":"2024-06-06T08:40:17.602634+0000","last_peered":"2024-06-06T08:40:17.602634+0000","last_clean":"2024-06-06T08:40:17.602634+0000","last_became_active":"2024-06-06T08:36:31.921845+0000","last_became_peered":"2024-06-06T08:36:31.921845+0000","last_unstale":"2024-06-06T08:40:17.602634+0000","last_undegraded":"2024-06-06T08:40:17.602634+0000","last_fullsized":"2024-06-06T08:40:17.602634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:17:31.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397399+0000","last_change":"2024-06-06T08:36:17.434750+0000","last_active":"2024-06-06T08:40:17.397399+0000","last_peered":"2024-06-06T08:40:17.397399+0000","last_clean":"2024-06-06T08:40:17.397399+0000","last_became_active":"2024-06-06T08:36:17.434267+0000","last_became_peered":"2024-06-06T08:36:17.434267+0000","last_unstale":"2024-06-06T08:40:17.397399+0000","last_undegraded":"2024-06-06T08:40:17.397399+0000","last_fullsized":"2024-06-06T08:40:17.397399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:51:33.218033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618791+0000","last_change":"2024-06-06T08:36:20.894167+0000","last_active":"2024-06-06T08:40:17.618791+0000","last_peered":"2024-06-06T08:40:17.618791+0000","last_clean":"2024-06-06T08:40:17.618791+0000","last_became_active":"2024-06-06T08:36:20.893786+0000","last_became_peered":"2024-06-06T08:36:20.893786+0000","last_unstale":"2024-06-06T08:40:17.618791+0000","last_undegraded":"2024-06-06T08:40:17.618791+0000","last_fullsized":"2024-06-06T08:40:17.618791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:30.234721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602522+0000","last_change":"2024-06-06T08:36:31.924592+0000","last_active":"2024-06-06T08:40:17.602522+0000","last_peered":"2024-06-06T08:40:17.602522+0000","last_clean":"2024-06-06T08:40:17.602522+0000","last_became_active":"2024-06-06T08:36:31.924068+0000","last_became_peered":"2024-06-06T08:36:31.924068+0000","last_unstale":"2024-06-06T08:40:17.602522+0000","last_undegraded":"2024-06-06T08:40:17.602522+0000","last_fullsized":"2024-06-06T08:40:17.602522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:29:14.560377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397350+0000","last_change":"2024-06-06T08:36:17.435189+0000","last_active":"2024-06-06T08:40:17.397350+0000","last_peered":"2024-06-06T08:40:17.397350+0000","last_clean":"2024-06-06T08:40:17.397350+0000","last_became_active":"2024-06-06T08:36:17.431621+0000","last_became_peered":"2024-06-06T08:36:17.431621+0000","last_unstale":"2024-06-06T08:40:17.397350+0000","last_undegraded":"2024-06-06T08:40:17.397350+0000","last_fullsized":"2024-06-06T08:40:17.397350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:53.164429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394983+0000","last_change":"2024-06-06T08:38:01.728398+0000","last_active":"2024-06-06T08:40:17.394983+0000","last_peered":"2024-06-06T08:40:17.394983+0000","last_clean":"2024-06-06T08:40:17.394983+0000","last_became_active":"2024-06-06T08:38:01.728004+0000","last_became_peered":"2024-06-06T08:38:01.728004+0000","last_unstale":"2024-06-06T08:40:17.394983+0000","last_undegraded":"2024-06-06T08:40:17.394983+0000","last_fullsized":"2024-06-06T08:40:17.394983+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:02:03.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291009+0000","last_change":"2024-06-06T08:36:31.927205+0000","last_active":"2024-06-06T08:40:17.291009+0000","last_peered":"2024-06-06T08:40:17.291009+0000","last_clean":"2024-06-06T08:40:17.291009+0000","last_became_active":"2024-06-06T08:36:31.926386+0000","last_became_peered":"2024-06-06T08:36:31.926386+0000","last_unstale":"2024-06-06T08:40:17.291009+0000","last_undegraded":"2024-06-06T08:40:17.291009+0000","last_fullsized":"2024-06-06T08:40:17.291009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:35:36.287767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":257,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:42.122550+0000","last_change":"2024-06-06T08:36:17.435804+0000","last_active":"2024-06-06T08:40:42.122550+0000","last_peered":"2024-06-06T08:40:42.122550+0000","last_clean":"2024-06-06T08:40:42.122550+0000","last_became_active":"2024-06-06T08:36:17.435243+0000","last_became_peered":"2024-06-06T08:36:17.435243+0000","last_unstale":"2024-06-06T08:40:42.122550+0000","last_undegraded":"2024-06-06T08:40:42.122550+0000","last_fullsized":"2024-06-06T08:40:42.122550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:23.240699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1320,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":203,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396976+0000","last_change":"2024-06-06T08:36:20.902444+0000","last_active":"2024-06-06T08:40:17.396976+0000","last_peered":"2024-06-06T08:40:17.396976+0000","last_clean":"2024-06-06T08:40:17.396976+0000","last_became_active":"2024-06-06T08:36:20.902122+0000","last_became_peered":"2024-06-06T08:36:20.902122+0000","last_unstale":"2024-06-06T08:40:17.396976+0000","last_undegraded":"2024-06-06T08:40:17.396976+0000","last_fullsized":"2024-06-06T08:40:17.396976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:53.780497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290894+0000","last_change":"2024-06-06T08:36:31.926106+0000","last_active":"2024-06-06T08:40:17.290894+0000","last_peered":"2024-06-06T08:40:17.290894+0000","last_clean":"2024-06-06T08:40:17.290894+0000","last_became_active":"2024-06-06T08:36:31.925560+0000","last_became_peered":"2024-06-06T08:36:31.925560+0000","last_unstale":"2024-06-06T08:40:17.290894+0000","last_undegraded":"2024-06-06T08:40:17.290894+0000","last_fullsized":"2024-06-06T08:40:17.290894+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:48:36.983779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406405+0000","last_change":"2024-06-06T08:36:17.431973+0000","last_active":"2024-06-06T08:40:17.406405+0000","last_peered":"2024-06-06T08:40:17.406405+0000","last_clean":"2024-06-06T08:40:17.406405+0000","last_became_active":"2024-06-06T08:36:17.431555+0000","last_became_peered":"2024-06-06T08:36:17.431555+0000","last_unstale":"2024-06-06T08:40:17.406405+0000","last_undegraded":"2024-06-06T08:40:17.406405+0000","last_fullsized":"2024-06-06T08:40:17.406405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:37:57.033080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394510+0000","last_change":"2024-06-06T08:38:01.728238+0000","last_active":"2024-06-06T08:40:17.394510+0000","last_peered":"2024-06-06T08:40:17.394510+0000","last_clean":"2024-06-06T08:40:17.394510+0000","last_became_active":"2024-06-06T08:38:01.727844+0000","last_became_peered":"2024-06-06T08:38:01.727844+0000","last_unstale":"2024-06-06T08:40:17.394510+0000","last_undegraded":"2024-06-06T08:40:17.394510+0000","last_fullsized":"2024-06-06T08:40:17.394510+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:19:46.825321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543559+0000","last_change":"2024-06-06T08:36:31.924654+0000","last_active":"2024-06-06T08:40:17.543559+0000","last_peered":"2024-06-06T08:40:17.543559+0000","last_clean":"2024-06-06T08:40:17.543559+0000","last_became_active":"2024-06-06T08:36:31.924265+0000","last_became_peered":"2024-06-06T08:36:31.924265+0000","last_unstale":"2024-06-06T08:40:17.543559+0000","last_undegraded":"2024-06-06T08:40:17.543559+0000","last_fullsized":"2024-06-06T08:40:17.543559+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:40.628390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288975+0000","last_change":"2024-06-06T08:36:17.428695+0000","last_active":"2024-06-06T08:40:17.288975+0000","last_peered":"2024-06-06T08:40:17.288975+0000","last_clean":"2024-06-06T08:40:17.288975+0000","last_became_active":"2024-06-06T08:36:17.428356+0000","last_became_peered":"2024-06-06T08:36:17.428356+0000","last_unstale":"2024-06-06T08:40:17.288975+0000","last_undegraded":"2024-06-06T08:40:17.288975+0000","last_fullsized":"2024-06-06T08:40:17.288975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:56:12.442583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396897+0000","last_change":"2024-06-06T08:36:20.894634+0000","last_active":"2024-06-06T08:40:17.396897+0000","last_peered":"2024-06-06T08:40:17.396897+0000","last_clean":"2024-06-06T08:40:17.396897+0000","last_became_active":"2024-06-06T08:36:20.894205+0000","last_became_peered":"2024-06-06T08:36:20.894205+0000","last_unstale":"2024-06-06T08:40:17.396897+0000","last_undegraded":"2024-06-06T08:40:17.396897+0000","last_fullsized":"2024-06-06T08:40:17.396897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:24:47.379850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603635+0000","last_change":"2024-06-06T08:36:31.923178+0000","last_active":"2024-06-06T08:40:17.603635+0000","last_peered":"2024-06-06T08:40:17.603635+0000","last_clean":"2024-06-06T08:40:17.603635+0000","last_became_active":"2024-06-06T08:36:31.922761+0000","last_became_peered":"2024-06-06T08:36:31.922761+0000","last_unstale":"2024-06-06T08:40:17.603635+0000","last_undegraded":"2024-06-06T08:40:17.603635+0000","last_fullsized":"2024-06-06T08:40:17.603635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:09:09.101049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'1","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403412+0000","last_change":"2024-06-06T08:36:17.434919+0000","last_active":"2024-06-06T08:40:17.403412+0000","last_peered":"2024-06-06T08:40:17.403412+0000","last_clean":"2024-06-06T08:40:17.403412+0000","last_became_active":"2024-06-06T08:36:17.434122+0000","last_became_peered":"2024-06-06T08:36:17.434122+0000","last_unstale":"2024-06-06T08:40:17.403412+0000","last_undegraded":"2024-06-06T08:40:17.403412+0000","last_fullsized":"2024-06-06T08:40:17.403412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:12:19.766768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394236+0000","last_change":"2024-06-06T08:36:20.895961+0000","last_active":"2024-06-06T08:40:17.394236+0000","last_peered":"2024-06-06T08:40:17.394236+0000","last_clean":"2024-06-06T08:40:17.394236+0000","last_became_active":"2024-06-06T08:36:20.895621+0000","last_became_peered":"2024-06-06T08:36:20.895621+0000","last_unstale":"2024-06-06T08:40:17.394236+0000","last_undegraded":"2024-06-06T08:40:17.394236+0000","last_fullsized":"2024-06-06T08:40:17.394236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:39:14.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:37:00.683085+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:37:00.682669+0000","last_became_peered":"2024-06-06T08:37:00.682669+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:54.403602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,8],"acting":[3,7,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'4","reported_seq":250,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:42.122074+0000","last_change":"2024-06-06T08:36:17.433286+0000","last_active":"2024-06-06T08:40:42.122074+0000","last_peered":"2024-06-06T08:40:42.122074+0000","last_clean":"2024-06-06T08:40:42.122074+0000","last_became_active":"2024-06-06T08:36:17.432866+0000","last_became_peered":"2024-06-06T08:36:17.432866+0000","last_unstale":"2024-06-06T08:40:42.122074+0000","last_undegraded":"2024-06-06T08:40:42.122074+0000","last_fullsized":"2024-06-06T08:40:42.122074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:43:02.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":199,"num_read_kb":199,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.619308+0000","last_change":"2024-06-06T08:36:20.895831+0000","last_active":"2024-06-06T08:40:17.619308+0000","last_peered":"2024-06-06T08:40:17.619308+0000","last_clean":"2024-06-06T08:40:17.619308+0000","last_became_active":"2024-06-06T08:36:20.893887+0000","last_became_peered":"2024-06-06T08:36:20.893887+0000","last_unstale":"2024-06-06T08:40:17.619308+0000","last_undegraded":"2024-06-06T08:40:17.619308+0000","last_fullsized":"2024-06-06T08:40:17.619308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:42.151928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394167+0000","last_change":"2024-06-06T08:36:31.929409+0000","last_active":"2024-06-06T08:40:17.394167+0000","last_peered":"2024-06-06T08:40:17.394167+0000","last_clean":"2024-06-06T08:40:17.394167+0000","last_became_active":"2024-06-06T08:36:31.928987+0000","last_became_peered":"2024-06-06T08:36:31.928987+0000","last_unstale":"2024-06-06T08:40:17.394167+0000","last_undegraded":"2024-06-06T08:40:17.394167+0000","last_fullsized":"2024-06-06T08:40:17.394167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:26:15.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290830+0000","last_change":"2024-06-06T08:36:17.435214+0000","last_active":"2024-06-06T08:40:17.290830+0000","last_peered":"2024-06-06T08:40:17.290830+0000","last_clean":"2024-06-06T08:40:17.290830+0000","last_became_active":"2024-06-06T08:36:17.434506+0000","last_became_peered":"2024-06-06T08:36:17.434506+0000","last_unstale":"2024-06-06T08:40:17.290830+0000","last_undegraded":"2024-06-06T08:40:17.290830+0000","last_fullsized":"2024-06-06T08:40:17.290830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:15:18.278673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603715+0000","last_change":"2024-06-06T08:36:20.902741+0000","last_active":"2024-06-06T08:40:17.603715+0000","last_peered":"2024-06-06T08:40:17.603715+0000","last_clean":"2024-06-06T08:40:17.603715+0000","last_became_active":"2024-06-06T08:36:20.902438+0000","last_became_peered":"2024-06-06T08:36:20.902438+0000","last_unstale":"2024-06-06T08:40:17.603715+0000","last_undegraded":"2024-06-06T08:40:17.603715+0000","last_fullsized":"2024-06-06T08:40:17.603715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:28.550048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602684+0000","last_change":"2024-06-06T08:36:31.922216+0000","last_active":"2024-06-06T08:40:17.602684+0000","last_peered":"2024-06-06T08:40:17.602684+0000","last_clean":"2024-06-06T08:40:17.602684+0000","last_became_active":"2024-06-06T08:36:31.920844+0000","last_became_peered":"2024-06-06T08:36:31.920844+0000","last_unstale":"2024-06-06T08:40:17.602684+0000","last_undegraded":"2024-06-06T08:40:17.602684+0000","last_fullsized":"2024-06-06T08:40:17.602684+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:50:04.872876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403562+0000","last_change":"2024-06-06T08:36:17.432992+0000","last_active":"2024-06-06T08:40:17.403562+0000","last_peered":"2024-06-06T08:40:17.403562+0000","last_clean":"2024-06-06T08:40:17.403562+0000","last_became_active":"2024-06-06T08:36:17.432044+0000","last_became_peered":"2024-06-06T08:36:17.432044+0000","last_unstale":"2024-06-06T08:40:17.403562+0000","last_undegraded":"2024-06-06T08:40:17.403562+0000","last_fullsized":"2024-06-06T08:40:17.403562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:55:07.376479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618947+0000","last_change":"2024-06-06T08:36:20.899001+0000","last_active":"2024-06-06T08:40:17.618947+0000","last_peered":"2024-06-06T08:40:17.618947+0000","last_clean":"2024-06-06T08:40:17.618947+0000","last_became_active":"2024-06-06T08:36:20.898618+0000","last_became_peered":"2024-06-06T08:36:20.898618+0000","last_unstale":"2024-06-06T08:40:17.618947+0000","last_undegraded":"2024-06-06T08:40:17.618947+0000","last_fullsized":"2024-06-06T08:40:17.618947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:41:10.527222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406287+0000","last_change":"2024-06-06T08:37:00.332002+0000","last_active":"2024-06-06T08:40:17.406287+0000","last_peered":"2024-06-06T08:40:17.406287+0000","last_clean":"2024-06-06T08:40:17.406287+0000","last_became_active":"2024-06-06T08:37:00.331672+0000","last_became_peered":"2024-06-06T08:37:00.331672+0000","last_unstale":"2024-06-06T08:40:17.406287+0000","last_undegraded":"2024-06-06T08:40:17.406287+0000","last_fullsized":"2024-06-06T08:40:17.406287+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:57:57.916476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"100'43","reported_seq":521,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:43.123504+0000","last_change":"2024-06-06T08:37:00.064165+0000","last_active":"2024-06-06T08:40:43.123504+0000","last_peered":"2024-06-06T08:40:43.123504+0000","last_clean":"2024-06-06T08:40:43.123504+0000","last_became_active":"2024-06-06T08:37:00.054887+0000","last_became_peered":"2024-06-06T08:37:00.054887+0000","last_unstale":"2024-06-06T08:40:43.123504+0000","last_undegraded":"2024-06-06T08:40:43.123504+0000","last_fullsized":"2024-06-06T08:40:43.123504+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:31:46.448417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71690,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":415,"num_read_kb":279,"num_write":44,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397172+0000","last_change":"2024-06-06T08:36:20.898705+0000","last_active":"2024-06-06T08:40:17.397172+0000","last_peered":"2024-06-06T08:40:17.397172+0000","last_clean":"2024-06-06T08:40:17.397172+0000","last_became_active":"2024-06-06T08:36:20.898336+0000","last_became_peered":"2024-06-06T08:36:20.898336+0000","last_unstale":"2024-06-06T08:40:17.397172+0000","last_undegraded":"2024-06-06T08:40:17.397172+0000","last_fullsized":"2024-06-06T08:40:17.397172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:50:29.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:31.928064+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:31.927407+0000","last_became_peered":"2024-06-06T08:36:31.927407+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:20:18.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291329+0000","last_change":"2024-06-06T08:36:17.429432+0000","last_active":"2024-06-06T08:40:17.291329+0000","last_peered":"2024-06-06T08:40:17.291329+0000","last_clean":"2024-06-06T08:40:17.291329+0000","last_became_active":"2024-06-06T08:36:17.429076+0000","last_became_peered":"2024-06-06T08:36:17.429076+0000","last_unstale":"2024-06-06T08:40:17.291329+0000","last_undegraded":"2024-06-06T08:40:17.291329+0000","last_fullsized":"2024-06-06T08:40:17.291329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:03:27.928167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289082+0000","last_change":"2024-06-06T08:36:20.898656+0000","last_active":"2024-06-06T08:40:17.289082+0000","last_peered":"2024-06-06T08:40:17.289082+0000","last_clean":"2024-06-06T08:40:17.289082+0000","last_became_active":"2024-06-06T08:36:20.898296+0000","last_became_peered":"2024-06-06T08:36:20.898296+0000","last_unstale":"2024-06-06T08:40:17.289082+0000","last_undegraded":"2024-06-06T08:40:17.289082+0000","last_fullsized":"2024-06-06T08:40:17.289082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:02.773343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543337+0000","last_change":"2024-06-06T08:37:00.683154+0000","last_active":"2024-06-06T08:40:17.543337+0000","last_peered":"2024-06-06T08:40:17.543337+0000","last_clean":"2024-06-06T08:40:17.543337+0000","last_became_active":"2024-06-06T08:37:00.682671+0000","last_became_peered":"2024-06-06T08:37:00.682671+0000","last_unstale":"2024-06-06T08:40:17.543337+0000","last_undegraded":"2024-06-06T08:40:17.543337+0000","last_fullsized":"2024-06-06T08:40:17.543337+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:15:32.982482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403613+0000","last_change":"2024-06-06T08:36:17.432098+0000","last_active":"2024-06-06T08:40:17.403613+0000","last_peered":"2024-06-06T08:40:17.403613+0000","last_clean":"2024-06-06T08:40:17.403613+0000","last_became_active":"2024-06-06T08:36:17.431656+0000","last_became_peered":"2024-06-06T08:36:17.431656+0000","last_unstale":"2024-06-06T08:40:17.403613+0000","last_undegraded":"2024-06-06T08:40:17.403613+0000","last_fullsized":"2024-06-06T08:40:17.403613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:30:29.820916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290581+0000","last_change":"2024-06-06T08:36:20.893683+0000","last_active":"2024-06-06T08:40:17.290581+0000","last_peered":"2024-06-06T08:40:17.290581+0000","last_clean":"2024-06-06T08:40:17.290581+0000","last_became_active":"2024-06-06T08:36:20.893347+0000","last_became_peered":"2024-06-06T08:36:20.893347+0000","last_unstale":"2024-06-06T08:40:17.290581+0000","last_undegraded":"2024-06-06T08:40:17.290581+0000","last_fullsized":"2024-06-06T08:40:17.290581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:08.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394932+0000","last_change":"2024-06-06T08:36:31.928420+0000","last_active":"2024-06-06T08:40:17.394932+0000","last_peered":"2024-06-06T08:40:17.394932+0000","last_clean":"2024-06-06T08:40:17.394932+0000","last_became_active":"2024-06-06T08:36:31.923494+0000","last_became_peered":"2024-06-06T08:36:31.923494+0000","last_unstale":"2024-06-06T08:40:17.394932+0000","last_undegraded":"2024-06-06T08:40:17.394932+0000","last_fullsized":"2024-06-06T08:40:17.394932+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:59:49.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":254,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:37.115001+0000","last_change":"2024-06-06T08:36:17.430307+0000","last_active":"2024-06-06T08:40:37.115001+0000","last_peered":"2024-06-06T08:40:37.115001+0000","last_clean":"2024-06-06T08:40:37.115001+0000","last_became_active":"2024-06-06T08:36:17.429945+0000","last_became_peered":"2024-06-06T08:36:17.429945+0000","last_unstale":"2024-06-06T08:40:37.115001+0000","last_undegraded":"2024-06-06T08:40:37.115001+0000","last_fullsized":"2024-06-06T08:40:37.115001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:13.331579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":201,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290588+0000","last_change":"2024-06-06T08:36:20.891241+0000","last_active":"2024-06-06T08:40:17.290588+0000","last_peered":"2024-06-06T08:40:17.290588+0000","last_clean":"2024-06-06T08:40:17.290588+0000","last_became_active":"2024-06-06T08:36:20.890346+0000","last_became_peered":"2024-06-06T08:36:20.890346+0000","last_unstale":"2024-06-06T08:40:17.290588+0000","last_undegraded":"2024-06-06T08:40:17.290588+0000","last_fullsized":"2024-06-06T08:40:17.290588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:49.346357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289726+0000","last_change":"2024-06-06T08:36:31.921538+0000","last_active":"2024-06-06T08:40:17.289726+0000","last_peered":"2024-06-06T08:40:17.289726+0000","last_clean":"2024-06-06T08:40:17.289726+0000","last_became_active":"2024-06-06T08:36:31.921125+0000","last_became_peered":"2024-06-06T08:36:31.921125+0000","last_unstale":"2024-06-06T08:40:17.289726+0000","last_undegraded":"2024-06-06T08:40:17.289726+0000","last_fullsized":"2024-06-06T08:40:17.289726+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:59.382051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'15","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396877+0000","last_change":"2024-06-06T08:36:17.430744+0000","last_active":"2024-06-06T08:40:17.396877+0000","last_peered":"2024-06-06T08:40:17.396877+0000","last_clean":"2024-06-06T08:40:17.396877+0000","last_became_active":"2024-06-06T08:36:17.429925+0000","last_became_peered":"2024-06-06T08:36:17.429925+0000","last_unstale":"2024-06-06T08:40:17.396877+0000","last_undegraded":"2024-06-06T08:40:17.396877+0000","last_fullsized":"2024-06-06T08:40:17.396877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:18.802442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403466+0000","last_change":"2024-06-06T08:36:20.898549+0000","last_active":"2024-06-06T08:40:17.403466+0000","last_peered":"2024-06-06T08:40:17.403466+0000","last_clean":"2024-06-06T08:40:17.403466+0000","last_became_active":"2024-06-06T08:36:20.898198+0000","last_became_peered":"2024-06-06T08:36:20.898198+0000","last_unstale":"2024-06-06T08:40:17.403466+0000","last_undegraded":"2024-06-06T08:40:17.403466+0000","last_fullsized":"2024-06-06T08:40:17.403466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:54.097622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403457+0000","last_change":"2024-06-06T08:36:31.929389+0000","last_active":"2024-06-06T08:40:17.403457+0000","last_peered":"2024-06-06T08:40:17.403457+0000","last_clean":"2024-06-06T08:40:17.403457+0000","last_became_active":"2024-06-06T08:36:31.928762+0000","last_became_peered":"2024-06-06T08:36:31.928762+0000","last_unstale":"2024-06-06T08:40:17.403457+0000","last_undegraded":"2024-06-06T08:40:17.403457+0000","last_fullsized":"2024-06-06T08:40:17.403457+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:35:39.125494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"100'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.933761+0000","last_change":"2024-06-06T08:36:17.432379+0000","last_active":"2024-06-06T08:40:17.933761+0000","last_peered":"2024-06-06T08:40:17.933761+0000","last_clean":"2024-06-06T08:40:17.933761+0000","last_became_active":"2024-06-06T08:36:17.430400+0000","last_became_peered":"2024-06-06T08:36:17.430400+0000","last_unstale":"2024-06-06T08:40:17.933761+0000","last_undegraded":"2024-06-06T08:40:17.933761+0000","last_fullsized":"2024-06-06T08:40:17.933761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:06.813438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543570+0000","last_change":"2024-06-06T08:36:20.907802+0000","last_active":"2024-06-06T08:40:17.543570+0000","last_peered":"2024-06-06T08:40:17.543570+0000","last_clean":"2024-06-06T08:40:17.543570+0000","last_became_active":"2024-06-06T08:36:20.907329+0000","last_became_peered":"2024-06-06T08:36:20.907329+0000","last_unstale":"2024-06-06T08:40:17.543570+0000","last_undegraded":"2024-06-06T08:40:17.543570+0000","last_fullsized":"2024-06-06T08:40:17.543570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:44:41.996038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396906+0000","last_change":"2024-06-06T08:36:32.047787+0000","last_active":"2024-06-06T08:40:17.396906+0000","last_peered":"2024-06-06T08:40:17.396906+0000","last_clean":"2024-06-06T08:40:17.396906+0000","last_became_active":"2024-06-06T08:36:32.047269+0000","last_became_peered":"2024-06-06T08:36:32.047269+0000","last_unstale":"2024-06-06T08:40:17.396906+0000","last_undegraded":"2024-06-06T08:40:17.396906+0000","last_fullsized":"2024-06-06T08:40:17.396906+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:38:10.424968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.394760+0000","last_change":"2024-06-06T08:36:17.431247+0000","last_active":"2024-06-06T08:40:17.394760+0000","last_peered":"2024-06-06T08:40:17.394760+0000","last_clean":"2024-06-06T08:40:17.394760+0000","last_became_active":"2024-06-06T08:36:17.430849+0000","last_became_peered":"2024-06-06T08:36:17.430849+0000","last_unstale":"2024-06-06T08:40:17.394760+0000","last_undegraded":"2024-06-06T08:40:17.394760+0000","last_fullsized":"2024-06-06T08:40:17.394760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:39:45.767753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406591+0000","last_change":"2024-06-06T08:36:20.894925+0000","last_active":"2024-06-06T08:40:17.406591+0000","last_peered":"2024-06-06T08:40:17.406591+0000","last_clean":"2024-06-06T08:40:17.406591+0000","last_became_active":"2024-06-06T08:36:20.893850+0000","last_became_peered":"2024-06-06T08:36:20.893850+0000","last_unstale":"2024-06-06T08:40:17.406591+0000","last_undegraded":"2024-06-06T08:40:17.406591+0000","last_fullsized":"2024-06-06T08:40:17.406591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:05.824596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289341+0000","last_change":"2024-06-06T08:36:31.927292+0000","last_active":"2024-06-06T08:40:17.289341+0000","last_peered":"2024-06-06T08:40:17.289341+0000","last_clean":"2024-06-06T08:40:17.289341+0000","last_became_active":"2024-06-06T08:36:31.926852+0000","last_became_peered":"2024-06-06T08:36:31.926852+0000","last_unstale":"2024-06-06T08:40:17.289341+0000","last_undegraded":"2024-06-06T08:40:17.289341+0000","last_fullsized":"2024-06-06T08:40:17.289341+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:19:05.201813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290643+0000","last_change":"2024-06-06T08:36:17.431632+0000","last_active":"2024-06-06T08:40:17.290643+0000","last_peered":"2024-06-06T08:40:17.290643+0000","last_clean":"2024-06-06T08:40:17.290643+0000","last_became_active":"2024-06-06T08:36:17.431302+0000","last_became_peered":"2024-06-06T08:36:17.431302+0000","last_unstale":"2024-06-06T08:40:17.290643+0000","last_undegraded":"2024-06-06T08:40:17.290643+0000","last_fullsized":"2024-06-06T08:40:17.290643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:36.999393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.406476+0000","last_change":"2024-06-06T08:36:20.903085+0000","last_active":"2024-06-06T08:40:17.406476+0000","last_peered":"2024-06-06T08:40:17.406476+0000","last_clean":"2024-06-06T08:40:17.406476+0000","last_became_active":"2024-06-06T08:36:20.902677+0000","last_became_peered":"2024-06-06T08:36:20.902677+0000","last_unstale":"2024-06-06T08:40:17.406476+0000","last_undegraded":"2024-06-06T08:40:17.406476+0000","last_fullsized":"2024-06-06T08:40:17.406476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:38:51.778742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289488+0000","last_change":"2024-06-06T08:36:32.047469+0000","last_active":"2024-06-06T08:40:17.289488+0000","last_peered":"2024-06-06T08:40:17.289488+0000","last_clean":"2024-06-06T08:40:17.289488+0000","last_became_active":"2024-06-06T08:36:32.046966+0000","last_became_peered":"2024-06-06T08:36:32.046966+0000","last_unstale":"2024-06-06T08:40:17.289488+0000","last_undegraded":"2024-06-06T08:40:17.289488+0000","last_fullsized":"2024-06-06T08:40:17.289488+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:33.163203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403823+0000","last_change":"2024-06-06T08:38:01.545781+0000","last_active":"2024-06-06T08:40:17.403823+0000","last_peered":"2024-06-06T08:40:17.403823+0000","last_clean":"2024-06-06T08:40:17.403823+0000","last_became_active":"2024-06-06T08:38:01.545462+0000","last_became_peered":"2024-06-06T08:38:01.545462+0000","last_unstale":"2024-06-06T08:40:17.403823+0000","last_undegraded":"2024-06-06T08:40:17.403823+0000","last_fullsized":"2024-06-06T08:40:17.403823+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:14:36.722960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":253,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:42.121957+0000","last_change":"2024-06-06T08:36:17.430984+0000","last_active":"2024-06-06T08:40:42.121957+0000","last_peered":"2024-06-06T08:40:42.121957+0000","last_clean":"2024-06-06T08:40:42.121957+0000","last_became_active":"2024-06-06T08:36:17.430666+0000","last_became_peered":"2024-06-06T08:36:17.430666+0000","last_unstale":"2024-06-06T08:40:42.121957+0000","last_undegraded":"2024-06-06T08:40:42.121957+0000","last_fullsized":"2024-06-06T08:40:42.121957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:59:33.772806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":199,"num_read_kb":199,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290959+0000","last_change":"2024-06-06T08:36:31.925812+0000","last_active":"2024-06-06T08:40:17.290959+0000","last_peered":"2024-06-06T08:40:17.290959+0000","last_clean":"2024-06-06T08:40:17.290959+0000","last_became_active":"2024-06-06T08:36:31.925329+0000","last_became_peered":"2024-06-06T08:36:31.925329+0000","last_unstale":"2024-06-06T08:40:17.290959+0000","last_undegraded":"2024-06-06T08:40:17.290959+0000","last_fullsized":"2024-06-06T08:40:17.290959+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:36.510652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397178+0000","last_change":"2024-06-06T08:36:20.886101+0000","last_active":"2024-06-06T08:40:17.397178+0000","last_peered":"2024-06-06T08:40:17.397178+0000","last_clean":"2024-06-06T08:40:17.397178+0000","last_became_active":"2024-06-06T08:36:20.885749+0000","last_became_peered":"2024-06-06T08:36:20.885749+0000","last_unstale":"2024-06-06T08:40:17.397178+0000","last_undegraded":"2024-06-06T08:40:17.397178+0000","last_fullsized":"2024-06-06T08:40:17.397178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:33:55.018017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'3","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602424+0000","last_change":"2024-06-06T08:36:17.428425+0000","last_active":"2024-06-06T08:40:17.602424+0000","last_peered":"2024-06-06T08:40:17.602424+0000","last_clean":"2024-06-06T08:40:17.602424+0000","last_became_active":"2024-06-06T08:36:17.428001+0000","last_became_peered":"2024-06-06T08:36:17.428001+0000","last_unstale":"2024-06-06T08:40:17.602424+0000","last_undegraded":"2024-06-06T08:40:17.602424+0000","last_fullsized":"2024-06-06T08:40:17.602424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:54:57.230011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"100'148","reported_seq":422,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:19.422865+0000","last_change":"2024-06-06T08:34:51.464367+0000","last_active":"2024-06-06T08:40:19.422865+0000","last_peered":"2024-06-06T08:40:19.422865+0000","last_clean":"2024-06-06T08:40:19.422865+0000","last_became_active":"2024-06-06T08:34:51.463871+0000","last_became_peered":"2024-06-06T08:34:51.463871+0000","last_unstale":"2024-06-06T08:40:19.422865+0000","last_undegraded":"2024-06-06T08:40:19.422865+0000","last_fullsized":"2024-06-06T08:40:19.422865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:31:22.877771+0000","last_clean_scrub_stamp":"2024-06-06T08:31:22.877771+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:25:40.692183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403949+0000","last_change":"2024-06-06T08:36:32.049299+0000","last_active":"2024-06-06T08:40:17.403949+0000","last_peered":"2024-06-06T08:40:17.403949+0000","last_clean":"2024-06-06T08:40:17.403949+0000","last_became_active":"2024-06-06T08:36:32.048776+0000","last_became_peered":"2024-06-06T08:36:32.048776+0000","last_unstale":"2024-06-06T08:40:17.403949+0000","last_undegraded":"2024-06-06T08:40:17.403949+0000","last_fullsized":"2024-06-06T08:40:17.403949+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:56:50.459146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.288880+0000","last_change":"2024-06-06T08:36:20.907852+0000","last_active":"2024-06-06T08:40:17.288880+0000","last_peered":"2024-06-06T08:40:17.288880+0000","last_clean":"2024-06-06T08:40:17.288880+0000","last_became_active":"2024-06-06T08:36:20.907500+0000","last_became_peered":"2024-06-06T08:36:20.907500+0000","last_unstale":"2024-06-06T08:40:17.288880+0000","last_undegraded":"2024-06-06T08:40:17.288880+0000","last_fullsized":"2024-06-06T08:40:17.288880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:32:28.511197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'3","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543213+0000","last_change":"2024-06-06T08:37:00.187174+0000","last_active":"2024-06-06T08:40:17.543213+0000","last_peered":"2024-06-06T08:40:17.543213+0000","last_clean":"2024-06-06T08:40:17.543213+0000","last_became_active":"2024-06-06T08:37:00.186657+0000","last_became_peered":"2024-06-06T08:37:00.186657+0000","last_unstale":"2024-06-06T08:40:17.543213+0000","last_undegraded":"2024-06-06T08:40:17.543213+0000","last_fullsized":"2024-06-06T08:40:17.543213+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:39:54.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543287+0000","last_change":"2024-06-06T08:36:31.924430+0000","last_active":"2024-06-06T08:40:17.543287+0000","last_peered":"2024-06-06T08:40:17.543287+0000","last_clean":"2024-06-06T08:40:17.543287+0000","last_became_active":"2024-06-06T08:36:31.924046+0000","last_became_peered":"2024-06-06T08:36:31.924046+0000","last_unstale":"2024-06-06T08:40:17.543287+0000","last_undegraded":"2024-06-06T08:40:17.543287+0000","last_fullsized":"2024-06-06T08:40:17.543287+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:04.839550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'1","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543496+0000","last_change":"2024-06-06T08:37:00.452181+0000","last_active":"2024-06-06T08:40:17.543496+0000","last_peered":"2024-06-06T08:40:17.543496+0000","last_clean":"2024-06-06T08:40:17.543496+0000","last_became_active":"2024-06-06T08:37:00.451862+0000","last_became_peered":"2024-06-06T08:37:00.451862+0000","last_unstale":"2024-06-06T08:40:17.543496+0000","last_undegraded":"2024-06-06T08:40:17.543496+0000","last_fullsized":"2024-06-06T08:40:17.543496+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:16:39.630293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289533+0000","last_change":"2024-06-06T08:36:20.904093+0000","last_active":"2024-06-06T08:40:17.289533+0000","last_peered":"2024-06-06T08:40:17.289533+0000","last_clean":"2024-06-06T08:40:17.289533+0000","last_became_active":"2024-06-06T08:36:20.903769+0000","last_became_peered":"2024-06-06T08:36:20.903769+0000","last_unstale":"2024-06-06T08:40:17.289533+0000","last_undegraded":"2024-06-06T08:40:17.289533+0000","last_fullsized":"2024-06-06T08:40:17.289533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:53:32.328870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397196+0000","last_change":"2024-06-06T08:36:32.047562+0000","last_active":"2024-06-06T08:40:17.397196+0000","last_peered":"2024-06-06T08:40:17.397196+0000","last_clean":"2024-06-06T08:40:17.397196+0000","last_became_active":"2024-06-06T08:36:32.047044+0000","last_became_peered":"2024-06-06T08:36:32.047044+0000","last_unstale":"2024-06-06T08:40:17.397196+0000","last_undegraded":"2024-06-06T08:40:17.397196+0000","last_fullsized":"2024-06-06T08:40:17.397196+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:36:50.357462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'102","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:33.017271+0000","last_change":"2024-06-06T08:36:17.431751+0000","last_active":"2024-06-06T08:40:33.017271+0000","last_peered":"2024-06-06T08:40:33.017271+0000","last_clean":"2024-06-06T08:40:33.017271+0000","last_became_active":"2024-06-06T08:36:17.431418+0000","last_became_peered":"2024-06-06T08:36:17.431418+0000","last_unstale":"2024-06-06T08:40:33.017271+0000","last_undegraded":"2024-06-06T08:40:33.017271+0000","last_fullsized":"2024-06-06T08:40:33.017271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:35.738614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":187,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397664+0000","last_change":"2024-06-06T08:36:20.896387+0000","last_active":"2024-06-06T08:40:17.397664+0000","last_peered":"2024-06-06T08:40:17.397664+0000","last_clean":"2024-06-06T08:40:17.397664+0000","last_became_active":"2024-06-06T08:36:20.895695+0000","last_became_peered":"2024-06-06T08:36:20.895695+0000","last_unstale":"2024-06-06T08:40:17.397664+0000","last_undegraded":"2024-06-06T08:40:17.397664+0000","last_fullsized":"2024-06-06T08:40:17.397664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:27:28.412146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290819+0000","last_change":"2024-06-06T08:36:31.921890+0000","last_active":"2024-06-06T08:40:17.290819+0000","last_peered":"2024-06-06T08:40:17.290819+0000","last_clean":"2024-06-06T08:40:17.290819+0000","last_became_active":"2024-06-06T08:36:31.921479+0000","last_became_peered":"2024-06-06T08:36:31.921479+0000","last_unstale":"2024-06-06T08:40:17.290819+0000","last_undegraded":"2024-06-06T08:40:17.290819+0000","last_fullsized":"2024-06-06T08:40:17.290819+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:58.978845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603116+0000","last_change":"2024-06-06T08:36:17.635016+0000","last_active":"2024-06-06T08:40:17.603116+0000","last_peered":"2024-06-06T08:40:17.603116+0000","last_clean":"2024-06-06T08:40:17.603116+0000","last_became_active":"2024-06-06T08:36:17.634652+0000","last_became_peered":"2024-06-06T08:36:17.634652+0000","last_unstale":"2024-06-06T08:40:17.603116+0000","last_undegraded":"2024-06-06T08:40:17.603116+0000","last_fullsized":"2024-06-06T08:40:17.603116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:49.606459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603756+0000","last_change":"2024-06-06T08:36:20.906480+0000","last_active":"2024-06-06T08:40:17.603756+0000","last_peered":"2024-06-06T08:40:17.603756+0000","last_clean":"2024-06-06T08:40:17.603756+0000","last_became_active":"2024-06-06T08:36:20.906087+0000","last_became_peered":"2024-06-06T08:36:20.906087+0000","last_unstale":"2024-06-06T08:40:17.603756+0000","last_undegraded":"2024-06-06T08:40:17.603756+0000","last_fullsized":"2024-06-06T08:40:17.603756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:29.047746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602722+0000","last_change":"2024-06-06T08:37:00.187379+0000","last_active":"2024-06-06T08:40:17.602722+0000","last_peered":"2024-06-06T08:40:17.602722+0000","last_clean":"2024-06-06T08:40:17.602722+0000","last_became_active":"2024-06-06T08:37:00.186870+0000","last_became_peered":"2024-06-06T08:37:00.186870+0000","last_unstale":"2024-06-06T08:40:17.602722+0000","last_undegraded":"2024-06-06T08:40:17.602722+0000","last_fullsized":"2024-06-06T08:40:17.602722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:32.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618429+0000","last_change":"2024-06-06T08:36:17.431340+0000","last_active":"2024-06-06T08:40:17.618429+0000","last_peered":"2024-06-06T08:40:17.618429+0000","last_clean":"2024-06-06T08:40:17.618429+0000","last_became_active":"2024-06-06T08:36:17.430896+0000","last_became_peered":"2024-06-06T08:36:17.430896+0000","last_unstale":"2024-06-06T08:40:17.618429+0000","last_undegraded":"2024-06-06T08:40:17.618429+0000","last_fullsized":"2024-06-06T08:40:17.618429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:24:50.963716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396892+0000","last_change":"2024-06-06T08:36:20.901334+0000","last_active":"2024-06-06T08:40:17.396892+0000","last_peered":"2024-06-06T08:40:17.396892+0000","last_clean":"2024-06-06T08:40:17.396892+0000","last_became_active":"2024-06-06T08:36:20.900073+0000","last_became_peered":"2024-06-06T08:36:20.900073+0000","last_unstale":"2024-06-06T08:40:17.396892+0000","last_undegraded":"2024-06-06T08:40:17.396892+0000","last_fullsized":"2024-06-06T08:40:17.396892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:44:20.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.395030+0000","last_change":"2024-06-06T08:36:31.929081+0000","last_active":"2024-06-06T08:40:17.395030+0000","last_peered":"2024-06-06T08:40:17.395030+0000","last_clean":"2024-06-06T08:40:17.395030+0000","last_became_active":"2024-06-06T08:36:31.924420+0000","last_became_peered":"2024-06-06T08:36:31.924420+0000","last_unstale":"2024-06-06T08:40:17.395030+0000","last_undegraded":"2024-06-06T08:40:17.395030+0000","last_fullsized":"2024-06-06T08:40:17.395030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:13:38.114520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397189+0000","last_change":"2024-06-06T08:36:17.429761+0000","last_active":"2024-06-06T08:40:17.397189+0000","last_peered":"2024-06-06T08:40:17.397189+0000","last_clean":"2024-06-06T08:40:17.397189+0000","last_became_active":"2024-06-06T08:36:17.429436+0000","last_became_peered":"2024-06-06T08:36:17.429436+0000","last_unstale":"2024-06-06T08:40:17.397189+0000","last_undegraded":"2024-06-06T08:40:17.397189+0000","last_fullsized":"2024-06-06T08:40:17.397189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:46:32.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543358+0000","last_change":"2024-06-06T08:36:20.907912+0000","last_active":"2024-06-06T08:40:17.543358+0000","last_peered":"2024-06-06T08:40:17.543358+0000","last_clean":"2024-06-06T08:40:17.543358+0000","last_became_active":"2024-06-06T08:36:20.907493+0000","last_became_peered":"2024-06-06T08:36:20.907493+0000","last_unstale":"2024-06-06T08:40:17.543358+0000","last_undegraded":"2024-06-06T08:40:17.543358+0000","last_fullsized":"2024-06-06T08:40:17.543358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:08.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602805+0000","last_change":"2024-06-06T08:36:31.922202+0000","last_active":"2024-06-06T08:40:17.602805+0000","last_peered":"2024-06-06T08:40:17.602805+0000","last_clean":"2024-06-06T08:40:17.602805+0000","last_became_active":"2024-06-06T08:36:31.920598+0000","last_became_peered":"2024-06-06T08:36:31.920598+0000","last_unstale":"2024-06-06T08:40:17.602805+0000","last_undegraded":"2024-06-06T08:40:17.602805+0000","last_fullsized":"2024-06-06T08:40:17.602805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:05:36.381987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.930424+0000","last_change":"2024-06-06T08:36:17.431848+0000","last_active":"2024-06-06T08:40:17.930424+0000","last_peered":"2024-06-06T08:40:17.930424+0000","last_clean":"2024-06-06T08:40:17.930424+0000","last_became_active":"2024-06-06T08:36:17.431439+0000","last_became_peered":"2024-06-06T08:36:17.431439+0000","last_unstale":"2024-06-06T08:40:17.930424+0000","last_undegraded":"2024-06-06T08:40:17.930424+0000","last_fullsized":"2024-06-06T08:40:17.930424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:11.378536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396937+0000","last_change":"2024-06-06T08:36:20.899235+0000","last_active":"2024-06-06T08:40:17.396937+0000","last_peered":"2024-06-06T08:40:17.396937+0000","last_clean":"2024-06-06T08:40:17.396937+0000","last_became_active":"2024-06-06T08:36:20.898901+0000","last_became_peered":"2024-06-06T08:36:20.898901+0000","last_unstale":"2024-06-06T08:40:17.396937+0000","last_undegraded":"2024-06-06T08:40:17.396937+0000","last_fullsized":"2024-06-06T08:40:17.396937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:12.400494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.602730+0000","last_change":"2024-06-06T08:36:32.047739+0000","last_active":"2024-06-06T08:40:17.602730+0000","last_peered":"2024-06-06T08:40:17.602730+0000","last_clean":"2024-06-06T08:40:17.602730+0000","last_became_active":"2024-06-06T08:36:32.047268+0000","last_became_peered":"2024-06-06T08:36:32.047268+0000","last_unstale":"2024-06-06T08:40:17.602730+0000","last_undegraded":"2024-06-06T08:40:17.602730+0000","last_fullsized":"2024-06-06T08:40:17.602730+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:21:24.810125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.396987+0000","last_change":"2024-06-06T08:36:17.435767+0000","last_active":"2024-06-06T08:40:17.396987+0000","last_peered":"2024-06-06T08:40:17.396987+0000","last_clean":"2024-06-06T08:40:17.396987+0000","last_became_active":"2024-06-06T08:36:17.434441+0000","last_became_peered":"2024-06-06T08:36:17.434441+0000","last_unstale":"2024-06-06T08:40:17.396987+0000","last_undegraded":"2024-06-06T08:40:17.396987+0000","last_fullsized":"2024-06-06T08:40:17.396987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.735535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290632+0000","last_change":"2024-06-06T08:36:20.896765+0000","last_active":"2024-06-06T08:40:17.290632+0000","last_peered":"2024-06-06T08:40:17.290632+0000","last_clean":"2024-06-06T08:40:17.290632+0000","last_became_active":"2024-06-06T08:36:20.896439+0000","last_became_peered":"2024-06-06T08:36:20.896439+0000","last_unstale":"2024-06-06T08:40:17.290632+0000","last_undegraded":"2024-06-06T08:40:17.290632+0000","last_fullsized":"2024-06-06T08:40:17.290632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:21:57.194920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618976+0000","last_change":"2024-06-06T08:36:31.924816+0000","last_active":"2024-06-06T08:40:17.618976+0000","last_peered":"2024-06-06T08:40:17.618976+0000","last_clean":"2024-06-06T08:40:17.618976+0000","last_became_active":"2024-06-06T08:36:31.924101+0000","last_became_peered":"2024-06-06T08:36:31.924101+0000","last_unstale":"2024-06-06T08:40:17.618976+0000","last_undegraded":"2024-06-06T08:40:17.618976+0000","last_fullsized":"2024-06-06T08:40:17.618976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:52.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'3","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291175+0000","last_change":"2024-06-06T08:36:17.635002+0000","last_active":"2024-06-06T08:40:17.291175+0000","last_peered":"2024-06-06T08:40:17.291175+0000","last_clean":"2024-06-06T08:40:17.291175+0000","last_became_active":"2024-06-06T08:36:17.634598+0000","last_became_peered":"2024-06-06T08:36:17.634598+0000","last_unstale":"2024-06-06T08:40:17.291175+0000","last_undegraded":"2024-06-06T08:40:17.291175+0000","last_fullsized":"2024-06-06T08:40:17.291175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T20:08:56.048919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291116+0000","last_change":"2024-06-06T08:36:20.904812+0000","last_active":"2024-06-06T08:40:17.291116+0000","last_peered":"2024-06-06T08:40:17.291116+0000","last_clean":"2024-06-06T08:40:17.291116+0000","last_became_active":"2024-06-06T08:36:20.904483+0000","last_became_peered":"2024-06-06T08:36:20.904483+0000","last_unstale":"2024-06-06T08:40:17.291116+0000","last_undegraded":"2024-06-06T08:40:17.291116+0000","last_fullsized":"2024-06-06T08:40:17.291116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:47:32.141475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603292+0000","last_change":"2024-06-06T08:36:31.931826+0000","last_active":"2024-06-06T08:40:17.603292+0000","last_peered":"2024-06-06T08:40:17.603292+0000","last_clean":"2024-06-06T08:40:17.603292+0000","last_became_active":"2024-06-06T08:36:31.931427+0000","last_became_peered":"2024-06-06T08:36:31.931427+0000","last_unstale":"2024-06-06T08:40:17.603292+0000","last_undegraded":"2024-06-06T08:40:17.603292+0000","last_fullsized":"2024-06-06T08:40:17.603292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:32.494954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'65","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:32.670293+0000","last_change":"2024-06-06T08:36:17.430354+0000","last_active":"2024-06-06T08:40:32.670293+0000","last_peered":"2024-06-06T08:40:32.670293+0000","last_clean":"2024-06-06T08:40:32.670293+0000","last_became_active":"2024-06-06T08:36:17.429951+0000","last_became_peered":"2024-06-06T08:36:17.429951+0000","last_unstale":"2024-06-06T08:40:32.670293+0000","last_undegraded":"2024-06-06T08:40:32.670293+0000","last_fullsized":"2024-06-06T08:40:32.670293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T19:12:40.411846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290659+0000","last_change":"2024-06-06T08:36:20.888409+0000","last_active":"2024-06-06T08:40:17.290659+0000","last_peered":"2024-06-06T08:40:17.290659+0000","last_clean":"2024-06-06T08:40:17.290659+0000","last_became_active":"2024-06-06T08:36:20.888015+0000","last_became_peered":"2024-06-06T08:36:20.888015+0000","last_unstale":"2024-06-06T08:40:17.290659+0000","last_undegraded":"2024-06-06T08:40:17.290659+0000","last_fullsized":"2024-06-06T08:40:17.290659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:47:10.216393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403261+0000","last_change":"2024-06-06T08:36:31.926627+0000","last_active":"2024-06-06T08:40:17.403261+0000","last_peered":"2024-06-06T08:40:17.403261+0000","last_clean":"2024-06-06T08:40:17.403261+0000","last_became_active":"2024-06-06T08:36:31.926204+0000","last_became_peered":"2024-06-06T08:36:31.926204+0000","last_unstale":"2024-06-06T08:40:17.403261+0000","last_undegraded":"2024-06-06T08:40:17.403261+0000","last_fullsized":"2024-06-06T08:40:17.403261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:30.902934+0000","last_clean_scrub_stamp":"2024-06-06T08:36:30.902934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:34:51.907623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289754+0000","last_change":"2024-06-06T08:37:00.066312+0000","last_active":"2024-06-06T08:40:17.289754+0000","last_peered":"2024-06-06T08:40:17.289754+0000","last_clean":"2024-06-06T08:40:17.289754+0000","last_became_active":"2024-06-06T08:37:00.055952+0000","last_became_peered":"2024-06-06T08:37:00.055952+0000","last_unstale":"2024-06-06T08:40:17.289754+0000","last_undegraded":"2024-06-06T08:40:17.289754+0000","last_fullsized":"2024-06-06T08:40:17.289754+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:16.405402+0000","last_clean_scrub_stamp":"2024-06-06T08:36:16.405402+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:32.243849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291007+0000","last_change":"2024-06-06T08:36:20.903264+0000","last_active":"2024-06-06T08:40:17.291007+0000","last_peered":"2024-06-06T08:40:17.291007+0000","last_clean":"2024-06-06T08:40:17.291007+0000","last_became_active":"2024-06-06T08:36:20.902744+0000","last_became_peered":"2024-06-06T08:36:20.902744+0000","last_unstale":"2024-06-06T08:40:17.291007+0000","last_undegraded":"2024-06-06T08:40:17.291007+0000","last_fullsized":"2024-06-06T08:40:17.291007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:35:27.779862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543518+0000","last_change":"2024-06-06T08:36:20.908033+0000","last_active":"2024-06-06T08:40:17.543518+0000","last_peered":"2024-06-06T08:40:17.543518+0000","last_clean":"2024-06-06T08:40:17.543518+0000","last_became_active":"2024-06-06T08:36:20.907411+0000","last_became_peered":"2024-06-06T08:36:20.907411+0000","last_unstale":"2024-06-06T08:40:17.543518+0000","last_undegraded":"2024-06-06T08:40:17.543518+0000","last_fullsized":"2024-06-06T08:40:17.543518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:20.854427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291058+0000","last_change":"2024-06-06T08:36:20.905029+0000","last_active":"2024-06-06T08:40:17.291058+0000","last_peered":"2024-06-06T08:40:17.291058+0000","last_clean":"2024-06-06T08:40:17.291058+0000","last_became_active":"2024-06-06T08:36:20.904567+0000","last_became_peered":"2024-06-06T08:36:20.904567+0000","last_unstale":"2024-06-06T08:40:17.291058+0000","last_undegraded":"2024-06-06T08:40:17.291058+0000","last_fullsized":"2024-06-06T08:40:17.291058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:02.792534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.290956+0000","last_change":"2024-06-06T08:36:20.908574+0000","last_active":"2024-06-06T08:40:17.290956+0000","last_peered":"2024-06-06T08:40:17.290956+0000","last_clean":"2024-06-06T08:40:17.290956+0000","last_became_active":"2024-06-06T08:36:20.908199+0000","last_became_peered":"2024-06-06T08:36:20.908199+0000","last_unstale":"2024-06-06T08:40:17.290956+0000","last_undegraded":"2024-06-06T08:40:17.290956+0000","last_fullsized":"2024-06-06T08:40:17.290956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:57:55.075447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.603418+0000","last_change":"2024-06-06T08:38:01.151003+0000","last_active":"2024-06-06T08:40:17.603418+0000","last_peered":"2024-06-06T08:40:17.603418+0000","last_clean":"2024-06-06T08:40:17.603418+0000","last_became_active":"2024-06-06T08:38:01.150557+0000","last_became_peered":"2024-06-06T08:38:01.150557+0000","last_unstale":"2024-06-06T08:40:17.603418+0000","last_undegraded":"2024-06-06T08:40:17.603418+0000","last_fullsized":"2024-06-06T08:40:17.603418+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:29:44.629894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289636+0000","last_change":"2024-06-06T08:36:20.894410+0000","last_active":"2024-06-06T08:40:17.289636+0000","last_peered":"2024-06-06T08:40:17.289636+0000","last_clean":"2024-06-06T08:40:17.289636+0000","last_became_active":"2024-06-06T08:36:20.894041+0000","last_became_peered":"2024-06-06T08:36:20.894041+0000","last_unstale":"2024-06-06T08:40:17.289636+0000","last_undegraded":"2024-06-06T08:40:17.289636+0000","last_fullsized":"2024-06-06T08:40:17.289636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:43:59.068344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543524+0000","last_change":"2024-06-06T08:36:20.902592+0000","last_active":"2024-06-06T08:40:17.543524+0000","last_peered":"2024-06-06T08:40:17.543524+0000","last_clean":"2024-06-06T08:40:17.543524+0000","last_became_active":"2024-06-06T08:36:20.902174+0000","last_became_peered":"2024-06-06T08:36:20.902174+0000","last_unstale":"2024-06-06T08:40:17.543524+0000","last_undegraded":"2024-06-06T08:40:17.543524+0000","last_fullsized":"2024-06-06T08:40:17.543524+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:50:13.707023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.291375+0000","last_change":"2024-06-06T08:36:20.907496+0000","last_active":"2024-06-06T08:40:17.291375+0000","last_peered":"2024-06-06T08:40:17.291375+0000","last_clean":"2024-06-06T08:40:17.291375+0000","last_became_active":"2024-06-06T08:36:20.906858+0000","last_became_peered":"2024-06-06T08:36:20.906858+0000","last_unstale":"2024-06-06T08:40:17.291375+0000","last_undegraded":"2024-06-06T08:40:17.291375+0000","last_fullsized":"2024-06-06T08:40:17.291375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:51:18.565412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397794+0000","last_change":"2024-06-06T08:37:00.332996+0000","last_active":"2024-06-06T08:40:17.397794+0000","last_peered":"2024-06-06T08:40:17.397794+0000","last_clean":"2024-06-06T08:40:17.397794+0000","last_became_active":"2024-06-06T08:37:00.332524+0000","last_became_peered":"2024-06-06T08:37:00.332524+0000","last_unstale":"2024-06-06T08:40:17.397794+0000","last_undegraded":"2024-06-06T08:40:17.397794+0000","last_fullsized":"2024-06-06T08:40:17.397794+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:37:49.066602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.403530+0000","last_change":"2024-06-06T08:36:20.887451+0000","last_active":"2024-06-06T08:40:17.403530+0000","last_peered":"2024-06-06T08:40:17.403530+0000","last_clean":"2024-06-06T08:40:17.403530+0000","last_became_active":"2024-06-06T08:36:20.887070+0000","last_became_peered":"2024-06-06T08:36:20.887070+0000","last_unstale":"2024-06-06T08:40:17.403530+0000","last_undegraded":"2024-06-06T08:40:17.403530+0000","last_fullsized":"2024-06-06T08:40:17.403530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:25:31.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.397272+0000","last_change":"2024-06-06T08:36:20.908506+0000","last_active":"2024-06-06T08:40:17.397272+0000","last_peered":"2024-06-06T08:40:17.397272+0000","last_clean":"2024-06-06T08:40:17.397272+0000","last_became_active":"2024-06-06T08:36:20.908140+0000","last_became_peered":"2024-06-06T08:36:20.908140+0000","last_unstale":"2024-06-06T08:40:17.397272+0000","last_undegraded":"2024-06-06T08:40:17.397272+0000","last_fullsized":"2024-06-06T08:40:17.397272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:06.291057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.618845+0000","last_change":"2024-06-06T08:36:20.895845+0000","last_active":"2024-06-06T08:40:17.618845+0000","last_peered":"2024-06-06T08:40:17.618845+0000","last_clean":"2024-06-06T08:40:17.618845+0000","last_became_active":"2024-06-06T08:36:20.892356+0000","last_became_peered":"2024-06-06T08:36:20.892356+0000","last_unstale":"2024-06-06T08:40:17.618845+0000","last_undegraded":"2024-06-06T08:40:17.618845+0000","last_fullsized":"2024-06-06T08:40:17.618845+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:35:57.275453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.289213+0000","last_change":"2024-06-06T08:36:20.896277+0000","last_active":"2024-06-06T08:40:17.289213+0000","last_peered":"2024-06-06T08:40:17.289213+0000","last_clean":"2024-06-06T08:40:17.289213+0000","last_became_active":"2024-06-06T08:36:20.894802+0000","last_became_peered":"2024-06-06T08:36:20.894802+0000","last_unstale":"2024-06-06T08:40:17.289213+0000","last_undegraded":"2024-06-06T08:40:17.289213+0000","last_fullsized":"2024-06-06T08:40:17.289213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:32:52.825788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T08:40:17.543444+0000","last_change":"2024-06-06T08:36:20.905231+0000","last_active":"2024-06-06T08:40:17.543444+0000","last_peered":"2024-06-06T08:40:17.543444+0000","last_clean":"2024-06-06T08:40:17.543444+0000","last_became_active":"2024-06-06T08:36:20.904355+0000","last_became_peered":"2024-06-06T08:36:20.904355+0000","last_unstale":"2024-06-06T08:40:17.543444+0000","last_undegraded":"2024-06-06T08:40:17.543444+0000","last_fullsized":"2024-06-06T08:40:17.543444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T08:36:19.868296+0000","last_clean_scrub_stamp":"2024-06-06T08:36:19.868296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:27:34.342084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107163,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798757,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1576,"num_read_kb":1011,"num_write":433,"num_write_kb":2367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":2396271,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":319,"ondisk_log_size":319,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383748,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":579875,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645386,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36620,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708532,"statfs":{"total":95995035648,"available":95957536768,"internally_reserved":0,"allocated":1306624,"data_stored":1568354,"data_compressed":86280,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808911,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":977401,"data_compressed":54646,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070547,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":973203,"data_compressed":54646,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332184,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":1177536,"data_compressed":71078,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495711,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":575265,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":646865,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":576653,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577223,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1183744,"data_stored":1296907,"data_compressed":69848,"data_compressed_allocated":94208,"data_compressed_original":798720,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920872,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1196032,"data_stored":652916,"data_compressed":35357,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757347,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1224704,"data_stored":1500045,"data_compressed":86280,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35721,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330406,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":402811,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T08:40:44.350 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:40:44.604 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:45.050 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:44 smithi027 ceph-mon[32335]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-06-06T08:40:45.050 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:44 smithi027 ceph-mon[32335]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:44 smithi203 ceph-mon[36579]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-06-06T08:40:45.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:44 smithi203 ceph-mon[36579]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:45.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:44 smithi176 ceph-mon[36786]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-06-06T08:40:45.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:44 smithi176 ceph-mon[36786]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T08:40:45.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:45 smithi027 ceph-mon[32335]: scrub summary: idle 2024-06-06T08:40:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.218+0000 7fc717273700 1 -- 172.21.15.27:0/1708781466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 msgr2=0x7fc71010e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.218+0000 7fc717273700 1 --2- 172.21.15.27:0/1708781466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 0x7fc71010e2d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b0d0 tx=0x7fc70800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.219+0000 7fc717273700 1 -- 172.21.15.27:0/1708781466 shutdown_connections 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.219+0000 7fc717273700 1 --2- 172.21.15.27:0/1708781466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 0x7fc71010e2d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.219+0000 7fc717273700 1 --2- 172.21.15.27:0/1708781466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc710101df0 0x7fc7101021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.219+0000 7fc717273700 1 --2- 172.21.15.27:0/1708781466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7101027a0 0x7fc710106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.219+0000 7fc717273700 1 -- 172.21.15.27:0/1708781466 >> 172.21.15.27:0/1708781466 conn(0x7fc710077a00 msgr2=0x7fc7100fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.219+0000 7fc717273700 1 -- 172.21.15.27:0/1708781466 shutdown_connections 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.219+0000 7fc717273700 1 -- 172.21.15.27:0/1708781466 wait complete. 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.219+0000 7fc717273700 1 Processor -- start 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.219+0000 7fc717273700 1 -- start start 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc717273700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710101df0 0x7fc71019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc717273700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7101027a0 0x7fc71019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc717273700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 0x7fc710198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc717273700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc710111690 con 0x7fc710106750 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc717273700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc710111510 con 0x7fc710101df0 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc717273700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc710111810 con 0x7fc7101027a0 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc715810700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 0x7fc710198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc715810700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 0x7fc710198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54754/0 (socket says 172.21.15.27:54754) 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc715810700 1 -- 172.21.15.27:0/3434592384 learned_addr learned my addr 172.21.15.27:0/3434592384 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc71500f700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710101df0 0x7fc71019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc71480e700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7101027a0 0x7fc71019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc715810700 1 -- 172.21.15.27:0/3434592384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7101027a0 msgr2=0x7fc71019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc715810700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7101027a0 0x7fc71019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc715810700 1 -- 172.21.15.27:0/3434592384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710101df0 msgr2=0x7fc71019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc715810700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710101df0 0x7fc71019e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc715810700 1 -- 172.21.15.27:0/3434592384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc710199080 con 0x7fc710106750 2024-06-06T08:40:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc71480e700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7101027a0 0x7fc71019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc71500f700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710101df0 0x7fc71019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.220+0000 7fc715810700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 0x7fc710198920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc708006070 tx=0x7fc708004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.221+0000 7fc7067fc700 1 -- 172.21.15.27:0/3434592384 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708021070 con 0x7fc710106750 2024-06-06T08:40:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.221+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc710199310 con 0x7fc710106750 2024-06-06T08:40:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.221+0000 7fc7067fc700 1 -- 172.21.15.27:0/3434592384 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc708007df0 con 0x7fc710106750 2024-06-06T08:40:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.221+0000 7fc7067fc700 1 -- 172.21.15.27:0/3434592384 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708005470 con 0x7fc710106750 2024-06-06T08:40:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.221+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a5630 con 0x7fc710106750 2024-06-06T08:40:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.222+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc710069180 con 0x7fc710106750 2024-06-06T08:40:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.223+0000 7fc7067fc700 1 -- 172.21.15.27:0/3434592384 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fc708024070 con 0x7fc710106750 2024-06-06T08:40:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.224+0000 7fc7067fc700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc6fc0a2eb0 0x7fc6fc0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.224+0000 7fc71500f700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc6fc0a2eb0 0x7fc6fc0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.224+0000 7fc7067fc700 1 -- 172.21.15.27:0/3434592384 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fc7080d0db0 con 0x7fc710106750 2024-06-06T08:40:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.224+0000 7fc71500f700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc6fc0a2eb0 0x7fc6fc0a5370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc70c005950 tx=0x7fc70c005f30 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.227+0000 7fc7067fc700 1 -- 172.21.15.27:0/3434592384 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc70809d450 con 0x7fc710106750 2024-06-06T08:40:46.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:45 smithi203 ceph-mon[36579]: scrub summary: idle 2024-06-06T08:40:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.249+0000 7f06f5309640 1 -- 172.21.15.27:0/1785443366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f010bf70 msgr2=0x7f06f010e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.249+0000 7f06f5309640 1 --2- 172.21.15.27:0/1785443366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f010bf70 0x7f06f010e410 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009ca0 tx=0x7f06e802a9e0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.249+0000 7f06f5309640 1 -- 172.21.15.27:0/1785443366 shutdown_connections 2024-06-06T08:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.249+0000 7f06f5309640 1 --2- 172.21.15.27:0/1785443366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f010bf70 0x7f06f010e410 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.249+0000 7f06f5309640 1 --2- 172.21.15.27:0/1785443366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06f0103550 0x7f06f010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.249+0000 7f06f5309640 1 --2- 172.21.15.27:0/1785443366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f0102c30 0x7f06f0103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.249+0000 7f06f5309640 1 -- 172.21.15.27:0/1785443366 >> 172.21.15.27:0/1785443366 conn(0x7f06f0077a20 msgr2=0x7f06f0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.249+0000 7f06f5309640 1 -- 172.21.15.27:0/1785443366 shutdown_connections 2024-06-06T08:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.249+0000 7f06f5309640 1 -- 172.21.15.27:0/1785443366 wait complete. 2024-06-06T08:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06f5309640 1 Processor -- start 2024-06-06T08:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06f5309640 1 -- start start 2024-06-06T08:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06f5309640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f0102c30 0x7f06f01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06f5309640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06f0103550 0x7f06f01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06f5309640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f010bf70 0x7f06f019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06f5309640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0111d70 con 0x7f06f0102c30 2024-06-06T08:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06f5309640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0111bf0 con 0x7f06f0103550 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06f5309640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0111ef0 con 0x7f06f010bf70 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06eeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f0102c30 0x7f06f01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06eeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f0102c30 0x7f06f01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54768/0 (socket says 172.21.15.27:54768) 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.250+0000 7f06eeffd640 1 -- 172.21.15.27:0/534594857 learned_addr learned my addr 172.21.15.27:0/534594857 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f010bf70 0x7f06f019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06eeffd640 1 -- 172.21.15.27:0/534594857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f010bf70 msgr2=0x7f06f019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06e67fc640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06f0103550 0x7f06f01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06eeffd640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f010bf70 0x7f06f019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06eeffd640 1 -- 172.21.15.27:0/534594857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06f0103550 msgr2=0x7f06f01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06eeffd640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06f0103550 0x7f06f01986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06eeffd640 1 -- 172.21.15.27:0/534594857 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f019d150 con 0x7f06f0102c30 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f010bf70 0x7f06f019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06eeffd640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f0102c30 0x7f06f01981b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f06e000e9f0 tx=0x7f06e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06ecff9640 1 -- 172.21.15.27:0/534594857 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e000cd80 con 0x7f06f0102c30 2024-06-06T08:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06f5309640 1 -- 172.21.15.27:0/534594857 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f019d440 con 0x7f06f0102c30 2024-06-06T08:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06ecff9640 1 -- 172.21.15.27:0/534594857 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e0005500 con 0x7f06f0102c30 2024-06-06T08:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06f5309640 1 -- 172.21.15.27:0/534594857 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f01a4cd0 con 0x7f06f0102c30 2024-06-06T08:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.251+0000 7f06ecff9640 1 -- 172.21.15.27:0/534594857 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0010690 con 0x7f06f0102c30 2024-06-06T08:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.253+0000 7f06ecff9640 1 -- 172.21.15.27:0/534594857 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f06e00046c0 con 0x7f06f0102c30 2024-06-06T08:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.253+0000 7f06f800b740 1 Processor -- start 2024-06-06T08:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06f800b740 1 -- start start 2024-06-06T08:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06f800b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fcff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06f800b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 0x55e74fd0e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06f800b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 0x55e74fd10b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06f800b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e74fc634f0 con 0x55e74fcff070 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06f800b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e74fc01870 con 0x55e74fcff990 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06f800b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e74fc5f4b0 con 0x55e74fd0e760 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fcff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fcff450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54772/0 (socket says 172.21.15.27:54772) 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06ef7fe640 1 -- 172.21.15.27:0/931467159 learned_addr learned my addr 172.21.15.27:0/931467159 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06e67fc640 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 0x55e74fd10b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06eeffd640 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 0x55e74fd0e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06ef7fe640 1 -- 172.21.15.27:0/931467159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 msgr2=0x55e74fd10b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.254+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 0x55e74fd10b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06ef7fe640 1 -- 172.21.15.27:0/931467159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 msgr2=0x55e74fd0e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 0x55e74fd0e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06ef7fe640 1 -- 172.21.15.27:0/931467159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74fc1d9b0 con 0x55e74fcff070 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06e67fc640 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 0x55e74fd10b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fcff450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f06e8005f70 tx=0x7f06e8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f643418d9a002a24 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06ecff9640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06dc0a2ec0 0x7f06dc0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06eeffd640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06dc0a2ec0 0x7f06dc0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06e57fa640 1 -- 172.21.15.27:0/931467159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8047070 con 0x55e74fcff070 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06e57fa640 1 -- 172.21.15.27:0/931467159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8003970 con 0x55e74fcff070 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06e57fa640 1 -- 172.21.15.27:0/931467159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8033050 con 0x55e74fcff070 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.255+0000 7f06ecff9640 1 -- 172.21.15.27:0/534594857 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f06e00d0010 con 0x7f06f0102c30 2024-06-06T08:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 -- 172.21.15.27:0/931467159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 msgr2=0x55e74fcff450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06eeffd640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06dc0a2ec0 0x7f06dc0a5380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06e00cd280 tx=0x7f06e0004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fcff450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f06e8005f70 tx=0x7f06e8005fa0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 -- 172.21.15.27:0/931467159 shutdown_connections 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 0x55e74fd10b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 0x55e74fd0e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 --2- 172.21.15.27:0/931467159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fcff450 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 -- 172.21.15.27:0/931467159 >> 172.21.15.27:0/931467159 conn(0x55e74fcfb680 msgr2=0x55e74fcfba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 -- 172.21.15.27:0/931467159 shutdown_connections 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 -- 172.21.15.27:0/931467159 wait complete. 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.256+0000 7f06f800b740 1 -- start start 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06f800b740 1 Processor -- start 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06f800b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff070 0x55e74fd0b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06f800b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 0x55e74fd0bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06f800b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fd0e760 0x55e74fd0c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06f800b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e74fd11330 con 0x55e74fd0e760 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06f800b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e74fbecae0 con 0x55e74fcff070 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06f800b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e74fd114b0 con 0x55e74fcff990 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06f800b740 10 client.0 _ll_register_callbacks cb 0x55e74fc259b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fd0e760 0x55e74fd0c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fd0e760 0x55e74fd0c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54778/0 (socket says 172.21.15.27:54778) 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06ef7fe640 1 -- 172.21.15.27:0/503013742 learned_addr learned my addr 172.21.15.27:0/503013742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff070 0x55e74fd0b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06e67fc640 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 0x55e74fd0bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06ef7fe640 1 -- 172.21.15.27:0/503013742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 msgr2=0x55e74fd0bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 0x55e74fd0bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06ef7fe640 1 -- 172.21.15.27:0/503013742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff070 msgr2=0x55e74fd0b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff070 0x55e74fd0b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.257+0000 7f06ef7fe640 1 -- 172.21.15.27:0/503013742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74f977210 con 0x55e74fd0e760 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fd0e760 0x55e74fd0c0f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009600 tx=0x7f06e802e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af16f8235dc1ba4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06d67fc640 1 -- 172.21.15.27:0/503013742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8047070 con 0x55e74fd0e760 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06d67fc640 1 -- 172.21.15.27:0/503013742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f06e802f640 con 0x55e74fd0e760 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06f800b740 1 -- 172.21.15.27:0/503013742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e8003970 con 0x55e74fd0e760 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06d67fc640 1 -- 172.21.15.27:0/503013742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06e805a050 con 0x55e74fd0e760 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06f800b740 10 client.15447 fetch_fsmap learned FSMap version 25 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06f800b740 1 -- 172.21.15.27:0/503013742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e74fa43620 con 0x55e74fd0e760 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06d67fc640 1 -- 172.21.15.27:0/503013742 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8003970 con 0x55e74fd0e760 2024-06-06T08:40:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06d67fc640 1 -- 172.21.15.27:0/503013742 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f06e80042e0 con 0x55e74fd0e760 2024-06-06T08:40:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06f800b740 10 client.15447 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:40:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06f800b740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:40:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06f800b740 20 client.15447 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06f800b740 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e74fd1e710 0x55e74fd20b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06f800b740 4 client.15447 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:40:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.258+0000 7f06f800b740 1 -- 172.21.15.27:0/503013742 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e74fc76f10 con 0x55e74fd1e710 2024-06-06T08:40:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.259+0000 7f06e67fc640 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e74fd1e710 0x55e74fd20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.259+0000 7f06e67fc640 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e74fd1e710 0x55e74fd20b00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a624406696f832 server_cookie=7194668fc565e49 in_seq=0 out_seq=0 2024-06-06T08:40:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.259+0000 7f06d67fc640 10 client.15447 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:46.264 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:45 smithi176 ceph-mon[36786]: scrub summary: idle 2024-06-06T08:40:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06d67fc640 1 -- 172.21.15.27:0/503013742 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e74fc76f10 con 0x55e74fd1e710 2024-06-06T08:40:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06d67fc640 10 client.15447 handle_command_reply: tid=0 2024-06-06T08:40:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 client.15447 shutdown 2024-06-06T08:40:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 -- 172.21.15.27:0/503013742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fd0e760 msgr2=0x55e74fd0c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fd0e760 0x55e74fd0c0f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009600 tx=0x7f06e802e5d0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 -- 172.21.15.27:0/503013742 shutdown_connections 2024-06-06T08:40:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e74fd1e710 0x55e74fd20b00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fd0e760 0x55e74fd0c0f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 0x55e74fd0bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 --2- 172.21.15.27:0/503013742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff070 0x55e74fd0b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 -- 172.21.15.27:0/503013742 >> 172.21.15.27:0/503013742 conn(0x55e74fcfb680 msgr2=0x55e74fd014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06d67fc640 0 client.15447 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 -- 172.21.15.27:0/503013742 shutdown_connections 2024-06-06T08:40:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 1 -- 172.21.15.27:0/503013742 wait complete. 2024-06-06T08:40:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.260+0000 7f06f800b740 20 client.15447 trim_cache size 0 max 16384 2024-06-06T08:40:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.275+0000 7f06f800b740 1 Processor -- start 2024-06-06T08:40:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.275+0000 7f06f800b740 1 -- start start 2024-06-06T08:40:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06f800b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06f800b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 0x55e74fd06ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06f800b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fd0e760 0x55e74fd09f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06f800b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01af480 con 0x55e74fcff070 2024-06-06T08:40:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06f800b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01acd50 con 0x55e74fd0e760 2024-06-06T08:40:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06f800b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01aaba0 con 0x55e74fcff990 2024-06-06T08:40:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54784/0 (socket says 172.21.15.27:54784) 2024-06-06T08:40:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 -- 172.21.15.27:0/78138555 learned_addr learned my addr 172.21.15.27:0/78138555 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06e67fc640 1 --2- 172.21.15.27:0/78138555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 0x55e74fd06ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/78138555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fd0e760 0x55e74fd09f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 -- 172.21.15.27:0/78138555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 msgr2=0x55e74fd06ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/78138555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 0x55e74fd06ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 -- 172.21.15.27:0/78138555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fd0e760 msgr2=0x55e74fd09f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/78138555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fd0e760 0x55e74fd09f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 -- 172.21.15.27:0/78138555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74fd0a480 con 0x55e74fcff070 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/78138555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd069a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000f80 tx=0x7f06e8009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124120d81aa25ad0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.276+0000 7f06e4ff9640 1 -- 172.21.15.27:0/78138555 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8047030 con 0x55e74fcff070 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06e4ff9640 1 -- 172.21.15.27:0/78138555 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8048040 con 0x55e74fcff070 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06e4ff9640 1 -- 172.21.15.27:0/78138555 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8033070 con 0x55e74fcff070 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06f800b740 1 -- 172.21.15.27:0/78138555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 msgr2=0x55e74fd069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06f800b740 1 --2- 172.21.15.27:0/78138555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd069a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000f80 tx=0x7f06e8009540 comp rx=0 tx=0).stop 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06f800b740 1 -- 172.21.15.27:0/78138555 shutdown_connections 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06f800b740 1 --2- 172.21.15.27:0/78138555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fd0e760 0x55e74fd09f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06f800b740 1 --2- 172.21.15.27:0/78138555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fcff990 0x55e74fd06ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06f800b740 1 --2- 172.21.15.27:0/78138555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd069a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06f800b740 1 -- 172.21.15.27:0/78138555 >> 172.21.15.27:0/78138555 conn(0x55e74fcfb680 msgr2=0x55e74fd002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06f800b740 1 -- 172.21.15.27:0/78138555 shutdown_connections 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.277+0000 7f06f800b740 1 -- 172.21.15.27:0/78138555 wait complete. 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06f800b740 1 -- start start 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06f800b740 1 Processor -- start 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06f800b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd20720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06f800b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 0x55e74fd20c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06f800b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 0x55e74fd211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06f800b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ab210 con 0x55e74fcff070 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06f800b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ac680 con 0x55e74fcff990 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06f800b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ab8e0 con 0x55e74fd0e760 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06f800b740 10 client.0 _ll_register_callbacks cb 0x7f06e8045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd20720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd20720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54788/0 (socket says 172.21.15.27:54788) 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06ef7fe640 1 -- 172.21.15.27:0/3158038140 learned_addr learned my addr 172.21.15.27:0/3158038140 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06e67fc640 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 0x55e74fd20c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06ef7fe640 1 -- 172.21.15.27:0/3158038140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 msgr2=0x55e74fd211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 0x55e74fd211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06ef7fe640 1 -- 172.21.15.27:0/3158038140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 msgr2=0x55e74fd20c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 0x55e74fd20c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.278+0000 7f06ef7fe640 1 -- 172.21.15.27:0/3158038140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74fd87040 con 0x55e74fcff070 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06ef7fe640 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd20720 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f06e8005cc0 tx=0x7f06e80367b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efec3f8d3ed34dfe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06d67fc640 1 -- 172.21.15.27:0/3158038140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8047040 con 0x55e74fcff070 2024-06-06T08:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06d67fc640 1 -- 172.21.15.27:0/3158038140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f06e803e070 con 0x55e74fcff070 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06f800b740 1 -- 172.21.15.27:0/3158038140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e8048040 con 0x55e74fcff070 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06d67fc640 1 -- 172.21.15.27:0/3158038140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06e8033040 con 0x55e74fcff070 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06f800b740 10 client.15453 fetch_fsmap learned FSMap version 25 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06f800b740 1 -- 172.21.15.27:0/3158038140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e74fd882f0 con 0x55e74fcff070 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06d67fc640 1 -- 172.21.15.27:0/3158038140 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8048040 con 0x55e74fcff070 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06d67fc640 1 -- 172.21.15.27:0/3158038140 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f06e8065030 con 0x55e74fcff070 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06f800b740 10 client.15453 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06f800b740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06f800b740 20 client.15453 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06f800b740 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e74fd887d0 0x55e74fd8abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06f800b740 4 client.15453 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.279+0000 7f06f800b740 1 -- 172.21.15.27:0/3158038140 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e74fc70c20 con 0x55e74fd887d0 2024-06-06T08:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.280+0000 7f06e67fc640 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e74fd887d0 0x55e74fd8abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.280+0000 7f06e67fc640 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e74fd887d0 0x55e74fd8abc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99298a70882e0f server_cookie=aa8335efee8a2378 in_seq=0 out_seq=0 2024-06-06T08:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.280+0000 7f06d67fc640 10 client.15453 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06d67fc640 1 -- 172.21.15.27:0/3158038140 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e74fc70c20 con 0x55e74fd887d0 2024-06-06T08:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06d67fc640 10 client.15453 handle_command_reply: tid=0 2024-06-06T08:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 client.15453 shutdown 2024-06-06T08:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 -- 172.21.15.27:0/3158038140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 msgr2=0x55e74fd20720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd20720 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f06e8005cc0 tx=0x7f06e80367b0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 -- 172.21.15.27:0/3158038140 shutdown_connections 2024-06-06T08:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e74fd887d0 0x55e74fd8abc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e74fd0e760 0x55e74fd211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e74fcff990 0x55e74fd20c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06d67fc640 0 client.15453 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 --2- 172.21.15.27:0/3158038140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e74fcff070 0x55e74fd20720 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 -- 172.21.15.27:0/3158038140 >> 172.21.15.27:0/3158038140 conn(0x55e74fcfb680 msgr2=0x55e74fd069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 -- 172.21.15.27:0/3158038140 shutdown_connections 2024-06-06T08:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 1 -- 172.21.15.27:0/3158038140 wait complete. 2024-06-06T08:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.295+0000 7f06f800b740 20 client.15453 trim_cache size 0 max 16384 2024-06-06T08:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:40:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:40:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 -- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06dc0a2ec0 msgr2=0x7f06dc0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06dc0a2ec0 0x7f06dc0a5380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f06e00cd280 tx=0x7f06e0004a40 comp rx=0 tx=0).stop 2024-06-06T08:40:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 -- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f0102c30 msgr2=0x7f06f01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f0102c30 0x7f06f01981b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f06e000e9f0 tx=0x7f06e000edf0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 -- 172.21.15.27:0/534594857 shutdown_connections 2024-06-06T08:40:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06dc0a2ec0 0x7f06dc0a5380 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f010bf70 0x7f06f019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06f0103550 0x7f06f01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 --2- 172.21.15.27:0/534594857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06f0102c30 0x7f06f01981b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 -- 172.21.15.27:0/534594857 >> 172.21.15.27:0/534594857 conn(0x7f06f0077a20 msgr2=0x7f06f00fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 -- 172.21.15.27:0/534594857 shutdown_connections 2024-06-06T08:40:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:46.296+0000 7f06f5309640 1 -- 172.21.15.27:0/534594857 wait complete. 2024-06-06T08:40:46.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.490+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7101a5800 con 0x7fc710106750 2024-06-06T08:40:46.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.491+0000 7fc7067fc700 1 -- 172.21.15.27:0/3434592384 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7fc70809cd50 con 0x7fc710106750 2024-06-06T08:40:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:46.490 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:40:46.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc6fc0a2eb0 msgr2=0x7fc6fc0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc6fc0a2eb0 0x7fc6fc0a5370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc70c005950 tx=0x7fc70c005f30 comp rx=0 tx=0).stop 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 msgr2=0x7fc710198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 0x7fc710198920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc708006070 tx=0x7fc708004ef0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 shutdown_connections 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc710106750 0x7fc710198920 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc6fc0a2eb0 0x7fc6fc0a5370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc710101df0 0x7fc71019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 --2- 172.21.15.27:0/3434592384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7101027a0 0x7fc71019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 >> 172.21.15.27:0/3434592384 conn(0x7fc710077a00 msgr2=0x7fc710100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 shutdown_connections 2024-06-06T08:40:46.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:46.493+0000 7fc717273700 1 -- 172.21.15.27:0/3434592384 wait complete. 2024-06-06T08:40:46.493 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:40:47.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:46 smithi027 ceph-mon[32335]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:40:47.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:46 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3434592384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:47.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:46 smithi203 ceph-mon[36579]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:40:47.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:46 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3434592384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:47.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:46 smithi176 ceph-mon[36786]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:40:47.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:46 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3434592384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:47.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:40:47.572 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:49.025 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:48 smithi027 ceph-mon[32335]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:40:49.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:48 smithi203 ceph-mon[36579]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:40:49.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:48 smithi176 ceph-mon[36786]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:40:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.381+0000 7f26a49a1700 1 -- 172.21.15.27:0/169531688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0101d50 msgr2=0x7f26a010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.381+0000 7f26a49a1700 1 --2- 172.21.15.27:0/169531688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0101d50 0x7f26a010e240 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f269800b0a0 tx=0x7f269800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.381+0000 7f26a49a1700 1 -- 172.21.15.27:0/169531688 shutdown_connections 2024-06-06T08:40:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.381+0000 7f26a49a1700 1 --2- 172.21.15.27:0/169531688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0101d50 0x7f26a010e240 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.381+0000 7f26a49a1700 1 --2- 172.21.15.27:0/169531688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26a0107400 0x7f26a01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.381+0000 7f26a49a1700 1 --2- 172.21.15.27:0/169531688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a0101390 0x7f26a0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.381+0000 7f26a49a1700 1 -- 172.21.15.27:0/169531688 >> 172.21.15.27:0/169531688 conn(0x7f26a0078340 msgr2=0x7f26a0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.381+0000 7f26a49a1700 1 -- 172.21.15.27:0/169531688 shutdown_connections 2024-06-06T08:40:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.381+0000 7f26a49a1700 1 -- 172.21.15.27:0/169531688 wait complete. 2024-06-06T08:40:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.382+0000 7f26a49a1700 1 Processor -- start 2024-06-06T08:40:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.382+0000 7f26a49a1700 1 -- start start 2024-06-06T08:40:49.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.382+0000 7f26a49a1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26a0101390 0x7f26a0199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:49.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.382+0000 7f26a49a1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a0101d50 0x7f26a0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:49.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.382+0000 7f26a49a1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0107400 0x7f26a01a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:49.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.382+0000 7f26a49a1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0111670 con 0x7f26a0107400 2024-06-06T08:40:49.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.382+0000 7f26a49a1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01114f0 con 0x7f26a0101390 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.382+0000 7f26a49a1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01117f0 con 0x7f26a0101d50 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.382+0000 7f269ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0107400 0x7f26a01a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0107400 0x7f26a01a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54806/0 (socket says 172.21.15.27:54806) 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269ed9d700 1 -- 172.21.15.27:0/2156175213 learned_addr learned my addr 172.21.15.27:0/2156175213 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269dd9b700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a0101d50 0x7f26a0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269e59c700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26a0101390 0x7f26a0199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269ed9d700 1 -- 172.21.15.27:0/2156175213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a0101d50 msgr2=0x7f26a0199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269ed9d700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a0101d50 0x7f26a0199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269ed9d700 1 -- 172.21.15.27:0/2156175213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26a0101390 msgr2=0x7f26a0199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269ed9d700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26a0101390 0x7f26a0199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269ed9d700 1 -- 172.21.15.27:0/2156175213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a01a5010 con 0x7f26a0107400 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f269ed9d700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0107400 0x7f26a01a4ad0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f26980062a0 tx=0x7f2698004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f268f7fe700 1 -- 172.21.15.27:0/2156175213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698021070 con 0x7f26a0107400 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f268f7fe700 1 -- 172.21.15.27:0/2156175213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2698003c00 con 0x7f26a0107400 2024-06-06T08:40:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a01a51e0 con 0x7f26a0107400 2024-06-06T08:40:49.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f268f7fe700 1 -- 172.21.15.27:0/2156175213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698004880 con 0x7f26a0107400 2024-06-06T08:40:49.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.383+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a0197840 con 0x7f26a0107400 2024-06-06T08:40:49.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.384+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a0069180 con 0x7f26a0107400 2024-06-06T08:40:49.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.386+0000 7f268f7fe700 1 -- 172.21.15.27:0/2156175213 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f2698007980 con 0x7f26a0107400 2024-06-06T08:40:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.386+0000 7f268f7fe700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26880a72c0 0x7f26880a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.386+0000 7f269e59c700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26880a72c0 0x7f26880a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.386+0000 7f268f7fe700 1 -- 172.21.15.27:0/2156175213 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f26980d0e40 con 0x7f26a0107400 2024-06-06T08:40:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.387+0000 7f269e59c700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26880a72c0 0x7f26880a9780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2690005950 tx=0x7f269000b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.390+0000 7f268f7fe700 1 -- 172.21.15.27:0/2156175213 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f269809d4e0 con 0x7f26a0107400 2024-06-06T08:40:49.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.646+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26a002dca0 con 0x7f26a0107400 2024-06-06T08:40:49.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.647+0000 7f268f7fe700 1 -- 172.21.15.27:0/2156175213 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7f269809cde0 con 0x7f26a0107400 2024-06-06T08:40:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:49.646 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:40:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.649+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26880a72c0 msgr2=0x7f26880a9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.649+0000 7f26a49a1700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26880a72c0 0x7f26880a9780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2690005950 tx=0x7f269000b410 comp rx=0 tx=0).stop 2024-06-06T08:40:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.649+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0107400 msgr2=0x7f26a01a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.649+0000 7f26a49a1700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0107400 0x7f26a01a4ad0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f26980062a0 tx=0x7f2698004d70 comp rx=0 tx=0).stop 2024-06-06T08:40:49.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.650+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 shutdown_connections 2024-06-06T08:40:49.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.650+0000 7f26a49a1700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26a0107400 0x7f26a01a4ad0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.650+0000 7f26a49a1700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26880a72c0 0x7f26880a9780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.650+0000 7f26a49a1700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26a0101390 0x7f26a0199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.650+0000 7f26a49a1700 1 --2- 172.21.15.27:0/2156175213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a0101d50 0x7f26a0199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:49.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.650+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 >> 172.21.15.27:0/2156175213 conn(0x7f26a0078340 msgr2=0x7f26a00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:49.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.650+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 shutdown_connections 2024-06-06T08:40:49.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:49.650+0000 7f26a49a1700 1 -- 172.21.15.27:0/2156175213 wait complete. 2024-06-06T08:40:49.649 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:40:49.949 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:49 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2156175213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:50.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:49 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2156175213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:50.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:49 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2156175213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:50.271 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:40:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:50.913 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:50 smithi027 ceph-mon[32335]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T08:40:51.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:50 smithi203 ceph-mon[36579]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T08:40:51.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:50 smithi176 ceph-mon[36786]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T08:40:52.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 -- 172.21.15.27:0/473588024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180107840 msgr2=0x7f718010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 --2- 172.21.15.27:0/473588024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180107840 0x7f718010f3c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f717800b0d0 tx=0x7f717800b3e0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 -- 172.21.15.27:0/473588024 shutdown_connections 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 --2- 172.21.15.27:0/473588024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180107840 0x7f718010f3c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 --2- 172.21.15.27:0/473588024 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7180102740 0x7f7180102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 --2- 172.21.15.27:0/473588024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71801030f0 0x7f7180107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 -- 172.21.15.27:0/473588024 >> 172.21.15.27:0/473588024 conn(0x7f71800795f0 msgr2=0x7f7180079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 -- 172.21.15.27:0/473588024 shutdown_connections 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 -- 172.21.15.27:0/473588024 wait complete. 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.430+0000 7f7184bfb700 1 Processor -- start 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f7184bfb700 1 -- start start 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f7184bfb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180102740 0x7f718019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f7184bfb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71801030f0 0x7f718019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f7184bfb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7180107840 0x7f7180198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f7184bfb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f71801127d0 con 0x7f7180102740 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f7184bfb700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7180112650 con 0x7f71801030f0 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f7184bfb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7180112950 con 0x7f7180107840 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180102740 0x7f718019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180102740 0x7f718019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59940/0 (socket says 172.21.15.27:59940) 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717e59c700 1 -- 172.21.15.27:0/3664946727 learned_addr learned my addr 172.21.15.27:0/3664946727 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717dd9b700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71801030f0 0x7f718019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717ed9d700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7180107840 0x7f7180198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717e59c700 1 -- 172.21.15.27:0/3664946727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7180107840 msgr2=0x7f7180198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717e59c700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7180107840 0x7f7180198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717e59c700 1 -- 172.21.15.27:0/3664946727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71801030f0 msgr2=0x7f718019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717e59c700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71801030f0 0x7f718019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717e59c700 1 -- 172.21.15.27:0/3664946727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71801990f0 con 0x7f7180102740 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717ed9d700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7180107840 0x7f7180198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.431+0000 7f717e59c700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180102740 0x7f718019e7d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f717000c920 tx=0x7f717000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.432+0000 7f71777fe700 1 -- 172.21.15.27:0/3664946727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170007b00 con 0x7f7180102740 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.432+0000 7f71777fe700 1 -- 172.21.15.27:0/3664946727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7170004d10 con 0x7f7180102740 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.432+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71801993e0 con 0x7f7180102740 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.432+0000 7f71777fe700 1 -- 172.21.15.27:0/3664946727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170005680 con 0x7f7180102740 2024-06-06T08:40:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.432+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801a56a0 con 0x7f7180102740 2024-06-06T08:40:52.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.433+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7180069180 con 0x7f7180102740 2024-06-06T08:40:52.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.434+0000 7f71777fe700 1 -- 172.21.15.27:0/3664946727 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f7170004750 con 0x7f7180102740 2024-06-06T08:40:52.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.435+0000 7f71777fe700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f716c0a2eb0 0x7f716c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:52.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.435+0000 7f717dd9b700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f716c0a2eb0 0x7f716c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:52.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.435+0000 7f71777fe700 1 -- 172.21.15.27:0/3664946727 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f71700cfa00 con 0x7f7180102740 2024-06-06T08:40:52.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.435+0000 7f717dd9b700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f716c0a2eb0 0x7f716c0a5370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7180108c60 tx=0x7f7168009190 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:52.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.438+0000 7f71777fe700 1 -- 172.21.15.27:0/3664946727 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7170014030 con 0x7f7180102740 2024-06-06T08:40:52.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.645+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f718002dca0 con 0x7f7180102740 2024-06-06T08:40:52.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.647+0000 7f71777fe700 1 -- 172.21.15.27:0/3664946727 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25213 (secure 0 0 0) 0x7f717009c010 con 0x7f7180102740 2024-06-06T08:40:52.646 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:52.646 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:40:17.280912+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/876705499":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/550980984":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/4174349888":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3053481437":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:40:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f716c0a2eb0 msgr2=0x7f716c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f716c0a2eb0 0x7f716c0a5370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7180108c60 tx=0x7f7168009190 comp rx=0 tx=0).stop 2024-06-06T08:40:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180102740 msgr2=0x7f718019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180102740 0x7f718019e7d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f717000c920 tx=0x7f717000cc30 comp rx=0 tx=0).stop 2024-06-06T08:40:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 shutdown_connections 2024-06-06T08:40:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7180107840 0x7f7180198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7180102740 0x7f718019e7d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f716c0a2eb0 0x7f716c0a5370 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 --2- 172.21.15.27:0/3664946727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71801030f0 0x7f718019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 >> 172.21.15.27:0/3664946727 conn(0x7f71800795f0 msgr2=0x7f71801004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 shutdown_connections 2024-06-06T08:40:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:52.649+0000 7f7184bfb700 1 -- 172.21.15.27:0/3664946727 wait complete. 2024-06-06T08:40:53.074 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:52 smithi027 ceph-mon[32335]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:40:53.075 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:52 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3664946727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:53.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:52 smithi203 ceph-mon[36579]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:40:53.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:52 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3664946727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:53.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:52 smithi176 ceph-mon[36786]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:40:53.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:52 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3664946727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:40:53.450 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-06-06T08:40:53.701 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:55.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:54 smithi027 ceph-mon[32335]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T08:40:55.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:54 smithi203 ceph-mon[36579]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T08:40:55.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:54 smithi176 ceph-mon[36786]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T08:40:56.018 INFO:teuthology.orchestra.run.smithi027.stdout:{"run_dir":"/var/run/ceph"} 2024-06-06T08:40:56.018 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-06-06T08:40:56.019 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 5050 2024-06-06T08:40:56.019 DEBUG:tasks.quiescer:Yielding 2024-06-06T08:40:56.019 INFO:teuthology.run_tasks:Running task workunit... 2024-06-06T08:40:56.034 INFO:tasks.workunit:Pulling workunits from ref 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T08:40:56.034 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-06T08:40:56.035 INFO:tasks.workunit:timeout=6h 2024-06-06T08:40:56.035 INFO:tasks.workunit:cleanup=True 2024-06-06T08:40:56.035 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-06T08:40:56.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:40:56.053 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-06T08:40:56.053 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-06T08:40:56.053 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-06-06T08:40:56.053 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T08:40:56.053 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-06T08:40:56.053 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-06-06 08:37:39.778131033 +0000 2024-06-06T08:40:56.053 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-06-06 08:37:39.778131033 +0000 2024-06-06T08:40:56.053 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-06-06 08:40:14.439048224 +0000 2024-06-06T08:40:56.054 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-06-06 08:37:39.778131033 +0000 2024-06-06T08:40:56.054 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-06T08:40:56.054 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-06T08:40:56.125 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T08:40:56.182 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-06T08:40:56.291 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:57.106 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:56 smithi027 ceph-mon[32335]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:40:57.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:56 smithi203 ceph-mon[36579]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:40:57.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:56 smithi176 ceph-mon[36786]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:40:58.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.060+0000 7fb40c276700 1 -- 172.21.15.27:0/1016554130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 msgr2=0x7fb4041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:58.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.060+0000 7fb40c276700 1 --2- 172.21.15.27:0/1016554130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 0x7fb4041029b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80099c0 tx=0x7fb3f8009cd0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.060+0000 7fb40c276700 1 -- 172.21.15.27:0/1016554130 shutdown_connections 2024-06-06T08:40:58.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.060+0000 7fb40c276700 1 --2- 172.21.15.27:0/1016554130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb404102ef0 0x7fb40410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.060+0000 7fb40c276700 1 --2- 172.21.15.27:0/1016554130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4041085d0 0x7fb4041089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.060+0000 7fb40c276700 1 --2- 172.21.15.27:0/1016554130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 0x7fb4041029b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.060+0000 7fb40c276700 1 -- 172.21.15.27:0/1016554130 >> 172.21.15.27:0/1016554130 conn(0x7fb404079640 msgr2=0x7fb404079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.060+0000 7fb40c276700 1 -- 172.21.15.27:0/1016554130 shutdown_connections 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.060+0000 7fb40c276700 1 -- 172.21.15.27:0/1016554130 wait complete. 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40c276700 1 Processor -- start 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40c276700 1 -- start start 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40c276700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 0x7fb40419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40c276700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb404102ef0 0x7fb40419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40c276700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4041085d0 0x7fb404198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40c276700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb404112810 con 0x7fb404102530 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40c276700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb404112690 con 0x7fb4041085d0 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40c276700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb404112990 con 0x7fb404102ef0 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40a012700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 0x7fb40419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40a012700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 0x7fb40419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59968/0 (socket says 172.21.15.27:59968) 2024-06-06T08:40:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.061+0000 7fb40a012700 1 -- 172.21.15.27:0/2516284708 learned_addr learned my addr 172.21.15.27:0/2516284708 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb409811700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb404102ef0 0x7fb40419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40a813700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4041085d0 0x7fb404198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40a012700 1 -- 172.21.15.27:0/2516284708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb404102ef0 msgr2=0x7fb40419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40a012700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb404102ef0 0x7fb40419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40a012700 1 -- 172.21.15.27:0/2516284708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4041085d0 msgr2=0x7fb404198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40a012700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4041085d0 0x7fb404198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40a012700 1 -- 172.21.15.27:0/2516284708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb404199190 con 0x7fb404102530 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40a813700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4041085d0 0x7fb404198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40a012700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 0x7fb40419e870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40060b0 tx=0x7fb3f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb4037fe700 1 -- 172.21.15.27:0/2516284708 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f401a070 con 0x7fb404102530 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb404199480 con 0x7fb404102530 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb4037fe700 1 -- 172.21.15.27:0/2516284708 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f400ddd0 con 0x7fb404102530 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb4037fe700 1 -- 172.21.15.27:0/2516284708 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40147a0 con 0x7fb404102530 2024-06-06T08:40:58.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.062+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a5740 con 0x7fb404102530 2024-06-06T08:40:58.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.063+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb404103c40 con 0x7fb404102530 2024-06-06T08:40:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.064+0000 7fb4037fe700 1 -- 172.21.15.27:0/2516284708 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fb3f401d070 con 0x7fb404102530 2024-06-06T08:40:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.065+0000 7fb4037fe700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb3ec0a2eb0 0x7fb3ec0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.065+0000 7fb409811700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb3ec0a2eb0 0x7fb3ec0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.065+0000 7fb4037fe700 1 -- 172.21.15.27:0/2516284708 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb3f40cf3b0 con 0x7fb404102530 2024-06-06T08:40:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.066+0000 7fb409811700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb3ec0a2eb0 0x7fb3ec0a5370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb404079cb0 tx=0x7fb3f8005fb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:58.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.070+0000 7fb4037fe700 1 -- 172.21.15.27:0/2516284708 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb3f4010070 con 0x7fb404102530 2024-06-06T08:40:58.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.324+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb404199f60 con 0x7fb404102530 2024-06-06T08:40:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.325+0000 7fb4037fe700 1 -- 172.21.15.27:0/2516284708 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7fb3f409b9c0 con 0x7fb404102530 2024-06-06T08:40:58.325 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:40:58.325 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:40:58.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb3ec0a2eb0 msgr2=0x7fb3ec0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb3ec0a2eb0 0x7fb3ec0a5370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb404079cb0 tx=0x7fb3f8005fb0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 msgr2=0x7fb40419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 0x7fb40419e870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40060b0 tx=0x7fb3f400d3b0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 shutdown_connections 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb3ec0a2eb0 0x7fb3ec0a5370 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4041085d0 0x7fb404198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb404102530 0x7fb40419e870 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 --2- 172.21.15.27:0/2516284708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb404102ef0 0x7fb40419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 >> 172.21.15.27:0/2516284708 conn(0x7fb404079640 msgr2=0x7fb404101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 shutdown_connections 2024-06-06T08:40:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:40:58.328+0000 7fb40c276700 1 -- 172.21.15.27:0/2516284708 wait complete. 2024-06-06T08:40:58.329 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:40:58.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:58 smithi027 ceph-mon[32335]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:40:58.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:40:58 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2516284708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:58.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T08:36:23.190518+0000', 'modified': '2024-06-06T08:40:15.140203+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24482, 'mds_1': 24460, 'mds_2': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/2823423002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 2823423002}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 2823423002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6835/1368203014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6834', 'nonce': 1368203014}, {'type': 'v1', 'addr': '172.21.15.176:6835', 'nonce': 1368203014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/266251643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 266251643}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 266251643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.176:6837/3697558303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 3697558303}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 3697558303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.203:6839/693474346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 693474346}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 693474346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.176:6839/1702529484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 1702529484}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 1702529484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24460, 24482, 24458]} 2024-06-06T08:40:58.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-06T08:40:58.957 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T08:40:59.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:58 smithi203 ceph-mon[36579]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:40:59.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:40:58 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2516284708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:59.250 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:40:59.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:58 smithi176 ceph-mon[36786]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:40:59.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:40:58 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2516284708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:40:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 -- 172.21.15.27:0/855831613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 msgr2=0x7fd2e810e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/855831613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 0x7fd2e810e860 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009ca0 tx=0x7fd2d802a8c0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 -- 172.21.15.27:0/855831613 shutdown_connections 2024-06-06T08:40:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/855831613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 0x7fd2e810e860 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/855831613 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2e806a180 0x7fd2e806a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/855831613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e81069d0 0x7fd2e8106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 -- 172.21.15.27:0/855831613 >> 172.21.15.27:0/855831613 conn(0x7fd2e8077680 msgr2=0x7fd2e8077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 -- 172.21.15.27:0/855831613 shutdown_connections 2024-06-06T08:40:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 -- 172.21.15.27:0/855831613 wait complete. 2024-06-06T08:40:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.387+0000 7fd2ef91b640 1 Processor -- start 2024-06-06T08:40:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2ef91b640 1 -- start start 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2ef91b640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2e806a180 0x7fd2e819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2ef91b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 0x7fd2e819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2ef91b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e81069d0 0x7fd2e819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2ef91b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e8111d90 con 0x7fd2e806ab40 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2ef91b640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e8111c10 con 0x7fd2e806a180 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2ef91b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e8111f10 con 0x7fd2e81069d0 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2eceb6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 0x7fd2e819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2eceb6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 0x7fd2e819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59980/0 (socket says 172.21.15.27:59980) 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2eceb6640 1 -- 172.21.15.27:0/4096396739 learned_addr learned my addr 172.21.15.27:0/4096396739 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2e806a180 0x7fd2e819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e81069d0 0x7fd2e819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2eceb6640 1 -- 172.21.15.27:0/4096396739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e81069d0 msgr2=0x7fd2e819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e81069d0 0x7fd2e819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2eceb6640 1 -- 172.21.15.27:0/4096396739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2e806a180 msgr2=0x7fd2e819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2e806a180 0x7fd2e819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2eceb6640 1 -- 172.21.15.27:0/4096396739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e819f2a0 con 0x7fd2e806ab40 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.388+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2e806a180 0x7fd2e819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.389+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 0x7fd2e819a7f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00d950 tx=0x7fd2dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.389+0000 7fd2e67fc640 1 -- 172.21.15.27:0/4096396739 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc017070 con 0x7fd2e806ab40 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.389+0000 7fd2ef91b640 1 -- 172.21.15.27:0/4096396739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e819f590 con 0x7fd2e806ab40 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.389+0000 7fd2e67fc640 1 -- 172.21.15.27:0/4096396739 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc009c30 con 0x7fd2e806ab40 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.389+0000 7fd2e67fc640 1 -- 172.21.15.27:0/4096396739 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00f6b0 con 0x7fd2e806ab40 2024-06-06T08:40:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.389+0000 7fd2ef91b640 1 -- 172.21.15.27:0/4096396739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e81a6e20 con 0x7fd2e806ab40 2024-06-06T08:40:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.391+0000 7fd2e67fc640 1 -- 172.21.15.27:0/4096396739 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fd2dc0109d0 con 0x7fd2e806ab40 2024-06-06T08:40:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2f262f740 1 Processor -- start 2024-06-06T08:40:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2f262f740 1 -- start start 2024-06-06T08:40:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 0x559ac94ff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9509e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac950a350 0x559ac950c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2f262f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559ac94634f0 con 0x559ac94ffa90 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2f262f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559ac9401870 con 0x559ac94ff170 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2f262f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559ac945f4b0 con 0x559ac950a350 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2e67fc640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd2d00a2ec0 0x7fd2d00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2ed6b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9509e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2ed6b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9509e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59982/0 (socket says 172.21.15.27:59982) 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2ed6b7640 1 -- 172.21.15.27:0/2436012953 learned_addr learned my addr 172.21.15.27:0/2436012953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.392+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd2d00a2ec0 0x7fd2d00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2e67fc640 1 -- 172.21.15.27:0/4096396739 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fd2dc0cf6d0 con 0x7fd2e806ab40 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 0x559ac94ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac950a350 0x559ac950c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2ed6b7640 1 -- 172.21.15.27:0/2436012953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac950a350 msgr2=0x559ac950c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac950a350 0x559ac950c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2ed6b7640 1 -- 172.21.15.27:0/2436012953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 msgr2=0x559ac94ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 0x559ac94ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2ed6b7640 1 -- 172.21.15.27:0/2436012953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac941d9b0 con 0x559ac94ffa90 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac950a350 0x559ac950c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9509e10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00104d0 tx=0x7fd2e0010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198de572ff70d70 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd2d00a2ec0 0x7fd2d00a5380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8005fc0 tx=0x7fd2d802f3d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2cf7fe640 1 -- 172.21.15.27:0/2436012953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0004220 con 0x559ac94ffa90 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2cf7fe640 1 -- 172.21.15.27:0/2436012953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e00043c0 con 0x559ac94ffa90 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.393+0000 7fd2cf7fe640 1 -- 172.21.15.27:0/2436012953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0011a70 con 0x559ac94ffa90 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 -- 172.21.15.27:0/2436012953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 msgr2=0x559ac9509e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9509e10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00104d0 tx=0x7fd2e0010990 comp rx=0 tx=0).stop 2024-06-06T08:40:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 -- 172.21.15.27:0/2436012953 shutdown_connections 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac950a350 0x559ac950c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9509e10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 --2- 172.21.15.27:0/2436012953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 0x559ac94ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 -- 172.21.15.27:0/2436012953 >> 172.21.15.27:0/2436012953 conn(0x559ac94fb680 msgr2=0x559ac94fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 -- 172.21.15.27:0/2436012953 shutdown_connections 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 -- 172.21.15.27:0/2436012953 wait complete. 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 -- start start 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.394+0000 7fd2f262f740 1 Processor -- start 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 0x559ac9507280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ffa90 0x559ac95077c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac9507d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2f262f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559ac950cf80 con 0x559ac950a350 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2f262f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x559ac93ecae0 con 0x559ac94ffa90 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2f262f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559ac950d100 con 0x559ac94ff170 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2f262f740 10 client.0 _ll_register_callbacks cb 0x559ac94259b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2edeb8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac9507d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2edeb8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac9507d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59998/0 (socket says 172.21.15.27:59998) 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2edeb8640 1 -- 172.21.15.27:0/4200528210 learned_addr learned my addr 172.21.15.27:0/4200528210 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ffa90 0x559ac95077c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 0x559ac9507280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2edeb8640 1 -- 172.21.15.27:0/4200528210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 msgr2=0x559ac9507280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 0x559ac9507280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2edeb8640 1 -- 172.21.15.27:0/4200528210 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ffa90 msgr2=0x559ac95077c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ffa90 0x559ac95077c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2edeb8640 1 -- 172.21.15.27:0/4200528210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac9177210 con 0x559ac950a350 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 0x559ac9507280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac9507d00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8035000 tx=0x7fd2d8048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed4983f735d449a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.395+0000 7fd2ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2ccff9640 1 -- 172.21.15.27:0/4200528210 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8050070 con 0x559ac950a350 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2f262f740 1 -- 172.21.15.27:0/4200528210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2e00043c0 con 0x559ac950a350 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2ccff9640 1 -- 172.21.15.27:0/4200528210 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fd2d8030950 con 0x559ac950a350 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2ccff9640 1 -- 172.21.15.27:0/4200528210 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2d805c050 con 0x559ac950a350 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2f262f740 10 client.15483 fetch_fsmap learned FSMap version 25 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2f262f740 1 -- 172.21.15.27:0/4200528210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ac9243620 con 0x559ac950a350 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2ccff9640 1 -- 172.21.15.27:0/4200528210 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e00043c0 con 0x559ac950a350 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2ccff9640 1 -- 172.21.15.27:0/4200528210 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd2d804a040 con 0x559ac950a350 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2f262f740 10 client.15483 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:40:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2f262f740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2f262f740 20 client.15483 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2f262f740 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x559ac951a4b0 0x559ac951c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2f262f740 4 client.15483 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.396+0000 7fd2f262f740 1 -- 172.21.15.27:0/4200528210 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ac9476f10 con 0x559ac951a4b0 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.397+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x559ac951a4b0 0x559ac951c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.397+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x559ac951a4b0 0x559ac951c930 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a15900422ede8bc5 server_cookie=c3bbc21e289ecfef in_seq=0 out_seq=0 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.397+0000 7fd2ccff9640 10 client.15483 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2ccff9640 1 -- 172.21.15.27:0/4200528210 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559ac9476f10 con 0x559ac951a4b0 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2ccff9640 10 client.15483 handle_command_reply: tid=0 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 client.15483 shutdown 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 -- 172.21.15.27:0/4200528210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 msgr2=0x559ac9507d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac9507d00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8035000 tx=0x7fd2d8048000 comp rx=0 tx=0).stop 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 -- 172.21.15.27:0/4200528210 shutdown_connections 2024-06-06T08:40:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x559ac951a4b0 0x559ac951c930 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac9507d00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ffa90 0x559ac95077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 --2- 172.21.15.27:0/4200528210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 0x559ac9507280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 -- 172.21.15.27:0/4200528210 >> 172.21.15.27:0/4200528210 conn(0x559ac94fb680 msgr2=0x559ac9501400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2ccff9640 0 client.15483 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 -- 172.21.15.27:0/4200528210 shutdown_connections 2024-06-06T08:40:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 1 -- 172.21.15.27:0/4200528210 wait complete. 2024-06-06T08:40:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.398+0000 7fd2f262f740 20 client.15483 trim_cache size 0 max 16384 2024-06-06T08:40:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2f262f740 1 Processor -- start 2024-06-06T08:40:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2f262f740 1 -- start start 2024-06-06T08:40:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 0x559ac9501b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9522240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac950a350 0x559ac9522780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2f262f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e81b3830 con 0x559ac94ffa90 2024-06-06T08:40:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2f262f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e81b1100 con 0x559ac950a350 2024-06-06T08:40:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2f262f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e81aef50 con 0x559ac94ff170 2024-06-06T08:40:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2eceb6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9522240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2eceb6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9522240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60000/0 (socket says 172.21.15.27:60000) 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2eceb6640 1 -- 172.21.15.27:0/894837192 learned_addr learned my addr 172.21.15.27:0/894837192 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 0x559ac9501b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac950a350 0x559ac9522780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2eceb6640 1 -- 172.21.15.27:0/894837192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 msgr2=0x559ac9501b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 0x559ac9501b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2eceb6640 1 -- 172.21.15.27:0/894837192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac950a350 msgr2=0x559ac9522780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac950a350 0x559ac9522780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2eceb6640 1 -- 172.21.15.27:0/894837192 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac94ff8c0 con 0x559ac94ffa90 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac950a350 0x559ac9522780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9522240 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc097fb0 tx=0x7fd2dc09af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1f98c7985192cc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.414+0000 7fd2ceffd640 1 -- 172.21.15.27:0/894837192 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc020040 con 0x559ac94ffa90 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2ceffd640 1 -- 172.21.15.27:0/894837192 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc09b5e0 con 0x559ac94ffa90 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2ceffd640 1 -- 172.21.15.27:0/894837192 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc0cce20 con 0x559ac94ffa90 2024-06-06T08:40:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 -- 172.21.15.27:0/894837192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 msgr2=0x559ac9522240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9522240 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc097fb0 tx=0x7fd2dc09af40 comp rx=0 tx=0).stop 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 -- 172.21.15.27:0/894837192 shutdown_connections 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac950a350 0x559ac9522780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac94ffa90 0x559ac9522240 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 --2- 172.21.15.27:0/894837192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ff170 0x559ac9501b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 -- 172.21.15.27:0/894837192 >> 172.21.15.27:0/894837192 conn(0x559ac94fb680 msgr2=0x559ac9502630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 -- 172.21.15.27:0/894837192 shutdown_connections 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 -- 172.21.15.27:0/894837192 wait complete. 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.415+0000 7fd2f262f740 1 -- start start 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2f262f740 1 Processor -- start 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 0x559ac9512710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ffa90 0x559ac9512c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2f262f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac950dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2f262f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e81af5c0 con 0x559ac950a350 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2f262f740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e81b0a30 con 0x559ac94ff170 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2f262f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e81afc90 con 0x559ac94ffa90 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2f262f740 10 client.0 _ll_register_callbacks cb 0x7fd2d802af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2edeb8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac950dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2edeb8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac950dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60012/0 (socket says 172.21.15.27:60012) 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2edeb8640 1 -- 172.21.15.27:0/1087577733 learned_addr learned my addr 172.21.15.27:0/1087577733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2eceb6640 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ffa90 0x559ac9512c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 0x559ac9512710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2edeb8640 1 -- 172.21.15.27:0/1087577733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ffa90 msgr2=0x559ac9512c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ffa90 0x559ac9512c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2edeb8640 1 -- 172.21.15.27:0/1087577733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 msgr2=0x559ac9512710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 0x559ac9512710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2edeb8640 1 -- 172.21.15.27:0/1087577733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac950e450 con 0x559ac950a350 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.416+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 0x559ac9512710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2edeb8640 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac950dc80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd2d802b0b0 tx=0x7fd2d804c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02be0d2a0be5c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2ccff9640 1 -- 172.21.15.27:0/1087577733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8050040 con 0x559ac950a350 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2ccff9640 1 -- 172.21.15.27:0/1087577733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fd2d8004d50 con 0x559ac950a350 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2f262f740 1 -- 172.21.15.27:0/1087577733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2dc09b5e0 con 0x559ac950a350 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2ccff9640 1 -- 172.21.15.27:0/1087577733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2d8069050 con 0x559ac950a350 2024-06-06T08:40:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2f262f740 10 client.15489 fetch_fsmap learned FSMap version 25 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2f262f740 1 -- 172.21.15.27:0/1087577733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ac9584df0 con 0x559ac950a350 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2ccff9640 1 -- 172.21.15.27:0/1087577733 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2dc09b5e0 con 0x559ac950a350 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2ccff9640 1 -- 172.21.15.27:0/1087577733 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd2d804c790 con 0x559ac950a350 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2f262f740 10 client.15489 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2f262f740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2f262f740 20 client.15489 populate_metadata read hostname 'smithi027' 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2f262f740 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x559ac9585540 0x559ac9587960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2f262f740 4 client.15489 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.417+0000 7fd2f262f740 1 -- 172.21.15.27:0/1087577733 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ac9470c20 con 0x559ac9585540 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.418+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x559ac9585540 0x559ac9587960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.418+0000 7fd2ccff9640 10 client.15489 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.418+0000 7fd2ed6b7640 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x559ac9585540 0x559ac9587960 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae343c97161d8887 server_cookie=8b81c401bad1c793 in_seq=0 out_seq=0 2024-06-06T08:40:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2ccff9640 1 -- 172.21.15.27:0/1087577733 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559ac9470c20 con 0x559ac9585540 2024-06-06T08:40:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2ccff9640 10 client.15489 handle_command_reply: tid=0 2024-06-06T08:40:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 client.15489 shutdown 2024-06-06T08:40:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 -- 172.21.15.27:0/1087577733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 msgr2=0x559ac950dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac950dc80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd2d802b0b0 tx=0x7fd2d804c720 comp rx=0 tx=0).stop 2024-06-06T08:40:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 -- 172.21.15.27:0/1087577733 shutdown_connections 2024-06-06T08:40:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x559ac9585540 0x559ac9587960 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559ac950a350 0x559ac950dc80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2ccff9640 0 client.15489 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:40:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559ac94ffa90 0x559ac9512c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 --2- 172.21.15.27:0/1087577733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559ac94ff170 0x559ac9512710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 -- 172.21.15.27:0/1087577733 >> 172.21.15.27:0/1087577733 conn(0x559ac94fb680 msgr2=0x559ac9508ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 -- 172.21.15.27:0/1087577733 shutdown_connections 2024-06-06T08:40:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 1 -- 172.21.15.27:0/1087577733 wait complete. 2024-06-06T08:40:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.427+0000 7fd2f262f740 20 client.15489 trim_cache size 0 max 16384 2024-06-06T08:40:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:40:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:40:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:40:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 -- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd2d00a2ec0 msgr2=0x7fd2d00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd2d00a2ec0 0x7fd2d00a5380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8005fc0 tx=0x7fd2d802f3d0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 -- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 msgr2=0x7fd2e819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:40:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 0x7fd2e819a7f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00d950 tx=0x7fd2dc00de10 comp rx=0 tx=0).stop 2024-06-06T08:40:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 -- 172.21.15.27:0/4096396739 shutdown_connections 2024-06-06T08:40:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd2d00a2ec0 0x7fd2d00a5380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e81069d0 0x7fd2e819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2e806ab40 0x7fd2e819a7f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 --2- 172.21.15.27:0/4096396739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd2e806a180 0x7fd2e819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:40:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 -- 172.21.15.27:0/4096396739 >> 172.21.15.27:0/4096396739 conn(0x7fd2e8077680 msgr2=0x7fd2e80fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:40:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 -- 172.21.15.27:0/4096396739 shutdown_connections 2024-06-06T08:40:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:40:59.428+0000 7fd2ef91b640 1 -- 172.21.15.27:0/4096396739 wait complete. 2024-06-06T08:41:00.691 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:41:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:41:00.302+0000 7f9048fbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:41:00.691 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:41:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:41:00.302+0000 7f9048fbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:41:00.691 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:41:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:41:00.302+0000 7f9048fbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:41:00.691 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:41:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:41:00.302+0000 7f9048fbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:41:00.691 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:41:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:41:00.302+0000 7f9048fbe700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T08:41:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:00 smithi027 ceph-mon[32335]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-06-06T08:41:01.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.072+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1139020620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78107430 msgr2=0x7f9f78107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:01.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.072+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1139020620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78107430 0x7f9f78107810 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009990 tx=0x7f9f68009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.072+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1139020620 shutdown_connections 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.072+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1139020620 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f78101d80 0x7f9f7810e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.072+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1139020620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78107430 0x7f9f78107810 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.072+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1139020620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f781013c0 0x7f9f78101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.072+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1139020620 >> 172.21.15.27:0/1139020620 conn(0x7f9f780783d0 msgr2=0x7f9f780787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.072+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1139020620 shutdown_connections 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.072+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1139020620 wait complete. 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f7caf0700 1 Processor -- start 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f7caf0700 1 -- start start 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f7caf0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f781013c0 0x7f9f78198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f7caf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78101d80 0x7f9f781992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f7caf0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f78107430 0x7f9f7819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f7caf0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78112840 con 0x7f9f78101d80 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f7caf0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781126c0 con 0x7f9f781013c0 2024-06-06T08:41:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f7caf0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781129c0 con 0x7f9f78107430 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f75d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78101d80 0x7f9f781992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f75d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78101d80 0x7f9f781992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37850/0 (socket says 172.21.15.27:37850) 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f75d9b700 1 -- 172.21.15.27:0/1791424009 learned_addr learned my addr 172.21.15.27:0/1791424009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f7659c700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f781013c0 0x7f9f78198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.073+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f78107430 0x7f9f7819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f75d9b700 1 -- 172.21.15.27:0/1791424009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f78107430 msgr2=0x7f9f7819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f78107430 0x7f9f7819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f75d9b700 1 -- 172.21.15.27:0/1791424009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f781013c0 msgr2=0x7f9f78198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f781013c0 0x7f9f78198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f75d9b700 1 -- 172.21.15.27:0/1791424009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7819ddc0 con 0x7f9f78101d80 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f7659c700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f781013c0 0x7f9f78198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78101d80 0x7f9f781992f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00b530 tx=0x7f9f6c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f677fe700 1 -- 172.21.15.27:0/1791424009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c015020 con 0x7f9f78101d80 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1791424009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7819e0b0 con 0x7f9f78101d80 2024-06-06T08:41:01.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f677fe700 1 -- 172.21.15.27:0/1791424009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c011d10 con 0x7f9f78101d80 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f677fe700 1 -- 172.21.15.27:0/1791424009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00f2e0 con 0x7f9f78101d80 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.074+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1791424009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f781a5930 con 0x7f9f78101d80 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.076+0000 7f9f677fe700 1 -- 172.21.15.27:0/1791424009 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f9f6c018020 con 0x7f9f78101d80 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.077+0000 7f9f677fe700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f600a2eb0 0x7f9f600a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.077+0000 7f9f7659c700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f600a2eb0 0x7f9f600a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.077+0000 7f9f677fe700 1 -- 172.21.15.27:0/1791424009 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9f6c0cf140 con 0x7f9f78101d80 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.077+0000 7f9f7659c700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f600a2eb0 0x7f9f600a5370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800b5c0 tx=0x7f9f68005960 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f87373b80 1 Processor -- start 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f87373b80 1 -- start start 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20f062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f00210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f00b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f87373b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562a20df2f10 con 0x562a20f00750 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f87373b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a20e51800 con 0x562a20f05f10 2024-06-06T08:41:01.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f87373b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562a20db9410 con 0x562a20f06830 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f76d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f00b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f76d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f00b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37854/0 (socket says 172.21.15.27:37854) 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f76d9d700 1 -- 172.21.15.27:0/541889085 learned_addr learned my addr 172.21.15.27:0/541889085 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.078+0000 7f9f7659c700 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20f062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f00210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f76d9d700 1 -- 172.21.15.27:0/541889085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 msgr2=0x562a20f00210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f00210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f76d9d700 1 -- 172.21.15.27:0/541889085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 msgr2=0x562a20f062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20f062f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f76d9d700 1 -- 172.21.15.27:0/541889085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a20da0880 con 0x562a20f00750 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f00210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f00b70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009dc0 tx=0x7f9f700077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8b531037c8339f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:01.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f64ff9700 1 -- 172.21.15.27:0/541889085 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70014610 con 0x562a20f00750 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f64ff9700 1 -- 172.21.15.27:0/541889085 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f70019070 con 0x562a20f00750 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.079+0000 7f9f64ff9700 1 -- 172.21.15.27:0/541889085 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70013a50 con 0x562a20f00750 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 -- 172.21.15.27:0/541889085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 msgr2=0x562a20f00b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f00b70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009dc0 tx=0x7f9f700077e0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 -- 172.21.15.27:0/541889085 shutdown_connections 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f00b70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20f062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 --2- 172.21.15.27:0/541889085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f00210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 -- 172.21.15.27:0/541889085 >> 172.21.15.27:0/541889085 conn(0x562a20e2d030 msgr2=0x562a20efb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 -- 172.21.15.27:0/541889085 shutdown_connections 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 -- 172.21.15.27:0/541889085 wait complete. 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 -- start start 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.080+0000 7f9f87373b80 1 Processor -- start 2024-06-06T08:41:01.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f0c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20f097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f09d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f87373b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562a20df1f70 con 0x562a20f00750 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f87373b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a20e4d7c0 con 0x562a20f05f10 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f87373b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562a20f0d210 con 0x562a20f06830 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f87373b80 10 client.0 _ll_register_callbacks cb 0x562a20de7560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f7659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f0c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f7659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f0c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37860/0 (socket says 172.21.15.27:37860) 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f7659c700 1 -- 172.21.15.27:0/1905612867 learned_addr learned my addr 172.21.15.27:0/1905612867 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20f097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f09d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f7659c700 1 -- 172.21.15.27:0/1905612867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 msgr2=0x562a20f09d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f7659c700 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f09d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f7659c700 1 -- 172.21.15.27:0/1905612867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 msgr2=0x562a20f097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f7659c700 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20f097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f7659c700 1 -- 172.21.15.27:0/1905612867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a20e17700 con 0x562a20f00750 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f09d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f7659c700 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f0c4e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009990 tx=0x7f9f68009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f196c9e1feef348 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.081+0000 7f9f5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.082+0000 7f9f5dffb700 1 -- 172.21.15.27:0/1905612867 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68034060 con 0x562a20f00750 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.082+0000 7f9f5dffb700 1 -- 172.21.15.27:0/1905612867 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9f6800f660 con 0x562a20f00750 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.082+0000 7f9f87373b80 1 -- 172.21.15.27:0/1905612867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f70019070 con 0x562a20f00750 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.082+0000 7f9f5dffb700 1 -- 172.21.15.27:0/1905612867 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f6800f860 con 0x562a20f00750 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.082+0000 7f9f87373b80 10 client.15507 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:01.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.082+0000 7f9f87373b80 1 -- 172.21.15.27:0/1905612867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a20f178b0 con 0x562a20f00750 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.082+0000 7f9f5dffb700 1 -- 172.21.15.27:0/1905612867 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f70019070 con 0x562a20f00750 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.082+0000 7f9f5dffb700 1 -- 172.21.15.27:0/1905612867 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f6802e070 con 0x562a20f00750 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.083+0000 7f9f87373b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.083+0000 7f9f87373b80 10 client.15507 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.083+0000 7f9f87373b80 20 client.15507 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.083+0000 7f9f87373b80 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x562a20f19eb0 0x562a20f1b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.083+0000 7f9f87373b80 4 client.15507 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.083+0000 7f9f87373b80 1 -- 172.21.15.27:0/1905612867 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a20e5ed00 con 0x562a20f19eb0 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.083+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x562a20f19eb0 0x562a20f1b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.083+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x562a20f19eb0 0x562a20f1b320 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd4e45614623c20e server_cookie=7fa2702304585fee in_seq=0 out_seq=0 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.083+0000 7f9f5dffb700 10 client.15507 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:01.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f5dffb700 1 -- 172.21.15.27:0/1905612867 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a20e5ed00 con 0x562a20f19eb0 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f5dffb700 10 client.15507 handle_command_reply: tid=0 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 client.15507 shutdown 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 -- 172.21.15.27:0/1905612867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 msgr2=0x562a20f0c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f0c4e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009990 tx=0x7f9f68009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 -- 172.21.15.27:0/1905612867 shutdown_connections 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x562a20f19eb0 0x562a20f1b320 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f06830 0x562a20f09d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f5dffb700 0 client.15507 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f00750 0x562a20f0c4e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 --2- 172.21.15.27:0/1905612867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20f097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 -- 172.21.15.27:0/1905612867 >> 172.21.15.27:0/1905612867 conn(0x562a20e2d030 msgr2=0x562a20efa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.084+0000 7f9f87373b80 1 -- 172.21.15.27:0/1905612867 shutdown_connections 2024-06-06T08:41:01.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.085+0000 7f9f87373b80 1 -- 172.21.15.27:0/1905612867 wait complete. 2024-06-06T08:41:01.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.085+0000 7f9f87373b80 20 client.15507 trim_cache size 0 max 16384 2024-06-06T08:41:01.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.106+0000 7f9f87373b80 1 Processor -- start 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.106+0000 7f9f87373b80 1 -- start start 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.106+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 0x562a20f00210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.106+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f05f10 0x562a20f07320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.106+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f06830 0x562a20efae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.106+0000 7f9f87373b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562a20f19660 con 0x562a20f05f10 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.106+0000 7f9f87373b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a20f19d30 con 0x562a20f06830 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.106+0000 7f9f87373b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562a20f1a400 con 0x562a20f00750 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f75d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f05f10 0x562a20f07320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f75d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f05f10 0x562a20f07320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37864/0 (socket says 172.21.15.27:37864) 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f75d9b700 1 -- 172.21.15.27:0/79193742 learned_addr learned my addr 172.21.15.27:0/79193742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f7659c700 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 0x562a20f00210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f06830 0x562a20efae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f75d9b700 1 -- 172.21.15.27:0/79193742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 msgr2=0x562a20f00210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 0x562a20f00210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f75d9b700 1 -- 172.21.15.27:0/79193742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f06830 msgr2=0x562a20efae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f06830 0x562a20efae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f75d9b700 1 -- 172.21.15.27:0/79193742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a20f07d50 con 0x562a20f05f10 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f06830 0x562a20efae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f05f10 0x562a20f07320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c000c00 tx=0x7f9f6c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1683f9f40b417d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f5ffff700 1 -- 172.21.15.27:0/79193742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c09f5e0 con 0x562a20f05f10 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f5ffff700 1 -- 172.21.15.27:0/79193742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c0a3ea0 con 0x562a20f05f10 2024-06-06T08:41:01.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.107+0000 7f9f5ffff700 1 -- 172.21.15.27:0/79193742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0bb310 con 0x562a20f05f10 2024-06-06T08:41:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 -- 172.21.15.27:0/79193742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f05f10 msgr2=0x562a20f07320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f05f10 0x562a20f07320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c000c00 tx=0x7f9f6c013040 comp rx=0 tx=0).stop 2024-06-06T08:41:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 -- 172.21.15.27:0/79193742 shutdown_connections 2024-06-06T08:41:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f06830 0x562a20efae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 0x562a20f00210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 --2- 172.21.15.27:0/79193742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f05f10 0x562a20f07320 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 -- 172.21.15.27:0/79193742 >> 172.21.15.27:0/79193742 conn(0x562a20e2d030 msgr2=0x562a20f06c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 -- 172.21.15.27:0/79193742 shutdown_connections 2024-06-06T08:41:01.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 -- 172.21.15.27:0/79193742 wait complete. 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 -- start start 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.108+0000 7f9f87373b80 1 Processor -- start 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 0x562a20efa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20efadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f87373b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f06830 0x562a20efb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f87373b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562a20f1b1a0 con 0x562a20f06830 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f87373b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562a20f1aad0 con 0x562a20f05f10 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f87373b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562a20f1b870 con 0x562a20f00750 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f87373b80 10 client.0 _ll_register_callbacks cb 0x562a20de7560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f76d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f06830 0x562a20efb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f7659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 0x562a20efa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f76d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f06830 0x562a20efb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37878/0 (socket says 172.21.15.27:37878) 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f76d9d700 1 -- 172.21.15.27:0/872111153 learned_addr learned my addr 172.21.15.27:0/872111153 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:01.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20efadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f76d9d700 1 -- 172.21.15.27:0/872111153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 msgr2=0x562a20efa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 0x562a20efa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f76d9d700 1 -- 172.21.15.27:0/872111153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 msgr2=0x562a20efadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20efadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f76d9d700 1 -- 172.21.15.27:0/872111153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a20f23110 con 0x562a20f06830 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f75d9b700 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20efadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f76d9d700 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f06830 0x562a20efb300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9f700090b0 tx=0x7f9f70005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a2cdf738758bfe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.109+0000 7f9f5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f5dffb700 1 -- 172.21.15.27:0/872111153 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f700058a0 con 0x562a20f06830 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f5dffb700 1 -- 172.21.15.27:0/872111153 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9f7001e430 con 0x562a20f06830 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f87373b80 1 -- 172.21.15.27:0/872111153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f6c0a3ea0 con 0x562a20f06830 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f5dffb700 1 -- 172.21.15.27:0/872111153 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f70030050 con 0x562a20f06830 2024-06-06T08:41:01.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f87373b80 10 client.15513 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f87373b80 1 -- 172.21.15.27:0/872111153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a20f85b90 con 0x562a20f06830 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f5dffb700 1 -- 172.21.15.27:0/872111153 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c0a3ea0 con 0x562a20f06830 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f5dffb700 1 -- 172.21.15.27:0/872111153 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f70014410 con 0x562a20f06830 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f87373b80 10 client.15513 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f87373b80 10 client.15513 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f87373b80 20 client.15513 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f87373b80 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x562a20f19eb0 0x562a20f891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f87373b80 4 client.15513 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.110+0000 7f9f87373b80 1 -- 172.21.15.27:0/872111153 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "status"}) -- 0x562a20e59280 con 0x562a20f19eb0 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f7659c700 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x562a20f19eb0 0x562a20f891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f5dffb700 10 client.15513 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f7659c700 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x562a20f19eb0 0x562a20f891e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e70e2325061fd72 server_cookie=9fa592e5600a5624 in_seq=0 out_seq=0 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f5dffb700 1 -- 172.21.15.27:0/872111153 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x562a20e59280 con 0x562a20f19eb0 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f5dffb700 10 client.15513 handle_command_reply: tid=0 2024-06-06T08:41:01.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f87373b80 1 client.15513 shutdown 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f87373b80 1 -- 172.21.15.27:0/872111153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f06830 msgr2=0x562a20efb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f87373b80 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f06830 0x562a20efb300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9f700090b0 tx=0x7f9f70005270 comp rx=0 tx=0).stop 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f87373b80 1 -- 172.21.15.27:0/872111153 shutdown_connections 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.111+0000 7f9f87373b80 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x562a20f19eb0 0x562a20f891e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f87373b80 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562a20f06830 0x562a20efb300 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f5dffb700 0 client.15513 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f87373b80 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a20f00750 0x562a20efa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f87373b80 1 --2- 172.21.15.27:0/872111153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562a20f05f10 0x562a20efadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f87373b80 1 -- 172.21.15.27:0/872111153 >> 172.21.15.27:0/872111153 conn(0x562a20e2d030 msgr2=0x562a20f0b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f87373b80 1 -- 172.21.15.27:0/872111153 shutdown_connections 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f87373b80 1 -- 172.21.15.27:0/872111153 wait complete. 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f87373b80 20 client.15513 trim_cache size 0 max 16384 2024-06-06T08:41:01.116 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24458, 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 219.2076606, 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 25, 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 100, 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 99, 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 308.75681005799999 2024-06-06T08:41:01.117 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:41:01.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f600a2eb0 msgr2=0x7f9f600a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:01.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f600a2eb0 0x7f9f600a5370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800b5c0 tx=0x7f9f68005960 comp rx=0 tx=0).stop 2024-06-06T08:41:01.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78101d80 msgr2=0x7f9f781992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:01.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78101d80 0x7f9f781992f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00b530 tx=0x7f9f6c00b840 comp rx=0 tx=0).stop 2024-06-06T08:41:01.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1791424009 shutdown_connections 2024-06-06T08:41:01.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f78107430 0x7f9f7819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9f600a2eb0 0x7f9f600a5370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f781013c0 0x7f9f78198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 --2- 172.21.15.27:0/1791424009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f78101d80 0x7f9f781992f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:01.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.112+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1791424009 >> 172.21.15.27:0/1791424009 conn(0x7f9f780783d0 msgr2=0x7f9f781001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:01.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.113+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1791424009 shutdown_connections 2024-06-06T08:41:01.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:01.113+0000 7f9f7caf0700 1 -- 172.21.15.27:0/1791424009 wait complete. 2024-06-06T08:41:01.242 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:00 smithi203 ceph-mon[36579]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-06-06T08:41:01.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:00 smithi176 ceph-mon[36786]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-06-06T08:41:01.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T08:41:02.119 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:03.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:02 smithi027 ceph-mon[32335]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-06T08:41:03.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:02 smithi027 ceph-mon[32335]: mgrmap e25: x(active, since 44s), standbys: z, y 2024-06-06T08:41:03.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:02 smithi203 ceph-mon[36579]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-06T08:41:03.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:02 smithi203 ceph-mon[36579]: mgrmap e25: x(active, since 44s), standbys: z, y 2024-06-06T08:41:03.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:02 smithi176 ceph-mon[36786]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-06T08:41:03.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:02 smithi176 ceph-mon[36786]: mgrmap e25: x(active, since 44s), standbys: z, y 2024-06-06T08:41:03.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.946+0000 7f2506fc2700 1 -- 172.21.15.27:0/84972305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2500102740 msgr2=0x7f2500102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.946+0000 7f2506fc2700 1 --2- 172.21.15.27:0/84972305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2500102740 0x7f2500102b20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f24f40099c0 tx=0x7f24f4009cd0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.946+0000 7f2506fc2700 1 -- 172.21.15.27:0/84972305 shutdown_connections 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.946+0000 7f2506fc2700 1 --2- 172.21.15.27:0/84972305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2500107840 0x7f250010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.946+0000 7f2506fc2700 1 --2- 172.21.15.27:0/84972305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2500102740 0x7f2500102b20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.946+0000 7f2506fc2700 1 --2- 172.21.15.27:0/84972305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25001030f0 0x7f2500107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.946+0000 7f2506fc2700 1 -- 172.21.15.27:0/84972305 >> 172.21.15.27:0/84972305 conn(0x7f25000795f0 msgr2=0x7f2500079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.946+0000 7f2506fc2700 1 -- 172.21.15.27:0/84972305 shutdown_connections 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.946+0000 7f2506fc2700 1 -- 172.21.15.27:0/84972305 wait complete. 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.947+0000 7f2506fc2700 1 Processor -- start 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.947+0000 7f2506fc2700 1 -- start start 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.947+0000 7f2506fc2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2500102740 0x7f250019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.947+0000 7f2506fc2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25001030f0 0x7f250019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.947+0000 7f2506fc2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2500107840 0x7f2500198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.947+0000 7f2506fc2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2500112850 con 0x7f25001030f0 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.947+0000 7f2506fc2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f25001126d0 con 0x7f2500107840 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.947+0000 7f2506fc2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f25001129d0 con 0x7f2500102740 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.947+0000 7f24fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25001030f0 0x7f250019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25001030f0 0x7f250019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37900/0 (socket says 172.21.15.27:37900) 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fffff700 1 -- 172.21.15.27:0/828862104 learned_addr learned my addr 172.21.15.27:0/828862104 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f2504d5e700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2500102740 0x7f250019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f250555f700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2500107840 0x7f2500198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fffff700 1 -- 172.21.15.27:0/828862104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2500102740 msgr2=0x7f250019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fffff700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2500102740 0x7f250019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fffff700 1 -- 172.21.15.27:0/828862104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2500107840 msgr2=0x7f2500198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:03.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fffff700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2500107840 0x7f2500198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fffff700 1 -- 172.21.15.27:0/828862104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25001992b0 con 0x7f25001030f0 2024-06-06T08:41:03.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fffff700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25001030f0 0x7f250019ee80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00b530 tx=0x7f24ec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:03.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fdffb700 1 -- 172.21.15.27:0/828862104 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec015020 con 0x7f25001030f0 2024-06-06T08:41:03.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fdffb700 1 -- 172.21.15.27:0/828862104 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24ec011d10 con 0x7f25001030f0 2024-06-06T08:41:03.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f2506fc2700 1 -- 172.21.15.27:0/828862104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25001995a0 con 0x7f25001030f0 2024-06-06T08:41:03.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f24fdffb700 1 -- 172.21.15.27:0/828862104 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00f2e0 con 0x7f25001030f0 2024-06-06T08:41:03.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.948+0000 7f2506fc2700 1 -- 172.21.15.27:0/828862104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25001a5810 con 0x7f25001030f0 2024-06-06T08:41:03.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.951+0000 7f24fdffb700 1 -- 172.21.15.27:0/828862104 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f24ec018020 con 0x7f25001030f0 2024-06-06T08:41:03.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.952+0000 7f24fdffb700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24f00a2fc0 0x7f24f00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.952+0000 7f2504d5e700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24f00a2fc0 0x7f24f00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.952+0000 7f24fdffb700 1 -- 172.21.15.27:0/828862104 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f24ec0cf250 con 0x7f25001030f0 2024-06-06T08:41:03.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.952+0000 7f2504d5e700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24f00a2fc0 0x7f24f00a5480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24f4000c00 tx=0x7f24f4005fb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:03.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.952+0000 7f2511845b80 1 Processor -- start 2024-06-06T08:41:03.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.952+0000 7f2511845b80 1 -- start start 2024-06-06T08:41:03.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f2511845b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 0x55b718a5a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f2511845b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a54aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f2511845b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f2511845b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b718942f10 con 0x55b718a54fe0 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f2511845b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b7189a1800 con 0x55b718a54620 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f2511845b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b718909410 con 0x55b718a5a420 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f250555f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f250555f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37916/0 (socket says 172.21.15.27:37916) 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f250555f700 1 -- 172.21.15.27:0/235108157 learned_addr learned my addr 172.21.15.27:0/235108157 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f24fffff700 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a54aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f2504d5e700 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 0x55b718a5a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f24fffff700 1 -- 172.21.15.27:0/235108157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 msgr2=0x55b718a5a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f24fffff700 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 0x55b718a5a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f24fffff700 1 -- 172.21.15.27:0/235108157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 msgr2=0x55b718a611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f24fffff700 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f24fffff700 1 -- 172.21.15.27:0/235108157 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7188f0880 con 0x55b718a54620 2024-06-06T08:41:03.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.953+0000 7f2504d5e700 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 0x55b718a5a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:03.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.954+0000 7f24fffff700 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a54aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0cc4b0 tx=0x7f24ec097650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5feeb62d623b5aa7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:03.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.954+0000 7f24eaffd700 1 -- 172.21.15.27:0/235108157 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec09f630 con 0x55b718a54620 2024-06-06T08:41:03.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.954+0000 7f24eaffd700 1 -- 172.21.15.27:0/235108157 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24ec0a3ef0 con 0x55b718a54620 2024-06-06T08:41:03.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.954+0000 7f24eaffd700 1 -- 172.21.15.27:0/235108157 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec0a3a60 con 0x55b718a54620 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 -- 172.21.15.27:0/235108157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 msgr2=0x55b718a54aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a54aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0cc4b0 tx=0x7f24ec097650 comp rx=0 tx=0).stop 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 -- 172.21.15.27:0/235108157 shutdown_connections 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 0x55b718a5a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 --2- 172.21.15.27:0/235108157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a54aa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 -- 172.21.15.27:0/235108157 >> 172.21.15.27:0/235108157 conn(0x55b71897d030 msgr2=0x55b718a4b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 -- 172.21.15.27:0/235108157 shutdown_connections 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 -- 172.21.15.27:0/235108157 wait complete. 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 -- start start 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.955+0000 7f2511845b80 1 Processor -- start 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f2511845b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a60ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f2511845b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a5e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f2511845b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 0x55b718a5e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f2511845b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b718941f70 con 0x55b718a54fe0 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f2511845b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b71899d7c0 con 0x55b718a54620 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f2511845b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b718a61700 con 0x55b718a5a420 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f24fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a5e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f2511845b80 10 client.0 _ll_register_callbacks cb 0x55b718937560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f2504d5e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a60ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f24fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a5e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37918/0 (socket says 172.21.15.27:37918) 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f24fffff700 1 -- 172.21.15.27:0/926485009 learned_addr learned my addr 172.21.15.27:0/926485009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f250555f700 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 0x55b718a5e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f24fffff700 1 -- 172.21.15.27:0/926485009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 msgr2=0x55b718a5e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f24fffff700 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 0x55b718a5e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f24fffff700 1 -- 172.21.15.27:0/926485009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 msgr2=0x55b718a60ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f24fffff700 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a60ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f24fffff700 1 -- 172.21.15.27:0/926485009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b718967700 con 0x55b718a54fe0 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.956+0000 7f2504d5e700 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a60ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f24fffff700 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a5e1e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f24ec010a50 tx=0x7f24ec09b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddf99820e33181 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f24dffff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f24dffff700 1 -- 172.21.15.27:0/926485009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec0b3070 con 0x55b718a54fe0 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f24dffff700 1 -- 172.21.15.27:0/926485009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f24ec010610 con 0x55b718a54fe0 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f2511845b80 1 -- 172.21.15.27:0/926485009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24ec0a3ef0 con 0x55b718a54fe0 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f24dffff700 1 -- 172.21.15.27:0/926485009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24ec0b4260 con 0x55b718a54fe0 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f2511845b80 10 client.15525 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f2511845b80 1 -- 172.21.15.27:0/926485009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b718a6bdc0 con 0x55b718a54fe0 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f24dffff700 1 -- 172.21.15.27:0/926485009 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24ec0a3ef0 con 0x55b718a54fe0 2024-06-06T08:41:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f24dffff700 1 -- 172.21.15.27:0/926485009 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f24ec0bb0b0 con 0x55b718a54fe0 2024-06-06T08:41:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f2511845b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f2511845b80 10 client.15525 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T08:41:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.957+0000 7f2511845b80 20 client.15525 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.958+0000 7f2511845b80 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55b718a6d110 0x55b718a6f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.958+0000 7f2511845b80 4 client.15525 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.958+0000 7f2511845b80 1 -- 172.21.15.27:0/926485009 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7189aed00 con 0x55b718a6d110 2024-06-06T08:41:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.958+0000 7f2504d5e700 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55b718a6d110 0x55b718a6f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.958+0000 7f2504d5e700 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55b718a6d110 0x55b718a6f2d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d6c71333f47dc49 server_cookie=4fe69bae9a54a47f in_seq=0 out_seq=0 2024-06-06T08:41:03.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.958+0000 7f24dffff700 10 client.15525 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.959+0000 7f24dffff700 1 -- 172.21.15.27:0/926485009 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b7189aed00 con 0x55b718a6d110 2024-06-06T08:41:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.959+0000 7f24dffff700 10 client.15525 handle_command_reply: tid=0 2024-06-06T08:41:03.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.959+0000 7f2511845b80 1 client.15525 shutdown 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.959+0000 7f2511845b80 1 -- 172.21.15.27:0/926485009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 msgr2=0x55b718a5e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.959+0000 7f2511845b80 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a5e1e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f24ec010a50 tx=0x7f24ec09b340 comp rx=0 tx=0).stop 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.959+0000 7f2511845b80 1 -- 172.21.15.27:0/926485009 shutdown_connections 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.959+0000 7f2511845b80 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a5a420 0x55b718a5e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.960+0000 7f2511845b80 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55b718a6d110 0x55b718a6f2d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.960+0000 7f2511845b80 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a54620 0x55b718a60ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.960+0000 7f2511845b80 1 --2- 172.21.15.27:0/926485009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a5e1e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.960+0000 7f24dffff700 0 client.15525 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.960+0000 7f2511845b80 1 -- 172.21.15.27:0/926485009 >> 172.21.15.27:0/926485009 conn(0x55b71897d030 msgr2=0x55b718a4a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.960+0000 7f2511845b80 1 -- 172.21.15.27:0/926485009 shutdown_connections 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.960+0000 7f2511845b80 1 -- 172.21.15.27:0/926485009 wait complete. 2024-06-06T08:41:03.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.960+0000 7f2511845b80 20 client.15525 trim_cache size 0 max 16384 2024-06-06T08:41:03.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2511845b80 1 Processor -- start 2024-06-06T08:41:03.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2511845b80 1 -- start start 2024-06-06T08:41:03.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2511845b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54620 0x55b718a58520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2511845b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54fe0 0x55b718a60a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2511845b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a5a420 0x55b718a4ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2511845b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b718a6d610 con 0x55b718a54620 2024-06-06T08:41:03.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2511845b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b718a6dce0 con 0x55b718a5a420 2024-06-06T08:41:03.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2511845b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b718a6e3b0 con 0x55b718a54fe0 2024-06-06T08:41:03.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2504d5e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54620 0x55b718a58520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2504d5e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54620 0x55b718a58520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37930/0 (socket says 172.21.15.27:37930) 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.982+0000 7f2504d5e700 1 -- 172.21.15.27:0/2626592870 learned_addr learned my addr 172.21.15.27:0/2626592870 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f2504d5e700 1 -- 172.21.15.27:0/2626592870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54fe0 msgr2=0x55b718a60a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f24fffff700 1 --2- 172.21.15.27:0/2626592870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54fe0 0x55b718a60a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f2504d5e700 1 --2- 172.21.15.27:0/2626592870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54fe0 0x55b718a60a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f2504d5e700 1 -- 172.21.15.27:0/2626592870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a5a420 msgr2=0x55b718a4ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f2504d5e700 1 --2- 172.21.15.27:0/2626592870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a5a420 0x55b718a4ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f2504d5e700 1 -- 172.21.15.27:0/2626592870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b718a5c770 con 0x55b718a54620 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f24fffff700 1 --2- 172.21.15.27:0/2626592870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54fe0 0x55b718a60a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:03.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f2504d5e700 1 --2- 172.21.15.27:0/2626592870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54620 0x55b718a58520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009750 tx=0x7f24f4023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1366da0b3d82892 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:03.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f24ea7fc700 1 -- 172.21.15.27:0/2626592870 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f403c420 con 0x55b718a54620 2024-06-06T08:41:03.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f24ea7fc700 1 -- 172.21.15.27:0/2626592870 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24f40248f0 con 0x55b718a54620 2024-06-06T08:41:03.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.983+0000 7f24ea7fc700 1 -- 172.21.15.27:0/2626592870 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f40339d0 con 0x55b718a54620 2024-06-06T08:41:03.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 -- 172.21.15.27:0/2626592870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54620 msgr2=0x55b718a58520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:03.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 --2- 172.21.15.27:0/2626592870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54620 0x55b718a58520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009750 tx=0x7f24f4023000 comp rx=0 tx=0).stop 2024-06-06T08:41:03.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 -- 172.21.15.27:0/2626592870 shutdown_connections 2024-06-06T08:41:03.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 --2- 172.21.15.27:0/2626592870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a5a420 0x55b718a4ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 --2- 172.21.15.27:0/2626592870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54620 0x55b718a58520 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 --2- 172.21.15.27:0/2626592870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54fe0 0x55b718a60a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 -- 172.21.15.27:0/2626592870 >> 172.21.15.27:0/2626592870 conn(0x55b71897d030 msgr2=0x55b718a57e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 -- 172.21.15.27:0/2626592870 shutdown_connections 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 -- 172.21.15.27:0/2626592870 wait complete. 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 -- start start 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 Processor -- start 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.984+0000 7f2511845b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54620 0x55b718a72640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f2511845b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a72b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f2511845b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a5a420 0x55b718a730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f2511845b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b718a6f150 con 0x55b718a54fe0 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f2511845b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b718a6ea80 con 0x55b718a5a420 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f2511845b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b718a6f820 con 0x55b718a54620 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f2511845b80 10 client.0 _ll_register_callbacks cb 0x55b718937560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:03.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a72b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a72b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37932/0 (socket says 172.21.15.27:37932) 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24fffff700 1 -- 172.21.15.27:0/2744258813 learned_addr learned my addr 172.21.15.27:0/2744258813 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f2504d5e700 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54620 0x55b718a72640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24fffff700 1 -- 172.21.15.27:0/2744258813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54620 msgr2=0x55b718a72640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f250555f700 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a5a420 0x55b718a730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24fffff700 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54620 0x55b718a72640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24fffff700 1 -- 172.21.15.27:0/2744258813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a5a420 msgr2=0x55b718a730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24fffff700 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a5a420 0x55b718a730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24fffff700 1 -- 172.21.15.27:0/2744258813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b718a76ef0 con 0x55b718a54fe0 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24fffff700 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a72b80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0976e0 tx=0x7f24ec010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929b375d4e353b74 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24dffff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24dffff700 1 -- 172.21.15.27:0/2744258813 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec0b3070 con 0x55b718a54fe0 2024-06-06T08:41:03.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f24dffff700 1 -- 172.21.15.27:0/2744258813 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f24ec09b4e0 con 0x55b718a54fe0 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.985+0000 7f2511845b80 1 -- 172.21.15.27:0/2744258813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f40248f0 con 0x55b718a54fe0 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f24dffff700 1 -- 172.21.15.27:0/2744258813 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24ec097520 con 0x55b718a54fe0 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f2511845b80 10 client.15531 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f2511845b80 1 -- 172.21.15.27:0/2744258813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b718ad9a90 con 0x55b718a54fe0 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f24dffff700 1 -- 172.21.15.27:0/2744258813 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24f40248f0 con 0x55b718a54fe0 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f24dffff700 1 -- 172.21.15.27:0/2744258813 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f24ec09a6d0 con 0x55b718a54fe0 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f2511845b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f2511845b80 10 client.15531 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f2511845b80 20 client.15531 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f2511845b80 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55b718ade480 0x55b718add190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f2511845b80 4 client.15531 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f2511845b80 1 -- 172.21.15.27:0/2744258813 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "status"}) -- 0x55b7189a9280 con 0x55b718ade480 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.986+0000 7f2504d5e700 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55b718ade480 0x55b718add190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2504d5e700 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55b718ade480 0x55b718add190 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a213d90f01d21ec7 server_cookie=e29faaf236a4fa8c in_seq=0 out_seq=0 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f24dffff700 10 client.15531 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f24dffff700 1 -- 172.21.15.27:0/2744258813 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55b7189a9280 con 0x55b718ade480 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f24dffff700 10 client.15531 handle_command_reply: tid=0 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 client.15531 shutdown 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 -- 172.21.15.27:0/2744258813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 msgr2=0x55b718a72b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a72b80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0976e0 tx=0x7f24ec010980 comp rx=0 tx=0).stop 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 -- 172.21.15.27:0/2744258813 shutdown_connections 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55b718ade480 0x55b718add190 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b718a5a420 0x55b718a730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b718a54620 0x55b718a72640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 --2- 172.21.15.27:0/2744258813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b718a54fe0 0x55b718a72b80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f24dffff700 0 client.15531 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:03.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 -- 172.21.15.27:0/2744258813 >> 172.21.15.27:0/2744258813 conn(0x55b71897d030 msgr2=0x55b718a586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:03.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.987+0000 7f2511845b80 1 -- 172.21.15.27:0/2744258813 shutdown_connections 2024-06-06T08:41:03.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2511845b80 1 -- 172.21.15.27:0/2744258813 wait complete. 2024-06-06T08:41:03.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2511845b80 20 client.15531 trim_cache size 0 max 16384 2024-06-06T08:41:03.991 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:41:03.991 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:41:03.991 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-06-06T08:41:03.991 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24460, 2024-06-06T08:41:03.991 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:41:03.992 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:41:03.992 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:41:03.992 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 223.085922982, 2024-06-06T08:41:03.992 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 25, 2024-06-06T08:41:03.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 100, 2024-06-06T08:41:03.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 99, 2024-06-06T08:41:03.992 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 308.82312647499998 2024-06-06T08:41:03.992 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 -- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24f00a2fc0 msgr2=0x7f24f00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24f00a2fc0 0x7f24f00a5480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24f4000c00 tx=0x7f24f4005fb0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 -- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25001030f0 msgr2=0x7f250019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25001030f0 0x7f250019ee80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00b530 tx=0x7f24ec00b840 comp rx=0 tx=0).stop 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 -- 172.21.15.27:0/828862104 shutdown_connections 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24f00a2fc0 0x7f24f00a5480 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2500107840 0x7f2500198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2500102740 0x7f250019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 --2- 172.21.15.27:0/828862104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25001030f0 0x7f250019ee80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 -- 172.21.15.27:0/828862104 >> 172.21.15.27:0/828862104 conn(0x7f25000795f0 msgr2=0x7f25001007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:03.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 -- 172.21.15.27:0/828862104 shutdown_connections 2024-06-06T08:41:03.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:03.988+0000 7f2506fc2700 1 -- 172.21.15.27:0/828862104 wait complete. 2024-06-06T08:41:04.809 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T08:41:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:05.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:04 smithi027 ceph-mon[32335]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-06T08:41:05.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:04 smithi203 ceph-mon[36579]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-06T08:41:05.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:04 smithi176 ceph-mon[36786]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-06T08:41:06.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.797+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1721198173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f8101610 msgr2=0x7f63f81019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:06.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.797+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1721198173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f8101610 0x7f63f81019f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f63e80099c0 tx=0x7f63e8009cd0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.797+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1721198173 shutdown_connections 2024-06-06T08:41:06.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.797+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1721198173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f81066f0 0x7f63f810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.797+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1721198173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f8101610 0x7f63f81019f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.797+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1721198173 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63f8101fc0 0x7f63f8106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.797+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1721198173 >> 172.21.15.27:0/1721198173 conn(0x7f63f8078430 msgr2=0x7f63f8078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.797+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1721198173 shutdown_connections 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.797+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1721198173 wait complete. 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fe2b4700 1 Processor -- start 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fe2b4700 1 -- start start 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fe2b4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63f8101610 0x7f63f8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fe2b4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8101fc0 0x7f63f8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fe2b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f81066f0 0x7f63f81a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fe2b4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81116d0 con 0x7f63f81066f0 2024-06-06T08:41:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fe2b4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8111550 con 0x7f63f8101610 2024-06-06T08:41:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fe2b4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8111850 con 0x7f63f8101fc0 2024-06-06T08:41:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fc851700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f81066f0 0x7f63f81a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fc851700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f81066f0 0x7f63f81a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37956/0 (socket says 172.21.15.27:37956) 2024-06-06T08:41:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63fc851700 1 -- 172.21.15.27:0/1679182503 learned_addr learned my addr 172.21.15.27:0/1679182503 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63f7fff700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63f8101610 0x7f63f8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.798+0000 7f63effff700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8101fc0 0x7f63f8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63fc851700 1 -- 172.21.15.27:0/1679182503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8101fc0 msgr2=0x7f63f8199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63fc851700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8101fc0 0x7f63f8199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63fc851700 1 -- 172.21.15.27:0/1679182503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63f8101610 msgr2=0x7f63f8199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63fc851700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63f8101610 0x7f63f8199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63fc851700 1 -- 172.21.15.27:0/1679182503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f81a5070 con 0x7f63f81066f0 2024-06-06T08:41:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63fc851700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f81066f0 0x7f63f81a4b30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f63f000ada0 tx=0x7f63f000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63f5ffb700 1 -- 172.21.15.27:0/1679182503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0017020 con 0x7f63f81066f0 2024-06-06T08:41:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63f5ffb700 1 -- 172.21.15.27:0/1679182503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f000cdb0 con 0x7f63f81066f0 2024-06-06T08:41:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63f5ffb700 1 -- 172.21.15.27:0/1679182503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f00053e0 con 0x7f63f81066f0 2024-06-06T08:41:06.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1679182503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f81a5240 con 0x7f63f81066f0 2024-06-06T08:41:06.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.799+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1679182503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f81a5750 con 0x7f63f81066f0 2024-06-06T08:41:06.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.801+0000 7f63f5ffb700 1 -- 172.21.15.27:0/1679182503 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f63f000c8d0 con 0x7f63f81066f0 2024-06-06T08:41:06.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.802+0000 7f63f5ffb700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63e40a2f40 0x7f63e40a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.802+0000 7f63f7fff700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63e40a2f40 0x7f63e40a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.802+0000 7f63f5ffb700 1 -- 172.21.15.27:0/1679182503 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f63f00cf140 con 0x7f63f81066f0 2024-06-06T08:41:06.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f63f7fff700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63e40a2f40 0x7f63e40a5400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009fd0 tx=0x7f63e8005fb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:06.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f6408b37b80 1 Processor -- start 2024-06-06T08:41:06.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f6408b37b80 1 -- start start 2024-06-06T08:41:06.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x5564355ea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355eaf10 0x5564355e4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355e4ed0 0x5564355e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f6408b37b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564355367c0 con 0x5564355eaf10 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f6408b37b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5564354de6d0 con 0x5564355e4ed0 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f6408b37b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556435532780 con 0x5564355ea5f0 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f63effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355eaf10 0x5564355e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.803+0000 7f63effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355eaf10 0x5564355e4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37970/0 (socket says 172.21.15.27:37970) 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63effff700 1 -- 172.21.15.27:0/12343292 learned_addr learned my addr 172.21.15.27:0/12343292 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63f7fff700 1 --2- 172.21.15.27:0/12343292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x5564355ea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63fc851700 1 --2- 172.21.15.27:0/12343292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355e4ed0 0x5564355e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63effff700 1 -- 172.21.15.27:0/12343292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 msgr2=0x5564355ea9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63effff700 1 --2- 172.21.15.27:0/12343292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x5564355ea9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63effff700 1 -- 172.21.15.27:0/12343292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355e4ed0 msgr2=0x5564355e52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63effff700 1 --2- 172.21.15.27:0/12343292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355e4ed0 0x5564355e52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63effff700 1 -- 172.21.15.27:0/12343292 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564354d4f00 con 0x5564355eaf10 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63effff700 1 --2- 172.21.15.27:0/12343292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355eaf10 0x5564355e4990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f63e000aca0 tx=0x7f63e000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9feebbcc651e5e35 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63ee7fc700 1 -- 172.21.15.27:0/12343292 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e000fe50 con 0x5564355eaf10 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63ee7fc700 1 -- 172.21.15.27:0/12343292 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63e0004510 con 0x5564355eaf10 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.804+0000 7f63ee7fc700 1 -- 172.21.15.27:0/12343292 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0018690 con 0x5564355eaf10 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 -- 172.21.15.27:0/12343292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355eaf10 msgr2=0x5564355e4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 --2- 172.21.15.27:0/12343292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355eaf10 0x5564355e4990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f63e000aca0 tx=0x7f63e000f3c0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 -- 172.21.15.27:0/12343292 shutdown_connections 2024-06-06T08:41:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 --2- 172.21.15.27:0/12343292 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355e4ed0 0x5564355e52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 --2- 172.21.15.27:0/12343292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x5564355ea9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 --2- 172.21.15.27:0/12343292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355eaf10 0x5564355e4990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 -- 172.21.15.27:0/12343292 >> 172.21.15.27:0/12343292 conn(0x55643550fc70 msgr2=0x5564355df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 -- 172.21.15.27:0/12343292 shutdown_connections 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 -- 172.21.15.27:0/12343292 wait complete. 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.805+0000 7f6408b37b80 1 -- start start 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f6408b37b80 1 Processor -- start 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x5564355edf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x5564355ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f6408b37b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564355f1980 con 0x5564355e4ed0 2024-06-06T08:41:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f6408b37b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5564354d6ba0 con 0x5564355eaf10 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f6408b37b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5564355f1b00 con 0x5564355ea5f0 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f6408b37b80 10 client.0 _ll_register_callbacks cb 0x5564354d08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63f7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63f7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355f0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37980/0 (socket says 172.21.15.27:37980) 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63f7fff700 1 -- 172.21.15.27:0/2852200759 learned_addr learned my addr 172.21.15.27:0/2852200759 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63fc851700 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x5564355ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63effff700 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x5564355edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63fc851700 1 -- 172.21.15.27:0/2852200759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 msgr2=0x5564355edf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63fc851700 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x5564355edf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63fc851700 1 -- 172.21.15.27:0/2852200759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 msgr2=0x5564355f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63fc851700 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355f0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63fc851700 1 -- 172.21.15.27:0/2852200759 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556435485880 con 0x5564355eaf10 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.806+0000 7f63f7fff700 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.807+0000 7f63fc851700 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x5564355ee4a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63f00cc290 tx=0x7f63f000a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df929dc8a722ecec server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.807+0000 7f63d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.807+0000 7f63d77fe700 1 -- 172.21.15.27:0/2852200759 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f00b4060 con 0x5564355eaf10 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.807+0000 7f63d77fe700 1 -- 172.21.15.27:0/2852200759 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f63f009b350 con 0x5564355eaf10 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.807+0000 7f6408b37b80 1 -- 172.21.15.27:0/2852200759 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e0004510 con 0x5564355eaf10 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.807+0000 7f63d77fe700 1 -- 172.21.15.27:0/2852200759 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f63f009b7f0 con 0x5564355eaf10 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.807+0000 7f6408b37b80 10 client.25178 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.807+0000 7f6408b37b80 1 -- 172.21.15.27:0/2852200759 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564355fc2c0 con 0x5564355eaf10 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.807+0000 7f63d77fe700 1 -- 172.21.15.27:0/2852200759 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63e0004510 con 0x5564355eaf10 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f63d77fe700 1 -- 172.21.15.27:0/2852200759 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f63f0012070 con 0x5564355eaf10 2024-06-06T08:41:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f6408b37b80 10 client.25178 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f6408b37b80 10 client.25178 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f6408b37b80 20 client.25178 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f6408b37b80 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5564355fe860 0x5564355ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f6408b37b80 4 client.25178 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f6408b37b80 1 -- 172.21.15.27:0/2852200759 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556435543cc0 con 0x5564355fe860 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f63f7fff700 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5564355fe860 0x5564355ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f63d77fe700 10 client.25178 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.808+0000 7f63f7fff700 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5564355fe860 0x5564355ffcd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ec8dd704faf7f server_cookie=7c5b9b60d2ff54d8 in_seq=0 out_seq=0 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.809+0000 7f63d77fe700 1 -- 172.21.15.27:0/2852200759 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556435543cc0 con 0x5564355fe860 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.809+0000 7f63d77fe700 10 client.25178 handle_command_reply: tid=0 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.809+0000 7f6408b37b80 1 client.25178 shutdown 2024-06-06T08:41:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.809+0000 7f6408b37b80 1 -- 172.21.15.27:0/2852200759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 msgr2=0x5564355ee4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:06.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.809+0000 7f6408b37b80 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x5564355ee4a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63f00cc290 tx=0x7f63f000a1f0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.809+0000 7f6408b37b80 1 -- 172.21.15.27:0/2852200759 shutdown_connections 2024-06-06T08:41:06.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.809+0000 7f6408b37b80 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x5564355ee4a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.810+0000 7f6408b37b80 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5564355fe860 0x5564355ffcd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.810+0000 7f6408b37b80 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.810+0000 7f6408b37b80 1 --2- 172.21.15.27:0/2852200759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x5564355edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.810+0000 7f63d77fe700 0 client.25178 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:06.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.810+0000 7f6408b37b80 1 -- 172.21.15.27:0/2852200759 >> 172.21.15.27:0/2852200759 conn(0x55643550fc70 msgr2=0x5564355e3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:06.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.810+0000 7f6408b37b80 1 -- 172.21.15.27:0/2852200759 shutdown_connections 2024-06-06T08:41:06.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.810+0000 7f6408b37b80 1 -- 172.21.15.27:0/2852200759 wait complete. 2024-06-06T08:41:06.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.810+0000 7f6408b37b80 20 client.25178 trim_cache size 0 max 16384 2024-06-06T08:41:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.832+0000 7f6408b37b80 1 Processor -- start 2024-06-06T08:41:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.832+0000 7f6408b37b80 1 -- start start 2024-06-06T08:41:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355e4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x556435609130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x556435609670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f6408b37b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564355fe010 con 0x5564355e4ed0 2024-06-06T08:41:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f6408b37b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5564355fe6e0 con 0x5564355eaf10 2024-06-06T08:41:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f6408b37b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5564355fedb0 con 0x5564355ea5f0 2024-06-06T08:41:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63f7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63f7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355e4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37992/0 (socket says 172.21.15.27:37992) 2024-06-06T08:41:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63f7fff700 1 -- 172.21.15.27:0/434430253 learned_addr learned my addr 172.21.15.27:0/434430253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63fc851700 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x556435609670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63effff700 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x556435609130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63f7fff700 1 -- 172.21.15.27:0/434430253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 msgr2=0x556435609130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:06.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63f7fff700 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x556435609130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63f7fff700 1 -- 172.21.15.27:0/434430253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 msgr2=0x556435609670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:06.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63f7fff700 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x556435609670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63f7fff700 1 -- 172.21.15.27:0/434430253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564355e5620 con 0x5564355e4ed0 2024-06-06T08:41:06.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63fc851700 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x556435609670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:06.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63effff700 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x556435609130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:06.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63f7fff700 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355e4990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f63e8000c00 tx=0x7f63e8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f07c6cdf3022f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:06.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.833+0000 7f63edffb700 1 -- 172.21.15.27:0/434430253 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e800f460 con 0x5564355e4ed0 2024-06-06T08:41:06.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f63edffb700 1 -- 172.21.15.27:0/434430253 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63e800fae0 con 0x5564355e4ed0 2024-06-06T08:41:06.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f63edffb700 1 -- 172.21.15.27:0/434430253 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e803c680 con 0x5564355e4ed0 2024-06-06T08:41:06.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f6408b37b80 1 -- 172.21.15.27:0/434430253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 msgr2=0x5564355e4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:06.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f6408b37b80 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355e4990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f63e8000c00 tx=0x7f63e8020000 comp rx=0 tx=0).stop 2024-06-06T08:41:06.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f6408b37b80 1 -- 172.21.15.27:0/434430253 shutdown_connections 2024-06-06T08:41:06.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f6408b37b80 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x556435609670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f6408b37b80 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355e4ed0 0x5564355e4990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f6408b37b80 1 --2- 172.21.15.27:0/434430253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355ea5f0 0x556435609130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f6408b37b80 1 -- 172.21.15.27:0/434430253 >> 172.21.15.27:0/434430253 conn(0x55643550fc70 msgr2=0x5564355ec890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:06.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f6408b37b80 1 -- 172.21.15.27:0/434430253 shutdown_connections 2024-06-06T08:41:06.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.834+0000 7f6408b37b80 1 -- 172.21.15.27:0/434430253 wait complete. 2024-06-06T08:41:06.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f6408b37b80 1 -- start start 2024-06-06T08:41:06.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f6408b37b80 1 Processor -- start 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355e4ed0 0x556435609150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355ea5f0 0x556435609690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f6408b37b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x556435604100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f6408b37b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564355ffb50 con 0x5564355ea5f0 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f6408b37b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5564355ff480 con 0x5564355eaf10 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f6408b37b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556435600220 con 0x5564355e4ed0 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f6408b37b80 10 client.0 _ll_register_callbacks cb 0x5564354d08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f63effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355ea5f0 0x556435609690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f63effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355ea5f0 0x556435609690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38006/0 (socket says 172.21.15.27:38006) 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f63effff700 1 -- 172.21.15.27:0/1671008914 learned_addr learned my addr 172.21.15.27:0/1671008914 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:06.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.835+0000 7f63fc851700 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x556435604100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63f7fff700 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355e4ed0 0x556435609150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63effff700 1 -- 172.21.15.27:0/1671008914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355e4ed0 msgr2=0x556435609150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63effff700 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355e4ed0 0x556435609150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63effff700 1 -- 172.21.15.27:0/1671008914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 msgr2=0x556435604100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63effff700 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x556435604100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63effff700 1 -- 172.21.15.27:0/1671008914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564355e4be0 con 0x5564355ea5f0 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63effff700 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355ea5f0 0x556435609690 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f63e00046b0 tx=0x7f63e0004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e678a310da5452cf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63d77fe700 1 -- 172.21.15.27:0/1671008914 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0027070 con 0x5564355ea5f0 2024-06-06T08:41:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63d77fe700 1 -- 172.21.15.27:0/1671008914 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f63e0005040 con 0x5564355ea5f0 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f6408b37b80 1 -- 172.21.15.27:0/1671008914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e800fae0 con 0x5564355ea5f0 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63d77fe700 1 -- 172.21.15.27:0/1671008914 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f63e0035050 con 0x5564355ea5f0 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f6408b37b80 10 client.15549 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f6408b37b80 1 -- 172.21.15.27:0/1671008914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55643566a5c0 con 0x5564355ea5f0 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.836+0000 7f63d77fe700 1 -- 172.21.15.27:0/1671008914 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63e800fae0 con 0x5564355ea5f0 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f63d77fe700 1 -- 172.21.15.27:0/1671008914 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f63e002cb30 con 0x5564355ea5f0 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f6408b37b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f6408b37b80 10 client.15549 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f6408b37b80 20 client.15549 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f6408b37b80 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55643566ee60 0x55643566db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f6408b37b80 4 client.15549 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f6408b37b80 1 -- 172.21.15.27:0/1671008914 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "status"}) -- 0x55643553e240 con 0x55643566ee60 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f63f7fff700 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55643566ee60 0x55643566db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f63f7fff700 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55643566ee60 0x55643566db70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745e9bc70d119c22 server_cookie=382538c746d13d0f in_seq=0 out_seq=0 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.837+0000 7f63d77fe700 10 client.15549 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f63d77fe700 1 -- 172.21.15.27:0/1671008914 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55643553e240 con 0x55643566ee60 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f63d77fe700 10 client.15549 handle_command_reply: tid=0 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 client.15549 shutdown 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 -- 172.21.15.27:0/1671008914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355ea5f0 msgr2=0x556435609690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355ea5f0 0x556435609690 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f63e00046b0 tx=0x7f63e0004940 comp rx=0 tx=0).stop 2024-06-06T08:41:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 -- 172.21.15.27:0/1671008914 shutdown_connections 2024-06-06T08:41:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5564355eaf10 0x556435604100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564355e4ed0 0x556435609150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55643566ee60 0x55643566db70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f63d77fe700 0 client.15549 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 --2- 172.21.15.27:0/1671008914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564355ea5f0 0x556435609690 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 -- 172.21.15.27:0/1671008914 >> 172.21.15.27:0/1671008914 conn(0x55643550fc70 msgr2=0x5564355d8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 -- 172.21.15.27:0/1671008914 shutdown_connections 2024-06-06T08:41:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 1 -- 172.21.15.27:0/1671008914 wait complete. 2024-06-06T08:41:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.838+0000 7f6408b37b80 20 client.15549 trim_cache size 0 max 16384 2024-06-06T08:41:06.846 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:41:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "8da40ada-23de-11ef-bc9b-c7b262605968", 2024-06-06T08:41:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-06-06T08:41:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24482, 2024-06-06T08:41:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-06-06T08:41:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-06-06T08:41:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-06-06T08:41:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 282.93896376700002, 2024-06-06T08:41:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 25, 2024-06-06T08:41:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 100, 2024-06-06T08:41:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 99, 2024-06-06T08:41:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 301.541888384 2024-06-06T08:41:06.847 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:41:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63e40a2f40 msgr2=0x7f63e40a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63e40a2f40 0x7f63e40a5400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009fd0 tx=0x7f63e8005fb0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f81066f0 msgr2=0x7f63f81a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f81066f0 0x7f63f81a4b30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f63f000ada0 tx=0x7f63f000c330 comp rx=0 tx=0).stop 2024-06-06T08:41:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1679182503 shutdown_connections 2024-06-06T08:41:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f81066f0 0x7f63f81a4b30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63e40a2f40 0x7f63e40a5400 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63f8101610 0x7f63f8199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 --2- 172.21.15.27:0/1679182503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f8101fc0 0x7f63f8199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:06.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1679182503 >> 172.21.15.27:0/1679182503 conn(0x7f63f8078430 msgr2=0x7f63f80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:06.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1679182503 shutdown_connections 2024-06-06T08:41:06.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:06.839+0000 7f63fe2b4700 1 -- 172.21.15.27:0/1679182503 wait complete. 2024-06-06T08:41:07.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:06 smithi027 ceph-mon[32335]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:41:07.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:06 smithi203 ceph-mon[36579]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:41:07.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:06 smithi176 ceph-mon[36786]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:41:07.669 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-06-06T08:41:08.587 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T08:41:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:09.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:08 smithi027 ceph-mon[32335]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:41:09.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:08 smithi203 ceph-mon[36579]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:41:09.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:08 smithi176 ceph-mon[36786]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:41:10.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.644+0000 7f799ae12700 1 -- 172.21.15.27:0/1853077491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79941066b0 msgr2=0x7f799410e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.644+0000 7f799ae12700 1 --2- 172.21.15.27:0/1853077491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79941066b0 0x7f799410e280 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f798c00b0d0 tx=0x7f798c00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.644+0000 7f799ae12700 1 -- 172.21.15.27:0/1853077491 shutdown_connections 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.644+0000 7f799ae12700 1 --2- 172.21.15.27:0/1853077491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79941066b0 0x7f799410e280 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.644+0000 7f799ae12700 1 --2- 172.21.15.27:0/1853077491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79941015d0 0x7f79941019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.644+0000 7f799ae12700 1 --2- 172.21.15.27:0/1853077491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7994101f80 0x7f7994105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.644+0000 7f799ae12700 1 -- 172.21.15.27:0/1853077491 >> 172.21.15.27:0/1853077491 conn(0x7f7994078400 msgr2=0x7f7994078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.644+0000 7f799ae12700 1 -- 172.21.15.27:0/1853077491 shutdown_connections 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.644+0000 7f799ae12700 1 -- 172.21.15.27:0/1853077491 wait complete. 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f799ae12700 1 Processor -- start 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f799ae12700 1 -- start start 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f799ae12700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79941015d0 0x7f799419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f799ae12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7994101f80 0x7f799419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f799ae12700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79941066b0 0x7f7994198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f799ae12700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7994111710 con 0x7f7994101f80 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f799ae12700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7994111590 con 0x7f79941066b0 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f799ae12700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7994111890 con 0x7f79941015d0 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f7993fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7994101f80 0x7f799419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f7993fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7994101f80 0x7f799419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33600/0 (socket says 172.21.15.27:33600) 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f7993fff700 1 -- 172.21.15.27:0/1127878593 learned_addr learned my addr 172.21.15.27:0/1127878593 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f79993af700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79941066b0 0x7f7994198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.645+0000 7f7998bae700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79941015d0 0x7f799419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7993fff700 1 -- 172.21.15.27:0/1127878593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79941015d0 msgr2=0x7f799419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7993fff700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79941015d0 0x7f799419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7993fff700 1 -- 172.21.15.27:0/1127878593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79941066b0 msgr2=0x7f7994198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7993fff700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79941066b0 0x7f7994198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7993fff700 1 -- 172.21.15.27:0/1127878593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7994199160 con 0x7f7994101f80 2024-06-06T08:41:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7998bae700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79941015d0 0x7f799419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7993fff700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7994101f80 0x7f799419ed80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f798000c9e0 tx=0x7f798000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7991ffb700 1 -- 172.21.15.27:0/1127878593 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980007aa0 con 0x7f7994101f80 2024-06-06T08:41:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7991ffb700 1 -- 172.21.15.27:0/1127878593 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7980007c40 con 0x7f7994101f80 2024-06-06T08:41:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f799ae12700 1 -- 172.21.15.27:0/1127878593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7994199450 con 0x7f7994101f80 2024-06-06T08:41:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f7991ffb700 1 -- 172.21.15.27:0/1127878593 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980018640 con 0x7f7994101f80 2024-06-06T08:41:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.646+0000 7f799ae12700 1 -- 172.21.15.27:0/1127878593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7994102fa0 con 0x7f7994101f80 2024-06-06T08:41:10.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.648+0000 7f7991ffb700 1 -- 172.21.15.27:0/1127878593 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f798000f450 con 0x7f7994101f80 2024-06-06T08:41:10.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.649+0000 7f7991ffb700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79840a2f40 0x7f79840a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.649+0000 7f7991ffb700 1 -- 172.21.15.27:0/1127878593 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f79800cfa70 con 0x7f7994101f80 2024-06-06T08:41:10.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.649+0000 7f7998bae700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79840a2f40 0x7f79840a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f7998bae700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79840a2f40 0x7f79840a5400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7994199b40 tx=0x7f7988008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79a5695b80 1 Processor -- start 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79a5695b80 1 -- start start 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda736a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda7309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda731340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79a5695b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564eda682810 con 0x564eda730f20 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79a5695b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564eda62a720 con 0x564eda736640 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79a5695b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564eda67e7d0 con 0x564eda736f60 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79993af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda731340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79993af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda731340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33604/0 (socket says 172.21.15.27:33604) 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.650+0000 7f79993af700 1 -- 172.21.15.27:0/3991755478 learned_addr learned my addr 172.21.15.27:0/3991755478 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f7993fff700 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda7309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f7998bae700 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda736a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f79993af700 1 -- 172.21.15.27:0/3991755478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 msgr2=0x564eda7309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f79993af700 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda7309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f79993af700 1 -- 172.21.15.27:0/3991755478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 msgr2=0x564eda736a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f79993af700 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda736a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f79993af700 1 -- 172.21.15.27:0/3991755478 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eda620f50 con 0x564eda730f20 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f7993fff700 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda7309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f79993af700 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda731340 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f798c00cfe0 tx=0x7f798c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef799807f1f5d217 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f797effd700 1 -- 172.21.15.27:0/3991755478 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c020070 con 0x564eda730f20 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f797effd700 1 -- 172.21.15.27:0/3991755478 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f798c0074e0 con 0x564eda730f20 2024-06-06T08:41:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.651+0000 7f797effd700 1 -- 172.21.15.27:0/3991755478 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0052a0 con 0x564eda730f20 2024-06-06T08:41:10.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 -- 172.21.15.27:0/3991755478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 msgr2=0x564eda731340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:10.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda731340 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f798c00cfe0 tx=0x7f798c009e20 comp rx=0 tx=0).stop 2024-06-06T08:41:10.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 -- 172.21.15.27:0/3991755478 shutdown_connections 2024-06-06T08:41:10.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda731340 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda736a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 --2- 172.21.15.27:0/3991755478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda7309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 -- 172.21.15.27:0/3991755478 >> 172.21.15.27:0/3991755478 conn(0x564eda65bcc0 msgr2=0x564eda72b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:10.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 -- 172.21.15.27:0/3991755478 shutdown_connections 2024-06-06T08:41:10.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 -- 172.21.15.27:0/3991755478 wait complete. 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 -- start start 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.652+0000 7f79a5695b80 1 Processor -- start 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda730f20 0x564eda73ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda739fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda736f60 0x564eda73a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79a5695b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564eda73d970 con 0x564eda736f60 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79a5695b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564eda622bf0 con 0x564eda736640 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79a5695b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564eda73daf0 con 0x564eda730f20 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79a5695b80 10 client.0 _ll_register_callbacks cb 0x564eda61c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79993af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda736f60 0x564eda73a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79993af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda736f60 0x564eda73a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33612/0 (socket says 172.21.15.27:33612) 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79993af700 1 -- 172.21.15.27:0/2480924085 learned_addr learned my addr 172.21.15.27:0/2480924085 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f7993fff700 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda739fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f7998bae700 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda730f20 0x564eda73ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79993af700 1 -- 172.21.15.27:0/2480924085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda730f20 msgr2=0x564eda73ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79993af700 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda730f20 0x564eda73ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79993af700 1 -- 172.21.15.27:0/2480924085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 msgr2=0x564eda739fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79993af700 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda739fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.653+0000 7f79993af700 1 -- 172.21.15.27:0/2480924085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eda5d18d0 con 0x564eda736f60 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79993af700 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda736f60 0x564eda73a510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f798c0078c0 tx=0x7f798c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a30b483de8b5c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f7973fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f7973fff700 1 -- 172.21.15.27:0/2480924085 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c02c070 con 0x564eda736f60 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79a5695b80 1 -- 172.21.15.27:0/2480924085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f798c0074e0 con 0x564eda736f60 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f7973fff700 1 -- 172.21.15.27:0/2480924085 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f798c003c10 con 0x564eda736f60 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f7973fff700 1 -- 172.21.15.27:0/2480924085 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f798c037050 con 0x564eda736f60 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79a5695b80 10 client.15561 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79a5695b80 1 -- 172.21.15.27:0/2480924085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564eda748270 con 0x564eda736f60 2024-06-06T08:41:10.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f7973fff700 1 -- 172.21.15.27:0/2480924085 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f798c0074e0 con 0x564eda736f60 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f7973fff700 1 -- 172.21.15.27:0/2480924085 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f798c011070 con 0x564eda736f60 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79a5695b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79a5695b80 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79a5695b80 20 client.15561 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564eda74a740 0x564eda74cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79a5695b80 4 client.15561 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.654+0000 7f79a5695b80 1 -- 172.21.15.27:0/2480924085 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564eda68fd10 con 0x564eda74a740 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.655+0000 7f7998bae700 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564eda74a740 0x564eda74cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.655+0000 7f7998bae700 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564eda74a740 0x564eda74cb30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b740b6cf9643cfa4 server_cookie=4b73d39114bfaee8 in_seq=0 out_seq=0 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.655+0000 7f7973fff700 10 client.15561 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f7973fff700 1 -- 172.21.15.27:0/2480924085 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564eda68fd10 con 0x564eda74a740 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f7973fff700 10 client.15561 handle_command_reply: tid=0 2024-06-06T08:41:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f79a5695b80 1 client.15561 shutdown 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f79a5695b80 1 -- 172.21.15.27:0/2480924085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda736f60 msgr2=0x564eda73a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda736f60 0x564eda73a510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f798c0078c0 tx=0x7f798c0079a0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f79a5695b80 1 -- 172.21.15.27:0/2480924085 shutdown_connections 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564eda74a740 0x564eda74cb30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda736f60 0x564eda73a510 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f7973fff700 0 client.15561 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda730f20 0x564eda73ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2480924085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda739fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.656+0000 7f79a5695b80 1 -- 172.21.15.27:0/2480924085 >> 172.21.15.27:0/2480924085 conn(0x564eda65bcc0 msgr2=0x564eda72f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.657+0000 7f79a5695b80 1 -- 172.21.15.27:0/2480924085 shutdown_connections 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.657+0000 7f79a5695b80 1 -- 172.21.15.27:0/2480924085 wait complete. 2024-06-06T08:41:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.657+0000 7f79a5695b80 20 client.15561 trim_cache size 0 max 16384 2024-06-06T08:41:10.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.677+0000 7f79a5695b80 1 Processor -- start 2024-06-06T08:41:10.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.677+0000 7f79a5695b80 1 -- start start 2024-06-06T08:41:10.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda7309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736640 0x564eda737a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736f60 0x564eda724e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f79a5695b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79941b2f00 con 0x564eda730f20 2024-06-06T08:41:10.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f79a5695b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f79941b0790 con 0x564eda736f60 2024-06-06T08:41:10.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f79a5695b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79941ae5e0 con 0x564eda736640 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7998bae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda7309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7998bae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda7309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33624/0 (socket says 172.21.15.27:33624) 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7998bae700 1 -- 172.21.15.27:0/2312050385 learned_addr learned my addr 172.21.15.27:0/2312050385 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7993fff700 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736640 0x564eda737a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f79993af700 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736f60 0x564eda724e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7998bae700 1 -- 172.21.15.27:0/2312050385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736640 msgr2=0x564eda737a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7998bae700 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736640 0x564eda737a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7998bae700 1 -- 172.21.15.27:0/2312050385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736f60 msgr2=0x564eda724e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7998bae700 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736f60 0x564eda724e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7998bae700 1 -- 172.21.15.27:0/2312050385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eda737f90 con 0x564eda730f20 2024-06-06T08:41:10.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f79993af700 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736f60 0x564eda724e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:10.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f7998bae700 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda7309e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f798800c430 tx=0x7f7988012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9953e876c2388038 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:10.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f797e7fc700 1 -- 172.21.15.27:0/2312050385 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988030070 con 0x564eda730f20 2024-06-06T08:41:10.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f797e7fc700 1 -- 172.21.15.27:0/2312050385 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79880219c0 con 0x564eda730f20 2024-06-06T08:41:10.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.678+0000 7f797e7fc700 1 -- 172.21.15.27:0/2312050385 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798802ba90 con 0x564eda730f20 2024-06-06T08:41:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.679+0000 7f79a5695b80 1 -- 172.21.15.27:0/2312050385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 msgr2=0x564eda7309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.679+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda7309e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f798800c430 tx=0x7f7988012000 comp rx=0 tx=0).stop 2024-06-06T08:41:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.679+0000 7f79a5695b80 1 -- 172.21.15.27:0/2312050385 shutdown_connections 2024-06-06T08:41:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.679+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736f60 0x564eda724e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.679+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda7309e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.679+0000 7f79a5695b80 1 --2- 172.21.15.27:0/2312050385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736640 0x564eda737a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.679+0000 7f79a5695b80 1 -- 172.21.15.27:0/2312050385 >> 172.21.15.27:0/2312050385 conn(0x564eda65bcc0 msgr2=0x564eda737340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.679+0000 7f79a5695b80 1 -- 172.21.15.27:0/2312050385 shutdown_connections 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.679+0000 7f79a5695b80 1 -- 172.21.15.27:0/2312050385 wait complete. 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79a5695b80 1 -- start start 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79a5695b80 1 Processor -- start 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda74d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda74de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79a5695b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda74e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79a5695b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79941aec50 con 0x564eda730f20 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79a5695b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f79941b00c0 con 0x564eda736640 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79a5695b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79941af320 con 0x564eda736f60 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79a5695b80 10 client.0 _ll_register_callbacks cb 0x564eda61c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f7998bae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda74d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f7998bae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda74d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33626/0 (socket says 172.21.15.27:33626) 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f7998bae700 1 -- 172.21.15.27:0/1192806236 learned_addr learned my addr 172.21.15.27:0/1192806236 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f7993fff700 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda74de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.680+0000 7f79993af700 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda74e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7998bae700 1 -- 172.21.15.27:0/1192806236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 msgr2=0x564eda74e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7998bae700 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda74e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7998bae700 1 -- 172.21.15.27:0/1192806236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 msgr2=0x564eda74de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7998bae700 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda74de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7998bae700 1 -- 172.21.15.27:0/1192806236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eda7b40b0 con 0x564eda730f20 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f79993af700 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda74e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7993fff700 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda74de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7998bae700 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda74d920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7988006b00 tx=0x7f798802b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f78188d4d85d17 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7973fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7973fff700 1 -- 172.21.15.27:0/1192806236 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988030070 con 0x564eda730f20 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7973fff700 1 -- 172.21.15.27:0/1192806236 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f798802b710 con 0x564eda730f20 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f79a5695b80 1 -- 172.21.15.27:0/1192806236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79880219c0 con 0x564eda730f20 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7973fff700 1 -- 172.21.15.27:0/1192806236 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7988042050 con 0x564eda730f20 2024-06-06T08:41:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f79a5695b80 10 client.15567 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f79a5695b80 1 -- 172.21.15.27:0/1192806236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564eda7b52f0 con 0x564eda730f20 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.681+0000 7f7973fff700 1 -- 172.21.15.27:0/1192806236 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79880219c0 con 0x564eda730f20 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f7973fff700 1 -- 172.21.15.27:0/1192806236 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f798803d030 con 0x564eda730f20 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f79a5695b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f79a5695b80 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f79a5695b80 20 client.15567 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f79a5695b80 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564eda74a740 0x564eda7b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f79a5695b80 4 client.15567 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f79a5695b80 1 -- 172.21.15.27:0/1192806236 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564eda68a290 con 0x564eda74a740 2024-06-06T08:41:10.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f7993fff700 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564eda74a740 0x564eda7b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f7993fff700 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564eda74a740 0x564eda7b7a40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a70d6aae279f864 server_cookie=9e6d3fc0b3369ed7 in_seq=0 out_seq=0 2024-06-06T08:41:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.682+0000 7f7973fff700 10 client.15567 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f7973fff700 1 -- 172.21.15.27:0/1192806236 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564eda68a290 con 0x564eda74a740 2024-06-06T08:41:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f7973fff700 10 client.15567 handle_command_reply: tid=0 2024-06-06T08:41:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 client.15567 shutdown 2024-06-06T08:41:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 -- 172.21.15.27:0/1192806236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 msgr2=0x564eda74d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda74d920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7988006b00 tx=0x7f798802b590 comp rx=0 tx=0).stop 2024-06-06T08:41:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 -- 172.21.15.27:0/1192806236 shutdown_connections 2024-06-06T08:41:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564eda736f60 0x564eda74e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564eda74a740 0x564eda7b7a40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564eda730f20 0x564eda74d920 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 --2- 172.21.15.27:0/1192806236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564eda736640 0x564eda74de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 -- 172.21.15.27:0/1192806236 >> 172.21.15.27:0/1192806236 conn(0x564eda65bcc0 msgr2=0x564eda724e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f7973fff700 0 client.15567 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 -- 172.21.15.27:0/1192806236 shutdown_connections 2024-06-06T08:41:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 1 -- 172.21.15.27:0/1192806236 wait complete. 2024-06-06T08:41:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.683+0000 7f79a5695b80 20 client.15567 trim_cache size 0 max 16384 2024-06-06T08:41:10.689 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:41:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-06-06T08:41:10.690 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {} 2024-06-06T08:41:10.690 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:41:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 -- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79840a2f40 msgr2=0x7f79840a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79840a2f40 0x7f79840a5400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7994199b40 tx=0x7f7988008040 comp rx=0 tx=0).stop 2024-06-06T08:41:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 -- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7994101f80 msgr2=0x7f799419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7994101f80 0x7f799419ed80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f798000c9e0 tx=0x7f798000ccf0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 -- 172.21.15.27:0/1127878593 shutdown_connections 2024-06-06T08:41:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79840a2f40 0x7f79840a5400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79941066b0 0x7f7994198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79941015d0 0x7f799419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 --2- 172.21.15.27:0/1127878593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7994101f80 0x7f799419ed80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:10.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 -- 172.21.15.27:0/1127878593 >> 172.21.15.27:0/1127878593 conn(0x7f7994078400 msgr2=0x7f79941051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:10.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 -- 172.21.15.27:0/1127878593 shutdown_connections 2024-06-06T08:41:10.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:10.684+0000 7f799ae12700 1 -- 172.21.15.27:0/1127878593 wait complete. 2024-06-06T08:41:11.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:10 smithi027 ceph-mon[32335]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-06T08:41:11.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:10 smithi203 ceph-mon[36579]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-06T08:41:11.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:10 smithi176 ceph-mon[36786]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-06T08:41:11.473 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T08:41:11.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:41:11.730 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.523+0000 7fb1d5476640 1 -- 172.21.15.27:0/559782266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0106890 msgr2=0x7fb1d0106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.523+0000 7fb1d5476640 1 --2- 172.21.15.27:0/559782266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0106890 0x7fb1d0106c70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009930 tx=0x7fb1c802a420 comp rx=0 tx=0).stop 2024-06-06T08:41:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.523+0000 7fb1d5476640 1 -- 172.21.15.27:0/559782266 shutdown_connections 2024-06-06T08:41:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.523+0000 7fb1d5476640 1 --2- 172.21.15.27:0/559782266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d0101290 0x7fb1d010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.523+0000 7fb1d5476640 1 --2- 172.21.15.27:0/559782266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d01008d0 0x7fb1d0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.523+0000 7fb1d5476640 1 --2- 172.21.15.27:0/559782266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0106890 0x7fb1d0106c70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.523+0000 7fb1d5476640 1 -- 172.21.15.27:0/559782266 >> 172.21.15.27:0/559782266 conn(0x7fb1d0077a20 msgr2=0x7fb1d0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.523+0000 7fb1d5476640 1 -- 172.21.15.27:0/559782266 shutdown_connections 2024-06-06T08:41:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.524+0000 7fb1d5476640 1 -- 172.21.15.27:0/559782266 wait complete. 2024-06-06T08:41:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.524+0000 7fb1d5476640 1 Processor -- start 2024-06-06T08:41:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.524+0000 7fb1d5476640 1 -- start start 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.524+0000 7fb1d5476640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d01008d0 0x7fb1d01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.524+0000 7fb1d5476640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0101290 0x7fb1d0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1d5476640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d0106890 0x7fb1d019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1d5476640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0110ca0 con 0x7fb1d0101290 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1d5476640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0110b20 con 0x7fb1d0106890 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1d5476640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0110e20 con 0x7fb1d01008d0 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ce7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0101290 0x7fb1d0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ce7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0101290 0x7fb1d0198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33652/0 (socket says 172.21.15.27:33652) 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ce7fc640 1 -- 172.21.15.27:0/4040536203 learned_addr learned my addr 172.21.15.27:0/4040536203 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d01008d0 0x7fb1d01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d0106890 0x7fb1d019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ce7fc640 1 -- 172.21.15.27:0/4040536203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d01008d0 msgr2=0x7fb1d01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d01008d0 0x7fb1d01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ce7fc640 1 -- 172.21.15.27:0/4040536203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d0106890 msgr2=0x7fb1d019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d0106890 0x7fb1d019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ce7fc640 1 -- 172.21.15.27:0/4040536203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d019d1b0 con 0x7fb1d0101290 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0101290 0x7fb1d0198700 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800e9f0 tx=0x7fb1b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1c7fff640 1 -- 172.21.15.27:0/4040536203 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800cd80 con 0x7fb1d0101290 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1d5476640 1 -- 172.21.15.27:0/4040536203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d019d4a0 con 0x7fb1d0101290 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1c7fff640 1 -- 172.21.15.27:0/4040536203 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b8005500 con 0x7fb1d0101290 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1c7fff640 1 -- 172.21.15.27:0/4040536203 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8010690 con 0x7fb1d0101290 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.525+0000 7fb1d5476640 1 -- 172.21.15.27:0/4040536203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d01a4d30 con 0x7fb1d0101290 2024-06-06T08:41:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.528+0000 7fb1c7fff640 1 -- 172.21.15.27:0/4040536203 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fb1b80046c0 con 0x7fb1d0101290 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.528+0000 7fb1d8188740 1 Processor -- start 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.528+0000 7fb1d8188740 1 -- start start 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.528+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f17550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.528+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 0x562b18f21e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.528+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f24740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1d8188740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b18e7b4f0 con 0x562b18f17170 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1d8188740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562b18e19870 con 0x562b18f17a90 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1d8188740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b18e774b0 con 0x562b18f22350 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f17550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f17550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33658/0 (socket says 172.21.15.27:33658) 2024-06-06T08:41:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 -- 172.21.15.27:0/4216818563 learned_addr learned my addr 172.21.15.27:0/4216818563 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1c7fff640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1bc0a2f50 0x7fb1bc0a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f24740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 0x562b18f21e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1bc0a2f50 0x7fb1bc0a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 -- 172.21.15.27:0/4216818563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 msgr2=0x562b18f24740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f24740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 -- 172.21.15.27:0/4216818563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 msgr2=0x562b18f21e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 0x562b18f21e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1c7fff640 1 -- 172.21.15.27:0/4040536203 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb1b80cffc0 con 0x7fb1d0101290 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 -- 172.21.15.27:0/4216818563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b18e359b0 con 0x562b18f17170 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f17550 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00129f0 tx=0x7fb1c0012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a148deaee919984 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.529+0000 7fb1c57fa640 1 -- 172.21.15.27:0/4216818563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0023070 con 0x562b18f17170 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1c57fa640 1 -- 172.21.15.27:0/4216818563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c0004710 con 0x562b18f17170 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1c57fa640 1 -- 172.21.15.27:0/4216818563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c001fa90 con 0x562b18f17170 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1bc0a2f50 0x7fb1bc0a5410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0022000 tx=0x7fb1c0021000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1d8188740 1 -- 172.21.15.27:0/4216818563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 msgr2=0x562b18f17550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1d8188740 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f17550 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00129f0 tx=0x7fb1c0012eb0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1d8188740 1 -- 172.21.15.27:0/4216818563 shutdown_connections 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1d8188740 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1d8188740 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 0x562b18f21e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1d8188740 1 --2- 172.21.15.27:0/4216818563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f17550 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1d8188740 1 -- 172.21.15.27:0/4216818563 >> 172.21.15.27:0/4216818563 conn(0x562b18f13680 msgr2=0x562b18f13a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1d8188740 1 -- 172.21.15.27:0/4216818563 shutdown_connections 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.530+0000 7fb1d8188740 1 -- 172.21.15.27:0/4216818563 wait complete. 2024-06-06T08:41:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1d8188740 1 -- start start 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1d8188740 1 Processor -- start 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f1f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 0x562b18f1f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f1fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1d8188740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b18f24f80 con 0x562b18f17170 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1d8188740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562b18e04ae0 con 0x562b18f17a90 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1d8188740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b18f25100 con 0x562b18f22350 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1d8188740 10 client.0 _ll_register_callbacks cb 0x562b18e3d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1ceffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f1f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1ceffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f1f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33660/0 (socket says 172.21.15.27:33660) 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1ceffd640 1 -- 172.21.15.27:0/2329281898 learned_addr learned my addr 172.21.15.27:0/2329281898 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f1fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 0x562b18f1f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1ceffd640 1 -- 172.21.15.27:0/2329281898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 msgr2=0x562b18f1fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f1fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1ceffd640 1 -- 172.21.15.27:0/2329281898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 msgr2=0x562b18f1f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 0x562b18f1f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1ceffd640 1 -- 172.21.15.27:0/2329281898 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b18b8f210 con 0x562b18f17170 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.531+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f1fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f1f2a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8000c00 tx=0x7fb1c8031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd34691402ee6b0c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1b67fc640 1 -- 172.21.15.27:0/2329281898 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8035040 con 0x562b18f17170 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1b67fc640 1 -- 172.21.15.27:0/2329281898 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb1c8004240 con 0x562b18f17170 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1d8188740 1 -- 172.21.15.27:0/2329281898 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c0004710 con 0x562b18f17170 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1b67fc640 1 -- 172.21.15.27:0/2329281898 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb1c804f050 con 0x562b18f17170 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1d8188740 10 client.15579 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1d8188740 1 -- 172.21.15.27:0/2329281898 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562b18c5b620 con 0x562b18f17170 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1b67fc640 1 -- 172.21.15.27:0/2329281898 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c0004710 con 0x562b18f17170 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1b67fc640 1 -- 172.21.15.27:0/2329281898 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb1c8046640 con 0x562b18f17170 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1d8188740 10 client.15579 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1d8188740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1d8188740 20 client.15579 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1d8188740 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x562b18f32480 0x562b18f34870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1d8188740 4 client.15579 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.532+0000 7fb1d8188740 1 -- 172.21.15.27:0/2329281898 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b18e8ef10 con 0x562b18f32480 2024-06-06T08:41:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.533+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x562b18f32480 0x562b18f34870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.533+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x562b18f32480 0x562b18f34870 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e20a3ab093010284 server_cookie=6e9fd981c6841f4c in_seq=0 out_seq=0 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.533+0000 7fb1b67fc640 10 client.15579 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1b67fc640 1 -- 172.21.15.27:0/2329281898 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b18e8ef10 con 0x562b18f32480 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1b67fc640 10 client.15579 handle_command_reply: tid=0 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 client.15579 shutdown 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 -- 172.21.15.27:0/2329281898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 msgr2=0x562b18f1f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f1f2a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8000c00 tx=0x7fb1c8031850 comp rx=0 tx=0).stop 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 -- 172.21.15.27:0/2329281898 shutdown_connections 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x562b18f32480 0x562b18f34870 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f1fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1b67fc640 0 client.15579 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17a90 0x562b18f1f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 --2- 172.21.15.27:0/2329281898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f1f2a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 -- 172.21.15.27:0/2329281898 >> 172.21.15.27:0/2329281898 conn(0x562b18f13680 msgr2=0x562b18f19400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 -- 172.21.15.27:0/2329281898 shutdown_connections 2024-06-06T08:41:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 1 -- 172.21.15.27:0/2329281898 wait complete. 2024-06-06T08:41:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.534+0000 7fb1d8188740 20 client.15579 trim_cache size 0 max 16384 2024-06-06T08:41:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1d8188740 1 Processor -- start 2024-06-06T08:41:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1d8188740 1 -- start start 2024-06-06T08:41:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f19b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f17a90 0x562b18f211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f22350 0x562b18f3a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1d8188740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01b1650 con 0x562b18f17170 2024-06-06T08:41:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1d8188740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01aef20 con 0x562b18f22350 2024-06-06T08:41:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1d8188740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01acd70 con 0x562b18f17a90 2024-06-06T08:41:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1ceffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f19b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1ceffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f19b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33668/0 (socket says 172.21.15.27:33668) 2024-06-06T08:41:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.550+0000 7fb1ceffd640 1 -- 172.21.15.27:0/1524047636 learned_addr learned my addr 172.21.15.27:0/1524047636 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/1524047636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f22350 0x562b18f3a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/1524047636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f17a90 0x562b18f211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1ceffd640 1 -- 172.21.15.27:0/1524047636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f17a90 msgr2=0x562b18f211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/1524047636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f17a90 0x562b18f211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1ceffd640 1 -- 172.21.15.27:0/1524047636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f22350 msgr2=0x562b18f3a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/1524047636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f22350 0x562b18f3a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1ceffd640 1 -- 172.21.15.27:0/1524047636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b18f178c0 con 0x562b18f17170 2024-06-06T08:41:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/1524047636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f19b30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8006010 tx=0x7fb1c8009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26e238ff00cc26f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1c4ff9640 1 -- 172.21.15.27:0/1524047636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c805a070 con 0x562b18f17170 2024-06-06T08:41:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1c4ff9640 1 -- 172.21.15.27:0/1524047636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c8046be0 con 0x562b18f17170 2024-06-06T08:41:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.551+0000 7fb1c4ff9640 1 -- 172.21.15.27:0/1524047636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8035070 con 0x562b18f17170 2024-06-06T08:41:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 -- 172.21.15.27:0/1524047636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 msgr2=0x562b18f19b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 --2- 172.21.15.27:0/1524047636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f19b30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8006010 tx=0x7fb1c8009230 comp rx=0 tx=0).stop 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 -- 172.21.15.27:0/1524047636 shutdown_connections 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 --2- 172.21.15.27:0/1524047636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f22350 0x562b18f3a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 --2- 172.21.15.27:0/1524047636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f17a90 0x562b18f211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 --2- 172.21.15.27:0/1524047636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17170 0x562b18f19b30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 -- 172.21.15.27:0/1524047636 >> 172.21.15.27:0/1524047636 conn(0x562b18f13680 msgr2=0x562b18f20ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 -- 172.21.15.27:0/1524047636 shutdown_connections 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 -- 172.21.15.27:0/1524047636 wait complete. 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.552+0000 7fb1d8188740 1 -- start start 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1d8188740 1 Processor -- start 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17170 0x562b18f2a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17a90 0x562b18f25cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1d8188740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f261f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1d8188740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01ad3e0 con 0x562b18f17a90 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1d8188740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01ae850 con 0x562b18f17170 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1d8188740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d01adab0 con 0x562b18f22350 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1d8188740 10 client.0 _ll_register_callbacks cb 0x7fb1c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ce7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17a90 0x562b18f25cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ce7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17a90 0x562b18f25cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33674/0 (socket says 172.21.15.27:33674) 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ce7fc640 1 -- 172.21.15.27:0/3664206020 learned_addr learned my addr 172.21.15.27:0/3664206020 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17170 0x562b18f2a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1cf7fe640 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f261f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ce7fc640 1 -- 172.21.15.27:0/3664206020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 msgr2=0x562b18f261f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f261f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ce7fc640 1 -- 172.21.15.27:0/3664206020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17170 msgr2=0x562b18f2a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17170 0x562b18f2a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ce7fc640 1 -- 172.21.15.27:0/3664206020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b18f26860 con 0x562b18f17a90 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17170 0x562b18f2a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1ce7fc640 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17a90 0x562b18f25cb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80049d0 tx=0x7fb1b806df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c319f2d8aee83e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.553+0000 7fb1b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1b67fc640 1 -- 172.21.15.27:0/3664206020 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b809bee0 con 0x562b18f17a90 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1b67fc640 1 -- 172.21.15.27:0/3664206020 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb1b8098550 con 0x562b18f17a90 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1d8188740 1 -- 172.21.15.27:0/3664206020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c8046be0 con 0x562b18f17a90 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1b67fc640 1 -- 172.21.15.27:0/3664206020 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb1b80a0290 con 0x562b18f17a90 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1d8188740 10 client.15585 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1d8188740 1 -- 172.21.15.27:0/3664206020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562b18f9cfd0 con 0x562b18f17a90 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1b67fc640 1 -- 172.21.15.27:0/3664206020 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c8046be0 con 0x562b18f17a90 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1b67fc640 1 -- 172.21.15.27:0/3664206020 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb1b809b330 con 0x562b18f17a90 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1d8188740 10 client.15585 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1d8188740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1d8188740 20 client.15585 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1d8188740 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x562b18f9d6b0 0x562b18f9fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1d8188740 4 client.15585 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1d8188740 1 -- 172.21.15.27:0/3664206020 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b18e88c20 con 0x562b18f9d6b0 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.554+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x562b18f9d6b0 0x562b18f9fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.555+0000 7fb1ceffd640 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x562b18f9d6b0 0x562b18f9fad0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b651973b73fbf1e7 server_cookie=2cf2885e99e47838 in_seq=0 out_seq=0 2024-06-06T08:41:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.555+0000 7fb1b67fc640 10 client.15585 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.562+0000 7fb1b67fc640 1 -- 172.21.15.27:0/3664206020 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562b18e88c20 con 0x562b18f9d6b0 2024-06-06T08:41:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.562+0000 7fb1b67fc640 10 client.15585 handle_command_reply: tid=0 2024-06-06T08:41:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.562+0000 7fb1d8188740 1 client.15585 shutdown 2024-06-06T08:41:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 -- 172.21.15.27:0/3664206020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17a90 msgr2=0x562b18f25cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17a90 0x562b18f25cb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80049d0 tx=0x7fb1b806df60 comp rx=0 tx=0).stop 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 -- 172.21.15.27:0/3664206020 shutdown_connections 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x562b18f9d6b0 0x562b18f9fad0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b18f22350 0x562b18f261f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b18f17a90 0x562b18f25cb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 --2- 172.21.15.27:0/3664206020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b18f17170 0x562b18f2a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1b67fc640 0 client.15585 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 -- 172.21.15.27:0/3664206020 >> 172.21.15.27:0/3664206020 conn(0x562b18f13680 msgr2=0x562b18f1fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 -- 172.21.15.27:0/3664206020 shutdown_connections 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 1 -- 172.21.15.27:0/3664206020 wait complete. 2024-06-06T08:41:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d8188740 20 client.15585 trim_cache size 0 max 16384 2024-06-06T08:41:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:41:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:41:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:41:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 -- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1bc0a2f50 msgr2=0x7fb1bc0a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1bc0a2f50 0x7fb1bc0a5410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0022000 tx=0x7fb1c0021000 comp rx=0 tx=0).stop 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 -- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0101290 msgr2=0x7fb1d0198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0101290 0x7fb1d0198700 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800e9f0 tx=0x7fb1b800edf0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 -- 172.21.15.27:0/4040536203 shutdown_connections 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1bc0a2f50 0x7fb1bc0a5410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d0106890 0x7fb1d019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1d0101290 0x7fb1d0198700 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 --2- 172.21.15.27:0/4040536203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1d01008d0 0x7fb1d01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.563+0000 7fb1d5476640 1 -- 172.21.15.27:0/4040536203 >> 172.21.15.27:0/4040536203 conn(0x7fb1d0077a20 msgr2=0x7fb1d00ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.564+0000 7fb1d5476640 1 -- 172.21.15.27:0/4040536203 shutdown_connections 2024-06-06T08:41:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:12.564+0000 7fb1d5476640 1 -- 172.21.15.27:0/4040536203 wait complete. 2024-06-06T08:41:13.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:12 smithi027 ceph-mon[32335]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:41:13.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:12 smithi203 ceph-mon[36579]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:41:13.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:12 smithi176 ceph-mon[36786]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:41:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.495+0000 7f2f25b23700 1 -- 172.21.15.27:0/1733576455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101390 msgr2=0x7f2f20101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.495+0000 7f2f25b23700 1 --2- 172.21.15.27:0/1733576455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101390 0x7f2f20101810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009990 tx=0x7f2f08009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.495+0000 7f2f25b23700 1 -- 172.21.15.27:0/1733576455 shutdown_connections 2024-06-06T08:41:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.495+0000 7f2f25b23700 1 --2- 172.21.15.27:0/1733576455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f20101d50 0x7f2f2010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.495+0000 7f2f25b23700 1 --2- 172.21.15.27:0/1733576455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f20107400 0x7f2f201077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.495+0000 7f2f25b23700 1 --2- 172.21.15.27:0/1733576455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101390 0x7f2f20101810 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.495+0000 7f2f25b23700 1 -- 172.21.15.27:0/1733576455 >> 172.21.15.27:0/1733576455 conn(0x7f2f20078340 msgr2=0x7f2f20078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.495+0000 7f2f25b23700 1 -- 172.21.15.27:0/1733576455 shutdown_connections 2024-06-06T08:41:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.495+0000 7f2f25b23700 1 -- 172.21.15.27:0/1733576455 wait complete. 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f25b23700 1 Processor -- start 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f25b23700 1 -- start start 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f25b23700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f20101390 0x7f2f20103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f25b23700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101d50 0x7f2f201035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f25b23700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f20107400 0x7f2f20103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f25b23700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201116f0 con 0x7f2f20101d50 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f25b23700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20111570 con 0x7f2f20101390 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f25b23700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20111870 con 0x7f2f20107400 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f1effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101d50 0x7f2f201035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f1effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101d50 0x7f2f201035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33702/0 (socket says 172.21.15.27:33702) 2024-06-06T08:41:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.496+0000 7f2f1effd700 1 -- 172.21.15.27:0/540143708 learned_addr learned my addr 172.21.15.27:0/540143708 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:13.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1f7fe700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f20101390 0x7f2f20103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1ffff700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f20107400 0x7f2f20103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1effd700 1 -- 172.21.15.27:0/540143708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f20107400 msgr2=0x7f2f20103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1effd700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f20107400 0x7f2f20103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1effd700 1 -- 172.21.15.27:0/540143708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f20101390 msgr2=0x7f2f20103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1effd700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f20101390 0x7f2f20103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1effd700 1 -- 172.21.15.27:0/540143708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f201a7180 con 0x7f2f20101d50 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1effd700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101d50 0x7f2f201035b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2f08000c00 tx=0x7f2f08003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1cff9700 1 -- 172.21.15.27:0/540143708 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08027070 con 0x7f2f20101d50 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f201a7410 con 0x7f2f20101d50 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1cff9700 1 -- 172.21.15.27:0/540143708 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f08004190 con 0x7f2f20101d50 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f1cff9700 1 -- 172.21.15.27:0/540143708 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800f460 con 0x7f2f20101d50 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.497+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201a7870 con 0x7f2f20101d50 2024-06-06T08:41:13.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.498+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f20069180 con 0x7f2f20101d50 2024-06-06T08:41:13.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.500+0000 7f2f1cff9700 1 -- 172.21.15.27:0/540143708 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f2f0802a070 con 0x7f2f20101d50 2024-06-06T08:41:13.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.501+0000 7f2f1cff9700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f0c0a2f40 0x7f2f0c0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:13.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.501+0000 7f2f1f7fe700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f0c0a2f40 0x7f2f0c0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:13.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.501+0000 7f2f1cff9700 1 -- 172.21.15.27:0/540143708 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f2f080d0e20 con 0x7f2f20101d50 2024-06-06T08:41:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.501+0000 7f2f1f7fe700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f0c0a2f40 0x7f2f0c0a5400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f100060b0 tx=0x7f2f10007b80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.504+0000 7f2f1cff9700 1 -- 172.21.15.27:0/540143708 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f08013070 con 0x7f2f20101d50 2024-06-06T08:41:13.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.758+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f201a7ad0 con 0x7f2f20101d50 2024-06-06T08:41:13.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.759+0000 7f2f1cff9700 1 -- 172.21.15.27:0/540143708 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7f2f0809d4d0 con 0x7f2f20101d50 2024-06-06T08:41:13.758 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:41:13.758 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.761+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f0c0a2f40 msgr2=0x7f2f0c0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.761+0000 7f2f25b23700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f0c0a2f40 0x7f2f0c0a5400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f100060b0 tx=0x7f2f10007b80 comp rx=0 tx=0).stop 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.761+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101d50 msgr2=0x7f2f201035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.761+0000 7f2f25b23700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101d50 0x7f2f201035b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2f08000c00 tx=0x7f2f08003730 comp rx=0 tx=0).stop 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.762+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 shutdown_connections 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.762+0000 7f2f25b23700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f20107400 0x7f2f20103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.762+0000 7f2f25b23700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f0c0a2f40 0x7f2f0c0a5400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.762+0000 7f2f25b23700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f20101390 0x7f2f20103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.762+0000 7f2f25b23700 1 --2- 172.21.15.27:0/540143708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f20101d50 0x7f2f201035b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.762+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 >> 172.21.15.27:0/540143708 conn(0x7f2f20078340 msgr2=0x7f2f201005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:13.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.762+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 shutdown_connections 2024-06-06T08:41:13.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:13.762+0000 7f2f25b23700 1 -- 172.21.15.27:0/540143708 wait complete. 2024-06-06T08:41:13.762 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:41:14.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:13 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/540143708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:14.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:13 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/540143708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:14.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:13 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/540143708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:14.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T08:41:14.767 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:15.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:14 smithi027 ceph-mon[32335]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-06-06T08:41:15.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:14 smithi203 ceph-mon[36579]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-06-06T08:41:15.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:14 smithi176 ceph-mon[36786]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-06-06T08:41:16.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.407+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2609167509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 msgr2=0x7f0f9c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.407+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2609167509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 0x7f0f9c102b20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0099c0 tx=0x7f0f8c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.407+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2609167509 shutdown_connections 2024-06-06T08:41:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.407+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2609167509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f9c107840 0x7f0f9c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.407+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2609167509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 0x7f0f9c102b20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.407+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2609167509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f9c1030f0 0x7f0f9c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.407+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2609167509 >> 172.21.15.27:0/2609167509 conn(0x7f0f9c0795f0 msgr2=0x7f0f9c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.408+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2609167509 shutdown_connections 2024-06-06T08:41:16.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.408+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2609167509 wait complete. 2024-06-06T08:41:16.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.408+0000 7f0fa09a6700 1 Processor -- start 2024-06-06T08:41:16.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.408+0000 7f0fa09a6700 1 -- start start 2024-06-06T08:41:16.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0fa09a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 0x7f0f9c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0fa09a6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f9c1030f0 0x7f0f9c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0fa09a6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f9c107840 0x7f0f9c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0fa09a6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c112780 con 0x7f0f9c102740 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0fa09a6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c112600 con 0x7f0f9c1030f0 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0fa09a6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c112900 con 0x7f0f9c107840 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 0x7f0f9c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 0x7f0f9c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33716/0 (socket says 172.21.15.27:33716) 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2692761281 learned_addr learned my addr 172.21.15.27:0/2692761281 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f9c1030f0 0x7f0f9c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9ad9d700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f9c107840 0x7f0f9c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2692761281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f9c107840 msgr2=0x7f0f9c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f9c107840 0x7f0f9c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2692761281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f9c1030f0 msgr2=0x7f0f9c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f9c1030f0 0x7f0f9c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2692761281 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c199240 con 0x7f0f9c102740 2024-06-06T08:41:16.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f9c1030f0 0x7f0f9c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:16.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 0x7f0f9c19e8d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009fd0 tx=0x7f0f8c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:16.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.409+0000 7f0f937fe700 1 -- 172.21.15.27:0/2692761281 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c027070 con 0x7f0f9c102740 2024-06-06T08:41:16.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.410+0000 7f0f937fe700 1 -- 172.21.15.27:0/2692761281 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8c004150 con 0x7f0f9c102740 2024-06-06T08:41:16.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.410+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2692761281 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c1994d0 con 0x7f0f9c102740 2024-06-06T08:41:16.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.410+0000 7f0f937fe700 1 -- 172.21.15.27:0/2692761281 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00f460 con 0x7f0f9c102740 2024-06-06T08:41:16.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.410+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2692761281 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c1a57a0 con 0x7f0f9c102740 2024-06-06T08:41:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.412+0000 7f0f937fe700 1 -- 172.21.15.27:0/2692761281 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f0f8c02a070 con 0x7f0f9c102740 2024-06-06T08:41:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.413+0000 7f0f937fe700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0f880a2f40 0x7f0f880a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.413+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0f880a2f40 0x7f0f880a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.413+0000 7f0f937fe700 1 -- 172.21.15.27:0/2692761281 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f0f8c0d0ac0 con 0x7f0f9c102740 2024-06-06T08:41:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.413+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0f880a2f40 0x7f0f880a5400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c102030 tx=0x7f0f84005f30 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0fab229b80 1 Processor -- start 2024-06-06T08:41:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0fab229b80 1 -- start start 2024-06-06T08:41:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0fab229b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ac8a7a7f60 con 0x55ac8a8baf60 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0fab229b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ac8a806850 con 0x55ac8a8bb880 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0fab229b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ac8a76e460 con 0x55ac8a8b57a0 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33724/0 (socket says 172.21.15.27:33724) 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.414+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2795002223 learned_addr learned my addr 172.21.15.27:0/2795002223 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2795002223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8b57a0 msgr2=0x55ac8a8b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2795002223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2795002223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2795002223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 msgr2=0x55ac8a8b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2795002223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2795002223 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8a7558d0 con 0x55ac8a8baf60 2024-06-06T08:41:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2795002223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2795002223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bb340 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c09ca00 tx=0x7f0f8c022a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2da9eb3f26b7537 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f90ff9700 1 -- 172.21.15.27:0/2795002223 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c09ca70 con 0x55ac8a8baf60 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f90ff9700 1 -- 172.21.15.27:0/2795002223 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8c0a0e20 con 0x55ac8a8baf60 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.415+0000 7f0f90ff9700 1 -- 172.21.15.27:0/2795002223 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0a30c0 con 0x55ac8a8baf60 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 -- 172.21.15.27:0/2795002223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 msgr2=0x55ac8a8bb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2795002223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bb340 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c09ca00 tx=0x7f0f8c022a10 comp rx=0 tx=0).stop 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 -- 172.21.15.27:0/2795002223 shutdown_connections 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2795002223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2795002223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bb340 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2795002223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 -- 172.21.15.27:0/2795002223 >> 172.21.15.27:0/2795002223 conn(0x55ac8a7e2080 msgr2=0x55ac8a8b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 -- 172.21.15.27:0/2795002223 shutdown_connections 2024-06-06T08:41:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 -- 172.21.15.27:0/2795002223 wait complete. 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 -- start start 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.416+0000 7f0fab229b80 1 Processor -- start 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8baf60 0x55ac8a8be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8bb880 0x55ac8a8beee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0fab229b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ac8a7a6fc0 con 0x55ac8a8b57a0 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0fab229b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ac8a802810 con 0x55ac8a8baf60 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0fab229b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ac8a8c2010 con 0x55ac8a8bb880 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0fab229b80 10 client.0 _ll_register_callbacks cb 0x55ac8a79c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8c1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33734/0 (socket says 172.21.15.27:33734) 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f9a59c700 1 -- 172.21.15.27:0/1529260006 learned_addr learned my addr 172.21.15.27:0/1529260006 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8baf60 0x55ac8a8be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f9ad9d700 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8bb880 0x55ac8a8beee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f99d9b700 1 -- 172.21.15.27:0/1529260006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8bb880 msgr2=0x55ac8a8beee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8bb880 0x55ac8a8beee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f99d9b700 1 -- 172.21.15.27:0/1529260006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 msgr2=0x55ac8a8c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.417+0000 7f0f99d9b700 1 -- 172.21.15.27:0/1529260006 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8a7cc750 con 0x55ac8a8baf60 2024-06-06T08:41:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8baf60 0x55ac8a8be9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400aa10 tx=0x7f0f84004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91975dc48af566b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0f81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:41:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0f81ffb700 1 -- 172.21.15.27:0/1529260006 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8401d8f0 con 0x55ac8a8baf60 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0fab229b80 1 -- 172.21.15.27:0/1529260006 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8c0a0e20 con 0x55ac8a8baf60 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0f81ffb700 1 -- 172.21.15.27:0/1529260006 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f0f8401da90 con 0x55ac8a8baf60 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0f81ffb700 1 -- 172.21.15.27:0/1529260006 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0f8401dc90 con 0x55ac8a8baf60 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0fab229b80 10 client.25214 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0fab229b80 1 -- 172.21.15.27:0/1529260006 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac8a8cc780 con 0x55ac8a8baf60 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0f81ffb700 1 -- 172.21.15.27:0/1529260006 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8c0a0e20 con 0x55ac8a8baf60 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.418+0000 7f0f81ffb700 1 -- 172.21.15.27:0/1529260006 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0f8400f450 con 0x55ac8a8baf60 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.419+0000 7f0fab229b80 10 client.25214 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.419+0000 7f0fab229b80 10 client.25214 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.419+0000 7f0fab229b80 20 client.25214 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.419+0000 7f0fab229b80 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55ac8a8ced10 0x55ac8a8d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.419+0000 7f0fab229b80 4 client.25214 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.419+0000 7f0fab229b80 1 -- 172.21.15.27:0/1529260006 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac8a813d50 con 0x55ac8a8ced10 2024-06-06T08:41:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.419+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55ac8a8ced10 0x55ac8a8d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.419+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55ac8a8ced10 0x55ac8a8d1100 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38c01a2f34ee9dbe server_cookie=2f3e57fc8f0c583f in_seq=0 out_seq=0 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.419+0000 7f0f81ffb700 10 client.25214 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0f81ffb700 1 -- 172.21.15.27:0/1529260006 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ac8a813d50 con 0x55ac8a8ced10 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0f81ffb700 10 client.25214 handle_command_reply: tid=0 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 client.25214 shutdown 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 -- 172.21.15.27:0/1529260006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8baf60 msgr2=0x55ac8a8be9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8baf60 0x55ac8a8be9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400aa10 tx=0x7f0f84004750 comp rx=0 tx=0).stop 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 -- 172.21.15.27:0/1529260006 shutdown_connections 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55ac8a8ced10 0x55ac8a8d1100 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8bb880 0x55ac8a8beee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0f81ffb700 0 client.25214 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 --2- 172.21.15.27:0/1529260006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8baf60 0x55ac8a8be9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 -- 172.21.15.27:0/1529260006 >> 172.21.15.27:0/1529260006 conn(0x55ac8a7e2080 msgr2=0x55ac8a8af700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:16.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 -- 172.21.15.27:0/1529260006 shutdown_connections 2024-06-06T08:41:16.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 1 -- 172.21.15.27:0/1529260006 wait complete. 2024-06-06T08:41:16.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.420+0000 7f0fab229b80 20 client.25214 trim_cache size 0 max 16384 2024-06-06T08:41:16.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0fab229b80 1 Processor -- start 2024-06-06T08:41:16.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0fab229b80 1 -- start start 2024-06-06T08:41:16.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8befd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0fab229b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1b0e40 con 0x55ac8a8b57a0 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0fab229b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1ae6d0 con 0x55ac8a8bb880 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0fab229b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1ac520 con 0x55ac8a8baf60 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8befd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8befd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33740/0 (socket says 172.21.15.27:33740) 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2618917757 learned_addr learned my addr 172.21.15.27:0/2618917757 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9ad9d700 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2618917757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8baf60 msgr2=0x55ac8a8bc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2618917757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 msgr2=0x55ac8a8afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9a59c700 1 -- 172.21.15.27:0/2618917757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8a8b6190 con 0x55ac8a8b57a0 2024-06-06T08:41:16.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f9ad9d700 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:16.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.436+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:16.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8befd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009350 tx=0x7f0f8c022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f1d298a2aea588 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:16.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0f83fff700 1 -- 172.21.15.27:0/2618917757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c09ca70 con 0x55ac8a8b57a0 2024-06-06T08:41:16.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0f83fff700 1 -- 172.21.15.27:0/2618917757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8c0a51c0 con 0x55ac8a8b57a0 2024-06-06T08:41:16.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0f83fff700 1 -- 172.21.15.27:0/2618917757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0a30b0 con 0x55ac8a8b57a0 2024-06-06T08:41:16.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0fab229b80 1 -- 172.21.15.27:0/2618917757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 msgr2=0x55ac8a8befd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:16.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8befd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009350 tx=0x7f0f8c022f90 comp rx=0 tx=0).stop 2024-06-06T08:41:16.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0fab229b80 1 -- 172.21.15.27:0/2618917757 shutdown_connections 2024-06-06T08:41:16.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8befd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2618917757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8baf60 0x55ac8a8bc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0fab229b80 1 -- 172.21.15.27:0/2618917757 >> 172.21.15.27:0/2618917757 conn(0x55ac8a7e2080 msgr2=0x55ac8a8bbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0fab229b80 1 -- 172.21.15.27:0/2618917757 shutdown_connections 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.437+0000 7f0fab229b80 1 -- 172.21.15.27:0/2618917757 wait complete. 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0fab229b80 1 -- start start 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0fab229b80 1 Processor -- start 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0fab229b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0fab229b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1acb90 con 0x55ac8a8baf60 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0fab229b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1ae000 con 0x55ac8a8bb880 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0fab229b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1ad260 con 0x55ac8a8b57a0 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0fab229b80 10 client.0 _ll_register_callbacks cb 0x55ac8a79c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:16.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0f99d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0f99d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8d6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33756/0 (socket says 172.21.15.27:33756) 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.438+0000 7f0f99d9b700 1 -- 172.21.15.27:0/2820639829 learned_addr learned my addr 172.21.15.27:0/2820639829 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f9ad9d700 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f99d9b700 1 -- 172.21.15.27:0/2820639829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8b57a0 msgr2=0x55ac8a8d65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8d65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f99d9b700 1 -- 172.21.15.27:0/2820639829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 msgr2=0x55ac8a8d7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8d7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f99d9b700 1 -- 172.21.15.27:0/2820639829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8a9389d0 con 0x55ac8a8baf60 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f99d9b700 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8d6b30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0f84005950 tx=0x7f0f8400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7185134be980f43 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f81ffb700 1 -- 172.21.15.27:0/2820639829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8401d6f0 con 0x55ac8a8baf60 2024-06-06T08:41:16.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f81ffb700 1 -- 172.21.15.27:0/2820639829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f0f8401d890 con 0x55ac8a8baf60 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0fab229b80 1 -- 172.21.15.27:0/2820639829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8c0a51c0 con 0x55ac8a8baf60 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f81ffb700 1 -- 172.21.15.27:0/2820639829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0f84046050 con 0x55ac8a8baf60 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0fab229b80 10 client.15609 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0fab229b80 1 -- 172.21.15.27:0/2820639829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac8a939c90 con 0x55ac8a8baf60 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.439+0000 7f0f81ffb700 1 -- 172.21.15.27:0/2820639829 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f8c0a51c0 con 0x55ac8a8baf60 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0f81ffb700 1 -- 172.21.15.27:0/2820639829 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0f84005380 con 0x55ac8a8baf60 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0fab229b80 10 client.15609 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0fab229b80 10 client.15609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0fab229b80 20 client.15609 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55ac8a93a3d0 0x55ac8a93c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0fab229b80 4 client.15609 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0fab229b80 1 -- 172.21.15.27:0/2820639829 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac8a80e2d0 con 0x55ac8a93a3d0 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55ac8a93a3d0 0x55ac8a93c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:16.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0f9a59c700 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55ac8a93a3d0 0x55ac8a93c7f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecac256900eb5041 server_cookie=8dff4cd2b01f503 in_seq=0 out_seq=0 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.440+0000 7f0f81ffb700 10 client.15609 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0f81ffb700 1 -- 172.21.15.27:0/2820639829 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac8a80e2d0 con 0x55ac8a93a3d0 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0f81ffb700 10 client.15609 handle_command_reply: tid=0 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 client.15609 shutdown 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 -- 172.21.15.27:0/2820639829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 msgr2=0x55ac8a8d6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8d6b30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0f84005950 tx=0x7f0f8400fb60 comp rx=0 tx=0).stop 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 -- 172.21.15.27:0/2820639829 shutdown_connections 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55ac8a93a3d0 0x55ac8a93c7f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ac8a8bb880 0x55ac8a8d7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac8a8b57a0 0x55ac8a8d65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 --2- 172.21.15.27:0/2820639829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ac8a8baf60 0x55ac8a8d6b30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 -- 172.21.15.27:0/2820639829 >> 172.21.15.27:0/2820639829 conn(0x55ac8a7e2080 msgr2=0x55ac8a8ba360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:16.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0f81ffb700 0 client.15609 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 -- 172.21.15.27:0/2820639829 shutdown_connections 2024-06-06T08:41:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 1 -- 172.21.15.27:0/2820639829 wait complete. 2024-06-06T08:41:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.441+0000 7f0fab229b80 20 client.15609 trim_cache size 0 max 16384 2024-06-06T08:41:16.445 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:41:16.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0f880a2f40 msgr2=0x7f0f880a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0f880a2f40 0x7f0f880a5400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c102030 tx=0x7f0f84005f30 comp rx=0 tx=0).stop 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 msgr2=0x7f0f9c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 0x7f0f9c19e8d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009fd0 tx=0x7f0f8c003730 comp rx=0 tx=0).stop 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2692761281 shutdown_connections 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f9c107840 0x7f0f9c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f9c102740 0x7f0f9c19e8d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0f880a2f40 0x7f0f880a5400 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 --2- 172.21.15.27:0/2692761281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0f9c1030f0 0x7f0f9c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2692761281 >> 172.21.15.27:0/2692761281 conn(0x7f0f9c0795f0 msgr2=0x7f0f9c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2692761281 shutdown_connections 2024-06-06T08:41:16.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:16.442+0000 7f0fa09a6700 1 -- 172.21.15.27:0/2692761281 wait complete. 2024-06-06T08:41:17.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:16 smithi027 ceph-mon[32335]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-06T08:41:17.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:16 smithi203 ceph-mon[36579]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-06T08:41:17.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:16 smithi176 ceph-mon[36786]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-06T08:41:17.261 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T08:41:17.516 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:19.078 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:18 smithi027 ceph-mon[32335]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-06T08:41:19.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:18 smithi203 ceph-mon[36579]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-06T08:41:19.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:18 smithi176 ceph-mon[36786]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-06T08:41:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.329+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/1282593264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406b3a0 msgr2=0x7fb5f410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.329+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/1282593264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406b3a0 0x7fb5f410f310 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b0a0 tx=0x7fb5ec00b3b0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.329+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/1282593264 shutdown_connections 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.329+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/1282593264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406b3a0 0x7fb5f410f310 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.329+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/1282593264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f41084d0 0x7fb5f41088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.329+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/1282593264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f406aa00 0x7fb5f406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.329+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/1282593264 >> 172.21.15.27:0/1282593264 conn(0x7fb5f4078330 msgr2=0x7fb5f4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.329+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/1282593264 shutdown_connections 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.329+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/1282593264 wait complete. 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.330+0000 7fb5fb7aa700 1 Processor -- start 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.330+0000 7fb5fb7aa700 1 -- start start 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.330+0000 7fb5fb7aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406aa00 0x7fb5f4101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.330+0000 7fb5fb7aa700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f406b3a0 0x7fb5f4101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.330+0000 7fb5fb7aa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f41084d0 0x7fb5f4101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.330+0000 7fb5fb7aa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41127d0 con 0x7fb5f406aa00 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.330+0000 7fb5fb7aa700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4112650 con 0x7fb5f406b3a0 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5fb7aa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4112950 con 0x7fb5f41084d0 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9546700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406aa00 0x7fb5f4101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9546700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406aa00 0x7fb5f4101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33770/0 (socket says 172.21.15.27:33770) 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9546700 1 -- 172.21.15.27:0/2482103441 learned_addr learned my addr 172.21.15.27:0/2482103441 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9d47700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f41084d0 0x7fb5f4101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f406b3a0 0x7fb5f4101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9546700 1 -- 172.21.15.27:0/2482103441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f41084d0 msgr2=0x7fb5f4101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9546700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f41084d0 0x7fb5f4101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9546700 1 -- 172.21.15.27:0/2482103441 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f406b3a0 msgr2=0x7fb5f4101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9546700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f406b3a0 0x7fb5f4101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9546700 1 -- 172.21.15.27:0/2482103441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f41a71d0 con 0x7fb5f406aa00 2024-06-06T08:41:19.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5f9546700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406aa00 0x7fb5f4101450 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000d950 tx=0x7fb5f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5ea7fc700 1 -- 172.21.15.27:0/2482103441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0016070 con 0x7fb5f406aa00 2024-06-06T08:41:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5ea7fc700 1 -- 172.21.15.27:0/2482103441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f000fd70 con 0x7fb5f406aa00 2024-06-06T08:41:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.331+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/2482103441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f41a74c0 con 0x7fb5f406aa00 2024-06-06T08:41:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.332+0000 7fb5ea7fc700 1 -- 172.21.15.27:0/2482103441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000e610 con 0x7fb5f406aa00 2024-06-06T08:41:19.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.332+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/2482103441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f41a7a00 con 0x7fb5f406aa00 2024-06-06T08:41:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.334+0000 7fb5ea7fc700 1 -- 172.21.15.27:0/2482103441 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fb5f0019070 con 0x7fb5f406aa00 2024-06-06T08:41:19.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.335+0000 7fb5ea7fc700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5e00a2f40 0x7fb5e00a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.335+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5e00a2f40 0x7fb5e00a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.335+0000 7fb5ea7fc700 1 -- 172.21.15.27:0/2482103441 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb5f00cf560 con 0x7fb5f406aa00 2024-06-06T08:41:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.336+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5e00a2f40 0x7fb5e00a5400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4078930 tx=0x7fb5e4009190 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.336+0000 7fb60602db80 1 Processor -- start 2024-06-06T08:41:19.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.336+0000 7fb60602db80 1 -- start start 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.336+0000 7fb60602db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef652eaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.336+0000 7fb60602db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef652e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.336+0000 7fb60602db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.336+0000 7fb60602db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ef65236810 con 0x55ef652e4f20 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.336+0000 7fb60602db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ef651de720 con 0x55ef652ea640 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.336+0000 7fb60602db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ef652327d0 con 0x55ef652eaf60 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9d47700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9d47700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33774/0 (socket says 172.21.15.27:33774) 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9d47700 1 -- 172.21.15.27:0/4264304976 learned_addr learned my addr 172.21.15.27:0/4264304976 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef652e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9546700 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef652eaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9d47700 1 -- 172.21.15.27:0/4264304976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 msgr2=0x55ef652e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9d47700 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef652e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9d47700 1 -- 172.21.15.27:0/4264304976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 msgr2=0x55ef652eaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9d47700 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef652eaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9d47700 1 -- 172.21.15.27:0/4264304976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef651d4f50 con 0x55ef652e4f20 2024-06-06T08:41:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9546700 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef652eaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5f9d47700 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e5340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009200 tx=0x7fb5ec009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c4ed59d2641c85 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5df7fe700 1 -- 172.21.15.27:0/4264304976 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec020070 con 0x55ef652e4f20 2024-06-06T08:41:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5df7fe700 1 -- 172.21.15.27:0/4264304976 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ec009450 con 0x55ef652e4f20 2024-06-06T08:41:19.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.337+0000 7fb5df7fe700 1 -- 172.21.15.27:0/4264304976 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0052a0 con 0x55ef652e4f20 2024-06-06T08:41:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.338+0000 7fb60602db80 1 -- 172.21.15.27:0/4264304976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 msgr2=0x55ef652e5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.338+0000 7fb60602db80 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e5340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009200 tx=0x7fb5ec009230 comp rx=0 tx=0).stop 2024-06-06T08:41:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.338+0000 7fb60602db80 1 -- 172.21.15.27:0/4264304976 shutdown_connections 2024-06-06T08:41:19.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.338+0000 7fb60602db80 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e5340 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.338+0000 7fb60602db80 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef652eaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.338+0000 7fb60602db80 1 --2- 172.21.15.27:0/4264304976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef652e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.338+0000 7fb60602db80 1 -- 172.21.15.27:0/4264304976 >> 172.21.15.27:0/4264304976 conn(0x55ef6520fcc0 msgr2=0x55ef652df850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.338+0000 7fb60602db80 1 -- 172.21.15.27:0/4264304976 shutdown_connections 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.339+0000 7fb60602db80 1 -- 172.21.15.27:0/4264304976 wait complete. 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.339+0000 7fb60602db80 1 -- start start 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.339+0000 7fb60602db80 1 Processor -- start 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb60602db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb60602db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652ea640 0x55ef652edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb60602db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652eaf60 0x55ef652ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb60602db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ef652f1970 con 0x55ef652e4f20 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb60602db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ef651d6bf0 con 0x55ef652eaf60 2024-06-06T08:41:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb60602db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ef652f1af0 con 0x55ef652ea640 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb60602db80 10 client.0 _ll_register_callbacks cb 0x55ef651d0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f9546700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f9546700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33786/0 (socket says 172.21.15.27:33786) 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f9546700 1 -- 172.21.15.27:0/2135207778 learned_addr learned my addr 172.21.15.27:0/2135207778 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652ea640 0x55ef652edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f9d47700 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652eaf60 0x55ef652ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f9546700 1 -- 172.21.15.27:0/2135207778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652ea640 msgr2=0x55ef652edfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f9546700 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652ea640 0x55ef652edfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f9546700 1 -- 172.21.15.27:0/2135207778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652eaf60 msgr2=0x55ef652ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f9546700 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652eaf60 0x55ef652ee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.340+0000 7fb5f9546700 1 -- 172.21.15.27:0/2135207778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef651858d0 con 0x55ef652e4f20 2024-06-06T08:41:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb5f9546700 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f0cb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000dfb0 tx=0x7fb5f009aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db634a6239c75980 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb5dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb5dcff9700 1 -- 172.21.15.27:0/2135207778 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00b4060 con 0x55ef652e4f20 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb5dcff9700 1 -- 172.21.15.27:0/2135207778 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb5f00cf150 con 0x55ef652e4f20 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb60602db80 1 -- 172.21.15.27:0/2135207778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ec009450 con 0x55ef652e4f20 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb5dcff9700 1 -- 172.21.15.27:0/2135207778 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5f0020040 con 0x55ef652e4f20 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb60602db80 10 client.15621 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb60602db80 1 -- 172.21.15.27:0/2135207778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef652fc2b0 con 0x55ef652e4f20 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb5dcff9700 1 -- 172.21.15.27:0/2135207778 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ec009450 con 0x55ef652e4f20 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.341+0000 7fb5dcff9700 1 -- 172.21.15.27:0/2135207778 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb5f00a3ca0 con 0x55ef652e4f20 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.342+0000 7fb60602db80 10 client.15621 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.342+0000 7fb60602db80 10 client.15621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.342+0000 7fb60602db80 20 client.15621 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.342+0000 7fb60602db80 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55ef652fe8f0 0x55ef65300ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.342+0000 7fb60602db80 4 client.15621 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.342+0000 7fb60602db80 1 -- 172.21.15.27:0/2135207778 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef65243d10 con 0x55ef652fe8f0 2024-06-06T08:41:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.342+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55ef652fe8f0 0x55ef65300ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.342+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55ef652fe8f0 0x55ef65300ce0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1e1b8818cacf14c server_cookie=125fa852c2c9757e in_seq=0 out_seq=0 2024-06-06T08:41:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.342+0000 7fb5dcff9700 10 client.15621 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.343+0000 7fb5dcff9700 1 -- 172.21.15.27:0/2135207778 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef65243d10 con 0x55ef652fe8f0 2024-06-06T08:41:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.343+0000 7fb5dcff9700 10 client.15621 handle_command_reply: tid=0 2024-06-06T08:41:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.343+0000 7fb60602db80 1 client.15621 shutdown 2024-06-06T08:41:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.343+0000 7fb60602db80 1 -- 172.21.15.27:0/2135207778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 msgr2=0x55ef652f0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.343+0000 7fb60602db80 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f0cb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000dfb0 tx=0x7fb5f009aea0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb60602db80 1 -- 172.21.15.27:0/2135207778 shutdown_connections 2024-06-06T08:41:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb60602db80 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55ef652fe8f0 0x55ef65300ce0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb5dcff9700 0 client.15621 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb60602db80 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652eaf60 0x55ef652ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb60602db80 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f0cb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb60602db80 1 --2- 172.21.15.27:0/2135207778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652ea640 0x55ef652edfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb60602db80 1 -- 172.21.15.27:0/2135207778 >> 172.21.15.27:0/2135207778 conn(0x55ef6520fcc0 msgr2=0x55ef652e39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb60602db80 1 -- 172.21.15.27:0/2135207778 shutdown_connections 2024-06-06T08:41:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb60602db80 1 -- 172.21.15.27:0/2135207778 wait complete. 2024-06-06T08:41:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.344+0000 7fb60602db80 20 client.15621 trim_cache size 0 max 16384 2024-06-06T08:41:19.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb60602db80 1 Processor -- start 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb60602db80 1 -- start start 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb60602db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb60602db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef652eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb60602db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef652eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb60602db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41b3000 con 0x55ef652e4f20 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb60602db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41b0890 con 0x55ef652ea640 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb60602db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41ae6e0 con 0x55ef652eaf60 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb5f9546700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.377+0000 7fb5f9546700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33802/0 (socket says 172.21.15.27:33802) 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f9546700 1 -- 172.21.15.27:0/1106811027 learned_addr learned my addr 172.21.15.27:0/1106811027 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef652eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f9d47700 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef652eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f9546700 1 -- 172.21.15.27:0/1106811027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 msgr2=0x55ef652eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:19.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f9546700 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef652eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f9546700 1 -- 172.21.15.27:0/1106811027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 msgr2=0x55ef652eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:19.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f9546700 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef652eb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f9546700 1 -- 172.21.15.27:0/1106811027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef652ec540 con 0x55ef652e4f20 2024-06-06T08:41:19.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f9d47700 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef652eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:19.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5f9546700 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e49e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00cf4c0 tx=0x7fb5f0005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7364bbc2e734cdfa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:19.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5deffd700 1 -- 172.21.15.27:0/1106811027 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00b4040 con 0x55ef652e4f20 2024-06-06T08:41:19.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5deffd700 1 -- 172.21.15.27:0/1106811027 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f00a4130 con 0x55ef652e4f20 2024-06-06T08:41:19.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.378+0000 7fb5deffd700 1 -- 172.21.15.27:0/1106811027 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f009b4c0 con 0x55ef652e4f20 2024-06-06T08:41:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 -- 172.21.15.27:0/1106811027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 msgr2=0x55ef652e49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e49e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00cf4c0 tx=0x7fb5f0005910 comp rx=0 tx=0).stop 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 -- 172.21.15.27:0/1106811027 shutdown_connections 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef652eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652e49e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 --2- 172.21.15.27:0/1106811027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef652eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 -- 172.21.15.27:0/1106811027 >> 172.21.15.27:0/1106811027 conn(0x55ef6520fcc0 msgr2=0x55ef652df380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 -- 172.21.15.27:0/1106811027 shutdown_connections 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 -- 172.21.15.27:0/1106811027 wait complete. 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.379+0000 7fb60602db80 1 -- start start 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb60602db80 1 Processor -- start 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb60602db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb60602db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef65305cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb60602db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef653061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb60602db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41aed50 con 0x55ef652e4f20 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb60602db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41b01c0 con 0x55ef652ea640 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb60602db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41af420 con 0x55ef652eaf60 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb60602db80 10 client.0 _ll_register_callbacks cb 0x55ef651d0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb5f9546700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb5f9546700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33814/0 (socket says 172.21.15.27:33814) 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb5f9546700 1 -- 172.21.15.27:0/3003330301 learned_addr learned my addr 172.21.15.27:0/3003330301 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb5f9d47700 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef653061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.380+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef65305cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5f9546700 1 -- 172.21.15.27:0/3003330301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 msgr2=0x55ef653061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5f9546700 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef653061f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5f9546700 1 -- 172.21.15.27:0/3003330301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 msgr2=0x55ef65305cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5f9546700 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef65305cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5f9546700 1 -- 172.21.15.27:0/3003330301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef65306830 con 0x55ef652e4f20 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5f9546700 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f7370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb5f009b450 tx=0x7fb5f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678de8d50d2bff09 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5dcff9700 1 -- 172.21.15.27:0/3003330301 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f009b4c0 con 0x55ef652e4f20 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5dcff9700 1 -- 172.21.15.27:0/3003330301 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb5f0066310 con 0x55ef652e4f20 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb60602db80 1 -- 172.21.15.27:0/3003330301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5f00a4130 con 0x55ef652e4f20 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5dcff9700 1 -- 172.21.15.27:0/3003330301 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5f0066510 con 0x55ef652e4f20 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb60602db80 10 client.15627 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb60602db80 1 -- 172.21.15.27:0/3003330301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef65369260 con 0x55ef652e4f20 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.381+0000 7fb5dcff9700 1 -- 172.21.15.27:0/3003330301 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f00a4130 con 0x55ef652e4f20 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb5dcff9700 1 -- 172.21.15.27:0/3003330301 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb5f00cca00 con 0x55ef652e4f20 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb60602db80 10 client.15627 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb60602db80 10 client.15627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:41:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb60602db80 20 client.15627 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb60602db80 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55ef652fe8f0 0x55ef6536b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb60602db80 4 client.15627 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb60602db80 1 -- 172.21.15.27:0/3003330301 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef6523e290 con 0x55ef652fe8f0 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55ef652fe8f0 0x55ef6536b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb5dcff9700 10 client.15627 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.382+0000 7fb5f8d45700 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55ef652fe8f0 0x55ef6536b6d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49537ec6ec03cc56 server_cookie=c9c06492451727a3 in_seq=0 out_seq=0 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb5dcff9700 1 -- 172.21.15.27:0/3003330301 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef6523e290 con 0x55ef652fe8f0 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb5dcff9700 10 client.15627 handle_command_reply: tid=0 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 client.15627 shutdown 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 -- 172.21.15.27:0/3003330301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 msgr2=0x55ef652f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f7370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb5f009b450 tx=0x7fb5f0000bc0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 -- 172.21.15.27:0/3003330301 shutdown_connections 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef652eaf60 0x55ef653061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ef652e4f20 0x55ef652f7370 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55ef652fe8f0 0x55ef6536b6d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 --2- 172.21.15.27:0/3003330301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ef652ea640 0x55ef65305cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 -- 172.21.15.27:0/3003330301 >> 172.21.15.27:0/3003330301 conn(0x55ef6520fcc0 msgr2=0x55ef652d8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb5dcff9700 0 client.15627 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 -- 172.21.15.27:0/3003330301 shutdown_connections 2024-06-06T08:41:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 1 -- 172.21.15.27:0/3003330301 wait complete. 2024-06-06T08:41:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.383+0000 7fb60602db80 20 client.15627 trim_cache size 0 max 16384 2024-06-06T08:41:19.386 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:41:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5e00a2f40 msgr2=0x7fb5e00a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5e00a2f40 0x7fb5e00a5400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4078930 tx=0x7fb5e4009190 comp rx=0 tx=0).stop 2024-06-06T08:41:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406aa00 msgr2=0x7fb5f4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406aa00 0x7fb5f4101450 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000d950 tx=0x7fb5f000dc60 comp rx=0 tx=0).stop 2024-06-06T08:41:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/2482103441 shutdown_connections 2024-06-06T08:41:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f41084d0 0x7fb5f4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5f406aa00 0x7fb5f4101450 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5e00a2f40 0x7fb5e00a5400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 --2- 172.21.15.27:0/2482103441 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5f406b3a0 0x7fb5f4101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/2482103441 >> 172.21.15.27:0/2482103441 conn(0x7fb5f4078330 msgr2=0x7fb5f40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/2482103441 shutdown_connections 2024-06-06T08:41:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:19.384+0000 7fb5fb7aa700 1 -- 172.21.15.27:0/2482103441 wait complete. 2024-06-06T08:41:20.296 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T08:41:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:21.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:20 smithi027 ceph-mon[32335]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:41:21.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:20 smithi203 ceph-mon[36579]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:41:21.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:20 smithi176 ceph-mon[36786]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.450+0000 7f13393d6700 1 -- 172.21.15.27:0/3424205477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13341084d0 msgr2=0x7f13341088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.450+0000 7f13393d6700 1 --2- 172.21.15.27:0/3424205477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13341084d0 0x7f13341088b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f132400b0a0 tx=0x7f132400b3b0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.450+0000 7f13393d6700 1 -- 172.21.15.27:0/3424205477 shutdown_connections 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.450+0000 7f13393d6700 1 --2- 172.21.15.27:0/3424205477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f133406b3a0 0x7f133410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.450+0000 7f13393d6700 1 --2- 172.21.15.27:0/3424205477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13341084d0 0x7f13341088b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.450+0000 7f13393d6700 1 --2- 172.21.15.27:0/3424205477 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f133406aa00 0x7f133406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.450+0000 7f13393d6700 1 -- 172.21.15.27:0/3424205477 >> 172.21.15.27:0/3424205477 conn(0x7f1334078330 msgr2=0x7f1334078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.451+0000 7f13393d6700 1 -- 172.21.15.27:0/3424205477 shutdown_connections 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.451+0000 7f13393d6700 1 -- 172.21.15.27:0/3424205477 wait complete. 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.451+0000 7f13393d6700 1 Processor -- start 2024-06-06T08:41:22.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13393d6700 1 -- start start 2024-06-06T08:41:22.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13393d6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f133406aa00 0x7f133419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13393d6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f133406b3a0 0x7f133419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13393d6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13341084d0 0x7f1334198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13393d6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1334112730 con 0x7f133406b3a0 2024-06-06T08:41:22.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13393d6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f13341125b0 con 0x7f133406aa00 2024-06-06T08:41:22.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13393d6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13341128b0 con 0x7f13341084d0 2024-06-06T08:41:22.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13327fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f133406b3a0 0x7f133419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13327fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f133406b3a0 0x7f133419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43436/0 (socket says 172.21.15.27:43436) 2024-06-06T08:41:22.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13327fc700 1 -- 172.21.15.27:0/428147675 learned_addr learned my addr 172.21.15.27:0/428147675 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f1332ffd700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f133406aa00 0x7f133419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13337fe700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13341084d0 0x7f1334198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13327fc700 1 -- 172.21.15.27:0/428147675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13341084d0 msgr2=0x7f1334198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13327fc700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13341084d0 0x7f1334198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13327fc700 1 -- 172.21.15.27:0/428147675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f133406aa00 msgr2=0x7f133419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13327fc700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f133406aa00 0x7f133419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.452+0000 7f13327fc700 1 -- 172.21.15.27:0/428147675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1334199230 con 0x7f133406b3a0 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.453+0000 7f13327fc700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f133406b3a0 0x7f133419ee50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f131c00d950 tx=0x7f131c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.453+0000 7f132bfff700 1 -- 172.21.15.27:0/428147675 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c016070 con 0x7f133406b3a0 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.453+0000 7f132bfff700 1 -- 172.21.15.27:0/428147675 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c00fd70 con 0x7f133406b3a0 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.453+0000 7f13393d6700 1 -- 172.21.15.27:0/428147675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1334199520 con 0x7f133406b3a0 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.453+0000 7f132bfff700 1 -- 172.21.15.27:0/428147675 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c00e610 con 0x7f133406b3a0 2024-06-06T08:41:22.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.453+0000 7f13393d6700 1 -- 172.21.15.27:0/428147675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13341a57e0 con 0x7f133406b3a0 2024-06-06T08:41:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.455+0000 7f132bfff700 1 -- 172.21.15.27:0/428147675 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f131c019070 con 0x7f133406b3a0 2024-06-06T08:41:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.456+0000 7f132bfff700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13200a2f40 0x7f13200a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.456+0000 7f1332ffd700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13200a2f40 0x7f13200a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.456+0000 7f132bfff700 1 -- 172.21.15.27:0/428147675 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f131c0cf980 con 0x7f133406b3a0 2024-06-06T08:41:22.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.456+0000 7f1332ffd700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13200a2f40 0x7f13200a5400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1324009e60 tx=0x7f132400b7b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:22.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f1343c59b80 1 Processor -- start 2024-06-06T08:41:22.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f1343c59b80 1 -- start start 2024-06-06T08:41:22.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0640 0x56377e4d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 0x56377e4ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f1343c59b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56377e41c810 con 0x56377e4caf20 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f1343c59b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56377e3c4720 con 0x56377e4d0640 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f1343c59b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56377e4187d0 con 0x56377e4d0f60 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f13337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f13337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43450/0 (socket says 172.21.15.27:43450) 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.457+0000 7f13337fe700 1 -- 172.21.15.27:0/3865034811 learned_addr learned my addr 172.21.15.27:0/3865034811 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13327fc700 1 --2- 172.21.15.27:0/3865034811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 0x56377e4ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f1332ffd700 1 --2- 172.21.15.27:0/3865034811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0640 0x56377e4d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13337fe700 1 -- 172.21.15.27:0/3865034811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 msgr2=0x56377e4ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13337fe700 1 --2- 172.21.15.27:0/3865034811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 0x56377e4ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13337fe700 1 -- 172.21.15.27:0/3865034811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0640 msgr2=0x56377e4d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13337fe700 1 --2- 172.21.15.27:0/3865034811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0640 0x56377e4d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13337fe700 1 -- 172.21.15.27:0/3865034811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56377e3baf50 con 0x56377e4caf20 2024-06-06T08:41:22.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13337fe700 1 --2- 172.21.15.27:0/3865034811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4cb340 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f132c00abd0 tx=0x7f132c00aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4935bbc1ab8dbf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:22.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13297fa700 1 -- 172.21.15.27:0/3865034811 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c00fd80 con 0x56377e4caf20 2024-06-06T08:41:22.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13297fa700 1 -- 172.21.15.27:0/3865034811 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f132c004500 con 0x56377e4caf20 2024-06-06T08:41:22.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.458+0000 7f13297fa700 1 -- 172.21.15.27:0/3865034811 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c0184a0 con 0x56377e4caf20 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 -- 172.21.15.27:0/3865034811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 msgr2=0x56377e4cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 --2- 172.21.15.27:0/3865034811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4cb340 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f132c00abd0 tx=0x7f132c00aee0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 -- 172.21.15.27:0/3865034811 shutdown_connections 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 --2- 172.21.15.27:0/3865034811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4cb340 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 --2- 172.21.15.27:0/3865034811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0640 0x56377e4d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 --2- 172.21.15.27:0/3865034811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 0x56377e4ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 -- 172.21.15.27:0/3865034811 >> 172.21.15.27:0/3865034811 conn(0x56377e3f5cc0 msgr2=0x56377e4c5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 -- 172.21.15.27:0/3865034811 shutdown_connections 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 -- 172.21.15.27:0/3865034811 wait complete. 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.459+0000 7f1343c59b80 1 -- start start 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1343c59b80 1 Processor -- start 2024-06-06T08:41:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 0x56377e4d6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0640 0x56377e4d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 0x56377e4d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1343c59b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56377e4d79d0 con 0x56377e4d0640 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1343c59b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x56377e3bcbf0 con 0x56377e4caf20 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1343c59b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56377e4d7b50 con 0x56377e4d0f60 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1343c59b80 10 client.0 _ll_register_callbacks cb 0x56377e3b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13327fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0640 0x56377e4d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13327fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0640 0x56377e4d3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43460/0 (socket says 172.21.15.27:43460) 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13327fc700 1 -- 172.21.15.27:0/797655106 learned_addr learned my addr 172.21.15.27:0/797655106 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1332ffd700 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 0x56377e4d6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13337fe700 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 0x56377e4d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13327fc700 1 -- 172.21.15.27:0/797655106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 msgr2=0x56377e4d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13327fc700 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 0x56377e4d44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13327fc700 1 -- 172.21.15.27:0/797655106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 msgr2=0x56377e4d6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13327fc700 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 0x56377e4d6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13327fc700 1 -- 172.21.15.27:0/797655106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56377e36b8d0 con 0x56377e4d0640 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f1332ffd700 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 0x56377e4d6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.460+0000 7f13337fe700 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 0x56377e4d44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f13327fc700 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0640 0x56377e4d3fb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f131c09b830 tx=0x7f131c09c0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989ef02f72dc2453 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f131a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f131a7fc700 1 -- 172.21.15.27:0/797655106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c020070 con 0x56377e4d0640 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f131a7fc700 1 -- 172.21.15.27:0/797655106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f131c0860b0 con 0x56377e4d0640 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f1343c59b80 1 -- 172.21.15.27:0/797655106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f132c004500 con 0x56377e4d0640 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f131a7fc700 1 -- 172.21.15.27:0/797655106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f131c010e10 con 0x56377e4d0640 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f1343c59b80 10 client.15639 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f1343c59b80 1 -- 172.21.15.27:0/797655106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56377e4e2330 con 0x56377e4d0640 2024-06-06T08:41:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f131a7fc700 1 -- 172.21.15.27:0/797655106 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f132c004500 con 0x56377e4d0640 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f131a7fc700 1 -- 172.21.15.27:0/797655106 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f131c09cbd0 con 0x56377e4d0640 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f1343c59b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f1343c59b80 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f1343c59b80 20 client.15639 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f1343c59b80 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56377e4e4970 0x56377e4e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f1343c59b80 4 client.15639 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.461+0000 7f1343c59b80 1 -- 172.21.15.27:0/797655106 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56377e429d10 con 0x56377e4e4970 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.462+0000 7f1332ffd700 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56377e4e4970 0x56377e4e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.462+0000 7f1332ffd700 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56377e4e4970 0x56377e4e6d60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36bf5a2bbe76da2 server_cookie=3b70a8aad4e102b3 in_seq=0 out_seq=0 2024-06-06T08:41:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.462+0000 7f131a7fc700 10 client.15639 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:22.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f131a7fc700 1 -- 172.21.15.27:0/797655106 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56377e429d10 con 0x56377e4e4970 2024-06-06T08:41:22.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f131a7fc700 10 client.15639 handle_command_reply: tid=0 2024-06-06T08:41:22.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 client.15639 shutdown 2024-06-06T08:41:22.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 -- 172.21.15.27:0/797655106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0640 msgr2=0x56377e4d3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0640 0x56377e4d3fb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f131c09b830 tx=0x7f131c09c0d0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 -- 172.21.15.27:0/797655106 shutdown_connections 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0f60 0x56377e4d44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 0x56377e4d6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56377e4e4970 0x56377e4e6d60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 --2- 172.21.15.27:0/797655106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0640 0x56377e4d3fb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f131a7fc700 0 client.15639 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 -- 172.21.15.27:0/797655106 >> 172.21.15.27:0/797655106 conn(0x56377e3f5cc0 msgr2=0x56377e4c99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 -- 172.21.15.27:0/797655106 shutdown_connections 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 1 -- 172.21.15.27:0/797655106 wait complete. 2024-06-06T08:41:22.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.463+0000 7f1343c59b80 20 client.15639 trim_cache size 0 max 16384 2024-06-06T08:41:22.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.478+0000 7f1343c59b80 1 Processor -- start 2024-06-06T08:41:22.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.478+0000 7f1343c59b80 1 -- start start 2024-06-06T08:41:22.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 0x56377e4ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 0x56377e4bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0f60 0x56377e4ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f1343c59b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f13341b0ee0 con 0x56377e4d0f60 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f1343c59b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f13341ae770 con 0x56377e4caf20 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f1343c59b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13341ac5c0 con 0x56377e4d0640 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f13337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0f60 0x56377e4ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f13337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0f60 0x56377e4ec610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43476/0 (socket says 172.21.15.27:43476) 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f13337fe700 1 -- 172.21.15.27:0/1747171577 learned_addr learned my addr 172.21.15.27:0/1747171577 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f1332ffd700 1 --2- 172.21.15.27:0/1747171577 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 0x56377e4ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f13327fc700 1 --2- 172.21.15.27:0/1747171577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 0x56377e4bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f13337fe700 1 -- 172.21.15.27:0/1747171577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 msgr2=0x56377e4bf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f13337fe700 1 --2- 172.21.15.27:0/1747171577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 0x56377e4bf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f13337fe700 1 -- 172.21.15.27:0/1747171577 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 msgr2=0x56377e4ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f13337fe700 1 --2- 172.21.15.27:0/1747171577 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 0x56377e4ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.479+0000 7f13337fe700 1 -- 172.21.15.27:0/1747171577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56377e4c6500 con 0x56377e4d0f60 2024-06-06T08:41:22.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.480+0000 7f13337fe700 1 --2- 172.21.15.27:0/1747171577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0f60 0x56377e4ec610 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f132c005d30 tx=0x7f132c005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f559c01cd9442d30 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:22.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.480+0000 7f1328ff9700 1 -- 172.21.15.27:0/1747171577 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c027070 con 0x56377e4d0f60 2024-06-06T08:41:22.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.480+0000 7f1328ff9700 1 -- 172.21.15.27:0/1747171577 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f132c00fd80 con 0x56377e4d0f60 2024-06-06T08:41:22.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.480+0000 7f1328ff9700 1 -- 172.21.15.27:0/1747171577 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c005040 con 0x56377e4d0f60 2024-06-06T08:41:22.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.480+0000 7f1343c59b80 1 -- 172.21.15.27:0/1747171577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0f60 msgr2=0x56377e4ec610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:22.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.480+0000 7f1343c59b80 1 --2- 172.21.15.27:0/1747171577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0f60 0x56377e4ec610 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f132c005d30 tx=0x7f132c005bf0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 -- 172.21.15.27:0/1747171577 shutdown_connections 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 --2- 172.21.15.27:0/1747171577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4d0f60 0x56377e4ec610 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 --2- 172.21.15.27:0/1747171577 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4caf20 0x56377e4ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 --2- 172.21.15.27:0/1747171577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 0x56377e4bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 -- 172.21.15.27:0/1747171577 >> 172.21.15.27:0/1747171577 conn(0x56377e3f5cc0 msgr2=0x56377e4d1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 -- 172.21.15.27:0/1747171577 shutdown_connections 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 -- 172.21.15.27:0/1747171577 wait complete. 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 -- start start 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 Processor -- start 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4dd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 0x56377e4ec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0f60 0x56377e4ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f13341acc30 con 0x56377e4caf20 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f13341ae0a0 con 0x56377e4d0f60 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13341ad300 con 0x56377e4d0640 2024-06-06T08:41:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1343c59b80 10 client.0 _ll_register_callbacks cb 0x56377e3b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1332ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1332ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4dd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43490/0 (socket says 172.21.15.27:43490) 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.481+0000 7f1332ffd700 1 -- 172.21.15.27:0/4090122647 learned_addr learned my addr 172.21.15.27:0/4090122647 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f13327fc700 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 0x56377e4ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f13337fe700 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0f60 0x56377e4ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f1332ffd700 1 -- 172.21.15.27:0/4090122647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 msgr2=0x56377e4ec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f1332ffd700 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 0x56377e4ec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f1332ffd700 1 -- 172.21.15.27:0/4090122647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0f60 msgr2=0x56377e4ec5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f1332ffd700 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0f60 0x56377e4ec5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f1332ffd700 1 -- 172.21.15.27:0/4090122647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56377e4ece00 con 0x56377e4caf20 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f1332ffd700 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4dd620 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f132400d010 tx=0x7f132401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35eaf76cd2c692c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f131a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f131a7fc700 1 -- 172.21.15.27:0/4090122647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324035070 con 0x56377e4caf20 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f131a7fc700 1 -- 172.21.15.27:0/4090122647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f1324007bf0 con 0x56377e4caf20 2024-06-06T08:41:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f1343c59b80 1 -- 172.21.15.27:0/4090122647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f132c00fd80 con 0x56377e4caf20 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f131a7fc700 1 -- 172.21.15.27:0/4090122647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1324041050 con 0x56377e4caf20 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f1343c59b80 10 client.15645 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.482+0000 7f1343c59b80 1 -- 172.21.15.27:0/4090122647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56377e553aa0 con 0x56377e4caf20 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.483+0000 7f131a7fc700 1 -- 172.21.15.27:0/4090122647 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f132c00fd80 con 0x56377e4caf20 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.483+0000 7f131a7fc700 1 -- 172.21.15.27:0/4090122647 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1324011070 con 0x56377e4caf20 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.483+0000 7f1343c59b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.483+0000 7f1343c59b80 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.483+0000 7f1343c59b80 20 client.15645 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.483+0000 7f1343c59b80 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56377e554160 0x56377e556580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.483+0000 7f1343c59b80 4 client.15645 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.483+0000 7f1343c59b80 1 -- 172.21.15.27:0/4090122647 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56377e424290 con 0x56377e554160 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.483+0000 7f13327fc700 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56377e554160 0x56377e556580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f131a7fc700 10 client.15645 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f13327fc700 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56377e554160 0x56377e556580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5eff3da613f6a8 server_cookie=13b053a1ebc468ad in_seq=0 out_seq=0 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f131a7fc700 1 -- 172.21.15.27:0/4090122647 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56377e424290 con 0x56377e554160 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f131a7fc700 10 client.15645 handle_command_reply: tid=0 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 client.15645 shutdown 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 -- 172.21.15.27:0/4090122647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 msgr2=0x56377e4dd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4dd620 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f132400d010 tx=0x7f132401d000 comp rx=0 tx=0).stop 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 -- 172.21.15.27:0/4090122647 shutdown_connections 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56377e4d0f60 0x56377e4ec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56377e554160 0x56377e556580 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56377e4caf20 0x56377e4dd620 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 --2- 172.21.15.27:0/4090122647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56377e4d0640 0x56377e4ec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f131a7fc700 0 client.15645 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 -- 172.21.15.27:0/4090122647 >> 172.21.15.27:0/4090122647 conn(0x56377e3f5cc0 msgr2=0x56377e4d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 -- 172.21.15.27:0/4090122647 shutdown_connections 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 1 -- 172.21.15.27:0/4090122647 wait complete. 2024-06-06T08:41:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.484+0000 7f1343c59b80 20 client.15645 trim_cache size 0 max 16384 2024-06-06T08:41:22.488 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:41:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 -- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13200a2f40 msgr2=0x7f13200a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13200a2f40 0x7f13200a5400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1324009e60 tx=0x7f132400b7b0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 -- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f133406b3a0 msgr2=0x7f133419ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f133406b3a0 0x7f133419ee50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f131c00d950 tx=0x7f131c00dc60 comp rx=0 tx=0).stop 2024-06-06T08:41:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 -- 172.21.15.27:0/428147675 shutdown_connections 2024-06-06T08:41:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13341084d0 0x7f1334198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13200a2f40 0x7f13200a5400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f133406aa00 0x7f133419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 --2- 172.21.15.27:0/428147675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f133406b3a0 0x7f133419ee50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 -- 172.21.15.27:0/428147675 >> 172.21.15.27:0/428147675 conn(0x7f1334078330 msgr2=0x7f13340ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 -- 172.21.15.27:0/428147675 shutdown_connections 2024-06-06T08:41:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:22.485+0000 7f13393d6700 1 -- 172.21.15.27:0/428147675 wait complete. 2024-06-06T08:41:22.671 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.4 2024-06-06T08:41:22.671 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:41:22.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:22 smithi027 ceph-mon[32335]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-06T08:41:23.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:22 smithi203 ceph-mon[36579]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-06T08:41:23.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:22 smithi176 ceph-mon[36786]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-06T08:41:23.266 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:24.180 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T08:41:24.180 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T08:41:24.528 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:25.050 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:24 smithi027 ceph-mon[32335]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-06T08:41:25.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:24 smithi203 ceph-mon[36579]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-06T08:41:25.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.249+0000 7f3003b5d700 1 -- 172.21.15.27:0/193381861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101390 msgr2=0x7f2ffc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.249+0000 7f3003b5d700 1 --2- 172.21.15.27:0/193381861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101390 0x7f2ffc101810 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009990 tx=0x7f2ff0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.249+0000 7f3003b5d700 1 -- 172.21.15.27:0/193381861 shutdown_connections 2024-06-06T08:41:25.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.249+0000 7f3003b5d700 1 --2- 172.21.15.27:0/193381861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ffc101d50 0x7f2ffc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.249+0000 7f3003b5d700 1 --2- 172.21.15.27:0/193381861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ffc107400 0x7f2ffc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.249+0000 7f3003b5d700 1 --2- 172.21.15.27:0/193381861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101390 0x7f2ffc101810 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.250+0000 7f3003b5d700 1 -- 172.21.15.27:0/193381861 >> 172.21.15.27:0/193381861 conn(0x7f2ffc078340 msgr2=0x7f2ffc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:25.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.250+0000 7f3003b5d700 1 -- 172.21.15.27:0/193381861 shutdown_connections 2024-06-06T08:41:25.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.250+0000 7f3003b5d700 1 -- 172.21.15.27:0/193381861 wait complete. 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.250+0000 7f3003b5d700 1 Processor -- start 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.250+0000 7f3003b5d700 1 -- start start 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f3003b5d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ffc101390 0x7f2ffc103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f3003b5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101d50 0x7f2ffc1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f3003b5d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ffc107400 0x7f2ffc103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f3003b5d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc111690 con 0x7f2ffc101d50 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f3003b5d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc111510 con 0x7f2ffc101390 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f3003b5d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc111810 con 0x7f2ffc107400 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30010f8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101d50 0x7f2ffc1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30010f8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101d50 0x7f2ffc1035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43506/0 (socket says 172.21.15.27:43506) 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30010f8700 1 -- 172.21.15.27:0/3981323443 learned_addr learned my addr 172.21.15.27:0/3981323443 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30018f9700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ffc101390 0x7f2ffc103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30020fa700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ffc107400 0x7f2ffc103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30010f8700 1 -- 172.21.15.27:0/3981323443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ffc107400 msgr2=0x7f2ffc103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30010f8700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ffc107400 0x7f2ffc103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30010f8700 1 -- 172.21.15.27:0/3981323443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ffc101390 msgr2=0x7f2ffc103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30010f8700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ffc101390 0x7f2ffc103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30010f8700 1 -- 172.21.15.27:0/3981323443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc1a7140 con 0x7f2ffc101d50 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30020fa700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ffc107400 0x7f2ffc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f30010f8700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101d50 0x7f2ffc1035b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000c00 tx=0x7f2ff00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f2feeffd700 1 -- 172.21.15.27:0/3981323443 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0027070 con 0x7f2ffc101d50 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.251+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc1a73d0 con 0x7f2ffc101d50 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.252+0000 7f2feeffd700 1 -- 172.21.15.27:0/3981323443 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff0003780 con 0x7f2ffc101d50 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.252+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1a7880 con 0x7f2ffc101d50 2024-06-06T08:41:25.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.252+0000 7f2feeffd700 1 -- 172.21.15.27:0/3981323443 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0022dd0 con 0x7f2ffc101d50 2024-06-06T08:41:25.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.253+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc069180 con 0x7f2ffc101d50 2024-06-06T08:41:25.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.254+0000 7f2feeffd700 1 -- 172.21.15.27:0/3981323443 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f2ff002a070 con 0x7f2ffc101d50 2024-06-06T08:41:25.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.254+0000 7f2feeffd700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2fe80a2fc0 0x7f2fe80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.254+0000 7f30018f9700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2fe80a2fc0 0x7f2fe80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.254+0000 7f2feeffd700 1 -- 172.21.15.27:0/3981323443 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f2ff00d0740 con 0x7f2ffc101d50 2024-06-06T08:41:25.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.255+0000 7f30018f9700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2fe80a2fc0 0x7f2fe80a5480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80045b0 tx=0x7f2ff800b3c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:25.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:24 smithi176 ceph-mon[36786]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-06T08:41:25.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.259+0000 7f2feeffd700 1 -- 172.21.15.27:0/3981323443 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2ff0013070 con 0x7f2ffc101d50 2024-06-06T08:41:25.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.512+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ffc1a7bf0 con 0x7f2ffc101d50 2024-06-06T08:41:25.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.512+0000 7f2feeffd700 1 -- 172.21.15.27:0/3981323443 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7f2ff009ce80 con 0x7f2ffc101d50 2024-06-06T08:41:25.511 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:41:25.511 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:41:25.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.514+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2fe80a2fc0 msgr2=0x7f2fe80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.514+0000 7f3003b5d700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2fe80a2fc0 0x7f2fe80a5480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80045b0 tx=0x7f2ff800b3c0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.514+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101d50 msgr2=0x7f2ffc1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.514+0000 7f3003b5d700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101d50 0x7f2ffc1035b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000c00 tx=0x7f2ff00047b0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.514+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 shutdown_connections 2024-06-06T08:41:25.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.514+0000 7f3003b5d700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ffc107400 0x7f2ffc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.514+0000 7f3003b5d700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2fe80a2fc0 0x7f2fe80a5480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.514+0000 7f3003b5d700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ffc101390 0x7f2ffc103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.514+0000 7f3003b5d700 1 --2- 172.21.15.27:0/3981323443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ffc101d50 0x7f2ffc1035b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.515+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 >> 172.21.15.27:0/3981323443 conn(0x7f2ffc078340 msgr2=0x7f2ffc1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:25.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.515+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 shutdown_connections 2024-06-06T08:41:25.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:25.515+0000 7f3003b5d700 1 -- 172.21.15.27:0/3981323443 wait complete. 2024-06-06T08:41:25.514 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:41:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 -- 172.21.15.27:0/4102191235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1008f0 msgr2=0x7fd4bc100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 --2- 172.21.15.27:0/4102191235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1008f0 0x7fd4bc100d70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009930 tx=0x7fd4ac02a420 comp rx=0 tx=0).stop 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 -- 172.21.15.27:0/4102191235 shutdown_connections 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 --2- 172.21.15.27:0/4102191235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4bc1012b0 0x7fd4bc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 --2- 172.21.15.27:0/4102191235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1008f0 0x7fd4bc100d70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 --2- 172.21.15.27:0/4102191235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4bc1068b0 0x7fd4bc106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 -- 172.21.15.27:0/4102191235 >> 172.21.15.27:0/4102191235 conn(0x7fd4bc0779f0 msgr2=0x7fd4bc077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 -- 172.21.15.27:0/4102191235 shutdown_connections 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 -- 172.21.15.27:0/4102191235 wait complete. 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 Processor -- start 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.655+0000 7fd4c3410640 1 -- start start 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c3410640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4bc1008f0 0x7fd4bc1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c3410640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1012b0 0x7fd4bc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c3410640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4bc1068b0 0x7fd4bc19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c3410640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc110c80 con 0x7fd4bc1012b0 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c3410640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc110b00 con 0x7fd4bc1008f0 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c3410640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc110e00 con 0x7fd4bc1068b0 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c09ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1012b0 0x7fd4bc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c09ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1012b0 0x7fd4bc1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43518/0 (socket says 172.21.15.27:43518) 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c09ab640 1 -- 172.21.15.27:0/1681423200 learned_addr learned my addr 172.21.15.27:0/1681423200 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4bc1008f0 0x7fd4bc1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4bc1068b0 0x7fd4bc19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c09ab640 1 -- 172.21.15.27:0/1681423200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4bc1068b0 msgr2=0x7fd4bc19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4bc1068b0 0x7fd4bc19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c09ab640 1 -- 172.21.15.27:0/1681423200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4bc1008f0 msgr2=0x7fd4bc1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4bc1008f0 0x7fd4bc1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c09ab640 1 -- 172.21.15.27:0/1681423200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4bc19d2a0 con 0x7fd4bc1012b0 2024-06-06T08:41:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4bc1068b0 0x7fd4bc19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1012b0 0x7fd4bc1987f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009230 tx=0x7fd4ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4b27fc640 1 -- 172.21.15.27:0/1681423200 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac03f030 con 0x7fd4bc1012b0 2024-06-06T08:41:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4b27fc640 1 -- 172.21.15.27:0/1681423200 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4ac003ad0 con 0x7fd4bc1012b0 2024-06-06T08:41:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4b27fc640 1 -- 172.21.15.27:0/1681423200 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac035070 con 0x7fd4bc1012b0 2024-06-06T08:41:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.656+0000 7fd4c3410640 1 -- 172.21.15.27:0/1681423200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4bc19d530 con 0x7fd4bc1012b0 2024-06-06T08:41:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.657+0000 7fd4c3410640 1 -- 172.21.15.27:0/1681423200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4bc1a4e20 con 0x7fd4bc1012b0 2024-06-06T08:41:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.659+0000 7fd4b27fc640 1 -- 172.21.15.27:0/1681423200 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fd4ac042030 con 0x7fd4bc1012b0 2024-06-06T08:41:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.659+0000 7fd4c61f4740 1 Processor -- start 2024-06-06T08:41:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.659+0000 7fd4c61f4740 1 -- start start 2024-06-06T08:41:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.659+0000 7fd4b27fc640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd4a80a2fd0 0x7fd4a80a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.659+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d8fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.659+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 0x55b29d90c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d90eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b29d8614f0 con 0x55b29d8fd070 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b29d7ff870 con 0x55b29d90c760 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b29d85d4b0 con 0x55b29d8fd990 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd4a80a2fd0 0x7fd4a80a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4b27fc640 1 -- 172.21.15.27:0/1681423200 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fd4ac0f00e0 con 0x7fd4bc1012b0 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c11ac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d8fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c11ac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d8fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43532/0 (socket says 172.21.15.27:43532) 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c11ac640 1 -- 172.21.15.27:0/4165642305 learned_addr learned my addr 172.21.15.27:0/4165642305 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/4165642305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 0x55b29d90c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/4165642305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d90eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c11ac640 1 -- 172.21.15.27:0/4165642305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 msgr2=0x55b29d90c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/4165642305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 0x55b29d90c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c11ac640 1 -- 172.21.15.27:0/4165642305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 msgr2=0x55b29d90eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/4165642305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d90eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c11ac640 1 -- 172.21.15.27:0/4165642305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b29d81b9b0 con 0x55b29d8fd070 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd4a80a2fd0 0x7fd4a80a5490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8005fd0 tx=0x7fd4b8012bb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/4165642305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d8fd450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400ac90 tx=0x7fd4b400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426cf47af49aa5a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4a77fe640 1 -- 172.21.15.27:0/4165642305 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b401d070 con 0x55b29d8fd070 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4a77fe640 1 -- 172.21.15.27:0/4165642305 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b4018b20 con 0x55b29d8fd070 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.660+0000 7fd4a77fe640 1 -- 172.21.15.27:0/4165642305 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4010ae0 con 0x55b29d8fd070 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.661+0000 7fd4c61f4740 1 -- 172.21.15.27:0/4165642305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 msgr2=0x55b29d8fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.661+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/4165642305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d8fd450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400ac90 tx=0x7fd4b400f790 comp rx=0 tx=0).stop 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.661+0000 7fd4c61f4740 1 -- 172.21.15.27:0/4165642305 shutdown_connections 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.661+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/4165642305 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d90eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.661+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/4165642305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 0x55b29d90c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.661+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/4165642305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d8fd450 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.661+0000 7fd4c61f4740 1 -- 172.21.15.27:0/4165642305 >> 172.21.15.27:0/4165642305 conn(0x55b29d8f9680 msgr2=0x55b29d8f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.661+0000 7fd4c61f4740 1 -- 172.21.15.27:0/4165642305 shutdown_connections 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.661+0000 7fd4c61f4740 1 -- 172.21.15.27:0/4165642305 wait complete. 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c61f4740 1 -- start start 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c61f4740 1 Processor -- start 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 0x55b29d9096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd990 0x55b29d909c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d90a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b29d90f390 con 0x55b29d8fd990 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b29d7eaae0 con 0x55b29d90c760 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b29d90f510 con 0x55b29d8fd070 2024-06-06T08:41:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c61f4740 10 client.0 _ll_register_callbacks cb 0x55b29d8239b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c09ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd990 0x55b29d909c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c09ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd990 0x55b29d909c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43538/0 (socket says 172.21.15.27:43538) 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c09ab640 1 -- 172.21.15.27:0/1612850157 learned_addr learned my addr 172.21.15.27:0/1612850157 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 0x55b29d9096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.662+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d90a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4c09ab640 1 -- 172.21.15.27:0/1612850157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 msgr2=0x55b29d9096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 0x55b29d9096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4c09ab640 1 -- 172.21.15.27:0/1612850157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 msgr2=0x55b29d90a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d90a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4c09ab640 1 -- 172.21.15.27:0/1612850157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29d575210 con 0x55b29d8fd990 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd990 0x55b29d909c00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0ed320 tx=0x7fd4ac0bbf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e7b4ebfe92da7d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/1612850157 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac049040 con 0x55b29d8fd990 2024-06-06T08:41:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/1612850157 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fd4ac04ccf0 con 0x55b29d8fd990 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4c61f4740 1 -- 172.21.15.27:0/1612850157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b4018b20 con 0x55b29d8fd990 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/1612850157 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd4ac04c560 con 0x55b29d8fd990 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4c61f4740 10 client.15663 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4c61f4740 1 -- 172.21.15.27:0/1612850157 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b29d641620 con 0x55b29d8fd990 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/1612850157 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b4018b20 con 0x55b29d8fd990 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.663+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/1612850157 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd4ac03bc80 con 0x55b29d8fd990 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.664+0000 7fd4c61f4740 10 client.15663 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.664+0000 7fd4c61f4740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.664+0000 7fd4c61f4740 20 client.15663 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.664+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b29d91c910 0x55b29d91ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.664+0000 7fd4c61f4740 4 client.15663 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.664+0000 7fd4c61f4740 1 -- 172.21.15.27:0/1612850157 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b29d874f10 con 0x55b29d91c910 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.664+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b29d91c910 0x55b29d91ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.664+0000 7fd4a4ff9640 10 client.15663 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.664+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b29d91c910 0x55b29d91ed20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c76a5f8d160242 server_cookie=70b2122332c2e2f5 in_seq=0 out_seq=0 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/1612850157 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b29d874f10 con 0x55b29d91c910 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4a4ff9640 10 client.15663 handle_command_reply: tid=0 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 client.15663 shutdown 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 -- 172.21.15.27:0/1612850157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd990 msgr2=0x55b29d909c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd990 0x55b29d909c00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0ed320 tx=0x7fd4ac0bbf00 comp rx=0 tx=0).stop 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 -- 172.21.15.27:0/1612850157 shutdown_connections 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b29d91c910 0x55b29d91ed20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d90a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4a4ff9640 0 client.15663 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd990 0x55b29d909c00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/1612850157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 0x55b29d9096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 -- 172.21.15.27:0/1612850157 >> 172.21.15.27:0/1612850157 conn(0x55b29d8f9680 msgr2=0x55b29d8ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 -- 172.21.15.27:0/1612850157 shutdown_connections 2024-06-06T08:41:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 1 -- 172.21.15.27:0/1612850157 wait complete. 2024-06-06T08:41:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.665+0000 7fd4c61f4740 20 client.15663 trim_cache size 0 max 16384 2024-06-06T08:41:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.680+0000 7fd4c61f4740 1 Processor -- start 2024-06-06T08:41:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c61f4740 1 -- start start 2024-06-06T08:41:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 0x55b29d924630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d8fd990 0x55b29d924b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d90c760 0x55b29d90aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc1af490 con 0x55b29d90c760 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc1acd60 con 0x55b29d8fd990 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc1aabb0 con 0x55b29d8fd070 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c19ad640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d90c760 0x55b29d90aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c19ad640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d90c760 0x55b29d90aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43540/0 (socket says 172.21.15.27:43540) 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c19ad640 1 -- 172.21.15.27:0/521962385 learned_addr learned my addr 172.21.15.27:0/521962385 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 0x55b29d924630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d8fd990 0x55b29d924b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c19ad640 1 -- 172.21.15.27:0/521962385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 msgr2=0x55b29d924630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 0x55b29d924630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c19ad640 1 -- 172.21.15.27:0/521962385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d8fd990 msgr2=0x55b29d924b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d8fd990 0x55b29d924b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c19ad640 1 -- 172.21.15.27:0/521962385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b29d8fded0 con 0x55b29d90c760 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d8fd990 0x55b29d924b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d90c760 0x55b29d90aef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800e720 tx=0x7fd4b801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222194628ca3f248 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.681+0000 7fd4a6ffd640 1 -- 172.21.15.27:0/521962385 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b802d070 con 0x55b29d90c760 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4a6ffd640 1 -- 172.21.15.27:0/521962385 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b80283f0 con 0x55b29d90c760 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4a6ffd640 1 -- 172.21.15.27:0/521962385 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b800f620 con 0x55b29d90c760 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 -- 172.21.15.27:0/521962385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d90c760 msgr2=0x55b29d90aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d90c760 0x55b29d90aef0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800e720 tx=0x7fd4b801e000 comp rx=0 tx=0).stop 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 -- 172.21.15.27:0/521962385 shutdown_connections 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d90c760 0x55b29d90aef0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d8fd990 0x55b29d924b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/521962385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd070 0x55b29d924630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 -- 172.21.15.27:0/521962385 >> 172.21.15.27:0/521962385 conn(0x55b29d8f9680 msgr2=0x55b29d9049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 -- 172.21.15.27:0/521962385 shutdown_connections 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 -- 172.21.15.27:0/521962385 wait complete. 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.682+0000 7fd4c61f4740 1 -- start start 2024-06-06T08:41:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c61f4740 1 Processor -- start 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d910690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 0x55b29d910bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c61f4740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d911110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc1ab220 con 0x55b29d8fd070 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc1ac690 con 0x55b29d90c760 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c61f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4bc1ab8f0 con 0x55b29d8fd990 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c61f4740 10 client.0 _ll_register_callbacks cb 0x7fd4ac0b8310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c11ac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d910690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c11ac640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d910690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43554/0 (socket says 172.21.15.27:43554) 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c11ac640 1 -- 172.21.15.27:0/3249440119 learned_addr learned my addr 172.21.15.27:0/3249440119 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 0x55b29d910bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c19ad640 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d911110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c11ac640 1 -- 172.21.15.27:0/3249440119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 msgr2=0x55b29d910bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 0x55b29d910bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c11ac640 1 -- 172.21.15.27:0/3249440119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 msgr2=0x55b29d911110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d911110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.683+0000 7fd4c11ac640 1 -- 172.21.15.27:0/3249440119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29d9867b0 con 0x55b29d8fd070 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c11ac640 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d910690 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4010730 tx=0x7fd4b4010620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc2a9dfeb3682d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/3249440119 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b401d040 con 0x55b29d8fd070 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/3249440119 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fd4b4018880 con 0x55b29d8fd070 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c61f4740 1 -- 172.21.15.27:0/3249440119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b80283f0 con 0x55b29d8fd070 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/3249440119 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd4b4018aa0 con 0x55b29d8fd070 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c61f4740 10 client.15669 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c61f4740 1 -- 172.21.15.27:0/3249440119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b29d9879f0 con 0x55b29d8fd070 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/3249440119 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b80283f0 con 0x55b29d8fd070 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/3249440119 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd4b4030070 con 0x55b29d8fd070 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c61f4740 10 client.15669 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c61f4740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c61f4740 20 client.15669 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b29d91c910 0x55b29d989fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c61f4740 4 client.15669 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:41:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.684+0000 7fd4c61f4740 1 -- 172.21.15.27:0/3249440119 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b29d86ec20 con 0x55b29d91c910 2024-06-06T08:41:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.685+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b29d91c910 0x55b29d989fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.685+0000 7fd4c09ab640 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b29d91c910 0x55b29d989fe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a820e1816d0b48e server_cookie=918582a1627451f in_seq=0 out_seq=0 2024-06-06T08:41:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.685+0000 7fd4a4ff9640 10 client.15669 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4a4ff9640 1 -- 172.21.15.27:0/3249440119 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b29d86ec20 con 0x55b29d91c910 2024-06-06T08:41:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4a4ff9640 10 client.15669 handle_command_reply: tid=0 2024-06-06T08:41:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 client.15669 shutdown 2024-06-06T08:41:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 -- 172.21.15.27:0/3249440119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 msgr2=0x55b29d910690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d910690 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4010730 tx=0x7fd4b4010620 comp rx=0 tx=0).stop 2024-06-06T08:41:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 -- 172.21.15.27:0/3249440119 shutdown_connections 2024-06-06T08:41:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b29d91c910 0x55b29d989fe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b29d90c760 0x55b29d911110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b29d8fd990 0x55b29d910bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4a4ff9640 0 client.15669 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 --2- 172.21.15.27:0/3249440119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b29d8fd070 0x55b29d910690 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 -- 172.21.15.27:0/3249440119 >> 172.21.15.27:0/3249440119 conn(0x55b29d8f9680 msgr2=0x55b29d8f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 -- 172.21.15.27:0/3249440119 shutdown_connections 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 1 -- 172.21.15.27:0/3249440119 wait complete. 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c61f4740 20 client.15669 trim_cache size 0 max 16384 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:41:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:41:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c3410640 1 -- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd4a80a2fd0 msgr2=0x7fd4a80a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c3410640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd4a80a2fd0 0x7fd4a80a5490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8005fd0 tx=0x7fd4b8012bb0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c3410640 1 -- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1012b0 msgr2=0x7fd4bc1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.689+0000 7fd4c3410640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1012b0 0x7fd4bc1987f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009230 tx=0x7fd4ac004270 comp rx=0 tx=0).stop 2024-06-06T08:41:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.690+0000 7fd4c3410640 1 -- 172.21.15.27:0/1681423200 shutdown_connections 2024-06-06T08:41:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.690+0000 7fd4c3410640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd4a80a2fd0 0x7fd4a80a5490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.690+0000 7fd4c3410640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4bc1068b0 0x7fd4bc19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.690+0000 7fd4c3410640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4bc1012b0 0x7fd4bc1987f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.690+0000 7fd4c3410640 1 --2- 172.21.15.27:0/1681423200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4bc1008f0 0x7fd4bc1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.690+0000 7fd4c3410640 1 -- 172.21.15.27:0/1681423200 >> 172.21.15.27:0/1681423200 conn(0x7fd4bc0779f0 msgr2=0x7fd4bc069f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.690+0000 7fd4c3410640 1 -- 172.21.15.27:0/1681423200 shutdown_connections 2024-06-06T08:41:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:25.690+0000 7fd4c3410640 1 -- 172.21.15.27:0/1681423200 wait complete. 2024-06-06T08:41:25.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:25 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3981323443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:26.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:25 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3981323443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:26.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:25 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3981323443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:26.940 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 154.4 --await-for 45.0' 2024-06-06T08:41:26.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 154.4 --await-for 45.0 2024-06-06T08:41:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:26 smithi027 ceph-mon[32335]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:41:27.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:26 smithi203 ceph-mon[36579]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:41:27.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:26 smithi176 ceph-mon[36786]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:41:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.456+0000 7f76097b6700 1 -- 172.21.15.27:0/2916893464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101380 msgr2=0x7f7604101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.456+0000 7f76097b6700 1 --2- 172.21.15.27:0/2916893464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101380 0x7f7604101800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009990 tx=0x7f75ec009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.456+0000 7f76097b6700 1 -- 172.21.15.27:0/2916893464 shutdown_connections 2024-06-06T08:41:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.456+0000 7f76097b6700 1 --2- 172.21.15.27:0/2916893464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7604101d40 0x7f760410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.456+0000 7f76097b6700 1 --2- 172.21.15.27:0/2916893464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76041073f0 0x7f76041077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.456+0000 7f76097b6700 1 --2- 172.21.15.27:0/2916893464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101380 0x7f7604101800 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.456+0000 7f76097b6700 1 -- 172.21.15.27:0/2916893464 >> 172.21.15.27:0/2916893464 conn(0x7f7604078300 msgr2=0x7f7604078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.456+0000 7f76097b6700 1 -- 172.21.15.27:0/2916893464 shutdown_connections 2024-06-06T08:41:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.456+0000 7f76097b6700 1 -- 172.21.15.27:0/2916893464 wait complete. 2024-06-06T08:41:27.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.457+0000 7f76097b6700 1 Processor -- start 2024-06-06T08:41:27.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76097b6700 1 -- start start 2024-06-06T08:41:27.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76097b6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7604101380 0x7f7604199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76097b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101d40 0x7f7604199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76097b6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76041073f0 0x7f76041a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76097b6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76041116e0 con 0x7f7604101d40 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76097b6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7604111560 con 0x7f7604101380 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76097b6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7604111860 con 0x7f76041073f0 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101d40 0x7f7604199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101d40 0x7f7604199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43576/0 (socket says 172.21.15.27:43576) 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76027fc700 1 -- 172.21.15.27:0/253947110 learned_addr learned my addr 172.21.15.27:0/253947110 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f7602ffd700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7604101380 0x7f7604199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76037fe700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76041073f0 0x7f76041a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76027fc700 1 -- 172.21.15.27:0/253947110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76041073f0 msgr2=0x7f76041a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76027fc700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76041073f0 0x7f76041a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76027fc700 1 -- 172.21.15.27:0/253947110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7604101380 msgr2=0x7f7604199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76027fc700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7604101380 0x7f7604199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76027fc700 1 -- 172.21.15.27:0/253947110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76041a5020 con 0x7f7604101d40 2024-06-06T08:41:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.458+0000 7f76037fe700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76041073f0 0x7f76041a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.459+0000 7f76027fc700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101d40 0x7f7604199fa0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f75ec000c00 tx=0x7f75ec003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.459+0000 7f75fbfff700 1 -- 172.21.15.27:0/253947110 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec027070 con 0x7f7604101d40 2024-06-06T08:41:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.459+0000 7f76097b6700 1 -- 172.21.15.27:0/253947110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76041a51f0 con 0x7f7604101d40 2024-06-06T08:41:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.459+0000 7f75fbfff700 1 -- 172.21.15.27:0/253947110 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ec004110 con 0x7f7604101d40 2024-06-06T08:41:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.459+0000 7f75fbfff700 1 -- 172.21.15.27:0/253947110 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec00f460 con 0x7f7604101d40 2024-06-06T08:41:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.459+0000 7f76097b6700 1 -- 172.21.15.27:0/253947110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76041a56a0 con 0x7f7604101d40 2024-06-06T08:41:27.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.462+0000 7f75fbfff700 1 -- 172.21.15.27:0/253947110 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f75ec02a070 con 0x7f7604101d40 2024-06-06T08:41:27.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.463+0000 7f75fbfff700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75f00a2f40 0x7f75f00a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.463+0000 7f7602ffd700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75f00a2f40 0x7f75f00a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.463+0000 7f7614039b80 1 Processor -- start 2024-06-06T08:41:27.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.463+0000 7f75fbfff700 1 -- 172.21.15.27:0/253947110 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f75ec0d0a70 con 0x7f7604101d40 2024-06-06T08:41:27.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.463+0000 7f7614039b80 1 -- start start 2024-06-06T08:41:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.463+0000 7f7602ffd700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75f00a2f40 0x7f75f00a5400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009a10 tx=0x7f75f4009190 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f7614039b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798828d0 0x556679882cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f7614039b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798831f0 0x556679887050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f7614039b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 0x5566798879f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f7614039b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556679770fe0 con 0x5566798831f0 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f7614039b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5566797cf8d0 con 0x556679887590 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f7614039b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5566797374e0 con 0x5566798828d0 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798831f0 0x556679887050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798831f0 0x556679887050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43592/0 (socket says 172.21.15.27:43592) 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76027fc700 1 -- 172.21.15.27:0/3540406187 learned_addr learned my addr 172.21.15.27:0/3540406187 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f7602ffd700 1 --2- 172.21.15.27:0/3540406187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798828d0 0x556679882cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76037fe700 1 --2- 172.21.15.27:0/3540406187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 0x5566798879f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76027fc700 1 -- 172.21.15.27:0/3540406187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798828d0 msgr2=0x556679882cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76027fc700 1 --2- 172.21.15.27:0/3540406187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798828d0 0x556679882cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76027fc700 1 -- 172.21.15.27:0/3540406187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 msgr2=0x5566798879f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76027fc700 1 --2- 172.21.15.27:0/3540406187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 0x5566798879f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76027fc700 1 -- 172.21.15.27:0/3540406187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55667971e950 con 0x5566798831f0 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.464+0000 7f76027fc700 1 --2- 172.21.15.27:0/3540406187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798831f0 0x556679887050 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f75ec09c9b0 tx=0x7f75ec09c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5643e98cb02608d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.465+0000 7f75f97fa700 1 -- 172.21.15.27:0/3540406187 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec09d120 con 0x5566798831f0 2024-06-06T08:41:27.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.465+0000 7f75f97fa700 1 -- 172.21.15.27:0/3540406187 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ec09ca20 con 0x5566798831f0 2024-06-06T08:41:27.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.465+0000 7f75f97fa700 1 -- 172.21.15.27:0/3540406187 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec09cee0 con 0x5566798831f0 2024-06-06T08:41:27.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.465+0000 7f7614039b80 1 -- 172.21.15.27:0/3540406187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798831f0 msgr2=0x556679887050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:27.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.465+0000 7f7614039b80 1 --2- 172.21.15.27:0/3540406187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798831f0 0x556679887050 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f75ec09c9b0 tx=0x7f75ec09c560 comp rx=0 tx=0).stop 2024-06-06T08:41:27.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 -- 172.21.15.27:0/3540406187 shutdown_connections 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 --2- 172.21.15.27:0/3540406187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 0x5566798879f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 --2- 172.21.15.27:0/3540406187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798828d0 0x556679882cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 --2- 172.21.15.27:0/3540406187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798831f0 0x556679887050 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 -- 172.21.15.27:0/3540406187 >> 172.21.15.27:0/3540406187 conn(0x5566797ab100 msgr2=0x5566798792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 -- 172.21.15.27:0/3540406187 shutdown_connections 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 -- 172.21.15.27:0/3540406187 wait complete. 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 -- start start 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 Processor -- start 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 0x55667988be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 0x55667988c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556679770040 con 0x5566798828d0 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5566797cb890 con 0x556679887590 2024-06-06T08:41:27.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55667988f490 con 0x5566798831f0 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.466+0000 7f7614039b80 10 client.0 _ll_register_callbacks cb 0x556679765630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f7602ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f7602ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43608/0 (socket says 172.21.15.27:43608) 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f7602ffd700 1 -- 172.21.15.27:0/634246905 learned_addr learned my addr 172.21.15.27:0/634246905 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f76027fc700 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 0x55667988be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f76037fe700 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 0x55667988c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f7602ffd700 1 -- 172.21.15.27:0/634246905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 msgr2=0x55667988be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f7602ffd700 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 0x55667988be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f7602ffd700 1 -- 172.21.15.27:0/634246905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 msgr2=0x55667988c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f7602ffd700 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 0x55667988c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f7602ffd700 1 -- 172.21.15.27:0/634246905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566797957d0 con 0x5566798828d0 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f76037fe700 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 0x55667988c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f7602ffd700 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988eb30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f75f40097f0 tx=0x7f75f4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea95f9698dfed7c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:27.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.467+0000 7f75ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f75ea7fc700 1 -- 172.21.15.27:0/634246905 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4025070 con 0x5566798828d0 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f7614039b80 1 -- 172.21.15.27:0/634246905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75ec09ca20 con 0x5566798828d0 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f75ea7fc700 1 -- 172.21.15.27:0/634246905 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f75f4015c80 con 0x5566798828d0 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f75ea7fc700 1 -- 172.21.15.27:0/634246905 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75f401b040 con 0x5566798828d0 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f7614039b80 10 client.15681 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f7614039b80 1 -- 172.21.15.27:0/634246905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556679899c80 con 0x5566798828d0 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f75ea7fc700 1 -- 172.21.15.27:0/634246905 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ec09ca20 con 0x5566798828d0 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f75ea7fc700 1 -- 172.21.15.27:0/634246905 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f75f4021440 con 0x5566798828d0 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f7614039b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f7614039b80 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f7614039b80 20 client.15681 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f7614039b80 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55667989c210 0x55667989e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f7614039b80 4 client.15681 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:27.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.468+0000 7f7614039b80 1 -- 172.21.15.27:0/634246905 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566797dcdd0 con 0x55667989c210 2024-06-06T08:41:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.469+0000 7f76027fc700 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55667989c210 0x55667989e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.469+0000 7f76027fc700 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55667989c210 0x55667989e660 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e4c6f52ff28332 server_cookie=ce20193602314f88 in_seq=0 out_seq=0 2024-06-06T08:41:27.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.469+0000 7f75ea7fc700 10 client.15681 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f75ea7fc700 1 -- 172.21.15.27:0/634246905 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5566797dcdd0 con 0x55667989c210 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f75ea7fc700 10 client.15681 handle_command_reply: tid=0 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 client.15681 shutdown 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 -- 172.21.15.27:0/634246905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 msgr2=0x55667988eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988eb30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f75f40097f0 tx=0x7f75f4004860 comp rx=0 tx=0).stop 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 -- 172.21.15.27:0/634246905 shutdown_connections 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556679887590 0x55667988c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55667989c210 0x55667989e660 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988eb30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 --2- 172.21.15.27:0/634246905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 0x55667988be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 -- 172.21.15.27:0/634246905 >> 172.21.15.27:0/634246905 conn(0x5566797ab100 msgr2=0x556679878780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:27.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f75ea7fc700 0 client.15681 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:27.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 -- 172.21.15.27:0/634246905 shutdown_connections 2024-06-06T08:41:27.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 1 -- 172.21.15.27:0/634246905 wait complete. 2024-06-06T08:41:27.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.470+0000 7f7614039b80 20 client.15681 trim_cache size 0 max 16384 2024-06-06T08:41:27.486 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:27.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.492+0000 7f7614039b80 1 Processor -- start 2024-06-06T08:41:27.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.492+0000 7f7614039b80 1 -- start start 2024-06-06T08:41:27.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7614039b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7614039b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798831f0 0x556679878f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7614039b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556679887590 0x556679879440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7614039b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76041aec00 con 0x5566798828d0 2024-06-06T08:41:27.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7614039b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f76041ac490 con 0x5566798831f0 2024-06-06T08:41:27.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7614039b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76041aa2e0 con 0x556679887590 2024-06-06T08:41:27.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7602ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7602ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43618/0 (socket says 172.21.15.27:43618) 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7602ffd700 1 -- 172.21.15.27:0/3633690696 learned_addr learned my addr 172.21.15.27:0/3633690696 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f76027fc700 1 --2- 172.21.15.27:0/3633690696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798831f0 0x556679878f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f76037fe700 1 --2- 172.21.15.27:0/3633690696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556679887590 0x556679879440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7602ffd700 1 -- 172.21.15.27:0/3633690696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556679887590 msgr2=0x556679879440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7602ffd700 1 --2- 172.21.15.27:0/3633690696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556679887590 0x556679879440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7602ffd700 1 -- 172.21.15.27:0/3633690696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798831f0 msgr2=0x556679878f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7602ffd700 1 --2- 172.21.15.27:0/3633690696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798831f0 0x556679878f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.493+0000 7f7602ffd700 1 -- 172.21.15.27:0/3633690696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556679887390 con 0x5566798828d0 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.494+0000 7f7602ffd700 1 --2- 172.21.15.27:0/3633690696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988dd70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f75f4010010 tx=0x7f75f4006ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8ac345180970e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.494+0000 7f75f8ff9700 1 -- 172.21.15.27:0/3633690696 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4025040 con 0x5566798828d0 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.494+0000 7f75f8ff9700 1 -- 172.21.15.27:0/3633690696 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f4021640 con 0x5566798828d0 2024-06-06T08:41:27.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.494+0000 7f75f8ff9700 1 -- 172.21.15.27:0/3633690696 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f40064e0 con 0x5566798828d0 2024-06-06T08:41:27.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 -- 172.21.15.27:0/3633690696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 msgr2=0x55667988dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:27.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 --2- 172.21.15.27:0/3633690696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988dd70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f75f4010010 tx=0x7f75f4006ee0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 -- 172.21.15.27:0/3633690696 shutdown_connections 2024-06-06T08:41:27.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 --2- 172.21.15.27:0/3633690696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556679887590 0x556679879440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 --2- 172.21.15.27:0/3633690696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566798828d0 0x55667988dd70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 --2- 172.21.15.27:0/3633690696 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798831f0 0x556679878f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 -- 172.21.15.27:0/3633690696 >> 172.21.15.27:0/3633690696 conn(0x5566797ab100 msgr2=0x55667988d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:27.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 -- 172.21.15.27:0/3633690696 shutdown_connections 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 -- 172.21.15.27:0/3633690696 wait complete. 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 -- start start 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 Processor -- start 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798828d0 0x556679894df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 0x5566798916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.495+0000 7f7614039b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556679887590 0x556679891bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f7614039b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76041aa950 con 0x556679887590 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f7614039b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f76041abdc0 con 0x5566798828d0 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f7614039b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76041ab020 con 0x5566798831f0 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f7614039b80 10 client.0 _ll_register_callbacks cb 0x556679765630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556679887590 0x556679891bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556679887590 0x556679891bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43624/0 (socket says 172.21.15.27:43624) 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76037fe700 1 -- 172.21.15.27:0/1806803773 learned_addr learned my addr 172.21.15.27:0/1806803773 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:27.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f7602ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798828d0 0x556679894df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76027fc700 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 0x5566798916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76037fe700 1 -- 172.21.15.27:0/1806803773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 msgr2=0x5566798916b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76037fe700 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 0x5566798916b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76037fe700 1 -- 172.21.15.27:0/1806803773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798828d0 msgr2=0x556679894df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76037fe700 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798828d0 0x556679894df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76037fe700 1 -- 172.21.15.27:0/1806803773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556679892460 con 0x556679887590 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f7602ffd700 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798828d0 0x556679894df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f76037fe700 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556679887590 0x556679891bf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f75fc015e30 tx=0x7f75fc0135b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d020b2dce50144b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.496+0000 7f75ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f75ea7fc700 1 -- 172.21.15.27:0/1806803773 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc015e60 con 0x556679887590 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f7614039b80 1 -- 172.21.15.27:0/1806803773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f4021640 con 0x556679887590 2024-06-06T08:41:27.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f75ea7fc700 1 -- 172.21.15.27:0/1806803773 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f75fc013ca0 con 0x556679887590 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f75ea7fc700 1 -- 172.21.15.27:0/1806803773 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75fc029050 con 0x556679887590 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f7614039b80 10 client.15687 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f7614039b80 1 -- 172.21.15.27:0/1806803773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556679908060 con 0x556679887590 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f75ea7fc700 1 -- 172.21.15.27:0/1806803773 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f4021640 con 0x556679887590 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f75ea7fc700 1 -- 172.21.15.27:0/1806803773 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f75fc0215d0 con 0x556679887590 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f7614039b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f7614039b80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f7614039b80 20 client.15687 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f7614039b80 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5566799087a0 0x55667990abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f7614039b80 4 client.15687 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.497+0000 7f7614039b80 1 -- 172.21.15.27:0/1806803773 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5566797d7350 con 0x5566799087a0 2024-06-06T08:41:27.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.498+0000 7f7602ffd700 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5566799087a0 0x55667990abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:27.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.498+0000 7f75ea7fc700 10 client.15687 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.498+0000 7f7602ffd700 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5566799087a0 0x55667990abc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3deda4f88248f2c server_cookie=8c721a267c5f9e3c in_seq=0 out_seq=0 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.499+0000 7f75ea7fc700 1 -- 172.21.15.27:0/1806803773 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5566797d7350 con 0x5566799087a0 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.499+0000 7f75ea7fc700 10 client.15687 handle_command_reply: tid=0 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.499+0000 7f7614039b80 1 client.15687 shutdown 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 -- 172.21.15.27:0/1806803773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556679887590 msgr2=0x556679891bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556679887590 0x556679891bf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f75fc015e30 tx=0x7f75fc0135b0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 -- 172.21.15.27:0/1806803773 shutdown_connections 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5566799087a0 0x55667990abc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556679887590 0x556679891bf0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5566798828d0 0x556679894df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 --2- 172.21.15.27:0/1806803773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566798831f0 0x5566798916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 -- 172.21.15.27:0/1806803773 >> 172.21.15.27:0/1806803773 conn(0x5566797ab100 msgr2=0x556679886c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f75ea7fc700 0 client.15687 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:27.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 -- 172.21.15.27:0/1806803773 shutdown_connections 2024-06-06T08:41:27.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 1 -- 172.21.15.27:0/1806803773 wait complete. 2024-06-06T08:41:27.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.500+0000 7f7614039b80 20 client.15687 trim_cache size 0 max 16384 2024-06-06T08:41:27.502 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:41:27.502 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-06-06T08:41:27.502 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "36686f06-d6d5-4401-ad0d-b3d97f658b4a", 2024-06-06T08:41:27.502 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-06-06T08:41:27.502 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 -- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75f00a2f40 msgr2=0x7f75f00a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75f00a2f40 0x7f75f00a5400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009a10 tx=0x7f75f4009190 comp rx=0 tx=0).stop 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 -- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101d40 msgr2=0x7f7604199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101d40 0x7f7604199fa0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f75ec000c00 tx=0x7f75ec003680 comp rx=0 tx=0).stop 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 -- 172.21.15.27:0/253947110 shutdown_connections 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76041073f0 0x7f76041a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f75f00a2f40 0x7f75f00a5400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7604101380 0x7f7604199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 --2- 172.21.15.27:0/253947110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7604101d40 0x7f7604199fa0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 -- 172.21.15.27:0/253947110 >> 172.21.15.27:0/253947110 conn(0x7f7604078300 msgr2=0x7f76040ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:27.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 -- 172.21.15.27:0/253947110 shutdown_connections 2024-06-06T08:41:27.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:27.501+0000 7f76097b6700 1 -- 172.21.15.27:0/253947110 wait complete. 2024-06-06T08:41:28.219 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T08:41:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:28 smithi027 ceph-mon[32335]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:41:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:28 smithi027 ceph-mon[32335]: scrub queued for path: / 2024-06-06T08:41:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:28 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:28 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:41:29.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:28 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:29.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:28 smithi203 ceph-mon[36579]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:41:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:28 smithi203 ceph-mon[36579]: scrub queued for path: / 2024-06-06T08:41:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:28 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:28 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:41:29.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:28 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:28 smithi176 ceph-mon[36786]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T08:41:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:28 smithi176 ceph-mon[36786]: scrub queued for path: / 2024-06-06T08:41:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:28 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:28 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:41:29.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:28 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:30.287 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:30 smithi027 ceph-mon[32335]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T08:41:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:30 smithi203 ceph-mon[36579]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T08:41:30.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:30 smithi176 ceph-mon[36786]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 -- 172.21.15.27:0/3210753951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 msgr2=0x7f22d4112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 --2- 172.21.15.27:0/3210753951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 0x7f22d4112e20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009990 tx=0x7f22c8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 -- 172.21.15.27:0/3210753951 shutdown_connections 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 --2- 172.21.15.27:0/3210753951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22d4113360 0x7f22d4115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 --2- 172.21.15.27:0/3210753951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d40769c0 0x7f22d4076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 --2- 172.21.15.27:0/3210753951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 0x7f22d4112e20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 -- 172.21.15.27:0/3210753951 >> 172.21.15.27:0/3210753951 conn(0x7f22d41001b0 msgr2=0x7f22d41025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 -- 172.21.15.27:0/3210753951 shutdown_connections 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 -- 172.21.15.27:0/3210753951 wait complete. 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.676+0000 7f22d98eb700 1 Processor -- start 2024-06-06T08:41:30.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d98eb700 1 -- start start 2024-06-06T08:41:30.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d98eb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22d40769c0 0x7f22d419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d98eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 0x7f22d419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d98eb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d4113360 0x7f22d41a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d98eb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22d4118e40 con 0x7f22d40772e0 2024-06-06T08:41:30.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d98eb700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f22d4118cc0 con 0x7f22d4113360 2024-06-06T08:41:30.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d98eb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22d4118fc0 con 0x7f22d40769c0 2024-06-06T08:41:30.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 0x7f22d419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 0x7f22d419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43914/0 (socket says 172.21.15.27:43914) 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d27fc700 1 -- 172.21.15.27:0/2888566665 learned_addr learned my addr 172.21.15.27:0/2888566665 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22d40769c0 0x7f22d419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d37fe700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d4113360 0x7f22d41a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d27fc700 1 -- 172.21.15.27:0/2888566665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22d40769c0 msgr2=0x7f22d419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d27fc700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22d40769c0 0x7f22d419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d27fc700 1 -- 172.21.15.27:0/2888566665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d4113360 msgr2=0x7f22d41a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d27fc700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d4113360 0x7f22d41a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.677+0000 7f22d27fc700 1 -- 172.21.15.27:0/2888566665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d41a4200 con 0x7f22d40772e0 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.678+0000 7f22d27fc700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 0x7f22d419f730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.678+0000 7f22d88e9700 1 -- 172.21.15.27:0/2888566665 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8027070 con 0x7f22d40772e0 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.678+0000 7f22d98eb700 1 -- 172.21.15.27:0/2888566665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d41a4490 con 0x7f22d40772e0 2024-06-06T08:41:30.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.678+0000 7f22d88e9700 1 -- 172.21.15.27:0/2888566665 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c8003e40 con 0x7f22d40772e0 2024-06-06T08:41:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.678+0000 7f22d88e9700 1 -- 172.21.15.27:0/2888566665 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8022d10 con 0x7f22d40772e0 2024-06-06T08:41:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.678+0000 7f22d98eb700 1 -- 172.21.15.27:0/2888566665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d41abd70 con 0x7f22d40772e0 2024-06-06T08:41:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.680+0000 7f22d88e9700 1 -- 172.21.15.27:0/2888566665 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f22c802a070 con 0x7f22d40772e0 2024-06-06T08:41:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.681+0000 7f22d88e9700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22bc0a2fe0 0x7f22bc0a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.681+0000 7f22d88e9700 1 -- 172.21.15.27:0/2888566665 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f22c80d08c0 con 0x7f22d40772e0 2024-06-06T08:41:30.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.681+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22bc0a2fe0 0x7f22bc0a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.681+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22bc0a2fe0 0x7f22bc0a54a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f22c4005950 tx=0x7f22c400b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:30.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22e416eb80 1 Processor -- start 2024-06-06T08:41:30.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22e416eb80 1 -- start start 2024-06-06T08:41:30.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1590 0x55f047af1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 0x55f047aeb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 0x55f047aec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22e416eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f0479de0f0 con 0x55f047af1590 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22e416eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f047a3c9e0 con 0x55f047aebce0 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22e416eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f0479a45f0 con 0x55f047af1eb0 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22d2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1590 0x55f047af1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22d2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1590 0x55f047af1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43926/0 (socket says 172.21.15.27:43926) 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1956266766 learned_addr learned my addr 172.21.15.27:0/1956266766 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.682+0000 7f22d37fe700 1 --2- 172.21.15.27:0/1956266766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 0x55f047aec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22d27fc700 1 --2- 172.21.15.27:0/1956266766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 0x55f047aeb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1956266766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 msgr2=0x55f047aeb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:30.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1956266766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 0x55f047aeb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1956266766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 msgr2=0x55f047aec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:30.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1956266766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 0x55f047aec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1956266766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f04798ba60 con 0x55f047af1590 2024-06-06T08:41:30.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1956266766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1590 0x55f047af1970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f22c4010fa0 tx=0x7f22c4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2c8588ab35d15c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:30.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22b9ffb700 1 -- 172.21.15.27:0/1956266766 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c400be60 con 0x55f047af1590 2024-06-06T08:41:30.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22b9ffb700 1 -- 172.21.15.27:0/1956266766 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c4011db0 con 0x55f047af1590 2024-06-06T08:41:30.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.683+0000 7f22b9ffb700 1 -- 172.21.15.27:0/1956266766 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c401c850 con 0x55f047af1590 2024-06-06T08:41:30.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 -- 172.21.15.27:0/1956266766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1590 msgr2=0x55f047af1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1956266766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1590 0x55f047af1970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f22c4010fa0 tx=0x7f22c4013000 comp rx=0 tx=0).stop 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 -- 172.21.15.27:0/1956266766 shutdown_connections 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1956266766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 0x55f047aec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1956266766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1590 0x55f047af1970 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1956266766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 0x55f047aeb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 -- 172.21.15.27:0/1956266766 >> 172.21.15.27:0/1956266766 conn(0x55f047a18210 msgr2=0x55f047ae6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 -- 172.21.15.27:0/1956266766 shutdown_connections 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 -- 172.21.15.27:0/1956266766 wait complete. 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.684+0000 7f22e416eb80 1 -- start start 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22e416eb80 1 Processor -- start 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 0x55f047af7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 0x55f047af4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1eb0 0x55f047af5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22e416eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f0479dd150 con 0x55f047af1eb0 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22e416eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f047a389a0 con 0x55f047aebce0 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22e416eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f047af8890 con 0x55f047af1590 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22d37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1eb0 0x55f047af5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22d37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1eb0 0x55f047af5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43938/0 (socket says 172.21.15.27:43938) 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22d37fe700 1 -- 172.21.15.27:0/1527305692 learned_addr learned my addr 172.21.15.27:0/1527305692 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.685+0000 7f22e416eb80 10 client.0 _ll_register_callbacks cb 0x55f0479d2740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22d27fc700 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 0x55f047af4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 0x55f047af7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22d37fe700 1 -- 172.21.15.27:0/1527305692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 msgr2=0x55f047af4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22d37fe700 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 0x55f047af4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22d37fe700 1 -- 172.21.15.27:0/1527305692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 msgr2=0x55f047af7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22d37fe700 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 0x55f047af7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22d37fe700 1 -- 172.21.15.27:0/1527305692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f047a028e0 con 0x55f047af1eb0 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22d27fc700 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 0x55f047af4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22d37fe700 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1eb0 0x55f047af5480 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f22cc004070 tx=0x7f22cc004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a25f0aad959531 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1527305692 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc01a070 con 0x55f047af1eb0 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1527305692 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f22cc004db0 con 0x55f047af1eb0 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.686+0000 7f22e416eb80 1 -- 172.21.15.27:0/1527305692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22c4011db0 con 0x55f047af1eb0 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1527305692 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22cc025050 con 0x55f047af1eb0 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22e416eb80 10 client.15699 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:30.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22e416eb80 1 -- 172.21.15.27:0/1527305692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f047b03080 con 0x55f047af1eb0 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1527305692 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c4011db0 con 0x55f047af1eb0 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1527305692 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f22cc016880 con 0x55f047af1eb0 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22e416eb80 10 client.15699 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22e416eb80 10 client.15699 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22e416eb80 20 client.15699 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f047b05660 0x55f047b07a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22e416eb80 4 client.15699 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22e416eb80 1 -- 172.21.15.27:0/1527305692 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f047a49ee0 con 0x55f047b05660 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.687+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f047b05660 0x55f047b07a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.688+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f047b05660 0x55f047b07a50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69ba1ad04b85e4d1 server_cookie=947181fe7e5732e0 in_seq=0 out_seq=0 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.688+0000 7f22b2ffd700 10 client.15699 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.688+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1527305692 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f047a49ee0 con 0x55f047b05660 2024-06-06T08:41:30.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.688+0000 7f22b2ffd700 10 client.15699 handle_command_reply: tid=0 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.688+0000 7f22e416eb80 1 client.15699 shutdown 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 -- 172.21.15.27:0/1527305692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1eb0 msgr2=0x55f047af5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1eb0 0x55f047af5480 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f22cc004070 tx=0x7f22cc004150 comp rx=0 tx=0).stop 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 -- 172.21.15.27:0/1527305692 shutdown_connections 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f047b05660 0x55f047b07a50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047af1eb0 0x55f047af5480 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047aebce0 0x55f047af7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1527305692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 0x55f047af4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 -- 172.21.15.27:0/1527305692 >> 172.21.15.27:0/1527305692 conn(0x55f047a18210 msgr2=0x55f047ae53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22b2ffd700 0 client.15699 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 -- 172.21.15.27:0/1527305692 shutdown_connections 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 1 -- 172.21.15.27:0/1527305692 wait complete. 2024-06-06T08:41:30.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.689+0000 7f22e416eb80 20 client.15699 trim_cache size 0 max 16384 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.709+0000 7f22e416eb80 1 Processor -- start 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.709+0000 7f22e416eb80 1 -- start start 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.709+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047af4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.709+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 0x55f047af07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.709+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1eb0 0x55f047af0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22e416eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22d41b72d0 con 0x55f047aebce0 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22e416eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f22d41b4b60 con 0x55f047af1eb0 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22e416eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22d41b29b0 con 0x55f047af1590 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047af4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047af4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43944/0 (socket says 172.21.15.27:43944) 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1614864933 learned_addr learned my addr 172.21.15.27:0/1614864933 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1614864933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 msgr2=0x55f047af07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d27fc700 1 --2- 172.21.15.27:0/1614864933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 0x55f047af07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1614864933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 0x55f047af07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1614864933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1eb0 msgr2=0x55f047af0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1614864933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1eb0 0x55f047af0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1614864933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f047af5e70 con 0x55f047aebce0 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1614864933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047af4840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f22c401ffd0 tx=0x7f22c401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaabc8e5bdb33a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22b97fa700 1 -- 172.21.15.27:0/1614864933 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4005040 con 0x55f047aebce0 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22b97fa700 1 -- 172.21.15.27:0/1614864933 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c40056c0 con 0x55f047aebce0 2024-06-06T08:41:30.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.710+0000 7f22b97fa700 1 -- 172.21.15.27:0/1614864933 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c40048f0 con 0x55f047aebce0 2024-06-06T08:41:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 -- 172.21.15.27:0/1614864933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 msgr2=0x55f047af4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1614864933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047af4840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f22c401ffd0 tx=0x7f22c401ed10 comp rx=0 tx=0).stop 2024-06-06T08:41:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 -- 172.21.15.27:0/1614864933 shutdown_connections 2024-06-06T08:41:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1614864933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1eb0 0x55f047af0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1614864933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047af4840 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1614864933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1590 0x55f047af07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 -- 172.21.15.27:0/1614864933 >> 172.21.15.27:0/1614864933 conn(0x55f047a18210 msgr2=0x55f047af00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 -- 172.21.15.27:0/1614864933 shutdown_connections 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 -- 172.21.15.27:0/1614864933 wait complete. 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.711+0000 7f22e416eb80 1 -- start start 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22e416eb80 1 Processor -- start 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047afdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1590 0x55f047afe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22e416eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 0x55f047afab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22e416eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f22d41b3020 con 0x55f047aebce0 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22e416eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f22d41b4490 con 0x55f047af1590 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22e416eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22d41b36f0 con 0x55f047af1eb0 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22e416eb80 10 client.0 _ll_register_callbacks cb 0x55f0479d2740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:30.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22d2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047afdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22d2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047afdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43952/0 (socket says 172.21.15.27:43952) 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1432040705 learned_addr learned my addr 172.21.15.27:0/1432040705 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22d27fc700 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1590 0x55f047afe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.712+0000 7f22d37fe700 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 0x55f047afab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1432040705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 msgr2=0x55f047afab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 0x55f047afab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1432040705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1590 msgr2=0x55f047afe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1590 0x55f047afe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22d2ffd700 1 -- 172.21.15.27:0/1432040705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f047afb360 con 0x55f047aebce0 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22d27fc700 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1590 0x55f047afe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22d37fe700 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 0x55f047afab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22d2ffd700 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047afdd30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f22c4005a80 tx=0x7f22c401ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b37cc9e6c1323f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:30.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1432040705 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c401e630 con 0x55f047aebce0 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1432040705 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f22c401e7d0 con 0x55f047aebce0 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22e416eb80 1 -- 172.21.15.27:0/1432040705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f047a44460 con 0x55f047aebce0 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1432040705 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22c4032410 con 0x55f047aebce0 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22e416eb80 10 client.15705 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.713+0000 7f22e416eb80 1 -- 172.21.15.27:0/1432040705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f047b70d40 con 0x55f047aebce0 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.714+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1432040705 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f047a44460 con 0x55f047aebce0 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.714+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1432040705 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f22c404e860 con 0x55f047aebce0 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.714+0000 7f22e416eb80 10 client.15705 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.714+0000 7f22e416eb80 10 client.15705 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.714+0000 7f22e416eb80 20 client.15705 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.714+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f047b71260 0x55f047b73650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.714+0000 7f22e416eb80 4 client.15705 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.4, "await_for": 45.0}] 2024-06-06T08:41:30.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.714+0000 7f22e416eb80 1 -- 172.21.15.27:0/1432040705 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.4, "await_for": 45.0}) -- 0x55f0479f80e0 con 0x55f047b71260 2024-06-06T08:41:30.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.714+0000 7f22d27fc700 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f047b71260 0x55f047b73650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:30.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.715+0000 7f22b2ffd700 10 client.15705 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:30.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.715+0000 7f22d27fc700 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f047b71260 0x55f047b73650 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16cdba92ddf18089 server_cookie=7728afecc3cae3fb in_seq=0 out_seq=0 2024-06-06T08:41:30.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22b2ffd700 1 -- 172.21.15.27:0/1432040705 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f0479f80e0 con 0x55f047b71260 2024-06-06T08:41:30.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22b2ffd700 10 client.15705 handle_command_reply: tid=0 2024-06-06T08:41:30.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 client.15705 shutdown 2024-06-06T08:41:30.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 -- 172.21.15.27:0/1432040705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 msgr2=0x55f047afdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:30.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047afdd30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f22c4005a80 tx=0x7f22c401ec20 comp rx=0 tx=0).stop 2024-06-06T08:41:30.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 -- 172.21.15.27:0/1432040705 shutdown_connections 2024-06-06T08:41:30.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f047af1eb0 0x55f047afab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f047b71260 0x55f047b73650 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f047aebce0 0x55f047afdd30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 --2- 172.21.15.27:0/1432040705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f047af1590 0x55f047afe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 -- 172.21.15.27:0/1432040705 >> 172.21.15.27:0/1432040705 conn(0x55f047a18210 msgr2=0x55f047af0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:30.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22b2ffd700 0 client.15705 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:30.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 -- 172.21.15.27:0/1432040705 shutdown_connections 2024-06-06T08:41:30.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 1 -- 172.21.15.27:0/1432040705 wait complete. 2024-06-06T08:41:30.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.732+0000 7f22e416eb80 20 client.15705 trim_cache size 0 max 16384 2024-06-06T08:41:30.734 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:41:30.734 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 25, 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24482, 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 2, 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "ef5f116f": { 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 2, 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 154.4, 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-06-06T08:41:30.735 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:41:30.736 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 -- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22bc0a2fe0 msgr2=0x7f22bc0a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22bc0a2fe0 0x7f22bc0a54a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f22c4005950 tx=0x7f22c400b410 comp rx=0 tx=0).stop 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 -- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 msgr2=0x7f22d419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 0x7f22d419f730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c80047b0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 -- 172.21.15.27:0/2888566665 shutdown_connections 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22bc0a2fe0 0x7f22bc0a54a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22d4113360 0x7f22d41a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22d40769c0 0x7f22d419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 --2- 172.21.15.27:0/2888566665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d40772e0 0x7f22d419f730 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:30.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 -- 172.21.15.27:0/2888566665 >> 172.21.15.27:0/2888566665 conn(0x7f22d41001b0 msgr2=0x7f22d41025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:30.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 -- 172.21.15.27:0/2888566665 shutdown_connections 2024-06-06T08:41:30.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:30.733+0000 7f22d98eb700 1 -- 172.21.15.27:0/2888566665 wait complete. 2024-06-06T08:41:31.264 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:31 smithi027 ceph-mon[32335]: scrub summary: idle 2024-06-06T08:41:31.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:31 smithi203 ceph-mon[36579]: scrub summary: idle 2024-06-06T08:41:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:31 smithi176 ceph-mon[36786]: scrub summary: idle 2024-06-06T08:41:31.934 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ef5f116f', quiesce took 9.3 seconds. Will release after: 25.099999999999998 2024-06-06T08:41:32.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:32 smithi027 ceph-mon[32335]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T08:41:32.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:32 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.468+0000 7fa45767f700 1 -- 172.21.15.27:0/2203392483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101d50 msgr2=0x7fa45010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.468+0000 7fa45767f700 1 --2- 172.21.15.27:0/2203392483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101d50 0x7fa45010e240 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa44800b0a0 tx=0x7fa44800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.468+0000 7fa45767f700 1 -- 172.21.15.27:0/2203392483 shutdown_connections 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.468+0000 7fa45767f700 1 --2- 172.21.15.27:0/2203392483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101d50 0x7fa45010e240 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.468+0000 7fa45767f700 1 --2- 172.21.15.27:0/2203392483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450107400 0x7fa4501077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.468+0000 7fa45767f700 1 --2- 172.21.15.27:0/2203392483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa450101390 0x7fa450101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.468+0000 7fa45767f700 1 -- 172.21.15.27:0/2203392483 >> 172.21.15.27:0/2203392483 conn(0x7fa450078340 msgr2=0x7fa450078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.468+0000 7fa45767f700 1 -- 172.21.15.27:0/2203392483 shutdown_connections 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.468+0000 7fa45767f700 1 -- 172.21.15.27:0/2203392483 wait complete. 2024-06-06T08:41:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.469+0000 7fa45767f700 1 Processor -- start 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.469+0000 7fa45767f700 1 -- start start 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.469+0000 7fa45767f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101390 0x7fa450197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.469+0000 7fa45767f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450101d50 0x7fa450197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.469+0000 7fa45767f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa450107400 0x7fa450198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.469+0000 7fa45767f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa450111690 con 0x7fa450101390 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.469+0000 7fa45767f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa450111510 con 0x7fa450107400 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.469+0000 7fa45767f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa450111810 con 0x7fa450101d50 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.469+0000 7fa45541b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101390 0x7fa450197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45541b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101390 0x7fa450197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43966/0 (socket says 172.21.15.27:43966) 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45541b700 1 -- 172.21.15.27:0/3137161716 learned_addr learned my addr 172.21.15.27:0/3137161716 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa454c1a700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450101d50 0x7fa450197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa455c1c700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa450107400 0x7fa450198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45541b700 1 -- 172.21.15.27:0/3137161716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450101d50 msgr2=0x7fa450197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45541b700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450101d50 0x7fa450197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45541b700 1 -- 172.21.15.27:0/3137161716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa450107400 msgr2=0x7fa450198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45541b700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa450107400 0x7fa450198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45541b700 1 -- 172.21.15.27:0/3137161716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4501a7070 con 0x7fa450101390 2024-06-06T08:41:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45541b700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101390 0x7fa450197800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00d980 tx=0x7fa44c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa4427fc700 1 -- 172.21.15.27:0/3137161716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c017070 con 0x7fa450101390 2024-06-06T08:41:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45767f700 1 -- 172.21.15.27:0/3137161716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4501a7360 con 0x7fa450101390 2024-06-06T08:41:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa4427fc700 1 -- 172.21.15.27:0/3137161716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa44c005020 con 0x7fa450101390 2024-06-06T08:41:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa45767f700 1 -- 172.21.15.27:0/3137161716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4501a78a0 con 0x7fa450101390 2024-06-06T08:41:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.470+0000 7fa4427fc700 1 -- 172.21.15.27:0/3137161716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00fbf0 con 0x7fa450101390 2024-06-06T08:41:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.473+0000 7fa4427fc700 1 -- 172.21.15.27:0/3137161716 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fa44c0051d0 con 0x7fa450101390 2024-06-06T08:41:32.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.474+0000 7fa4427fc700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa43c0a2fc0 0x7fa43c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.474+0000 7fa454c1a700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa43c0a2fc0 0x7fa43c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.474+0000 7fa4427fc700 1 -- 172.21.15.27:0/3137161716 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fa44c0cf850 con 0x7fa450101390 2024-06-06T08:41:32.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.474+0000 7fa461f02b80 1 Processor -- start 2024-06-06T08:41:32.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.474+0000 7fa461f02b80 1 -- start start 2024-06-06T08:41:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a4257ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa454c1a700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa43c0a2fc0 0x7fa43c0a5480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa450199960 tx=0x7fa44400b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a425769e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42577340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa461f02b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556a424c8810 con 0x556a42576f20 2024-06-06T08:41:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa461f02b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556a42470720 con 0x556a4257c640 2024-06-06T08:41:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa461f02b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556a424c47d0 con 0x556a4257cf60 2024-06-06T08:41:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa455c1c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42577340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa455c1c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42577340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43974/0 (socket says 172.21.15.27:43974) 2024-06-06T08:41:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa455c1c700 1 -- 172.21.15.27:0/3325977114 learned_addr learned my addr 172.21.15.27:0/3325977114 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa45541b700 1 --2- 172.21.15.27:0/3325977114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a4257ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa454c1a700 1 --2- 172.21.15.27:0/3325977114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a425769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa455c1c700 1 -- 172.21.15.27:0/3325977114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 msgr2=0x556a425769e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa455c1c700 1 --2- 172.21.15.27:0/3325977114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a425769e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa455c1c700 1 -- 172.21.15.27:0/3325977114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 msgr2=0x556a4257ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa455c1c700 1 --2- 172.21.15.27:0/3325977114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a4257ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.475+0000 7fa455c1c700 1 -- 172.21.15.27:0/3325977114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a42466f50 con 0x556a42576f20 2024-06-06T08:41:32.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.476+0000 7fa455c1c700 1 --2- 172.21.15.27:0/3325977114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42577340 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa448009150 tx=0x7fa448009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfa609f89d5a4e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:32.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.476+0000 7fa43b7fe700 1 -- 172.21.15.27:0/3325977114 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448020070 con 0x556a42576f20 2024-06-06T08:41:32.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.476+0000 7fa43b7fe700 1 -- 172.21.15.27:0/3325977114 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4480074e0 con 0x556a42576f20 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.476+0000 7fa43b7fe700 1 -- 172.21.15.27:0/3325977114 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4480052a0 con 0x556a42576f20 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 -- 172.21.15.27:0/3325977114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 msgr2=0x556a42577340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 --2- 172.21.15.27:0/3325977114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42577340 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa448009150 tx=0x7fa448009e20 comp rx=0 tx=0).stop 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 -- 172.21.15.27:0/3325977114 shutdown_connections 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 --2- 172.21.15.27:0/3325977114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42577340 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 --2- 172.21.15.27:0/3325977114 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a4257ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 --2- 172.21.15.27:0/3325977114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a425769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 -- 172.21.15.27:0/3325977114 >> 172.21.15.27:0/3325977114 conn(0x556a424a1cc0 msgr2=0x556a42571850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 -- 172.21.15.27:0/3325977114 shutdown_connections 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 -- 172.21.15.27:0/3325977114 wait complete. 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 -- start start 2024-06-06T08:41:32.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.477+0000 7fa461f02b80 1 Processor -- start 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42582d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a42580580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa461f02b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556a42583970 con 0x556a42576f20 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa461f02b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556a42468bf0 con 0x556a4257c640 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa461f02b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556a42583af0 con 0x556a4257cf60 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa45541b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42582d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa454c1a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa454c1a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:46436/0 (socket says 172.21.15.27:46436) 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa454c1a700 1 -- 172.21.15.27:0/1605440756 learned_addr learned my addr 172.21.15.27:0/1605440756 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa45541b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42582d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43990/0 (socket says 172.21.15.27:43990) 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa461f02b80 10 client.0 _ll_register_callbacks cb 0x556a42462920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:32.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa455c1c700 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a42580580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa454c1a700 1 -- 172.21.15.27:0/1605440756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 msgr2=0x556a42580580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:32.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa454c1a700 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a42580580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa454c1a700 1 -- 172.21.15.27:0/1605440756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 msgr2=0x556a42582d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:32.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa454c1a700 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42582d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa454c1a700 1 -- 172.21.15.27:0/1605440756 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a424178d0 con 0x556a4257c640 2024-06-06T08:41:32.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa45541b700 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42582d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:32.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.478+0000 7fa454c1a700 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa444010e90 tx=0x7fa4440047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68310090b175ce63 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:32.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.479+0000 7fa438ff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:41:32.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.481+0000 7fa438ff9700 1 -- 172.21.15.27:0/1605440756 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444004a30 con 0x556a4257c640 2024-06-06T08:41:32.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.481+0000 7fa461f02b80 1 -- 172.21.15.27:0/1605440756 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4480074e0 con 0x556a4257c640 2024-06-06T08:41:32.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.481+0000 7fa438ff9700 1 -- 172.21.15.27:0/1605440756 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fa444004bd0 con 0x556a4257c640 2024-06-06T08:41:32.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.481+0000 7fa438ff9700 1 -- 172.21.15.27:0/1605440756 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa444011e30 con 0x556a4257c640 2024-06-06T08:41:32.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.481+0000 7fa461f02b80 10 client.25313 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:32.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.481+0000 7fa461f02b80 1 -- 172.21.15.27:0/1605440756 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a4258e3b0 con 0x556a4257c640 2024-06-06T08:41:32.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.481+0000 7fa438ff9700 1 -- 172.21.15.27:0/1605440756 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4480074e0 con 0x556a4257c640 2024-06-06T08:41:32.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa438ff9700 1 -- 172.21.15.27:0/1605440756 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa44401e970 con 0x556a4257c640 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa461f02b80 10 client.25313 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa461f02b80 10 client.25313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa461f02b80 20 client.25313 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a425909f0 0x556a42592e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa461f02b80 4 client.25313 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa461f02b80 1 -- 172.21.15.27:0/1605440756 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a424d5d10 con 0x556a425909f0 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa45541b700 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a425909f0 0x556a42592e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa45541b700 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a425909f0 0x556a42592e00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d11b0b034c321a server_cookie=62c0f1fc479b9462 in_seq=0 out_seq=0 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.482+0000 7fa438ff9700 10 client.25313 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa438ff9700 1 -- 172.21.15.27:0/1605440756 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556a424d5d10 con 0x556a425909f0 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa438ff9700 10 client.25313 handle_command_reply: tid=0 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa461f02b80 1 client.25313 shutdown 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa461f02b80 1 -- 172.21.15.27:0/1605440756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 msgr2=0x556a42580040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa444010e90 tx=0x7fa4440047c0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa461f02b80 1 -- 172.21.15.27:0/1605440756 shutdown_connections 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a42580580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a425909f0 0x556a42592e00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a42582d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1605440756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa438ff9700 0 client.25313 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.483+0000 7fa461f02b80 1 -- 172.21.15.27:0/1605440756 >> 172.21.15.27:0/1605440756 conn(0x556a424a1cc0 msgr2=0x556a425759d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.484+0000 7fa461f02b80 1 -- 172.21.15.27:0/1605440756 shutdown_connections 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.484+0000 7fa461f02b80 1 -- 172.21.15.27:0/1605440756 wait complete. 2024-06-06T08:41:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.484+0000 7fa461f02b80 20 client.25313 trim_cache size 0 max 16384 2024-06-06T08:41:32.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:32 smithi203 ceph-mon[36579]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T08:41:32.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:32 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:41:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa461f02b80 1 Processor -- start 2024-06-06T08:41:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa461f02b80 1 -- start start 2024-06-06T08:41:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a425769e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a4258cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa461f02b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501aeb60 con 0x556a42576f20 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa461f02b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501ac3f0 con 0x556a4257c640 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa461f02b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501aa260 con 0x556a4257cf60 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa45541b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a425769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa45541b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a425769e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44004/0 (socket says 172.21.15.27:44004) 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa45541b700 1 -- 172.21.15.27:0/266166029 learned_addr learned my addr 172.21.15.27:0/266166029 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.505+0000 7fa455c1c700 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a4258cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa454c1a700 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa455c1c700 1 -- 172.21.15.27:0/266166029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 msgr2=0x556a42580710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa455c1c700 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa455c1c700 1 -- 172.21.15.27:0/266166029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 msgr2=0x556a425769e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:32.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa455c1c700 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a425769e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa455c1c700 1 -- 172.21.15.27:0/266166029 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a42582980 con 0x556a4257cf60 2024-06-06T08:41:32.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa45541b700 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a425769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:32.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa455c1c700 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a4258cc70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa448009a20 tx=0x7fa448007e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e4edd9dbe40685 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:32.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa43affd700 1 -- 172.21.15.27:0/266166029 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44802d070 con 0x556a4257cf60 2024-06-06T08:41:32.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa43affd700 1 -- 172.21.15.27:0/266166029 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4480074e0 con 0x556a4257cf60 2024-06-06T08:41:32.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.506+0000 7fa43affd700 1 -- 172.21.15.27:0/266166029 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448020040 con 0x556a4257cf60 2024-06-06T08:41:32.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:32 smithi176 ceph-mon[36786]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T08:41:32.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:32 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:41:32.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 -- 172.21.15.27:0/266166029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 msgr2=0x556a4258cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:32.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a4258cc70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa448009a20 tx=0x7fa448007e40 comp rx=0 tx=0).stop 2024-06-06T08:41:32.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 -- 172.21.15.27:0/266166029 shutdown_connections 2024-06-06T08:41:32.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a4257cf60 0x556a4258cc70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a42576f20 0x556a425769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 --2- 172.21.15.27:0/266166029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257c640 0x556a42580710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 -- 172.21.15.27:0/266166029 >> 172.21.15.27:0/266166029 conn(0x556a424a1cc0 msgr2=0x556a42580000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 -- 172.21.15.27:0/266166029 shutdown_connections 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 -- 172.21.15.27:0/266166029 wait complete. 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.507+0000 7fa461f02b80 1 -- start start 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa461f02b80 1 Processor -- start 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a42576f20 0x556a42588de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a4257c640 0x556a42589320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa461f02b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257cf60 0x556a42585db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa461f02b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501aa8b0 con 0x556a4257c640 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa461f02b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501abd20 con 0x556a4257cf60 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa461f02b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501aaf80 con 0x556a42576f20 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa461f02b80 10 client.0 _ll_register_callbacks cb 0x556a42462920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa454c1a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a4257c640 0x556a42589320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa454c1a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a4257c640 0x556a42589320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44006/0 (socket says 172.21.15.27:44006) 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa454c1a700 1 -- 172.21.15.27:0/1216089132 learned_addr learned my addr 172.21.15.27:0/1216089132 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa455c1c700 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257cf60 0x556a42585db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.508+0000 7fa45541b700 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a42576f20 0x556a42588de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa455c1c700 1 -- 172.21.15.27:0/1216089132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a42576f20 msgr2=0x556a42588de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa455c1c700 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a42576f20 0x556a42588de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa455c1c700 1 -- 172.21.15.27:0/1216089132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a4257c640 msgr2=0x556a42589320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa455c1c700 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a4257c640 0x556a42589320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa455c1c700 1 -- 172.21.15.27:0/1216089132 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a425862f0 con 0x556a4257cf60 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa455c1c700 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257cf60 0x556a42585db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x556a42580ca0 tx=0x7fa448009410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad1ed84ed3bbf6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa438ff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa438ff9700 1 -- 172.21.15.27:0/1216089132 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448020040 con 0x556a4257cf60 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa438ff9700 1 -- 172.21.15.27:0/1216089132 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fa448007aa0 con 0x556a4257cf60 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa461f02b80 1 -- 172.21.15.27:0/1216089132 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4480074e0 con 0x556a4257cf60 2024-06-06T08:41:32.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa438ff9700 1 -- 172.21.15.27:0/1216089132 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa44802d040 con 0x556a4257cf60 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa461f02b80 10 client.25319 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.509+0000 7fa461f02b80 1 -- 172.21.15.27:0/1216089132 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556a425fbe10 con 0x556a4257cf60 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa438ff9700 1 -- 172.21.15.27:0/1216089132 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4480074e0 con 0x556a4257cf60 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa438ff9700 1 -- 172.21.15.27:0/1216089132 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa448011070 con 0x556a4257cf60 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa461f02b80 10 client.25319 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa461f02b80 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa461f02b80 20 client.25319 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a425fc090 0x556a425fe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa461f02b80 4 client.25319 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa461f02b80 1 -- 172.21.15.27:0/1216089132 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a424d0290 con 0x556a425fc090 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa45541b700 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a425fc090 0x556a425fe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.510+0000 7fa438ff9700 10 client.25319 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa45541b700 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a425fc090 0x556a425fe480 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610bd6721c29febe server_cookie=cfb009681db65596 in_seq=0 out_seq=0 2024-06-06T08:41:32.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa438ff9700 1 -- 172.21.15.27:0/1216089132 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a424d0290 con 0x556a425fc090 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa438ff9700 10 client.25319 handle_command_reply: tid=0 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa461f02b80 1 client.25319 shutdown 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa461f02b80 1 -- 172.21.15.27:0/1216089132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257cf60 msgr2=0x556a42585db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257cf60 0x556a42585db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x556a42580ca0 tx=0x7fa448009410 comp rx=0 tx=0).stop 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa461f02b80 1 -- 172.21.15.27:0/1216089132 shutdown_connections 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a4257cf60 0x556a42585db0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a42576f20 0x556a42588de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a425fc090 0x556a425fe480 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa461f02b80 1 --2- 172.21.15.27:0/1216089132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a4257c640 0x556a42589320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa438ff9700 0 client.25319 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.511+0000 7fa461f02b80 1 -- 172.21.15.27:0/1216089132 >> 172.21.15.27:0/1216089132 conn(0x556a424a1cc0 msgr2=0x556a4256ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.512+0000 7fa461f02b80 1 -- 172.21.15.27:0/1216089132 shutdown_connections 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.512+0000 7fa461f02b80 1 -- 172.21.15.27:0/1216089132 wait complete. 2024-06-06T08:41:32.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.512+0000 7fa461f02b80 20 client.25319 trim_cache size 0 max 16384 2024-06-06T08:41:32.515 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:41:32.515 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-06-06T08:41:32.515 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {} 2024-06-06T08:41:32.515 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:41:32.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 -- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa43c0a2fc0 msgr2=0x7fa43c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:32.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa43c0a2fc0 0x7fa43c0a5480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa450199960 tx=0x7fa44400b410 comp rx=0 tx=0).stop 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 -- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101390 msgr2=0x7fa450197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101390 0x7fa450197800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00d980 tx=0x7fa44c00dc90 comp rx=0 tx=0).stop 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 -- 172.21.15.27:0/3137161716 shutdown_connections 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa43c0a2fc0 0x7fa43c0a5480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa450107400 0x7fa450198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa450101390 0x7fa450197800 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 --2- 172.21.15.27:0/3137161716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450101d50 0x7fa450197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 -- 172.21.15.27:0/3137161716 >> 172.21.15.27:0/3137161716 conn(0x7fa450078340 msgr2=0x7fa4500ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 -- 172.21.15.27:0/3137161716 shutdown_connections 2024-06-06T08:41:32.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:32.513+0000 7fa45767f700 1 -- 172.21.15.27:0/3137161716 wait complete. 2024-06-06T08:41:33.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:33 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:41:33.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:33 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:41:33.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:33 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:41:33.393 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T08:41:33.393 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:41:33.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:33 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:41:33.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:33 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:41:33.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:33 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:41:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:33 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:41:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:33 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:41:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:33 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:41:33.645 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:34.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:34 smithi027 ceph-mon[32335]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-06T08:41:34.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:34 smithi203 ceph-mon[36579]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-06T08:41:34.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:34 smithi176 ceph-mon[36786]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 9 op/s 2024-06-06T08:41:35.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.435+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/3262427834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 msgr2=0x7f6cec06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:35.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/3262427834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 0x7f6cec06a5f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009990 tx=0x7f6ce0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/3262427834 shutdown_connections 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/3262427834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec06ab30 0x7f6cec10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/3262427834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cec108500 0x7f6cec1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/3262427834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 0x7f6cec06a5f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/3262427834 >> 172.21.15.27:0/3262427834 conn(0x7f6cec078340 msgr2=0x7f6cec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/3262427834 shutdown_connections 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/3262427834 wait complete. 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 Processor -- start 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.436+0000 7f6cf1c4d700 1 -- start start 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6cf1c4d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 0x7f6cec19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6cf1c4d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cec06ab30 0x7f6cec19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6cf1c4d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec108500 0x7f6cec198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:35.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6cf1c4d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec1127b0 con 0x7f6cec06a170 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6cf1c4d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec112630 con 0x7f6cec06ab30 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6cf1c4d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec112930 con 0x7f6cec108500 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 0x7f6cec19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 0x7f6cec19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44024/0 (socket says 172.21.15.27:44024) 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceb7fe700 1 -- 172.21.15.27:0/2868943318 learned_addr learned my addr 172.21.15.27:0/2868943318 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6cebfff700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec108500 0x7f6cec198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceaffd700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cec06ab30 0x7f6cec19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceb7fe700 1 -- 172.21.15.27:0/2868943318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec108500 msgr2=0x7f6cec198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceb7fe700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec108500 0x7f6cec198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceb7fe700 1 -- 172.21.15.27:0/2868943318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cec06ab30 msgr2=0x7f6cec19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceb7fe700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cec06ab30 0x7f6cec19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceb7fe700 1 -- 172.21.15.27:0/2868943318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec1991d0 con 0x7f6cec06a170 2024-06-06T08:41:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceaffd700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cec06ab30 0x7f6cec19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.437+0000 7f6ceb7fe700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 0x7f6cec19e8b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00d950 tx=0x7f6cdc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.438+0000 7f6ce8ff9700 1 -- 172.21.15.27:0/2868943318 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc016070 con 0x7f6cec06a170 2024-06-06T08:41:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.438+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cec1994c0 con 0x7f6cec06a170 2024-06-06T08:41:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.438+0000 7f6ce8ff9700 1 -- 172.21.15.27:0/2868943318 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cdc009e20 con 0x7f6cec06a170 2024-06-06T08:41:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.438+0000 7f6ce8ff9700 1 -- 172.21.15.27:0/2868943318 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00f620 con 0x7f6cec06a170 2024-06-06T08:41:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.438+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cec1a5780 con 0x7f6cec06a170 2024-06-06T08:41:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.439+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cec069180 con 0x7f6cec06a170 2024-06-06T08:41:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.440+0000 7f6ce8ff9700 1 -- 172.21.15.27:0/2868943318 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f6cdc019070 con 0x7f6cec06a170 2024-06-06T08:41:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.440+0000 7f6ce8ff9700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6cd40a2f40 0x7f6cd40a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.441+0000 7f6ceaffd700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6cd40a2f40 0x7f6cd40a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.441+0000 7f6ce8ff9700 1 -- 172.21.15.27:0/2868943318 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f6cdc0cf600 con 0x7f6cec06a170 2024-06-06T08:41:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.441+0000 7f6ceaffd700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6cd40a2f40 0x7f6cd40a5400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0006010 tx=0x7f6ce000b540 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:35.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.445+0000 7f6ce8ff9700 1 -- 172.21.15.27:0/2868943318 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6cdc09bc60 con 0x7f6cec06a170 2024-06-06T08:41:35.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.696+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cec02dca0 con 0x7f6cec06a170 2024-06-06T08:41:35.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.697+0000 7f6ce8ff9700 1 -- 172.21.15.27:0/2868943318 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7f6cdc09b560 con 0x7f6cec06a170 2024-06-06T08:41:35.696 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:41:35.696 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:41:35.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6cd40a2f40 msgr2=0x7f6cd40a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:35.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6cd40a2f40 0x7f6cd40a5400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0006010 tx=0x7f6ce000b540 comp rx=0 tx=0).stop 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 msgr2=0x7f6cec19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 0x7f6cec19e8b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00d950 tx=0x7f6cdc00dc60 comp rx=0 tx=0).stop 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 shutdown_connections 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec108500 0x7f6cec198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec06a170 0x7f6cec19e8b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6cd40a2f40 0x7f6cd40a5400 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 --2- 172.21.15.27:0/2868943318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cec06ab30 0x7f6cec19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 >> 172.21.15.27:0/2868943318 conn(0x7f6cec078340 msgr2=0x7f6cec0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 shutdown_connections 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:35.699+0000 7f6cf1c4d700 1 -- 172.21.15.27:0/2868943318 wait complete. 2024-06-06T08:41:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:41:36.504 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T08:41:36.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:36 smithi027 ceph-mon[32335]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-06-06T08:41:36.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:36 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2868943318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:36.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:36 smithi203 ceph-mon[36579]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-06-06T08:41:36.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:36 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2868943318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:36.755 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:36.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:36 smithi176 ceph-mon[36786]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-06-06T08:41:36.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:36 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2868943318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:38.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.448+0000 7f9a3ddab700 1 -- 172.21.15.27:0/232006757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a381073f0 msgr2=0x7f9a381077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.448+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/232006757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a381073f0 0x7f9a381077d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009990 tx=0x7f9a28009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.448+0000 7f9a3ddab700 1 -- 172.21.15.27:0/232006757 shutdown_connections 2024-06-06T08:41:38.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.448+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/232006757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a38101d40 0x7f9a3810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.448+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/232006757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a381073f0 0x7f9a381077d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.448+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/232006757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a38101380 0x7f9a38101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.448+0000 7f9a3ddab700 1 -- 172.21.15.27:0/232006757 >> 172.21.15.27:0/232006757 conn(0x7f9a38078300 msgr2=0x7f9a38078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.448+0000 7f9a3ddab700 1 -- 172.21.15.27:0/232006757 shutdown_connections 2024-06-06T08:41:38.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.448+0000 7f9a3ddab700 1 -- 172.21.15.27:0/232006757 wait complete. 2024-06-06T08:41:38.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a3ddab700 1 Processor -- start 2024-06-06T08:41:38.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a3ddab700 1 -- start start 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a3ddab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38101380 0x7f9a3819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a3ddab700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a38101d40 0x7f9a3819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a3ddab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a381073f0 0x7f9a3819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a3ddab700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a38111690 con 0x7f9a38101380 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a3ddab700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a38111510 con 0x7f9a38101d40 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a3ddab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a38111810 con 0x7f9a381073f0 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38101380 0x7f9a3819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38101380 0x7f9a3819ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44034/0 (socket says 172.21.15.27:44034) 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.449+0000 7f9a377fe700 1 -- 172.21.15.27:0/1280723111 learned_addr learned my addr 172.21.15.27:0/1280723111 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a37fff700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a381073f0 0x7f9a3819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a38101d40 0x7f9a3819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a377fe700 1 -- 172.21.15.27:0/1280723111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a381073f0 msgr2=0x7f9a3819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a377fe700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a381073f0 0x7f9a3819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a377fe700 1 -- 172.21.15.27:0/1280723111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a38101d40 msgr2=0x7f9a3819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a377fe700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a38101d40 0x7f9a3819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a377fe700 1 -- 172.21.15.27:0/1280723111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3819fd90 con 0x7f9a38101380 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a38101d40 0x7f9a3819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a377fe700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38101380 0x7f9a3819ad80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9a28000c00 tx=0x7f9a280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a34ff9700 1 -- 172.21.15.27:0/1280723111 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28027070 con 0x7f9a38101380 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a3ddab700 1 -- 172.21.15.27:0/1280723111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a381a0020 con 0x7f9a38101380 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a34ff9700 1 -- 172.21.15.27:0/1280723111 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a28003780 con 0x7f9a38101380 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.450+0000 7f9a34ff9700 1 -- 172.21.15.27:0/1280723111 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28022dd0 con 0x7f9a38101380 2024-06-06T08:41:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.451+0000 7f9a3ddab700 1 -- 172.21.15.27:0/1280723111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a381a7900 con 0x7f9a38101380 2024-06-06T08:41:38.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.453+0000 7f9a34ff9700 1 -- 172.21.15.27:0/1280723111 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f9a2802a070 con 0x7f9a38101380 2024-06-06T08:41:38.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:38 smithi027 ceph-mon[32335]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-06-06T08:41:38.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.454+0000 7f9a34ff9700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9a200a2f40 0x7f9a200a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.454+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9a200a2f40 0x7f9a200a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.454+0000 7f9a34ff9700 1 -- 172.21.15.27:0/1280723111 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9a280d07d0 con 0x7f9a38101380 2024-06-06T08:41:38.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.454+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9a200a2f40 0x7f9a200a5400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c005fd0 tx=0x7f9a2c005e30 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a4862eb80 1 Processor -- start 2024-06-06T08:41:38.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a4862eb80 1 -- start start 2024-06-06T08:41:38.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248529340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 0x556248523260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 0x556248523bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556248415f60 con 0x556248528f60 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556248474850 con 0x5562485237a0 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5562483dc460 con 0x556248529880 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248529340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248529340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44044/0 (socket says 172.21.15.27:44044) 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.455+0000 7f9a377fe700 1 -- 172.21.15.27:0/1148292531 learned_addr learned my addr 172.21.15.27:0/1148292531 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 0x556248523260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a37fff700 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 0x556248523bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a377fe700 1 -- 172.21.15.27:0/1148292531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 msgr2=0x556248523260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a377fe700 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 0x556248523260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a377fe700 1 -- 172.21.15.27:0/1148292531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 msgr2=0x556248523bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a377fe700 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 0x556248523bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a377fe700 1 -- 172.21.15.27:0/1148292531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562483c38d0 con 0x556248528f60 2024-06-06T08:41:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 0x556248523260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a37fff700 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 0x556248523bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a377fe700 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248529340 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9a280991e0 tx=0x7f9a2809cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c97d2b854f4267 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a1dffb700 1 -- 172.21.15.27:0/1148292531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280a53f0 con 0x556248528f60 2024-06-06T08:41:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a1dffb700 1 -- 172.21.15.27:0/1148292531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a2809ab10 con 0x556248528f60 2024-06-06T08:41:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.456+0000 7f9a1dffb700 1 -- 172.21.15.27:0/1148292531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280a4f60 con 0x556248528f60 2024-06-06T08:41:38.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.457+0000 7f9a4862eb80 1 -- 172.21.15.27:0/1148292531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 msgr2=0x556248529340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.457+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248529340 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9a280991e0 tx=0x7f9a2809cf90 comp rx=0 tx=0).stop 2024-06-06T08:41:38.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.457+0000 7f9a4862eb80 1 -- 172.21.15.27:0/1148292531 shutdown_connections 2024-06-06T08:41:38.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.457+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 0x556248523bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.457+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248529340 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.457+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/1148292531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 0x556248523260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.457+0000 7f9a4862eb80 1 -- 172.21.15.27:0/1148292531 >> 172.21.15.27:0/1148292531 conn(0x556248450080 msgr2=0x55624851e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.457+0000 7f9a4862eb80 1 -- 172.21.15.27:0/1148292531 shutdown_connections 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.457+0000 7f9a4862eb80 1 -- 172.21.15.27:0/1148292531 wait complete. 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a4862eb80 1 -- start start 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a4862eb80 1 Processor -- start 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248528f60 0x55624852c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 0x55624852cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556248414fc0 con 0x5562485237a0 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556248470810 con 0x556248529880 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556248530010 con 0x556248528f60 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a4862eb80 10 client.0 _ll_register_callbacks cb 0x55624840a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44054/0 (socket says 172.21.15.27:44054) 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.458+0000 7f9a377fe700 1 -- 172.21.15.27:0/176906644 learned_addr learned my addr 172.21.15.27:0/176906644 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a37fff700 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 0x55624852cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a377fe700 1 -- 172.21.15.27:0/176906644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248528f60 msgr2=0x55624852c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a377fe700 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248528f60 0x55624852c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a377fe700 1 -- 172.21.15.27:0/176906644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 msgr2=0x55624852cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a377fe700 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 0x55624852cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a377fe700 1 -- 172.21.15.27:0/176906644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55624843a750 con 0x5562485237a0 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a37fff700 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 0x55624852cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a377fe700 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852f5b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9a280cdaa0 tx=0x7f9a28005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490a7b9f7a4bac20 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a16ffd700 1 -- 172.21.15.27:0/176906644 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280ba060 con 0x5562485237a0 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a16ffd700 1 -- 172.21.15.27:0/176906644 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9a2809cc70 con 0x5562485237a0 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a4862eb80 1 -- 172.21.15.27:0/176906644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a2809ab10 con 0x5562485237a0 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a16ffd700 1 -- 172.21.15.27:0/176906644 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a280b4070 con 0x5562485237a0 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a4862eb80 10 client.15741 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a4862eb80 1 -- 172.21.15.27:0/176906644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55624853a630 con 0x5562485237a0 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.459+0000 7f9a16ffd700 1 -- 172.21.15.27:0/176906644 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a2809ab10 con 0x5562485237a0 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.460+0000 7f9a16ffd700 1 -- 172.21.15.27:0/176906644 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a2809bc60 con 0x5562485237a0 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.460+0000 7f9a4862eb80 10 client.15741 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.460+0000 7f9a4862eb80 10 client.15741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.460+0000 7f9a4862eb80 20 client.15741 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.460+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55624853cb00 0x55624853eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.460+0000 7f9a4862eb80 4 client.15741 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.460+0000 7f9a4862eb80 1 -- 172.21.15.27:0/176906644 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556248481d50 con 0x55624853cb00 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.460+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55624853cb00 0x55624853eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.461+0000 7f9a16ffd700 10 client.15741 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.461+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55624853cb00 0x55624853eef0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac4d3d0754913899 server_cookie=53ebc7f4e97b8880 in_seq=0 out_seq=0 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.461+0000 7f9a16ffd700 1 -- 172.21.15.27:0/176906644 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556248481d50 con 0x55624853cb00 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.461+0000 7f9a16ffd700 10 client.15741 handle_command_reply: tid=0 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.461+0000 7f9a4862eb80 1 client.15741 shutdown 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 -- 172.21.15.27:0/176906644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 msgr2=0x55624852f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852f5b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9a280cdaa0 tx=0x7f9a28005c00 comp rx=0 tx=0).stop 2024-06-06T08:41:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 -- 172.21.15.27:0/176906644 shutdown_connections 2024-06-06T08:41:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55624853cb00 0x55624853eef0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 0x55624852cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852f5b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/176906644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248528f60 0x55624852c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 -- 172.21.15.27:0/176906644 >> 172.21.15.27:0/176906644 conn(0x556248450080 msgr2=0x55624851d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a16ffd700 0 client.15741 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 -- 172.21.15.27:0/176906644 shutdown_connections 2024-06-06T08:41:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 1 -- 172.21.15.27:0/176906644 wait complete. 2024-06-06T08:41:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.462+0000 7f9a4862eb80 20 client.15741 trim_cache size 0 max 16384 2024-06-06T08:41:38.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a4862eb80 1 Processor -- start 2024-06-06T08:41:38.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a4862eb80 1 -- start start 2024-06-06T08:41:38.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248528f60 0x55624851de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 0x55624851e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381b2e90 con 0x5562485237a0 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381b0720 con 0x556248529880 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381ae570 con 0x556248528f60 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44068/0 (socket says 172.21.15.27:44068) 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.477+0000 7f9a377fe700 1 -- 172.21.15.27:0/342711615 learned_addr learned my addr 172.21.15.27:0/342711615 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/342711615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248528f60 0x55624851de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a377fe700 1 -- 172.21.15.27:0/342711615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248528f60 msgr2=0x55624851de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a377fe700 1 --2- 172.21.15.27:0/342711615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248528f60 0x55624851de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a377fe700 1 -- 172.21.15.27:0/342711615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 msgr2=0x55624851e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a377fe700 1 --2- 172.21.15.27:0/342711615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 0x55624851e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a377fe700 1 -- 172.21.15.27:0/342711615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556248523260 con 0x5562485237a0 2024-06-06T08:41:38.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a377fe700 1 --2- 172.21.15.27:0/342711615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852a370 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9a2809b6f0 tx=0x7f9a28005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3f518c5b08d7c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a1d7fa700 1 -- 172.21.15.27:0/342711615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280ba050 con 0x5562485237a0 2024-06-06T08:41:38.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a1d7fa700 1 -- 172.21.15.27:0/342711615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a2809ab10 con 0x5562485237a0 2024-06-06T08:41:38.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.478+0000 7f9a1d7fa700 1 -- 172.21.15.27:0/342711615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280a4f60 con 0x5562485237a0 2024-06-06T08:41:38.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 -- 172.21.15.27:0/342711615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 msgr2=0x55624852a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/342711615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852a370 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9a2809b6f0 tx=0x7f9a28005fb0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 -- 172.21.15.27:0/342711615 shutdown_connections 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/342711615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556248529880 0x55624851e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/342711615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5562485237a0 0x55624852a370 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/342711615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248528f60 0x55624851de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 -- 172.21.15.27:0/342711615 >> 172.21.15.27:0/342711615 conn(0x556248450080 msgr2=0x556248529c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 -- 172.21.15.27:0/342711615 shutdown_connections 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 -- 172.21.15.27:0/342711615 wait complete. 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 -- start start 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.479+0000 7f9a4862eb80 1 Processor -- start 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 0x556248535600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248535b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a4862eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 0x556248543e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381aebe0 con 0x556248528f60 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381b0050 con 0x5562485237a0 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a4862eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9a381af2b0 con 0x556248529880 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a4862eb80 10 client.0 _ll_register_callbacks cb 0x55624840a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248535b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248535b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44072/0 (socket says 172.21.15.27:44072) 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a36ffd700 1 -- 172.21.15.27:0/2763979958 learned_addr learned my addr 172.21.15.27:0/2763979958 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a377fe700 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 0x556248535600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a37fff700 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 0x556248543e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a36ffd700 1 -- 172.21.15.27:0/2763979958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 msgr2=0x556248543e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 0x556248543e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a36ffd700 1 -- 172.21.15.27:0/2763979958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 msgr2=0x556248535600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 0x556248535600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a36ffd700 1 -- 172.21.15.27:0/2763979958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556248544640 con 0x556248528f60 2024-06-06T08:41:38.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a36ffd700 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248535b40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00a5c0 tx=0x7f9a2c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78529b730d6e8be8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a16ffd700 1 -- 172.21.15.27:0/2763979958 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c029070 con 0x556248528f60 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a4862eb80 1 -- 172.21.15.27:0/2763979958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a2809ab10 con 0x556248528f60 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.480+0000 7f9a16ffd700 1 -- 172.21.15.27:0/2763979958 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9a2c010430 con 0x556248528f60 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a16ffd700 1 -- 172.21.15.27:0/2763979958 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a2c036050 con 0x556248528f60 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a4862eb80 10 client.15747 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a4862eb80 1 -- 172.21.15.27:0/2763979958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5562485a73c0 con 0x556248528f60 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a16ffd700 1 -- 172.21.15.27:0/2763979958 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a2809ab10 con 0x556248528f60 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a16ffd700 1 -- 172.21.15.27:0/2763979958 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a2c00f720 con 0x556248528f60 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a4862eb80 10 client.15747 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a4862eb80 10 client.15747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a4862eb80 20 client.15747 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x5562485a7ab0 0x5562485a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a4862eb80 4 client.15747 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a4862eb80 1 -- 172.21.15.27:0/2763979958 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55624847c2d0 con 0x5562485a7ab0 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.481+0000 7f9a377fe700 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x5562485a7ab0 0x5562485a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a16ffd700 10 client.15747 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a377fe700 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x5562485a7ab0 0x5562485a9ed0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adfc1350c8c2c812 server_cookie=5e30b6c0a79afe01 in_seq=0 out_seq=0 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a16ffd700 1 -- 172.21.15.27:0/2763979958 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55624847c2d0 con 0x5562485a7ab0 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a16ffd700 10 client.15747 handle_command_reply: tid=0 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 client.15747 shutdown 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 -- 172.21.15.27:0/2763979958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 msgr2=0x556248535b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248535b40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00a5c0 tx=0x7f9a2c0047c0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 -- 172.21.15.27:0/2763979958 shutdown_connections 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x5562485a7ab0 0x5562485a9ed0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556248529880 0x556248543e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562485237a0 0x556248535600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 --2- 172.21.15.27:0/2763979958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556248528f60 0x556248535b40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a16ffd700 0 client.15747 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:38.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 -- 172.21.15.27:0/2763979958 >> 172.21.15.27:0/2763979958 conn(0x556248450080 msgr2=0x55624852e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 -- 172.21.15.27:0/2763979958 shutdown_connections 2024-06-06T08:41:38.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.482+0000 7f9a4862eb80 1 -- 172.21.15.27:0/2763979958 wait complete. 2024-06-06T08:41:38.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a4862eb80 20 client.15747 trim_cache size 0 max 16384 2024-06-06T08:41:38.487 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 -- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9a200a2f40 msgr2=0x7f9a200a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9a200a2f40 0x7f9a200a5400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c005fd0 tx=0x7f9a2c005e30 comp rx=0 tx=0).stop 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 -- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38101380 msgr2=0x7f9a3819ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38101380 0x7f9a3819ad80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9a28000c00 tx=0x7f9a280047b0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 -- 172.21.15.27:0/1280723111 shutdown_connections 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a381073f0 0x7f9a3819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38101380 0x7f9a3819ad80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9a200a2f40 0x7f9a200a5400 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 --2- 172.21.15.27:0/1280723111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a38101d40 0x7f9a3819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 -- 172.21.15.27:0/1280723111 >> 172.21.15.27:0/1280723111 conn(0x7f9a38078300 msgr2=0x7f9a3806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 -- 172.21.15.27:0/1280723111 shutdown_connections 2024-06-06T08:41:38.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:38.483+0000 7f9a3ddab700 1 -- 172.21.15.27:0/1280723111 wait complete. 2024-06-06T08:41:38.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:38 smithi203 ceph-mon[36579]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-06-06T08:41:38.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:38 smithi176 ceph-mon[36786]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 6 op/s 2024-06-06T08:41:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 -- 172.21.15.27:0/30997256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6401079c0 msgr2=0x7fc640107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 --2- 172.21.15.27:0/30997256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6401079c0 0x7fc640107da0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc630009930 tx=0x7fc63002a420 comp rx=0 tx=0).stop 2024-06-06T08:41:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 -- 172.21.15.27:0/30997256 shutdown_connections 2024-06-06T08:41:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 --2- 172.21.15.27:0/30997256 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc640102370 0x7fc64010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 --2- 172.21.15.27:0/30997256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6401019b0 0x7fc640101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 --2- 172.21.15.27:0/30997256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6401079c0 0x7fc640107da0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 -- 172.21.15.27:0/30997256 >> 172.21.15.27:0/30997256 conn(0x7fc640078e30 msgr2=0x7fc640079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 -- 172.21.15.27:0/30997256 shutdown_connections 2024-06-06T08:41:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 -- 172.21.15.27:0/30997256 wait complete. 2024-06-06T08:41:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 Processor -- start 2024-06-06T08:41:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.784+0000 7fc645612640 1 -- start start 2024-06-06T08:41:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc645612640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6401019b0 0x7fc6401981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc645612640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc640102370 0x7fc640198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc645612640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6401079c0 0x7fc64019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc645612640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc640111dd0 con 0x7fc640102370 2024-06-06T08:41:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc645612640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc640111c50 con 0x7fc6401079c0 2024-06-06T08:41:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc645612640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc640111f50 con 0x7fc6401019b0 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc640102370 0x7fc640198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc640102370 0x7fc640198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44096/0 (socket says 172.21.15.27:44096) 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63e7fc640 1 -- 172.21.15.27:0/1683998134 learned_addr learned my addr 172.21.15.27:0/1683998134 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6401079c0 0x7fc64019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63effd640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6401019b0 0x7fc6401981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63e7fc640 1 -- 172.21.15.27:0/1683998134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6401019b0 msgr2=0x7fc6401981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6401019b0 0x7fc6401981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63e7fc640 1 -- 172.21.15.27:0/1683998134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6401079c0 msgr2=0x7fc64019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6401079c0 0x7fc64019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63e7fc640 1 -- 172.21.15.27:0/1683998134 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64019d1c0 con 0x7fc640102370 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc640102370 0x7fc640198710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc62800c970 tx=0x7fc62800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc637fff640 1 -- 172.21.15.27:0/1683998134 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628007bb0 con 0x7fc640102370 2024-06-06T08:41:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc637fff640 1 -- 172.21.15.27:0/1683998134 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc628007d50 con 0x7fc640102370 2024-06-06T08:41:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.785+0000 7fc645612640 1 -- 172.21.15.27:0/1683998134 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64019d4b0 con 0x7fc640102370 2024-06-06T08:41:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.786+0000 7fc637fff640 1 -- 172.21.15.27:0/1683998134 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62800f450 con 0x7fc640102370 2024-06-06T08:41:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.786+0000 7fc645612640 1 -- 172.21.15.27:0/1683998134 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6401a4d40 con 0x7fc640102370 2024-06-06T08:41:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.788+0000 7fc637fff640 1 -- 172.21.15.27:0/1683998134 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fc628005020 con 0x7fc640102370 2024-06-06T08:41:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.788+0000 7fc6483f8740 1 Processor -- start 2024-06-06T08:41:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.788+0000 7fc6483f8740 1 -- start start 2024-06-06T08:41:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.788+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43146550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.788+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43150e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.788+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43153740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.788+0000 7fc6483f8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563b430aa4f0 con 0x563b43146170 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.788+0000 7fc6483f8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563b43048870 con 0x563b43146a90 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.788+0000 7fc6483f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563b430a64b0 con 0x563b43151350 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc637fff640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc62c0a2f50 0x7fc62c0a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63e7fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44108/0 (socket says 172.21.15.27:44108) 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 -- 172.21.15.27:0/2902926150 learned_addr learned my addr 172.21.15.27:0/2902926150 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc62c0a2f50 0x7fc62c0a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63effd640 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc637fff640 1 -- 172.21.15.27:0/1683998134 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fc6280cf8e0 con 0x7fc640102370 2024-06-06T08:41:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 -- 172.21.15.27:0/2902926150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 msgr2=0x563b43153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 -- 172.21.15.27:0/2902926150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 msgr2=0x563b43150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 -- 172.21.15.27:0/2902926150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b430649b0 con 0x563b43146170 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63effd640 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43146550 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc638009930 tx=0x7fc6380117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67cba2553526bd7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc62c0a2f50 0x7fc62c0a5410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc638011cb0 tx=0x7fc638015000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.789+0000 7fc6357fa640 1 -- 172.21.15.27:0/2902926150 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638013520 con 0x563b43146170 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6357fa640 1 -- 172.21.15.27:0/2902926150 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638003e00 con 0x563b43146170 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6357fa640 1 -- 172.21.15.27:0/2902926150 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63801fc00 con 0x563b43146170 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6483f8740 1 -- 172.21.15.27:0/2902926150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 msgr2=0x563b43146550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6483f8740 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43146550 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc638009930 tx=0x7fc6380117b0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6483f8740 1 -- 172.21.15.27:0/2902926150 shutdown_connections 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6483f8740 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6483f8740 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6483f8740 1 --2- 172.21.15.27:0/2902926150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43146550 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6483f8740 1 -- 172.21.15.27:0/2902926150 >> 172.21.15.27:0/2902926150 conn(0x563b43142680 msgr2=0x563b43142a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6483f8740 1 -- 172.21.15.27:0/2902926150 shutdown_connections 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.790+0000 7fc6483f8740 1 -- 172.21.15.27:0/2902926150 wait complete. 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc6483f8740 1 -- start start 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc6483f8740 1 Processor -- start 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146170 0x563b4314e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146a90 0x563b4314e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b4314ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc6483f8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563b43153f20 con 0x563b43146a90 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc6483f8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563b43033ae0 con 0x563b43146170 2024-06-06T08:41:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc6483f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563b431540a0 con 0x563b43151350 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc6483f8740 10 client.0 _ll_register_callbacks cb 0x563b4306c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc63e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146a90 0x563b4314e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc63e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146a90 0x563b4314e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44124/0 (socket says 172.21.15.27:44124) 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc63e7fc640 1 -- 172.21.15.27:0/3939118051 learned_addr learned my addr 172.21.15.27:0/3939118051 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc63effd640 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146170 0x563b4314e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b4314ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc63e7fc640 1 -- 172.21.15.27:0/3939118051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 msgr2=0x563b4314ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.791+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b4314ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc63e7fc640 1 -- 172.21.15.27:0/3939118051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146170 msgr2=0x563b4314e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146170 0x563b4314e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc63e7fc640 1 -- 172.21.15.27:0/3939118051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b42dbe210 con 0x563b43146a90 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146a90 0x563b4314e920 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc628007590 tx=0x7fc62800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421a12e3f3352aa9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6267fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6267fc640 1 -- 172.21.15.27:0/3939118051 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62801d040 con 0x563b43146a90 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6267fc640 1 -- 172.21.15.27:0/3939118051 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fc628005310 con 0x563b43146a90 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6483f8740 1 -- 172.21.15.27:0/3939118051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc638003e00 con 0x563b43146a90 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6267fc640 1 -- 172.21.15.27:0/3939118051 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc62809fbb0 con 0x563b43146a90 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6483f8740 10 client.15759 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6483f8740 1 -- 172.21.15.27:0/3939118051 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b42e8a620 con 0x563b43146a90 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6267fc640 1 -- 172.21.15.27:0/3939118051 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638003e00 con 0x563b43146a90 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6267fc640 1 -- 172.21.15.27:0/3939118051 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc62809cbc0 con 0x563b43146a90 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6483f8740 10 client.15759 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6483f8740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6483f8740 20 client.15759 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563b43161530 0x563b43163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6483f8740 4 client.15759 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.792+0000 7fc6483f8740 1 -- 172.21.15.27:0/3939118051 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b430bdf10 con 0x563b43161530 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.793+0000 7fc63effd640 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563b43161530 0x563b43163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.793+0000 7fc63effd640 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563b43161530 0x563b43163920 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c69a3d9c5645700 server_cookie=7ea4e47d7c980eed in_seq=0 out_seq=0 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.793+0000 7fc6267fc640 10 client.15759 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6267fc640 1 -- 172.21.15.27:0/3939118051 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563b430bdf10 con 0x563b43161530 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6267fc640 10 client.15759 handle_command_reply: tid=0 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 client.15759 shutdown 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 -- 172.21.15.27:0/3939118051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146a90 msgr2=0x563b4314e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146a90 0x563b4314e920 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc628007590 tx=0x7fc62800a570 comp rx=0 tx=0).stop 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 -- 172.21.15.27:0/3939118051 shutdown_connections 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563b43161530 0x563b43163920 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6267fc640 0 client.15759 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b4314ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146a90 0x563b4314e920 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3939118051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146170 0x563b4314e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 -- 172.21.15.27:0/3939118051 >> 172.21.15.27:0/3939118051 conn(0x563b43142680 msgr2=0x563b431483b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 -- 172.21.15.27:0/3939118051 shutdown_connections 2024-06-06T08:41:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 1 -- 172.21.15.27:0/3939118051 wait complete. 2024-06-06T08:41:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.794+0000 7fc6483f8740 20 client.15759 trim_cache size 0 max 16384 2024-06-06T08:41:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.809+0000 7fc6483f8740 1 Processor -- start 2024-06-06T08:41:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.809+0000 7fc6483f8740 1 -- start start 2024-06-06T08:41:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43141f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43146a90 0x563b43150320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43151350 0x563b43169210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc6483f8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401b16c0 con 0x563b43146170 2024-06-06T08:41:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc6483f8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401aef90 con 0x563b43151350 2024-06-06T08:41:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc6483f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401acde0 con 0x563b43146a90 2024-06-06T08:41:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44138/0 (socket says 172.21.15.27:44138) 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63effd640 1 -- 172.21.15.27:0/2196580420 learned_addr learned my addr 172.21.15.27:0/2196580420 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43146a90 0x563b43150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63effd640 1 -- 172.21.15.27:0/2196580420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43146a90 msgr2=0x563b43150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43151350 0x563b43169210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63effd640 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43146a90 0x563b43150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63effd640 1 -- 172.21.15.27:0/2196580420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43151350 msgr2=0x563b43169210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63effd640 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43151350 0x563b43169210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63effd640 1 -- 172.21.15.27:0/2196580420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4315d180 con 0x563b43146170 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43151350 0x563b43169210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc63effd640 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43141f60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc630009930 tx=0x7fc6300095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d039b16fb696220 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc634ff9640 1 -- 172.21.15.27:0/2196580420 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630048070 con 0x563b43146170 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc634ff9640 1 -- 172.21.15.27:0/2196580420 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc630005520 con 0x563b43146170 2024-06-06T08:41:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.810+0000 7fc634ff9640 1 -- 172.21.15.27:0/2196580420 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630035070 con 0x563b43146170 2024-06-06T08:41:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 -- 172.21.15.27:0/2196580420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 msgr2=0x563b43141f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43141f60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc630009930 tx=0x7fc6300095f0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 -- 172.21.15.27:0/2196580420 shutdown_connections 2024-06-06T08:41:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43151350 0x563b43169210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43146a90 0x563b43150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 --2- 172.21.15.27:0/2196580420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b43141f60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 -- 172.21.15.27:0/2196580420 >> 172.21.15.27:0/2196580420 conn(0x563b43142680 msgr2=0x563b4314fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 -- 172.21.15.27:0/2196580420 shutdown_connections 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 -- 172.21.15.27:0/2196580420 wait complete. 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.811+0000 7fc6483f8740 1 -- start start 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc6483f8740 1 Processor -- start 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b431599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43154d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc6483f8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43155260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc6483f8740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401ad450 con 0x563b43146170 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc6483f8740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401ae8c0 con 0x563b43146a90 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc6483f8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401adb20 con 0x563b43151350 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc6483f8740 10 client.0 _ll_register_callbacks cb 0x7fc6280058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b431599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b431599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44150/0 (socket says 172.21.15.27:44150) 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63effd640 1 -- 172.21.15.27:0/3585745498 learned_addr learned my addr 172.21.15.27:0/3585745498 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63effd640 1 -- 172.21.15.27:0/3585745498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 msgr2=0x563b43155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63effd640 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63effd640 1 -- 172.21.15.27:0/3585745498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 msgr2=0x563b43154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63effd640 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63effd640 1 -- 172.21.15.27:0/3585745498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b43155a60 con 0x563b43146170 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63f7fe640 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc63effd640 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b431599b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc630031520 tx=0x7fc630004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd5eb22f72bc18 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.812+0000 7fc6267fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6267fc640 1 -- 172.21.15.27:0/3585745498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630050020 con 0x563b43146170 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6267fc640 1 -- 172.21.15.27:0/3585745498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fc6300048c0 con 0x563b43146170 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6483f8740 1 -- 172.21.15.27:0/3585745498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc630005520 con 0x563b43146170 2024-06-06T08:41:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6267fc640 1 -- 172.21.15.27:0/3585745498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc630048040 con 0x563b43146170 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6483f8740 10 client.15765 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6483f8740 1 -- 172.21.15.27:0/3585745498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b431cc0d0 con 0x563b43146170 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6267fc640 1 -- 172.21.15.27:0/3585745498 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc630005520 con 0x563b43146170 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6267fc640 1 -- 172.21.15.27:0/3585745498 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc63004b030 con 0x563b43146170 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6483f8740 10 client.15765 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6483f8740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6483f8740 20 client.15765 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563b43161530 0x563b431ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6483f8740 4 client.15765 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc6483f8740 1 -- 172.21.15.27:0/3585745498 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b430b7c20 con 0x563b43161530 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.813+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563b43161530 0x563b431ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.814+0000 7fc63e7fc640 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563b43161530 0x563b431ce7d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50cd1f3d5248d9c2 server_cookie=da7d169caabe50fd in_seq=0 out_seq=0 2024-06-06T08:41:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.814+0000 7fc6267fc640 10 client.15765 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6267fc640 1 -- 172.21.15.27:0/3585745498 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563b430b7c20 con 0x563b43161530 2024-06-06T08:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6267fc640 10 client.15765 handle_command_reply: tid=0 2024-06-06T08:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 client.15765 shutdown 2024-06-06T08:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 -- 172.21.15.27:0/3585745498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 msgr2=0x563b431599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b431599b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc630031520 tx=0x7fc630004490 comp rx=0 tx=0).stop 2024-06-06T08:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 -- 172.21.15.27:0/3585745498 shutdown_connections 2024-06-06T08:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563b43161530 0x563b431ce7d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b43151350 0x563b43155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563b43146a90 0x563b43154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6267fc640 0 client.15765 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 --2- 172.21.15.27:0/3585745498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563b43146170 0x563b431599b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 -- 172.21.15.27:0/3585745498 >> 172.21.15.27:0/3585745498 conn(0x563b43142680 msgr2=0x563b4315ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 -- 172.21.15.27:0/3585745498 shutdown_connections 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.821+0000 7fc6483f8740 1 -- 172.21.15.27:0/3585745498 wait complete. 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc6483f8740 20 client.15765 trim_cache size 0 max 16384 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:41:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:41:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 -- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc62c0a2f50 msgr2=0x7fc62c0a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc62c0a2f50 0x7fc62c0a5410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc638011cb0 tx=0x7fc638015000 comp rx=0 tx=0).stop 2024-06-06T08:41:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 -- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc640102370 msgr2=0x7fc640198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc640102370 0x7fc640198710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc62800c970 tx=0x7fc62800ce30 comp rx=0 tx=0).stop 2024-06-06T08:41:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 -- 172.21.15.27:0/1683998134 shutdown_connections 2024-06-06T08:41:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fc62c0a2f50 0x7fc62c0a5410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6401079c0 0x7fc64019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc640102370 0x7fc640198710 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 --2- 172.21.15.27:0/1683998134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6401019b0 0x7fc6401981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 -- 172.21.15.27:0/1683998134 >> 172.21.15.27:0/1683998134 conn(0x7fc640078e30 msgr2=0x7fc6400ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 -- 172.21.15.27:0/1683998134 shutdown_connections 2024-06-06T08:41:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:38.822+0000 7fc645612640 1 -- 172.21.15.27:0/1683998134 wait complete. 2024-06-06T08:41:39.221 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T08:41:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:40.647 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:40 smithi027 ceph-mon[32335]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-06T08:41:40.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:40 smithi203 ceph-mon[36579]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-06T08:41:40.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:40 smithi176 ceph-mon[36786]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-06T08:41:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.272+0000 7fd89ab1b700 1 -- 172.21.15.27:0/2038773444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8941009e0 msgr2=0x7fd894100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.272+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/2038773444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8941009e0 0x7fd894100e60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd880009990 tx=0x7fd880009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.272+0000 7fd89ab1b700 1 -- 172.21.15.27:0/2038773444 shutdown_connections 2024-06-06T08:41:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.272+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/2038773444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8941013a0 0x7fd89410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.272+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/2038773444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd894108560 0x7fd894108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.272+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/2038773444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8941009e0 0x7fd894100e60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.272+0000 7fd89ab1b700 1 -- 172.21.15.27:0/2038773444 >> 172.21.15.27:0/2038773444 conn(0x7fd894077990 msgr2=0x7fd8940fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:41.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.272+0000 7fd89ab1b700 1 -- 172.21.15.27:0/2038773444 shutdown_connections 2024-06-06T08:41:41.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.272+0000 7fd89ab1b700 1 -- 172.21.15.27:0/2038773444 wait complete. 2024-06-06T08:41:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd89ab1b700 1 Processor -- start 2024-06-06T08:41:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd89ab1b700 1 -- start start 2024-06-06T08:41:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd89ab1b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8941009e0 0x7fd89419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd89ab1b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd8941013a0 0x7fd89419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd89ab1b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd894108560 0x7fd89410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd89ab1b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941127f0 con 0x7fd894108560 2024-06-06T08:41:41.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd89ab1b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd894112670 con 0x7fd8941013a0 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd89ab1b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd894112970 con 0x7fd8941009e0 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd8990b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd894108560 0x7fd89410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd8988b7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8941009e0 0x7fd89419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd8990b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd894108560 0x7fd89410e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56472/0 (socket says 172.21.15.27:56472) 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd8988b7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8941009e0 0x7fd89419f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:42724/0 (socket says 172.21.15.27:42724) 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd8990b8700 1 -- 172.21.15.27:0/554632583 learned_addr learned my addr 172.21.15.27:0/554632583 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.273+0000 7fd893fff700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd8941013a0 0x7fd89419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd8990b8700 1 -- 172.21.15.27:0/554632583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8941009e0 msgr2=0x7fd89419f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd8990b8700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8941009e0 0x7fd89419f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd8990b8700 1 -- 172.21.15.27:0/554632583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd8941013a0 msgr2=0x7fd89419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd8990b8700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd8941013a0 0x7fd89419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd8990b8700 1 -- 172.21.15.27:0/554632583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89410eda0 con 0x7fd894108560 2024-06-06T08:41:41.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd8988b7700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8941009e0 0x7fd89419f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd8990b8700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd894108560 0x7fd89410e4d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00ef30 tx=0x7fd88c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd891ffb700 1 -- 172.21.15.27:0/554632583 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0092e0 con 0x7fd894108560 2024-06-06T08:41:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd891ffb700 1 -- 172.21.15.27:0/554632583 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88c004bb0 con 0x7fd894108560 2024-06-06T08:41:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd89ab1b700 1 -- 172.21.15.27:0/554632583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89410f030 con 0x7fd894108560 2024-06-06T08:41:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd891ffb700 1 -- 172.21.15.27:0/554632583 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c004280 con 0x7fd894108560 2024-06-06T08:41:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.274+0000 7fd89ab1b700 1 -- 172.21.15.27:0/554632583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89419b4a0 con 0x7fd894108560 2024-06-06T08:41:41.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.277+0000 7fd891ffb700 1 -- 172.21.15.27:0/554632583 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fd88c01b030 con 0x7fd894108560 2024-06-06T08:41:41.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.277+0000 7fd891ffb700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd8840a2f40 0x7fd8840a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.277+0000 7fd891ffb700 1 -- 172.21.15.27:0/554632583 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fd88c0cf4d0 con 0x7fd894108560 2024-06-06T08:41:41.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.277+0000 7fd8988b7700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd8840a2f40 0x7fd8840a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8988b7700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd8840a2f40 0x7fd8840a5400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd88800afa0 tx=0x7fd88800a380 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8a539eb80 1 Processor -- start 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8a539eb80 1 -- start start 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b62149f60 0x562b6214a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 0x562b62144260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b62036f60 con 0x562b62149f60 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562b62095850 con 0x562b6214a880 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b61ffd460 con 0x562b621447a0 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8988b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b62149f60 0x562b6214a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8988b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b62149f60 0x562b6214a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56480/0 (socket says 172.21.15.27:56480) 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.278+0000 7fd8988b7700 1 -- 172.21.15.27:0/2029397512 learned_addr learned my addr 172.21.15.27:0/2029397512 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd8990b8700 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd893fff700 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 0x562b62144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd8990b8700 1 -- 172.21.15.27:0/2029397512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 msgr2=0x562b62144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd8990b8700 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 0x562b62144260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd8990b8700 1 -- 172.21.15.27:0/2029397512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b62149f60 msgr2=0x562b6214a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd8990b8700 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b62149f60 0x562b6214a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd8990b8700 1 -- 172.21.15.27:0/2029397512 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b61fe48d0 con 0x562b621447a0 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd8988b7700 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b62149f60 0x562b6214a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd8990b8700 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0cc6c0 tx=0x7fd88c00cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b2de48817613aa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd87effd700 1 -- 172.21.15.27:0/2029397512 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c09f8d0 con 0x562b621447a0 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd87effd700 1 -- 172.21.15.27:0/2029397512 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88c0a4100 con 0x562b621447a0 2024-06-06T08:41:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.279+0000 7fd87effd700 1 -- 172.21.15.27:0/2029397512 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0a3ed0 con 0x562b621447a0 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2029397512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 msgr2=0x562b62144bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0cc6c0 tx=0x7fd88c00cf00 comp rx=0 tx=0).stop 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2029397512 shutdown_connections 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144bc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b62149f60 0x562b6214a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2029397512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 0x562b62144260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2029397512 >> 172.21.15.27:0/2029397512 conn(0x562b62071080 msgr2=0x562b6213f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2029397512 shutdown_connections 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2029397512 wait complete. 2024-06-06T08:41:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 -- start start 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.280+0000 7fd8a539eb80 1 Processor -- start 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62150670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 0x562b6214d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6214deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562b62035fc0 con 0x562b6214a880 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x562b62091810 con 0x562b62149f60 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b62151260 con 0x562b621447a0 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8990b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6214deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8a539eb80 10 client.0 _ll_register_callbacks cb 0x562b6202b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8990b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6214deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56488/0 (socket says 172.21.15.27:56488) 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8990b8700 1 -- 172.21.15.27:0/2056299681 learned_addr learned my addr 172.21.15.27:0/2056299681 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd893fff700 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 0x562b6214d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8988b7700 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8990b8700 1 -- 172.21.15.27:0/2056299681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 msgr2=0x562b62150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8990b8700 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8990b8700 1 -- 172.21.15.27:0/2056299681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 msgr2=0x562b6214d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8990b8700 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 0x562b6214d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.281+0000 7fd8990b8700 1 -- 172.21.15.27:0/2056299681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b6205b750 con 0x562b6214a880 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8990b8700 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6214deb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0cc720 tx=0x7fd88c09b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9000d712110158 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd873fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd873fff700 1 -- 172.21.15.27:0/2056299681 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0b3070 con 0x562b6214a880 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd873fff700 1 -- 172.21.15.27:0/2056299681 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fd88c0b4260 con 0x562b6214a880 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2056299681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd88c0a4100 con 0x562b6214a880 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd873fff700 1 -- 172.21.15.27:0/2056299681 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd88c0b4460 con 0x562b6214a880 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8a539eb80 10 client.15777 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2056299681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562b6215ba60 con 0x562b6214a880 2024-06-06T08:41:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd873fff700 1 -- 172.21.15.27:0/2056299681 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88c0a4100 con 0x562b6214a880 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd873fff700 1 -- 172.21.15.27:0/2056299681 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd88c003a00 con 0x562b6214a880 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8a539eb80 10 client.15777 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8a539eb80 10 client.15777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8a539eb80 20 client.15777 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x562b6215df30 0x562b62160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8a539eb80 4 client.15777 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.282+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2056299681 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b620a2d50 con 0x562b6215df30 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.283+0000 7fd8988b7700 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x562b6215df30 0x562b62160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.283+0000 7fd8988b7700 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x562b6215df30 0x562b62160320 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eeb2b35571705e32 server_cookie=a80d84ec7ef02b6f in_seq=0 out_seq=0 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.283+0000 7fd873fff700 10 client.15777 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd873fff700 1 -- 172.21.15.27:0/2056299681 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b620a2d50 con 0x562b6215df30 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd873fff700 10 client.15777 handle_command_reply: tid=0 2024-06-06T08:41:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 client.15777 shutdown 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2056299681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 msgr2=0x562b6214deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6214deb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0cc720 tx=0x7fd88c09b5e0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2056299681 shutdown_connections 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6214deb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x562b6215df30 0x562b62160320 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd873fff700 0 client.15777 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2056299681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 0x562b6214d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2056299681 >> 172.21.15.27:0/2056299681 conn(0x562b62071080 msgr2=0x562b6213e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.284+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2056299681 shutdown_connections 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.285+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2056299681 wait complete. 2024-06-06T08:41:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.285+0000 7fd8a539eb80 20 client.15777 trim_cache size 0 max 16384 2024-06-06T08:41:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8a539eb80 1 Processor -- start 2024-06-06T08:41:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8a539eb80 1 -- start start 2024-06-06T08:41:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 0x562b6214c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6213ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941bfc10 con 0x562b6214a880 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941bd4a0 con 0x562b62149f60 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941bb2f0 con 0x562b621447a0 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8990b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6213ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8988b7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8988b7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:42748/0 (socket says 172.21.15.27:42748) 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.300+0000 7fd8988b7700 1 -- 172.21.15.27:0/1508203097 learned_addr learned my addr 172.21.15.27:0/1508203097 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd893fff700 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 0x562b6214c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd8988b7700 1 -- 172.21.15.27:0/1508203097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 msgr2=0x562b6214c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd8988b7700 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 0x562b6214c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd8988b7700 1 -- 172.21.15.27:0/1508203097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 msgr2=0x562b6213ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd8988b7700 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6213ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd8988b7700 1 -- 172.21.15.27:0/1508203097 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6214bee0 con 0x562b621447a0 2024-06-06T08:41:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd893fff700 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 0x562b6214c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd8990b8700 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6213ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd8988b7700 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562b61ffd580 tx=0x7fd888012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356eb35197c3442a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd87e7fc700 1 -- 172.21.15.27:0/1508203097 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888016070 con 0x562b621447a0 2024-06-06T08:41:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd87e7fc700 1 -- 172.21.15.27:0/1508203097 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd888003b60 con 0x562b621447a0 2024-06-06T08:41:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.301+0000 7fd87e7fc700 1 -- 172.21.15.27:0/1508203097 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888028690 con 0x562b621447a0 2024-06-06T08:41:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 -- 172.21.15.27:0/1508203097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 msgr2=0x562b62144260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562b61ffd580 tx=0x7fd888012000 comp rx=0 tx=0).stop 2024-06-06T08:41:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 -- 172.21.15.27:0/1508203097 shutdown_connections 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b6214a880 0x562b6213ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b621447a0 0x562b62144260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/1508203097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b62149f60 0x562b6214c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 -- 172.21.15.27:0/1508203097 >> 172.21.15.27:0/1508203097 conn(0x562b62071080 msgr2=0x562b6214c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 -- 172.21.15.27:0/1508203097 shutdown_connections 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 -- 172.21.15.27:0/1508203097 wait complete. 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 -- start start 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.302+0000 7fd8a539eb80 1 Processor -- start 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b621447a0 0x562b62161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b62149f60 0x562b62161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8a539eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 0x562b62161b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941bb960 con 0x562b621447a0 2024-06-06T08:41:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941bcdd0 con 0x562b6214a880 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8a539eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941bc030 con 0x562b62149f60 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8a539eb80 10 client.0 _ll_register_callbacks cb 0x562b6202b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8988b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b621447a0 0x562b62161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8988b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b621447a0 0x562b62161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56508/0 (socket says 172.21.15.27:56508) 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd893fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b62149f60 0x562b62161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8988b7700 1 -- 172.21.15.27:0/2118334921 learned_addr learned my addr 172.21.15.27:0/2118334921 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8990b8700 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 0x562b62161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd893fff700 1 -- 172.21.15.27:0/2118334921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 msgr2=0x562b62161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd893fff700 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 0x562b62161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd893fff700 1 -- 172.21.15.27:0/2118334921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b621447a0 msgr2=0x562b62161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd893fff700 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b621447a0 0x562b62161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd893fff700 1 -- 172.21.15.27:0/2118334921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b621c7870 con 0x562b62149f60 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd8988b7700 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b621447a0 0x562b62161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd893fff700 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b62149f60 0x562b62161640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8800059d0 tx=0x7fd88000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9f555b9cbccc84c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.303+0000 7fd873fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd873fff700 1 -- 172.21.15.27:0/2118334921 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880005690 con 0x562b62149f60 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2118334921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd888003b60 con 0x562b62149f60 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd873fff700 1 -- 172.21.15.27:0/2118334921 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fd880003680 con 0x562b62149f60 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd873fff700 1 -- 172.21.15.27:0/2118334921 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd880032050 con 0x562b62149f60 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8a539eb80 10 client.25339 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2118334921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562b621c8b10 con 0x562b62149f60 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd873fff700 1 -- 172.21.15.27:0/2118334921 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd888003b60 con 0x562b62149f60 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd873fff700 1 -- 172.21.15.27:0/2118334921 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd880024ac0 con 0x562b62149f60 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8a539eb80 10 client.25339 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8a539eb80 10 client.25339 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8a539eb80 20 client.25339 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x562b621c91c0 0x562b621cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8a539eb80 4 client.25339 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2118334921 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b6209d2d0 con 0x562b621c91c0 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.304+0000 7fd8988b7700 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x562b621c91c0 0x562b621cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8988b7700 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x562b621c91c0 0x562b621cb5e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec364f081227b4be server_cookie=3fae0fd37a6a9241 in_seq=0 out_seq=0 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd873fff700 10 client.25339 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd873fff700 1 -- 172.21.15.27:0/2118334921 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b6209d2d0 con 0x562b621c91c0 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd873fff700 10 client.25339 handle_command_reply: tid=0 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 client.25339 shutdown 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2118334921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b62149f60 msgr2=0x562b62161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b62149f60 0x562b62161640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8800059d0 tx=0x7fd88000bfd0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2118334921 shutdown_connections 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x562b621c91c0 0x562b621cb5e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562b6214a880 0x562b62161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562b621447a0 0x562b62161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd873fff700 0 client.25339 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:41:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 --2- 172.21.15.27:0/2118334921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b62149f60 0x562b62161640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2118334921 >> 172.21.15.27:0/2118334921 conn(0x562b62071080 msgr2=0x562b6214c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2118334921 shutdown_connections 2024-06-06T08:41:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 1 -- 172.21.15.27:0/2118334921 wait complete. 2024-06-06T08:41:41.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.305+0000 7fd8a539eb80 20 client.25339 trim_cache size 0 max 16384 2024-06-06T08:41:41.310 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 -- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd8840a2f40 msgr2=0x7fd8840a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd8840a2f40 0x7fd8840a5400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd88800afa0 tx=0x7fd88800a380 comp rx=0 tx=0).stop 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 -- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd894108560 msgr2=0x7fd89410e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd894108560 0x7fd89410e4d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00ef30 tx=0x7fd88c00c3a0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 -- 172.21.15.27:0/554632583 shutdown_connections 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd894108560 0x7fd89410e4d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8941009e0 0x7fd89419f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fd8840a2f40 0x7fd8840a5400 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 --2- 172.21.15.27:0/554632583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd8941013a0 0x7fd89419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:41.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 -- 172.21.15.27:0/554632583 >> 172.21.15.27:0/554632583 conn(0x7fd894077990 msgr2=0x7fd894105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 -- 172.21.15.27:0/554632583 shutdown_connections 2024-06-06T08:41:41.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:41.306+0000 7fd89ab1b700 1 -- 172.21.15.27:0/554632583 wait complete. 2024-06-06T08:41:42.098 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T08:41:42.354 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:42.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:42 smithi027 ceph-mon[32335]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-06-06T08:41:42.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:42 smithi203 ceph-mon[36579]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-06-06T08:41:42.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:42 smithi176 ceph-mon[36786]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-06-06T08:41:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.102+0000 7f1fe35f6700 1 -- 172.21.15.27:0/4287612350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 msgr2=0x7f1fdc10ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.102+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/4287612350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 0x7f1fdc10ea30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009990 tx=0x7f1fcc009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.102+0000 7f1fe35f6700 1 -- 172.21.15.27:0/4287612350 shutdown_connections 2024-06-06T08:41:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.102+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/4287612350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fdc10ef70 0x7f1fdc111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.102+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/4287612350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fdc06abe0 0x7f1fdc06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.102+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/4287612350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 0x7f1fdc10ea30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.102+0000 7f1fe35f6700 1 -- 172.21.15.27:0/4287612350 >> 172.21.15.27:0/4287612350 conn(0x7f1fdc078360 msgr2=0x7f1fdc078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.102+0000 7f1fe35f6700 1 -- 172.21.15.27:0/4287612350 shutdown_connections 2024-06-06T08:41:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.102+0000 7f1fe35f6700 1 -- 172.21.15.27:0/4287612350 wait complete. 2024-06-06T08:41:44.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe35f6700 1 Processor -- start 2024-06-06T08:41:44.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe35f6700 1 -- start start 2024-06-06T08:41:44.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe35f6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fdc06abe0 0x7f1fdc19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe35f6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 0x7f1fdc19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe35f6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fdc10ef70 0x7f1fdc19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe35f6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1149a0 con 0x7f1fdc06b500 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe35f6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc114820 con 0x7f1fdc10ef70 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe35f6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc114b20 con 0x7f1fdc06abe0 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe0b91700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 0x7f1fdc19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe0b91700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 0x7f1fdc19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56532/0 (socket says 172.21.15.27:56532) 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.103+0000 7f1fe0b91700 1 -- 172.21.15.27:0/1841456564 learned_addr learned my addr 172.21.15.27:0/1841456564 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe1392700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fdc06abe0 0x7f1fdc19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe1b93700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fdc10ef70 0x7f1fdc19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe0b91700 1 -- 172.21.15.27:0/1841456564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fdc06abe0 msgr2=0x7f1fdc19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fdc06abe0 0x7f1fdc19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe0b91700 1 -- 172.21.15.27:0/1841456564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fdc10ef70 msgr2=0x7f1fdc19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fdc10ef70 0x7f1fdc19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe0b91700 1 -- 172.21.15.27:0/1841456564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc19fef0 con 0x7f1fdc06b500 2024-06-06T08:41:44.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe1b93700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fdc10ef70 0x7f1fdc19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:44.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 0x7f1fdc19b470 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0038b0 tx=0x7f1fcc009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:44.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fd27fc700 1 -- 172.21.15.27:0/1841456564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc027070 con 0x7f1fdc06b500 2024-06-06T08:41:44.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe35f6700 1 -- 172.21.15.27:0/1841456564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fdc1a0180 con 0x7f1fdc06b500 2024-06-06T08:41:44.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fd27fc700 1 -- 172.21.15.27:0/1841456564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fcc004050 con 0x7f1fdc06b500 2024-06-06T08:41:44.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fd27fc700 1 -- 172.21.15.27:0/1841456564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc022cd0 con 0x7f1fdc06b500 2024-06-06T08:41:44.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.104+0000 7f1fe35f6700 1 -- 172.21.15.27:0/1841456564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fdc1a7a60 con 0x7f1fdc06b500 2024-06-06T08:41:44.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.107+0000 7f1fd27fc700 1 -- 172.21.15.27:0/1841456564 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f1fcc02a070 con 0x7f1fdc06b500 2024-06-06T08:41:44.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.107+0000 7f1fd27fc700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1fc80a2f40 0x7f1fc80a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.108+0000 7f1fe1392700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1fc80a2f40 0x7f1fc80a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.108+0000 7f1fd27fc700 1 -- 172.21.15.27:0/1841456564 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f1fcc0d0910 con 0x7f1fdc06b500 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.108+0000 7f1fe1392700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1fc80a2f40 0x7f1fc80a5400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8007be0 tx=0x7f1fd80073d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.108+0000 7f1fede79b80 1 Processor -- start 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.108+0000 7f1fede79b80 1 -- start start 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.108+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e92f640 0x55724e92fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.108+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e9299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e929f20 0x55724e92a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fede79b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55724e87b810 con 0x55724e92f640 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fede79b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55724e823720 con 0x55724e929f20 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fede79b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55724e8777d0 con 0x55724e92ff60 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe1392700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e92f640 0x55724e92fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe1392700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e92f640 0x55724e92fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56544/0 (socket says 172.21.15.27:56544) 2024-06-06T08:41:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe1392700 1 -- 172.21.15.27:0/1545006949 learned_addr learned my addr 172.21.15.27:0/1545006949 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/1545006949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e9299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe1392700 1 -- 172.21.15.27:0/1545006949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 msgr2=0x55724e9299e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe1392700 1 --2- 172.21.15.27:0/1545006949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e9299e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe1392700 1 -- 172.21.15.27:0/1545006949 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e929f20 msgr2=0x55724e92a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe1392700 1 --2- 172.21.15.27:0/1545006949 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e929f20 0x55724e92a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe1392700 1 -- 172.21.15.27:0/1545006949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724e819f50 con 0x55724e92f640 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fe1392700 1 --2- 172.21.15.27:0/1545006949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e92f640 0x55724e92fa20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8010e80 tx=0x7f1fd8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92d38844b014100 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fc77fe700 1 -- 172.21.15.27:0/1545006949 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8024070 con 0x55724e92f640 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fc77fe700 1 -- 172.21.15.27:0/1545006949 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd8015070 con 0x55724e92f640 2024-06-06T08:41:44.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.109+0000 7f1fc77fe700 1 -- 172.21.15.27:0/1545006949 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011a40 con 0x55724e92f640 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 -- 172.21.15.27:0/1545006949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e92f640 msgr2=0x55724e92fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1545006949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e92f640 0x55724e92fa20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8010e80 tx=0x7f1fd8013000 comp rx=0 tx=0).stop 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 -- 172.21.15.27:0/1545006949 shutdown_connections 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1545006949 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e929f20 0x55724e92a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1545006949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e92f640 0x55724e92fa20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1545006949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e9299e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 -- 172.21.15.27:0/1545006949 >> 172.21.15.27:0/1545006949 conn(0x55724e854cc0 msgr2=0x55724e924850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 -- 172.21.15.27:0/1545006949 shutdown_connections 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 -- 172.21.15.27:0/1545006949 wait complete. 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.110+0000 7f1fede79b80 1 -- start start 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fede79b80 1 Processor -- start 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e935d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 0x55724e933060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e9335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fede79b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55724e9369d0 con 0x55724e929f20 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fede79b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55724e81bbf0 con 0x55724e92f640 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fede79b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55724e936b50 con 0x55724e92ff60 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fede79b80 10 client.0 _ll_register_callbacks cb 0x55724e815920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fe1392700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e935d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fe1392700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e935d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56558/0 (socket says 172.21.15.27:56558) 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fe1392700 1 -- 172.21.15.27:0/1963306344 learned_addr learned my addr 172.21.15.27:0/1963306344 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 0x55724e933060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.111+0000 7f1fe1b93700 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e9335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fe1392700 1 -- 172.21.15.27:0/1963306344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 msgr2=0x55724e9335a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fe1392700 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e9335a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fe1392700 1 -- 172.21.15.27:0/1963306344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 msgr2=0x55724e933060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fe1392700 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 0x55724e933060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fe1392700 1 -- 172.21.15.27:0/1963306344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724e7ca8d0 con 0x55724e929f20 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fe1b93700 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e9335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fe1392700 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e935d60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8011600 tx=0x7f1fd8005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bb7f5b38611b92 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/1963306344 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd802e020 con 0x55724e929f20 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/1963306344 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f1fd8003b50 con 0x55724e929f20 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fede79b80 1 -- 172.21.15.27:0/1963306344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd8015070 con 0x55724e929f20 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/1963306344 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1fd8039050 con 0x55724e929f20 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fede79b80 10 client.15795 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.112+0000 7f1fede79b80 1 -- 172.21.15.27:0/1963306344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55724e9412f0 con 0x55724e929f20 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.113+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/1963306344 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd8015070 con 0x55724e929f20 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.113+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/1963306344 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1fd801f730 con 0x55724e929f20 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.113+0000 7f1fede79b80 10 client.15795 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.113+0000 7f1fede79b80 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:44.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.113+0000 7f1fede79b80 20 client.15795 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:44.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.113+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55724e943870 0x55724e945c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.113+0000 7f1fede79b80 4 client.15795 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:44.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.113+0000 7f1fede79b80 1 -- 172.21.15.27:0/1963306344 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55724e888d10 con 0x55724e943870 2024-06-06T08:41:44.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.113+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55724e943870 0x55724e945c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.114+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55724e943870 0x55724e945c60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8964059165c56908 server_cookie=d2225e26d420944f in_seq=0 out_seq=0 2024-06-06T08:41:44.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.114+0000 7f1fc4ff9700 10 client.15795 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:44.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/1963306344 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55724e888d10 con 0x55724e943870 2024-06-06T08:41:44.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fc4ff9700 10 client.15795 handle_command_reply: tid=0 2024-06-06T08:41:44.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 client.15795 shutdown 2024-06-06T08:41:44.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 -- 172.21.15.27:0/1963306344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 msgr2=0x55724e935d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:44.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e935d60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8011600 tx=0x7f1fd8005700 comp rx=0 tx=0).stop 2024-06-06T08:41:44.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 -- 172.21.15.27:0/1963306344 shutdown_connections 2024-06-06T08:41:44.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e9335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55724e943870 0x55724e945c60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e935d60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 --2- 172.21.15.27:0/1963306344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 0x55724e933060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fc4ff9700 0 client.15795 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 -- 172.21.15.27:0/1963306344 >> 172.21.15.27:0/1963306344 conn(0x55724e854cc0 msgr2=0x55724e9289d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 -- 172.21.15.27:0/1963306344 shutdown_connections 2024-06-06T08:41:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 1 -- 172.21.15.27:0/1963306344 wait complete. 2024-06-06T08:41:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.115+0000 7f1fede79b80 20 client.15795 trim_cache size 0 max 16384 2024-06-06T08:41:44.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fede79b80 1 Processor -- start 2024-06-06T08:41:44.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fede79b80 1 -- start start 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 0x55724e9309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e91e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fede79b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1b3150 con 0x55724e929f20 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fede79b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1b09e0 con 0x55724e92f640 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fede79b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1ae830 con 0x55724e92ff60 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fe1392700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.134+0000 7f1fe1392700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9299e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56564/0 (socket says 172.21.15.27:56564) 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe1392700 1 -- 172.21.15.27:0/770639055 learned_addr learned my addr 172.21.15.27:0/770639055 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 0x55724e9309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe1b93700 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e91e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe1392700 1 -- 172.21.15.27:0/770639055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 msgr2=0x55724e91e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe1392700 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e91e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe1392700 1 -- 172.21.15.27:0/770639055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 msgr2=0x55724e9309b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:44.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe1392700 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 0x55724e9309b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe1392700 1 -- 172.21.15.27:0/770639055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724e925460 con 0x55724e929f20 2024-06-06T08:41:44.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 0x55724e9309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:44.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fe1392700 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9299e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1fd801fb00 tx=0x7f1fd8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21e753fb2d0ba76 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:44.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fc6ffd700 1 -- 172.21.15.27:0/770639055 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8042070 con 0x55724e929f20 2024-06-06T08:41:44.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fc6ffd700 1 -- 172.21.15.27:0/770639055 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd8015030 con 0x55724e929f20 2024-06-06T08:41:44.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.135+0000 7f1fc6ffd700 1 -- 172.21.15.27:0/770639055 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd802e040 con 0x55724e929f20 2024-06-06T08:41:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 -- 172.21.15.27:0/770639055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 msgr2=0x55724e9299e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9299e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1fd801fb00 tx=0x7f1fd8003f10 comp rx=0 tx=0).stop 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 -- 172.21.15.27:0/770639055 shutdown_connections 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92ff60 0x55724e91e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9299e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 --2- 172.21.15.27:0/770639055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92f640 0x55724e9309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 -- 172.21.15.27:0/770639055 >> 172.21.15.27:0/770639055 conn(0x55724e854cc0 msgr2=0x55724e931950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 -- 172.21.15.27:0/770639055 shutdown_connections 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 -- 172.21.15.27:0/770639055 wait complete. 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 -- start start 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.136+0000 7f1fede79b80 1 Processor -- start 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9491f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92f640 0x55724e946590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fede79b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92ff60 0x55724e946ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fede79b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1aeea0 con 0x55724e929f20 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fede79b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1b0310 con 0x55724e92ff60 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fede79b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc1af570 con 0x55724e92f640 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fede79b80 10 client.0 _ll_register_callbacks cb 0x55724e815920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1392700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1392700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56580/0 (socket says 172.21.15.27:56580) 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1392700 1 -- 172.21.15.27:0/4162124579 learned_addr learned my addr 172.21.15.27:0/4162124579 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92f640 0x55724e946590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1b93700 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92ff60 0x55724e946ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1392700 1 -- 172.21.15.27:0/4162124579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92f640 msgr2=0x55724e946590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1392700 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92f640 0x55724e946590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1392700 1 -- 172.21.15.27:0/4162124579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92ff60 msgr2=0x55724e946ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1392700 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92ff60 0x55724e946ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1392700 1 -- 172.21.15.27:0/4162124579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724e947110 con 0x55724e929f20 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fe1392700 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9491f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8005ba0 tx=0x7f1fd8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc4015e3b5e92b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.137+0000 7f1fc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/4162124579 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8042040 con 0x55724e929f20 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/4162124579 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f1fd801f430 con 0x55724e929f20 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fede79b80 1 -- 172.21.15.27:0/4162124579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd8015030 con 0x55724e929f20 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/4162124579 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1fd8051050 con 0x55724e929f20 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fede79b80 10 client.15801 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fede79b80 1 -- 172.21.15.27:0/4162124579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55724e9adfe0 con 0x55724e929f20 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/4162124579 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd8015030 con 0x55724e929f20 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/4162124579 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1fd802a4a0 con 0x55724e929f20 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fede79b80 10 client.15801 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fede79b80 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fede79b80 20 client.15801 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fede79b80 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55724e9ae5f0 0x55724e9b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fede79b80 4 client.15801 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.138+0000 7f1fede79b80 1 -- 172.21.15.27:0/4162124579 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55724e883290 con 0x55724e9ae5f0 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55724e9ae5f0 0x55724e9b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fe0b91700 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55724e9ae5f0 0x55724e9b0a10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efdd24097f6cdb4 server_cookie=e6a2b2cbec89d1ae in_seq=0 out_seq=0 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fc4ff9700 10 client.15801 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fc4ff9700 1 -- 172.21.15.27:0/4162124579 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55724e883290 con 0x55724e9ae5f0 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fc4ff9700 10 client.15801 handle_command_reply: tid=0 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 client.15801 shutdown 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 -- 172.21.15.27:0/4162124579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 msgr2=0x55724e9491f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9491f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8005ba0 tx=0x7f1fd8005bd0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 -- 172.21.15.27:0/4162124579 shutdown_connections 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55724e92ff60 0x55724e946ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55724e9ae5f0 0x55724e9b0a10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55724e929f20 0x55724e9491f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 --2- 172.21.15.27:0/4162124579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55724e92f640 0x55724e946590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fc4ff9700 0 client.15801 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 -- 172.21.15.27:0/4162124579 >> 172.21.15.27:0/4162124579 conn(0x55724e854cc0 msgr2=0x55724e92ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 -- 172.21.15.27:0/4162124579 shutdown_connections 2024-06-06T08:41:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.139+0000 7f1fede79b80 1 -- 172.21.15.27:0/4162124579 wait complete. 2024-06-06T08:41:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fede79b80 20 client.15801 trim_cache size 0 max 16384 2024-06-06T08:41:44.143 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:41:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 -- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1fc80a2f40 msgr2=0x7f1fc80a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1fc80a2f40 0x7f1fc80a5400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8007be0 tx=0x7f1fd80073d0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 -- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 msgr2=0x7f1fdc19b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 0x7f1fdc19b470 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0038b0 tx=0x7f1fcc009f50 comp rx=0 tx=0).stop 2024-06-06T08:41:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 -- 172.21.15.27:0/1841456564 shutdown_connections 2024-06-06T08:41:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1fc80a2f40 0x7f1fc80a5400 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fdc10ef70 0x7f1fdc19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fdc06abe0 0x7f1fdc19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 --2- 172.21.15.27:0/1841456564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fdc06b500 0x7f1fdc19b470 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 -- 172.21.15.27:0/1841456564 >> 172.21.15.27:0/1841456564 conn(0x7f1fdc078360 msgr2=0x7f1fdc110460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 -- 172.21.15.27:0/1841456564 shutdown_connections 2024-06-06T08:41:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:44.140+0000 7f1fe35f6700 1 -- 172.21.15.27:0/1841456564 wait complete. 2024-06-06T08:41:44.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:44 smithi027 ceph-mon[32335]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:41:44.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:44 smithi203 ceph-mon[36579]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:41:44.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:44 smithi176 ceph-mon[36786]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T08:41:45.961 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T08:41:45.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T08:41:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:46.610 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:46 smithi027 ceph-mon[32335]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T08:41:46.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:46 smithi203 ceph-mon[36579]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T08:41:46.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:46 smithi176 ceph-mon[36786]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T08:41:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.015+0000 7ff67a679700 1 -- 172.21.15.27:0/3529727244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67406b830 msgr2=0x7ff674107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.015+0000 7ff67a679700 1 --2- 172.21.15.27:0/3529727244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67406b830 0x7ff674107120 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff6680099c0 tx=0x7ff668009cd0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.015+0000 7ff67a679700 1 -- 172.21.15.27:0/3529727244 shutdown_connections 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.015+0000 7ff67a679700 1 --2- 172.21.15.27:0/3529727244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff674107850 0x7ff67410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.015+0000 7ff67a679700 1 --2- 172.21.15.27:0/3529727244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67406ae80 0x7ff67406b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.015+0000 7ff67a679700 1 --2- 172.21.15.27:0/3529727244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff67406b830 0x7ff674107120 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.015+0000 7ff67a679700 1 -- 172.21.15.27:0/3529727244 >> 172.21.15.27:0/3529727244 conn(0x7ff674078400 msgr2=0x7ff674078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.016+0000 7ff67a679700 1 -- 172.21.15.27:0/3529727244 shutdown_connections 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.016+0000 7ff67a679700 1 -- 172.21.15.27:0/3529727244 wait complete. 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.016+0000 7ff67a679700 1 Processor -- start 2024-06-06T08:41:48.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.016+0000 7ff67a679700 1 -- start start 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff67a679700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67406ae80 0x7ff674101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff67a679700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff67406b830 0x7ff674101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff67a679700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff674107850 0x7ff674101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff67a679700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741127e0 con 0x7ff674107850 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff67a679700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff674112660 con 0x7ff67406ae80 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff67a679700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff674112960 con 0x7ff67406b830 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff678c16700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff674107850 0x7ff674101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff673fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67406ae80 0x7ff674101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff678c16700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff674107850 0x7ff674101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56592/0 (socket says 172.21.15.27:56592) 2024-06-06T08:41:48.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff678c16700 1 -- 172.21.15.27:0/325447117 learned_addr learned my addr 172.21.15.27:0/325447117 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff6737fe700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff67406b830 0x7ff674101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff678c16700 1 -- 172.21.15.27:0/325447117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff67406b830 msgr2=0x7ff674101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff678c16700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff67406b830 0x7ff674101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff678c16700 1 -- 172.21.15.27:0/325447117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67406ae80 msgr2=0x7ff674101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff678c16700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67406ae80 0x7ff674101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff678c16700 1 -- 172.21.15.27:0/325447117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6741a70a0 con 0x7ff674107850 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff6737fe700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff67406b830 0x7ff674101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.017+0000 7ff673fff700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67406ae80 0x7ff674101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.018+0000 7ff678c16700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff674107850 0x7ff674101e80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00b970 tx=0x7ff66c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.018+0000 7ff6717fa700 1 -- 172.21.15.27:0/325447117 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c019070 con 0x7ff674107850 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.018+0000 7ff67a679700 1 -- 172.21.15.27:0/325447117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6741a7390 con 0x7ff674107850 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.018+0000 7ff6717fa700 1 -- 172.21.15.27:0/325447117 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff66c00ce10 con 0x7ff674107850 2024-06-06T08:41:48.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.018+0000 7ff6717fa700 1 -- 172.21.15.27:0/325447117 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c007370 con 0x7ff674107850 2024-06-06T08:41:48.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.018+0000 7ff67a679700 1 -- 172.21.15.27:0/325447117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6741a7850 con 0x7ff674107850 2024-06-06T08:41:48.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.020+0000 7ff6717fa700 1 -- 172.21.15.27:0/325447117 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7ff66c01c070 con 0x7ff674107850 2024-06-06T08:41:48.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.021+0000 7ff6717fa700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff65c0a2f40 0x7ff65c0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.021+0000 7ff673fff700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff65c0a2f40 0x7ff65c0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.021+0000 7ff6717fa700 1 -- 172.21.15.27:0/325447117 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7ff66c0cedb0 con 0x7ff674107850 2024-06-06T08:41:48.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff673fff700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff65c0a2f40 0x7ff65c0a5400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff66400afd0 tx=0x7ff66400a380 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:48.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff684efcb80 1 Processor -- start 2024-06-06T08:41:48.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff684efcb80 1 -- start start 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 0x565471c663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c60c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff684efcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565471b52fe0 con 0x565471c60820 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff684efcb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x565471bb18d0 con 0x565471c66900 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff684efcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x565471b194e0 con 0x565471c65fe0 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff678c16700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c60c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff678c16700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c60c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56604/0 (socket says 172.21.15.27:56604) 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.022+0000 7ff678c16700 1 -- 172.21.15.27:0/297553102 learned_addr learned my addr 172.21.15.27:0/297553102 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff673fff700 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 0x565471c663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff6737fe700 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff678c16700 1 -- 172.21.15.27:0/297553102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 msgr2=0x565471c663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff678c16700 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 0x565471c663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff678c16700 1 -- 172.21.15.27:0/297553102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 msgr2=0x565471c602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff678c16700 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff678c16700 1 -- 172.21.15.27:0/297553102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565471b00950 con 0x565471c60820 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff6737fe700 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff678c16700 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c60c40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff66c09aae0 tx=0x7ff66c09b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1a007657c17751 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff65a7fc700 1 -- 172.21.15.27:0/297553102 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0a3a50 con 0x565471c60820 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff65a7fc700 1 -- 172.21.15.27:0/297553102 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff66c0990e0 con 0x565471c60820 2024-06-06T08:41:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.023+0000 7ff65a7fc700 1 -- 172.21.15.27:0/297553102 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c09a490 con 0x565471c60820 2024-06-06T08:41:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 -- 172.21.15.27:0/297553102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 msgr2=0x565471c60c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c60c40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff66c09aae0 tx=0x7ff66c09b4e0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 -- 172.21.15.27:0/297553102 shutdown_connections 2024-06-06T08:41:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c60c40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 0x565471c663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 --2- 172.21.15.27:0/297553102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 -- 172.21.15.27:0/297553102 >> 172.21.15.27:0/297553102 conn(0x565471b8d100 msgr2=0x565471c5b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 -- 172.21.15.27:0/297553102 shutdown_connections 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 -- 172.21.15.27:0/297553102 wait complete. 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.024+0000 7ff684efcb80 1 -- start start 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff684efcb80 1 Processor -- start 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 0x565471c6c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c69f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff684efcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565471b52040 con 0x565471c65fe0 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff684efcb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x565471bad890 con 0x565471c66900 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff684efcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x565471c6d2e0 con 0x565471c60820 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff684efcb80 10 client.0 _ll_register_callbacks cb 0x565471b47630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff6737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff6737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56616/0 (socket says 172.21.15.27:56616) 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff6737fe700 1 -- 172.21.15.27:0/4237435469 learned_addr learned my addr 172.21.15.27:0/4237435469 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff673fff700 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 0x565471c6c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff678c16700 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c69f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff6737fe700 1 -- 172.21.15.27:0/4237435469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 msgr2=0x565471c6c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff6737fe700 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 0x565471c6c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.025+0000 7ff6737fe700 1 -- 172.21.15.27:0/4237435469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 msgr2=0x565471c69f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff6737fe700 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c69f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff6737fe700 1 -- 172.21.15.27:0/4237435469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565471b777d0 con 0x565471c65fe0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff6737fe700 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c699d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff66800b580 tx=0x7ff668009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7924a824c93c34 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff6537fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff6537fe700 1 -- 172.21.15.27:0/4237435469 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6680054d0 con 0x565471c65fe0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff684efcb80 1 -- 172.21.15.27:0/4237435469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff66c0990e0 con 0x565471c65fe0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff6537fe700 1 -- 172.21.15.27:0/4237435469 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7ff668005670 con 0x565471c65fe0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff6537fe700 1 -- 172.21.15.27:0/4237435469 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff668030050 con 0x565471c65fe0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff684efcb80 10 client.15813 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff684efcb80 1 -- 172.21.15.27:0/4237435469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565471c77b30 con 0x565471c65fe0 2024-06-06T08:41:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.026+0000 7ff6537fe700 1 -- 172.21.15.27:0/4237435469 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff66c0990e0 con 0x565471c65fe0 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff6537fe700 1 -- 172.21.15.27:0/4237435469 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff66800f980 con 0x565471c65fe0 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff684efcb80 10 client.15813 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff684efcb80 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff684efcb80 20 client.15813 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff684efcb80 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x565471c7a110 0x565471c7c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff684efcb80 4 client.15813 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff684efcb80 1 -- 172.21.15.27:0/4237435469 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565471bbedd0 con 0x565471c7a110 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff673fff700 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x565471c7a110 0x565471c7c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff673fff700 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x565471c7a110 0x565471c7c500 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1f382326c1d837 server_cookie=e215cfba039df3cf in_seq=0 out_seq=0 2024-06-06T08:41:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.027+0000 7ff6537fe700 10 client.15813 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff6537fe700 1 -- 172.21.15.27:0/4237435469 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565471bbedd0 con 0x565471c7a110 2024-06-06T08:41:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff6537fe700 10 client.15813 handle_command_reply: tid=0 2024-06-06T08:41:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff684efcb80 1 client.15813 shutdown 2024-06-06T08:41:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff684efcb80 1 -- 172.21.15.27:0/4237435469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 msgr2=0x565471c699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff684efcb80 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c699d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff66800b580 tx=0x7ff668009610 comp rx=0 tx=0).stop 2024-06-06T08:41:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff684efcb80 1 -- 172.21.15.27:0/4237435469 shutdown_connections 2024-06-06T08:41:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff684efcb80 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c69f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff684efcb80 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 0x565471c6c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff684efcb80 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x565471c7a110 0x565471c7c500 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff684efcb80 1 --2- 172.21.15.27:0/4237435469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c699d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff6537fe700 0 client.15813 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.028+0000 7ff684efcb80 1 -- 172.21.15.27:0/4237435469 >> 172.21.15.27:0/4237435469 conn(0x565471b8d100 msgr2=0x565471c5a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.029+0000 7ff684efcb80 1 -- 172.21.15.27:0/4237435469 shutdown_connections 2024-06-06T08:41:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.029+0000 7ff684efcb80 1 -- 172.21.15.27:0/4237435469 wait complete. 2024-06-06T08:41:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.029+0000 7ff684efcb80 20 client.15813 trim_cache size 0 max 16384 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.049+0000 7ff684efcb80 1 Processor -- start 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.049+0000 7ff684efcb80 1 -- start start 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 0x565471c640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c67570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff684efcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741b2fd0 con 0x565471c60820 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff684efcb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741b0860 con 0x565471c66900 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff684efcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741ae6b0 con 0x565471c65fe0 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff673fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff673fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56624/0 (socket says 172.21.15.27:56624) 2024-06-06T08:41:48.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff673fff700 1 -- 172.21.15.27:0/3843059765 learned_addr learned my addr 172.21.15.27:0/3843059765 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff6737fe700 1 --2- 172.21.15.27:0/3843059765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 0x565471c640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff678c16700 1 --2- 172.21.15.27:0/3843059765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c67570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff673fff700 1 -- 172.21.15.27:0/3843059765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 msgr2=0x565471c640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff673fff700 1 --2- 172.21.15.27:0/3843059765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 0x565471c640d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff673fff700 1 -- 172.21.15.27:0/3843059765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 msgr2=0x565471c67570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff673fff700 1 --2- 172.21.15.27:0/3843059765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c67570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff673fff700 1 -- 172.21.15.27:0/3843059765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565471c67f60 con 0x565471c60820 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff673fff700 1 --2- 172.21.15.27:0/3843059765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c602e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff66400fb30 tx=0x7ff664012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80514f69b3264029 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.050+0000 7ff659ffb700 1 -- 172.21.15.27:0/3843059765 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664016070 con 0x565471c60820 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff659ffb700 1 -- 172.21.15.27:0/3843059765 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff664004500 con 0x565471c60820 2024-06-06T08:41:48.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff659ffb700 1 -- 172.21.15.27:0/3843059765 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664026340 con 0x565471c60820 2024-06-06T08:41:48.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff684efcb80 1 -- 172.21.15.27:0/3843059765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 msgr2=0x565471c602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:48.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3843059765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c602e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff66400fb30 tx=0x7ff664012000 comp rx=0 tx=0).stop 2024-06-06T08:41:48.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff684efcb80 1 -- 172.21.15.27:0/3843059765 shutdown_connections 2024-06-06T08:41:48.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3843059765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c67570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3843059765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c60820 0x565471c602e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3843059765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c65fe0 0x565471c640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff684efcb80 1 -- 172.21.15.27:0/3843059765 >> 172.21.15.27:0/3843059765 conn(0x565471b8d100 msgr2=0x565471c6b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff684efcb80 1 -- 172.21.15.27:0/3843059765 shutdown_connections 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.051+0000 7ff684efcb80 1 -- 172.21.15.27:0/3843059765 wait complete. 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff684efcb80 1 -- start start 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff684efcb80 1 Processor -- start 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 0x565471c72570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c72ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff684efcb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c6f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff684efcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741aed20 con 0x565471c65fe0 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff684efcb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741b0190 con 0x565471c66900 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff684efcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741af3f0 con 0x565471c60820 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff684efcb80 10 client.0 _ll_register_callbacks cb 0x565471b47630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:48.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff6737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c72ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff6737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c72ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56638/0 (socket says 172.21.15.27:56638) 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff6737fe700 1 -- 172.21.15.27:0/3177287087 learned_addr learned my addr 172.21.15.27:0/3177287087 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff673fff700 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 0x565471c72570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.052+0000 7ff678c16700 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c6f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6737fe700 1 -- 172.21.15.27:0/3177287087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 msgr2=0x565471c72570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6737fe700 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 0x565471c72570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6737fe700 1 -- 172.21.15.27:0/3177287087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 msgr2=0x565471c6f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6737fe700 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c6f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6737fe700 1 -- 172.21.15.27:0/3177287087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565471c6fb30 con 0x565471c65fe0 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff678c16700 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c6f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6737fe700 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c72ab0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff668005090 tx=0x7ff6680226c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b9e4180775e787 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6537fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6537fe700 1 -- 172.21.15.27:0/3177287087 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668003e60 con 0x565471c65fe0 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6537fe700 1 -- 172.21.15.27:0/3177287087 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7ff668022870 con 0x565471c65fe0 2024-06-06T08:41:48.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff684efcb80 1 -- 172.21.15.27:0/3177287087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff664004500 con 0x565471c65fe0 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6537fe700 1 -- 172.21.15.27:0/3177287087 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6680056e0 con 0x565471c65fe0 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff684efcb80 10 client.15819 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff684efcb80 1 -- 172.21.15.27:0/3177287087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565471ce5640 con 0x565471c65fe0 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6537fe700 1 -- 172.21.15.27:0/3177287087 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff664004500 con 0x565471c65fe0 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff6537fe700 1 -- 172.21.15.27:0/3177287087 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff66800b730 con 0x565471c65fe0 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.053+0000 7ff684efcb80 10 client.15819 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.054+0000 7ff684efcb80 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.054+0000 7ff684efcb80 20 client.15819 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.054+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x565471ce5cb0 0x565471ce80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.054+0000 7ff684efcb80 4 client.15819 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.054+0000 7ff684efcb80 1 -- 172.21.15.27:0/3177287087 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565471bb9350 con 0x565471ce5cb0 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.054+0000 7ff673fff700 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x565471ce5cb0 0x565471ce80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.054+0000 7ff673fff700 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x565471ce5cb0 0x565471ce80d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f54b499bee71e8d7 server_cookie=8cb0b9f71a44547 in_seq=0 out_seq=0 2024-06-06T08:41:48.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.054+0000 7ff6537fe700 10 client.15819 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.055+0000 7ff6537fe700 1 -- 172.21.15.27:0/3177287087 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565471bb9350 con 0x565471ce5cb0 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.055+0000 7ff6537fe700 10 client.15819 handle_command_reply: tid=0 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.055+0000 7ff684efcb80 1 client.15819 shutdown 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 -- 172.21.15.27:0/3177287087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 msgr2=0x565471c72ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c72ab0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff668005090 tx=0x7ff6680226c0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 -- 172.21.15.27:0/3177287087 shutdown_connections 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565471c66900 0x565471c6f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565471c60820 0x565471c72570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x565471ce5cb0 0x565471ce80d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 --2- 172.21.15.27:0/3177287087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565471c65fe0 0x565471c72ab0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 -- 172.21.15.27:0/3177287087 >> 172.21.15.27:0/3177287087 conn(0x565471b8d100 msgr2=0x565471c6b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff6537fe700 0 client.15819 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 -- 172.21.15.27:0/3177287087 shutdown_connections 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 1 -- 172.21.15.27:0/3177287087 wait complete. 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.056+0000 7ff684efcb80 20 client.15819 trim_cache size 0 max 16384 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "85b95b7f-f310-4736-9630-7b512920aa0e", 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-06-06T08:41:48.059 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:41:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 -- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff65c0a2f40 msgr2=0x7ff65c0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff65c0a2f40 0x7ff65c0a5400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff66400afd0 tx=0x7ff66400a380 comp rx=0 tx=0).stop 2024-06-06T08:41:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 -- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff674107850 msgr2=0x7ff674101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff674107850 0x7ff674101e80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00b970 tx=0x7ff66c00bc80 comp rx=0 tx=0).stop 2024-06-06T08:41:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 -- 172.21.15.27:0/325447117 shutdown_connections 2024-06-06T08:41:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff674107850 0x7ff674101e80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff65c0a2f40 0x7ff65c0a5400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67406ae80 0x7ff674101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 --2- 172.21.15.27:0/325447117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff67406b830 0x7ff674101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 -- 172.21.15.27:0/325447117 >> 172.21.15.27:0/325447117 conn(0x7ff674078400 msgr2=0x7ff6740ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 -- 172.21.15.27:0/325447117 shutdown_connections 2024-06-06T08:41:48.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:48.057+0000 7ff67a679700 1 -- 172.21.15.27:0/325447117 wait complete. 2024-06-06T08:41:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:48 smithi027 ceph-mon[32335]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T08:41:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:48 smithi027 ceph-mon[32335]: scrub queued for path: / 2024-06-06T08:41:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:48 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:48 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:41:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:48 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:48.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:48 smithi203 ceph-mon[36579]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T08:41:48.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:48 smithi203 ceph-mon[36579]: scrub queued for path: / 2024-06-06T08:41:48.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:48 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:48.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:48 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:41:48.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:48 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:48.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:48 smithi176 ceph-mon[36786]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T08:41:48.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:48 smithi176 ceph-mon[36786]: scrub queued for path: / 2024-06-06T08:41:48.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:48 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:48.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:48 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:41:48.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:48 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:41:48.797 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T08:41:49.050 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:50.610 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:50 smithi027 ceph-mon[32335]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-06T08:41:50.610 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:50 smithi027 ceph-mon[32335]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-06-06T08:41:50.610 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:50 smithi027 ceph-mon[32335]: scrub summary: idle 2024-06-06T08:41:50.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:50 smithi203 ceph-mon[36579]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-06T08:41:50.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:50 smithi203 ceph-mon[36579]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-06-06T08:41:50.742 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:50 smithi203 ceph-mon[36579]: scrub summary: idle 2024-06-06T08:41:50.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:50 smithi176 ceph-mon[36786]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-06T08:41:50.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:50 smithi176 ceph-mon[36786]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-06-06T08:41:50.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:50 smithi176 ceph-mon[36786]: scrub summary: idle 2024-06-06T08:41:50.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.855+0000 7f9d89eb2700 1 -- 172.21.15.27:0/4163902407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84101380 msgr2=0x7f9d84101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:50.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.855+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/4163902407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84101380 0x7f9d84101800 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009990 tx=0x7f9d6c009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.855+0000 7f9d89eb2700 1 -- 172.21.15.27:0/4163902407 shutdown_connections 2024-06-06T08:41:50.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.855+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/4163902407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d84101d40 0x7f9d8410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.855+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/4163902407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d841073f0 0x7f9d841077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.855+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/4163902407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84101380 0x7f9d84101800 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.855+0000 7f9d89eb2700 1 -- 172.21.15.27:0/4163902407 >> 172.21.15.27:0/4163902407 conn(0x7f9d84078300 msgr2=0x7f9d84078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:50.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.855+0000 7f9d89eb2700 1 -- 172.21.15.27:0/4163902407 shutdown_connections 2024-06-06T08:41:50.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.855+0000 7f9d89eb2700 1 -- 172.21.15.27:0/4163902407 wait complete. 2024-06-06T08:41:50.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.856+0000 7f9d89eb2700 1 Processor -- start 2024-06-06T08:41:50.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.856+0000 7f9d89eb2700 1 -- start start 2024-06-06T08:41:50.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.856+0000 7f9d89eb2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d84101380 0x7f9d84199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.856+0000 7f9d89eb2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d84101d40 0x7f9d84199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.856+0000 7f9d89eb2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d841073f0 0x7f9d841a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d89eb2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84111690 con 0x7f9d841073f0 2024-06-06T08:41:50.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d89eb2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84111510 con 0x7f9d84101d40 2024-06-06T08:41:50.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d89eb2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84111810 con 0x7f9d84101380 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d841073f0 0x7f9d841a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d841073f0 0x7f9d841a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42638/0 (socket says 172.21.15.27:42638) 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d83fff700 1 -- 172.21.15.27:0/1359293074 learned_addr learned my addr 172.21.15.27:0/1359293074 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d7affd700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d84101d40 0x7f9d84199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d837fe700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d84101380 0x7f9d84199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d83fff700 1 -- 172.21.15.27:0/1359293074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d84101380 msgr2=0x7f9d84199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d83fff700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d84101380 0x7f9d84199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d83fff700 1 -- 172.21.15.27:0/1359293074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d84101d40 msgr2=0x7f9d84199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d83fff700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d84101d40 0x7f9d84199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d83fff700 1 -- 172.21.15.27:0/1359293074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d841a5010 con 0x7f9d841073f0 2024-06-06T08:41:50.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d83fff700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d841073f0 0x7f9d841a4ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c007ee0 tx=0x7f9d7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:50.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.857+0000 7f9d817fa700 1 -- 172.21.15.27:0/1359293074 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c004bb0 con 0x7f9d841073f0 2024-06-06T08:41:50.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.858+0000 7f9d817fa700 1 -- 172.21.15.27:0/1359293074 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c004d50 con 0x7f9d841073f0 2024-06-06T08:41:50.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.858+0000 7f9d89eb2700 1 -- 172.21.15.27:0/1359293074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d841a5240 con 0x7f9d841073f0 2024-06-06T08:41:50.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.858+0000 7f9d817fa700 1 -- 172.21.15.27:0/1359293074 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c005960 con 0x7f9d841073f0 2024-06-06T08:41:50.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.858+0000 7f9d89eb2700 1 -- 172.21.15.27:0/1359293074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d841a5700 con 0x7f9d841073f0 2024-06-06T08:41:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.860+0000 7f9d817fa700 1 -- 172.21.15.27:0/1359293074 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9d7c01a020 con 0x7f9d841073f0 2024-06-06T08:41:50.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.861+0000 7f9d817fa700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9d700a2d90 0x7f9d700a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.862+0000 7f9d837fe700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9d700a2d90 0x7f9d700a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.862+0000 7f9d817fa700 1 -- 172.21.15.27:0/1359293074 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9d7c0cefa0 con 0x7f9d841073f0 2024-06-06T08:41:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.862+0000 7f9d94735b80 1 Processor -- start 2024-06-06T08:41:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.862+0000 7f9d94735b80 1 -- start start 2024-06-06T08:41:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.862+0000 7f9d837fe700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9d700a2d90 0x7f9d700a5250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005950 tx=0x7f9d7400b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.862+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc775340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc76f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc76fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d94735b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555ecc661f60 con 0x555ecc76f7a0 2024-06-06T08:41:50.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d94735b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555ecc6c0850 con 0x555ecc774f60 2024-06-06T08:41:50.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d94735b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555ecc628460 con 0x555ecc775880 2024-06-06T08:41:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc76fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d7affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc76f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d7affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc76f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:43400/0 (socket says 172.21.15.27:43400) 2024-06-06T08:41:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d7affd700 1 -- 172.21.15.27:0/4174487525 learned_addr learned my addr 172.21.15.27:0/4174487525 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d837fe700 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc775340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d83fff700 1 -- 172.21.15.27:0/4174487525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 msgr2=0x555ecc76f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d83fff700 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc76f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d83fff700 1 -- 172.21.15.27:0/4174487525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 msgr2=0x555ecc775340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:50.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d83fff700 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc775340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d83fff700 1 -- 172.21.15.27:0/4174487525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecc60f8d0 con 0x555ecc76f7a0 2024-06-06T08:41:50.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d7affd700 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc76f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:50.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d837fe700 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc775340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:50.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.863+0000 7f9d83fff700 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc76fbc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0cc330 tx=0x7f9d7c09acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b05a9e175923f86 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:50.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.864+0000 7f9d79ffb700 1 -- 172.21.15.27:0/4174487525 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c09b100 con 0x555ecc76f7a0 2024-06-06T08:41:50.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.864+0000 7f9d79ffb700 1 -- 172.21.15.27:0/4174487525 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c09f4b0 con 0x555ecc76f7a0 2024-06-06T08:41:50.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.864+0000 7f9d79ffb700 1 -- 172.21.15.27:0/4174487525 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0192c0 con 0x555ecc76f7a0 2024-06-06T08:41:50.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 -- 172.21.15.27:0/4174487525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 msgr2=0x555ecc76fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc76fbc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0cc330 tx=0x7f9d7c09acc0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 -- 172.21.15.27:0/4174487525 shutdown_connections 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc76fbc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc775340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 --2- 172.21.15.27:0/4174487525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc76f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 -- 172.21.15.27:0/4174487525 >> 172.21.15.27:0/4174487525 conn(0x555ecc69c080 msgr2=0x555ecc76a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 -- 172.21.15.27:0/4174487525 shutdown_connections 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 -- 172.21.15.27:0/4174487525 wait complete. 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.865+0000 7f9d94735b80 1 -- start start 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d94735b80 1 Processor -- start 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc76f7a0 0x555ecc77b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc774f60 0x555ecc778910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc778e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d94735b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555ecc660fc0 con 0x555ecc774f60 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d94735b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x555ecc6bc810 con 0x555ecc76f7a0 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d94735b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555ecc77c250 con 0x555ecc775880 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d94735b80 10 client.0 _ll_register_callbacks cb 0x555ecc6565b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d7affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc774f60 0x555ecc778910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d7affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc774f60 0x555ecc778910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42650/0 (socket says 172.21.15.27:42650) 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.866+0000 7f9d7affd700 1 -- 172.21.15.27:0/2276237877 learned_addr learned my addr 172.21.15.27:0/2276237877 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d837fe700 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc76f7a0 0x555ecc77b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d83fff700 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc778e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d7affd700 1 -- 172.21.15.27:0/2276237877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 msgr2=0x555ecc778e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d7affd700 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc778e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d7affd700 1 -- 172.21.15.27:0/2276237877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc76f7a0 msgr2=0x555ecc77b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d7affd700 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc76f7a0 0x555ecc77b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d7affd700 1 -- 172.21.15.27:0/2276237877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecc686750 con 0x555ecc774f60 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d83fff700 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc778e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d837fe700 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc76f7a0 0x555ecc77b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d7affd700 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc774f60 0x555ecc778910 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c005cb0 tx=0x7f9d6c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73abff547398aba server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d62ffd700 1 -- 172.21.15.27:0/2276237877 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c02e060 con 0x555ecc774f60 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d62ffd700 1 -- 172.21.15.27:0/2276237877 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9d6c0054d0 con 0x555ecc774f60 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d94735b80 1 -- 172.21.15.27:0/2276237877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c09f4b0 con 0x555ecc774f60 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d62ffd700 1 -- 172.21.15.27:0/2276237877 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d6c035050 con 0x555ecc774f60 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d94735b80 10 client.15831 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.867+0000 7f9d94735b80 1 -- 172.21.15.27:0/2276237877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ecc786a60 con 0x555ecc774f60 2024-06-06T08:41:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.868+0000 7f9d62ffd700 1 -- 172.21.15.27:0/2276237877 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c09f4b0 con 0x555ecc774f60 2024-06-06T08:41:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.868+0000 7f9d62ffd700 1 -- 172.21.15.27:0/2276237877 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d6c028070 con 0x555ecc774f60 2024-06-06T08:41:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.868+0000 7f9d94735b80 10 client.15831 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.868+0000 7f9d94735b80 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.868+0000 7f9d94735b80 20 client.15831 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.868+0000 7f9d94735b80 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x555ecc7890a0 0x555ecc78b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.868+0000 7f9d94735b80 4 client.15831 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.868+0000 7f9d94735b80 1 -- 172.21.15.27:0/2276237877 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ecc6cdd50 con 0x555ecc7890a0 2024-06-06T08:41:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.868+0000 7f9d837fe700 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x555ecc7890a0 0x555ecc78b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.869+0000 7f9d62ffd700 10 client.15831 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.869+0000 7f9d837fe700 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x555ecc7890a0 0x555ecc78b490 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d7706e4cd2284c server_cookie=35f141810c42aceb in_seq=0 out_seq=0 2024-06-06T08:41:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d62ffd700 1 -- 172.21.15.27:0/2276237877 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ecc6cdd50 con 0x555ecc7890a0 2024-06-06T08:41:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d62ffd700 10 client.15831 handle_command_reply: tid=0 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 client.15831 shutdown 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 -- 172.21.15.27:0/2276237877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc774f60 msgr2=0x555ecc778910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc774f60 0x555ecc778910 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c005cb0 tx=0x7f9d6c009f90 comp rx=0 tx=0).stop 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 -- 172.21.15.27:0/2276237877 shutdown_connections 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc778e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc76f7a0 0x555ecc77b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x555ecc7890a0 0x555ecc78b490 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 --2- 172.21.15.27:0/2276237877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc774f60 0x555ecc778910 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d62ffd700 0 client.15831 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 -- 172.21.15.27:0/2276237877 >> 172.21.15.27:0/2276237877 conn(0x555ecc69c080 msgr2=0x555ecc769700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 -- 172.21.15.27:0/2276237877 shutdown_connections 2024-06-06T08:41:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 1 -- 172.21.15.27:0/2276237877 wait complete. 2024-06-06T08:41:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.870+0000 7f9d94735b80 20 client.15831 trim_cache size 0 max 16384 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.905+0000 7f9d94735b80 1 Processor -- start 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.905+0000 7f9d94735b80 1 -- start start 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.905+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc76f7a0 0x555ecc76f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.905+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc776370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.905+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc775880 0x555ecc769e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.905+0000 7f9d94735b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841aec30 con 0x555ecc775880 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.905+0000 7f9d94735b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841ac4c0 con 0x555ecc774f60 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.905+0000 7f9d94735b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841aa310 con 0x555ecc76f7a0 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc775880 0x555ecc769e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d837fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc76f7a0 0x555ecc76f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc775880 0x555ecc769e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42662/0 (socket says 172.21.15.27:42662) 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d83fff700 1 -- 172.21.15.27:0/1487906343 learned_addr learned my addr 172.21.15.27:0/1487906343 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d7affd700 1 --2- 172.21.15.27:0/1487906343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc776370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d837fe700 1 -- 172.21.15.27:0/1487906343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 msgr2=0x555ecc776370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d837fe700 1 --2- 172.21.15.27:0/1487906343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc776370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d837fe700 1 -- 172.21.15.27:0/1487906343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc775880 msgr2=0x555ecc769e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d837fe700 1 --2- 172.21.15.27:0/1487906343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc775880 0x555ecc769e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d837fe700 1 -- 172.21.15.27:0/1487906343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecc78b7d0 con 0x555ecc76f7a0 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d837fe700 1 --2- 172.21.15.27:0/1487906343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc76f7a0 0x555ecc76f260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9d74029fd0 tx=0x7f9d74013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99ecbdc0238ac494 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.906+0000 7f9d797fa700 1 -- 172.21.15.27:0/1487906343 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74020dc0 con 0x555ecc76f7a0 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.907+0000 7f9d797fa700 1 -- 172.21.15.27:0/1487906343 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d74004d10 con 0x555ecc76f7a0 2024-06-06T08:41:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.907+0000 7f9d797fa700 1 -- 172.21.15.27:0/1487906343 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d740293a0 con 0x555ecc76f7a0 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.907+0000 7f9d94735b80 1 -- 172.21.15.27:0/1487906343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc76f7a0 msgr2=0x555ecc76f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.907+0000 7f9d94735b80 1 --2- 172.21.15.27:0/1487906343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc76f7a0 0x555ecc76f260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9d74029fd0 tx=0x7f9d74013000 comp rx=0 tx=0).stop 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.908+0000 7f9d94735b80 1 -- 172.21.15.27:0/1487906343 shutdown_connections 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.908+0000 7f9d94735b80 1 --2- 172.21.15.27:0/1487906343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc775880 0x555ecc769e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.908+0000 7f9d94735b80 1 --2- 172.21.15.27:0/1487906343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc76f7a0 0x555ecc76f260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.908+0000 7f9d94735b80 1 --2- 172.21.15.27:0/1487906343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc776370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.908+0000 7f9d94735b80 1 -- 172.21.15.27:0/1487906343 >> 172.21.15.27:0/1487906343 conn(0x555ecc69c080 msgr2=0x555ecc775c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.908+0000 7f9d94735b80 1 -- 172.21.15.27:0/1487906343 shutdown_connections 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.908+0000 7f9d94735b80 1 -- 172.21.15.27:0/1487906343 wait complete. 2024-06-06T08:41:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.908+0000 7f9d94735b80 1 -- start start 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.908+0000 7f9d94735b80 1 Processor -- start 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc781ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d94735b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc7904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d837fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc781ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d837fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc781ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42672/0 (socket says 172.21.15.27:42672) 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d837fe700 1 -- 172.21.15.27:0/857035800 learned_addr learned my addr 172.21.15.27:0/857035800 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d94735b80 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc790a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841aa980 con 0x555ecc76f7a0 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841abdf0 con 0x555ecc774f60 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841ab050 con 0x555ecc775880 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d94735b80 10 client.0 _ll_register_callbacks cb 0x555ecc6565b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d837fe700 1 -- 172.21.15.27:0/857035800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 msgr2=0x555ecc790a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d837fe700 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc790a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d837fe700 1 -- 172.21.15.27:0/857035800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 msgr2=0x555ecc7904f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.909+0000 7f9d7affd700 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc7904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d837fe700 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc7904f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d837fe700 1 -- 172.21.15.27:0/857035800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecc791110 con 0x555ecc76f7a0 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d7affd700 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc7904f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d837fe700 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc781ba0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9d74010c60 tx=0x7f9d74020c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed930ce8ec47752 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d62ffd700 1 -- 172.21.15.27:0/857035800 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74004b60 con 0x555ecc76f7a0 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d62ffd700 1 -- 172.21.15.27:0/857035800 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9d74005000 con 0x555ecc76f7a0 2024-06-06T08:41:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d74004d10 con 0x555ecc76f7a0 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d62ffd700 1 -- 172.21.15.27:0/857035800 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d74041050 con 0x555ecc76f7a0 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d94735b80 10 client.15837 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.910+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ecc7fc370 con 0x555ecc76f7a0 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.911+0000 7f9d62ffd700 1 -- 172.21.15.27:0/857035800 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d74038070 con 0x555ecc76f7a0 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.911+0000 7f9d62ffd700 1 -- 172.21.15.27:0/857035800 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d74029b00 con 0x555ecc76f7a0 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.911+0000 7f9d94735b80 10 client.15837 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.911+0000 7f9d94735b80 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.911+0000 7f9d94735b80 20 client.15837 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.911+0000 7f9d94735b80 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x555ecc7fca10 0x555ecc7fee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.911+0000 7f9d94735b80 4 client.15837 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.911+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9d74004d10 con 0x555ecc7fca10 2024-06-06T08:41:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.911+0000 7f9d7affd700 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x555ecc7fca10 0x555ecc7fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.912+0000 7f9d62ffd700 10 client.15837 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.912+0000 7f9d7affd700 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x555ecc7fca10 0x555ecc7fee30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87b71ee7effcb75 server_cookie=8e34381253cc0a42 in_seq=0 out_seq=0 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.912+0000 7f9d62ffd700 1 -- 172.21.15.27:0/857035800 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f9d74004d10 con 0x555ecc7fca10 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.912+0000 7f9d62ffd700 10 client.15837 handle_command_reply: tid=0 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.912+0000 7f9d94735b80 1 client.15837 shutdown 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.912+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 msgr2=0x555ecc781ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.912+0000 7f9d94735b80 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc781ba0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9d74010c60 tx=0x7f9d74020c70 comp rx=0 tx=0).stop 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 shutdown_connections 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d94735b80 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ecc775880 0x555ecc790a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d94735b80 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x555ecc7fca10 0x555ecc7fee30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d94735b80 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ecc76f7a0 0x555ecc781ba0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d62ffd700 0 client.15837 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d94735b80 1 --2- 172.21.15.27:0/857035800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555ecc774f60 0x555ecc7904f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 >> 172.21.15.27:0/857035800 conn(0x555ecc69c080 msgr2=0x555ecc7764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 shutdown_connections 2024-06-06T08:41:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d94735b80 1 -- 172.21.15.27:0/857035800 wait complete. 2024-06-06T08:41:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.913+0000 7f9d94735b80 20 client.15837 trim_cache size 0 max 16384 2024-06-06T08:41:50.918 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:41:50.918 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-06-06T08:41:50.918 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {} 2024-06-06T08:41:50.919 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:41:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 -- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9d700a2d90 msgr2=0x7f9d700a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9d700a2d90 0x7f9d700a5250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005950 tx=0x7f9d7400b410 comp rx=0 tx=0).stop 2024-06-06T08:41:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 -- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d841073f0 msgr2=0x7f9d841a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d841073f0 0x7f9d841a4ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c007ee0 tx=0x7f9d7c00d3b0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 -- 172.21.15.27:0/1359293074 shutdown_connections 2024-06-06T08:41:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d841073f0 0x7f9d841a4ad0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d84101380 0x7f9d84199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9d700a2d90 0x7f9d700a5250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 --2- 172.21.15.27:0/1359293074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d84101d40 0x7f9d84199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.914+0000 7f9d89eb2700 1 -- 172.21.15.27:0/1359293074 >> 172.21.15.27:0/1359293074 conn(0x7f9d84078300 msgr2=0x7f9d840ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.915+0000 7f9d89eb2700 1 -- 172.21.15.27:0/1359293074 shutdown_connections 2024-06-06T08:41:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:50.915+0000 7f9d89eb2700 1 -- 172.21.15.27:0/1359293074 wait complete. 2024-06-06T08:41:51.731 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T08:41:51.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:41:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.923+0000 7f83ac05d640 1 -- 172.21.15.27:0/3114124245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a41079c0 msgr2=0x7f83a4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.923+0000 7f83ac05d640 1 --2- 172.21.15.27:0/3114124245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a41079c0 0x7f83a4107da0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009930 tx=0x7f83a002a420 comp rx=0 tx=0).stop 2024-06-06T08:41:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.923+0000 7f83ac05d640 1 -- 172.21.15.27:0/3114124245 shutdown_connections 2024-06-06T08:41:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.923+0000 7f83ac05d640 1 --2- 172.21.15.27:0/3114124245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83a4102370 0x7f83a410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.923+0000 7f83ac05d640 1 --2- 172.21.15.27:0/3114124245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83a41019b0 0x7f83a4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.923+0000 7f83ac05d640 1 --2- 172.21.15.27:0/3114124245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a41079c0 0x7f83a4107da0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.923+0000 7f83ac05d640 1 -- 172.21.15.27:0/3114124245 >> 172.21.15.27:0/3114124245 conn(0x7f83a40fd460 msgr2=0x7f83a40ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.923+0000 7f83ac05d640 1 -- 172.21.15.27:0/3114124245 shutdown_connections 2024-06-06T08:41:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.923+0000 7f83ac05d640 1 -- 172.21.15.27:0/3114124245 wait complete. 2024-06-06T08:41:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.924+0000 7f83ac05d640 1 Processor -- start 2024-06-06T08:41:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.924+0000 7f83ac05d640 1 -- start start 2024-06-06T08:41:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.924+0000 7f83ac05d640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83a41019b0 0x7f83a4073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.924+0000 7f83ac05d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a4102370 0x7f83a40741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.924+0000 7f83ac05d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83a41079c0 0x7f83a4074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.924+0000 7f83ac05d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83a4111dd0 con 0x7f83a4102370 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.924+0000 7f83ac05d640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83a4111c50 con 0x7f83a41019b0 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.924+0000 7f83ac05d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83a4111f50 con 0x7f83a41079c0 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.924+0000 7f83a95f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a4102370 0x7f83a40741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83a95f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a4102370 0x7f83a40741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42688/0 (socket says 172.21.15.27:42688) 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83a95f8640 1 -- 172.21.15.27:0/449580727 learned_addr learned my addr 172.21.15.27:0/449580727 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83aa5fa640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83a41079c0 0x7f83a4074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83a9df9640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83a41019b0 0x7f83a4073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83a95f8640 1 -- 172.21.15.27:0/449580727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83a41079c0 msgr2=0x7f83a4074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83a95f8640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83a41079c0 0x7f83a4074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83a95f8640 1 -- 172.21.15.27:0/449580727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83a41019b0 msgr2=0x7f83a4073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83a95f8640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83a41019b0 0x7f83a4073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83a95f8640 1 -- 172.21.15.27:0/449580727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a41a66f0 con 0x7f83a4102370 2024-06-06T08:41:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83a95f8640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a4102370 0x7f83a40741e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f839400d770 tx=0x7f839400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f839affd640 1 -- 172.21.15.27:0/449580727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394017070 con 0x7f83a4102370 2024-06-06T08:41:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f839affd640 1 -- 172.21.15.27:0/449580727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8394005500 con 0x7f83a4102370 2024-06-06T08:41:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83ac05d640 1 -- 172.21.15.27:0/449580727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a41a69e0 con 0x7f83a4102370 2024-06-06T08:41:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f839affd640 1 -- 172.21.15.27:0/449580727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394004e00 con 0x7f83a4102370 2024-06-06T08:41:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.925+0000 7f83ac05d640 1 -- 172.21.15.27:0/449580727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a41a6f30 con 0x7f83a4102370 2024-06-06T08:41:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.928+0000 7f83aed6b740 1 Processor -- start 2024-06-06T08:41:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.928+0000 7f839affd640 1 -- 172.21.15.27:0/449580727 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8394004fa0 con 0x7f83a4102370 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.928+0000 7f83aed6b740 1 -- start start 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 0x560c73dd8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73ddb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aed6b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560c73d324f0 con 0x560c73dce170 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aed6b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560c73cd0870 con 0x560c73dcea90 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aed6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560c73d2e4b0 con 0x560c73dd9350 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aa5fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aa5fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42690/0 (socket says 172.21.15.27:42690) 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aa5fa640 1 -- 172.21.15.27:0/2911854047 learned_addr learned my addr 172.21.15.27:0/2911854047 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f839affd640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f83900a2e20 0x7f83900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83a9df9640 1 --2- 172.21.15.27:0/2911854047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 0x560c73dd8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83a95f8640 1 --2- 172.21.15.27:0/2911854047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73ddb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f839affd640 1 -- 172.21.15.27:0/449580727 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f83940cf130 con 0x7f83a4102370 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83aa5fa640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f83900a2e20 0x7f83900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83a9df9640 1 -- 172.21.15.27:0/2911854047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 msgr2=0x560c73ddb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83a9df9640 1 --2- 172.21.15.27:0/2911854047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73ddb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83a9df9640 1 -- 172.21.15.27:0/2911854047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 msgr2=0x560c73dce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83a9df9640 1 --2- 172.21.15.27:0/2911854047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.929+0000 7f83a9df9640 1 -- 172.21.15.27:0/2911854047 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c73cec9b0 con 0x560c73dcea90 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.930+0000 7f83a9df9640 1 --2- 172.21.15.27:0/2911854047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 0x560c73dd8e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005940 tx=0x7f83a0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152d3e0e45d5b311 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.930+0000 7f838ffff640 1 -- 172.21.15.27:0/2911854047 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0040070 con 0x560c73dcea90 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.930+0000 7f838ffff640 1 -- 172.21.15.27:0/2911854047 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83a00319f0 con 0x560c73dcea90 2024-06-06T08:41:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.930+0000 7f838ffff640 1 -- 172.21.15.27:0/2911854047 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0035040 con 0x560c73dcea90 2024-06-06T08:41:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.930+0000 7f83aa5fa640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f83900a2e20 0x7f83900a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f839c005f40 tx=0x7f839c004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 -- 172.21.15.27:0/2911854047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 msgr2=0x560c73dd8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 --2- 172.21.15.27:0/2911854047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 0x560c73dd8e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005940 tx=0x7f83a0005c00 comp rx=0 tx=0).stop 2024-06-06T08:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 -- 172.21.15.27:0/2911854047 shutdown_connections 2024-06-06T08:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 --2- 172.21.15.27:0/2911854047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73ddb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 --2- 172.21.15.27:0/2911854047 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 0x560c73dd8e10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 --2- 172.21.15.27:0/2911854047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 -- 172.21.15.27:0/2911854047 >> 172.21.15.27:0/2911854047 conn(0x560c73dca680 msgr2=0x560c73dcaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 -- 172.21.15.27:0/2911854047 shutdown_connections 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 -- 172.21.15.27:0/2911854047 wait complete. 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 -- start start 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.931+0000 7f83aed6b740 1 Processor -- start 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dce170 0x560c73dd61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dcea90 0x560c73dd66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73dd6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83a9df9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dce170 0x560c73dd61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83a9df9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dce170 0x560c73dd61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:51634/0 (socket says 172.21.15.27:51634) 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83a9df9640 1 -- 172.21.15.27:0/807724850 learned_addr learned my addr 172.21.15.27:0/807724850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560c73ddbf20 con 0x560c73dcea90 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560c73cbbae0 con 0x560c73dce170 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560c73ddc0a0 con 0x560c73dd9350 2024-06-06T08:41:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.932+0000 7f83aed6b740 10 client.0 _ll_register_callbacks cb 0x560c73cf49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.933+0000 7f83aa5fa640 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73dd6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.933+0000 7f83a9df9640 1 -- 172.21.15.27:0/807724850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 msgr2=0x560c73dd6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.933+0000 7f83a9df9640 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73dd6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.933+0000 7f83a9df9640 1 -- 172.21.15.27:0/807724850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dcea90 msgr2=0x560c73dd66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.933+0000 7f83a9df9640 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dcea90 0x560c73dd66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.933+0000 7f83a9df9640 1 -- 172.21.15.27:0/807724850 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c73a46210 con 0x560c73dce170 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.933+0000 7f83a9df9640 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dce170 0x560c73dd61b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f83a00052d0 tx=0x7f83a0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2e499ad30c4b80 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.933+0000 7f838d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.934+0000 7f838d7fa640 1 -- 172.21.15.27:0/807724850 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0040030 con 0x560c73dce170 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.934+0000 7f838d7fa640 1 -- 172.21.15.27:0/807724850 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f83a0004240 con 0x560c73dce170 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.934+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a00319f0 con 0x560c73dce170 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.934+0000 7f838d7fa640 1 -- 172.21.15.27:0/807724850 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83a0057050 con 0x560c73dce170 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.934+0000 7f83aed6b740 10 client.25403 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.934+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560c73b12620 con 0x560c73dce170 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.935+0000 7f838d7fa640 1 -- 172.21.15.27:0/807724850 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83a00319f0 con 0x560c73dce170 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.935+0000 7f838d7fa640 1 -- 172.21.15.27:0/807724850 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f83a0003890 con 0x560c73dce170 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.935+0000 7f83aed6b740 10 client.25403 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.935+0000 7f83aed6b740 10 client.25403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.935+0000 7f83aed6b740 20 client.25403 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.935+0000 7f83aed6b740 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560c73de91b0 0x560c73deb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.935+0000 7f83aed6b740 4 client.25403 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.935+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c73d45f10 con 0x560c73de91b0 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.935+0000 7f83a95f8640 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560c73de91b0 0x560c73deb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.936+0000 7f83a95f8640 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560c73de91b0 0x560c73deb5a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6de6efaaf1678168 server_cookie=231f8b65cb73e32a in_seq=0 out_seq=0 2024-06-06T08:41:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.936+0000 7f838d7fa640 10 client.25403 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.936+0000 7f838d7fa640 1 -- 172.21.15.27:0/807724850 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560c73d45f10 con 0x560c73de91b0 2024-06-06T08:41:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.936+0000 7f838d7fa640 10 client.25403 handle_command_reply: tid=0 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 client.25403 shutdown 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dce170 msgr2=0x560c73dd61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dce170 0x560c73dd61b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f83a00052d0 tx=0x7f83a0000bc0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 shutdown_connections 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560c73de91b0 0x560c73deb5a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73dd6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dcea90 0x560c73dd66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 --2- 172.21.15.27:0/807724850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dce170 0x560c73dd61b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f838d7fa640 0 client.25403 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 >> 172.21.15.27:0/807724850 conn(0x560c73dca680 msgr2=0x560c73dd0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 shutdown_connections 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 1 -- 172.21.15.27:0/807724850 wait complete. 2024-06-06T08:41:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.937+0000 7f83aed6b740 20 client.25403 trim_cache size 0 max 16384 2024-06-06T08:41:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.954+0000 7f83aed6b740 1 Processor -- start 2024-06-06T08:41:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.954+0000 7f83aed6b740 1 -- start start 2024-06-06T08:41:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dd0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dcea90 0x560c73dd5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dd9350 0x560c73df0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83aed6b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41b39d0 con 0x560c73dce170 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83aed6b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41b12a0 con 0x560c73dd9350 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83aed6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41af0f0 con 0x560c73dcea90 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a9df9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dd0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a9df9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dd0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42704/0 (socket says 172.21.15.27:42704) 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a9df9640 1 -- 172.21.15.27:0/4285417332 learned_addr learned my addr 172.21.15.27:0/4285417332 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83aa5fa640 1 --2- 172.21.15.27:0/4285417332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dd9350 0x560c73df0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a95f8640 1 --2- 172.21.15.27:0/4285417332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dcea90 0x560c73dd5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a9df9640 1 -- 172.21.15.27:0/4285417332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dcea90 msgr2=0x560c73dd5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a9df9640 1 --2- 172.21.15.27:0/4285417332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dcea90 0x560c73dd5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a9df9640 1 -- 172.21.15.27:0/4285417332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dd9350 msgr2=0x560c73df0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a9df9640 1 --2- 172.21.15.27:0/4285417332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dd9350 0x560c73df0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a9df9640 1 -- 172.21.15.27:0/4285417332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c73de5180 con 0x560c73dce170 2024-06-06T08:41:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.955+0000 7f83a9df9640 1 --2- 172.21.15.27:0/4285417332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dd0b30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f83a002a940 tx=0x7f83a0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51170ab93b271e40 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f838f7fe640 1 -- 172.21.15.27:0/4285417332 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0040040 con 0x560c73dce170 2024-06-06T08:41:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f838f7fe640 1 -- 172.21.15.27:0/4285417332 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83a0005850 con 0x560c73dce170 2024-06-06T08:41:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f838f7fe640 1 -- 172.21.15.27:0/4285417332 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0035040 con 0x560c73dce170 2024-06-06T08:41:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f83aed6b740 1 -- 172.21.15.27:0/4285417332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 msgr2=0x560c73dd0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f83aed6b740 1 --2- 172.21.15.27:0/4285417332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dd0b30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f83a002a940 tx=0x7f83a0005a20 comp rx=0 tx=0).stop 2024-06-06T08:41:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f83aed6b740 1 -- 172.21.15.27:0/4285417332 shutdown_connections 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f83aed6b740 1 --2- 172.21.15.27:0/4285417332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dd9350 0x560c73df0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f83aed6b740 1 --2- 172.21.15.27:0/4285417332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dcea90 0x560c73dd5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f83aed6b740 1 --2- 172.21.15.27:0/4285417332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73dd0b30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f83aed6b740 1 -- 172.21.15.27:0/4285417332 >> 172.21.15.27:0/4285417332 conn(0x560c73dca680 msgr2=0x560c73dd6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f83aed6b740 1 -- 172.21.15.27:0/4285417332 shutdown_connections 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.956+0000 7f83aed6b740 1 -- 172.21.15.27:0/4285417332 wait complete. 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.957+0000 7f83aed6b740 1 -- start start 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.957+0000 7f83aed6b740 1 Processor -- start 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.957+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73ddcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 0x560c73ddcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aed6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73e4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83a95f8640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 0x560c73ddcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aed6b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41af760 con 0x560c73dce170 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83a9df9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73ddcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83a9df9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73ddcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42710/0 (socket says 172.21.15.27:42710) 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83a9df9640 1 -- 172.21.15.27:0/1329772793 learned_addr learned my addr 172.21.15.27:0/1329772793 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41b0bd0 con 0x560c73dcea90 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41afe30 con 0x560c73dd9350 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aa5fa640 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73e4bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aa5fa640 1 -- 172.21.15.27:0/1329772793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 msgr2=0x560c73ddcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aa5fa640 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 0x560c73ddcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aa5fa640 1 -- 172.21.15.27:0/1329772793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 msgr2=0x560c73ddcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aa5fa640 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73ddcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83aa5fa640 1 -- 172.21.15.27:0/1329772793 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c73e4c180 con 0x560c73dd9350 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.958+0000 7f83a9df9640 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73ddcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f83aa5fa640 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73e4bb40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f839c005f10 tx=0x7f839c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f02fcca9e421da server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f83aed6b740 10 client.0 _ll_register_callbacks cb 0x7f83a0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a0005850 con 0x560c73dd9350 2024-06-06T08:41:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f838d7fa640 10 client.25405 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:41:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f838d7fa640 1 -- 172.21.15.27:0/1329772793 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c01a040 con 0x560c73dd9350 2024-06-06T08:41:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f838d7fa640 1 -- 172.21.15.27:0/1329772793 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f839c011c40 con 0x560c73dd9350 2024-06-06T08:41:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f838d7fa640 1 -- 172.21.15.27:0/1329772793 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f839c03e430 con 0x560c73dd9350 2024-06-06T08:41:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f83aed6b740 10 client.25405 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=25}) -- 0x560c73e533c0 con 0x560c73dd9350 2024-06-06T08:41:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.959+0000 7f838d7fa640 1 -- 172.21.15.27:0/1329772793 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f839c00f040 con 0x560c73dd9350 2024-06-06T08:41:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.960+0000 7f83aed6b740 10 client.25405 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.960+0000 7f83aed6b740 10 client.25405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:41:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.960+0000 7f83aed6b740 20 client.25405 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.960+0000 7f83aed6b740 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560c73e53ab0 0x560c73e55f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.960+0000 7f83aed6b740 4 client.25405 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:41:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.960+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c73d3fc20 con 0x560c73e53ab0 2024-06-06T08:41:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.960+0000 7f83a9df9640 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560c73e53ab0 0x560c73e55f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.960+0000 7f838d7fa640 10 client.25405 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.960+0000 7f83a9df9640 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560c73e53ab0 0x560c73e55f70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f47021da09cbae1 server_cookie=e83596ef8bbdf00e in_seq=0 out_seq=0 2024-06-06T08:41:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.966+0000 7f838d7fa640 1 -- 172.21.15.27:0/1329772793 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560c73d3fc20 con 0x560c73e53ab0 2024-06-06T08:41:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.966+0000 7f838d7fa640 10 client.25405 handle_command_reply: tid=0 2024-06-06T08:41:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.966+0000 7f83aed6b740 1 client.25405 shutdown 2024-06-06T08:41:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 msgr2=0x560c73e4bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73e4bb40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f839c005f10 tx=0x7f839c004a40 comp rx=0 tx=0).stop 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 shutdown_connections 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560c73e53ab0 0x560c73e55f70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560c73dd9350 0x560c73e4bb40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560c73dcea90 0x560c73ddcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f838d7fa640 0 client.25405 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 --2- 172.21.15.27:0/1329772793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560c73dce170 0x560c73ddcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 >> 172.21.15.27:0/1329772793 conn(0x560c73dca680 msgr2=0x560c73dd6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 shutdown_connections 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 1 -- 172.21.15.27:0/1329772793 wait complete. 2024-06-06T08:41:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83aed6b740 20 client.25405 trim_cache size 0 max 16384 2024-06-06T08:41:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:41:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:41:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:41:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83ac05d640 1 -- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f83900a2e20 msgr2=0x7f83900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.967+0000 7f83ac05d640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f83900a2e20 0x7f83900a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f839c005f40 tx=0x7f839c004ab0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 -- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a4102370 msgr2=0x7f83a40741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a4102370 0x7f83a40741e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f839400d770 tx=0x7f839400dc30 comp rx=0 tx=0).stop 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 -- 172.21.15.27:0/449580727 shutdown_connections 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f83900a2e20 0x7f83900a52e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83a41079c0 0x7f83a4074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a4102370 0x7f83a40741e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 --2- 172.21.15.27:0/449580727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83a41019b0 0x7f83a4073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 -- 172.21.15.27:0/449580727 >> 172.21.15.27:0/449580727 conn(0x7f83a40fd460 msgr2=0x7f83a40fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 -- 172.21.15.27:0/449580727 shutdown_connections 2024-06-06T08:41:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:41:51.968+0000 7f83ac05d640 1 -- 172.21.15.27:0/449580727 wait complete. 2024-06-06T08:41:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:52.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:52 smithi027 ceph-mon[32335]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-06-06T08:41:52.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:52 smithi203 ceph-mon[36579]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-06-06T08:41:52.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:52 smithi176 ceph-mon[36786]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-06-06T08:41:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.933+0000 7f224c992700 1 -- 172.21.15.27:0/4107107714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22481027a0 msgr2=0x7f2248102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.933+0000 7f224c992700 1 --2- 172.21.15.27:0/4107107714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22481027a0 0x7f2248102b80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f22380099c0 tx=0x7f2238009cd0 comp rx=0 tx=0).stop 2024-06-06T08:41:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.933+0000 7f224c992700 1 -- 172.21.15.27:0/4107107714 shutdown_connections 2024-06-06T08:41:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.933+0000 7f224c992700 1 --2- 172.21.15.27:0/4107107714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2248107870 0x7f224810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.933+0000 7f224c992700 1 --2- 172.21.15.27:0/4107107714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22481027a0 0x7f2248102b80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.933+0000 7f224c992700 1 --2- 172.21.15.27:0/4107107714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2248103150 0x7f2248107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.933+0000 7f224c992700 1 -- 172.21.15.27:0/4107107714 >> 172.21.15.27:0/4107107714 conn(0x7f2248079520 msgr2=0x7f2248079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.933+0000 7f224c992700 1 -- 172.21.15.27:0/4107107714 shutdown_connections 2024-06-06T08:41:53.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.933+0000 7f224c992700 1 -- 172.21.15.27:0/4107107714 wait complete. 2024-06-06T08:41:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.934+0000 7f224c992700 1 Processor -- start 2024-06-06T08:41:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.934+0000 7f224c992700 1 -- start start 2024-06-06T08:41:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.934+0000 7f224c992700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22481027a0 0x7f224819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.934+0000 7f224c992700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2248103150 0x7f224819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.934+0000 7f224c992700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248107870 0x7f22481989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.934+0000 7f224c992700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2248112790 con 0x7f2248107870 2024-06-06T08:41:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.934+0000 7f224c992700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2248112610 con 0x7f22481027a0 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.934+0000 7f224c992700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2248112910 con 0x7f2248103150 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2246d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248107870 0x7f22481989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2245d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2248103150 0x7f224819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2246d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248107870 0x7f22481989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42730/0 (socket says 172.21.15.27:42730) 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2245d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2248103150 0x7f224819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:43504/0 (socket says 172.21.15.27:43504) 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2246d9d700 1 -- 172.21.15.27:0/3230363224 learned_addr learned my addr 172.21.15.27:0/3230363224 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f224659c700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22481027a0 0x7f224819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2246d9d700 1 -- 172.21.15.27:0/3230363224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2248103150 msgr2=0x7f224819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2246d9d700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2248103150 0x7f224819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2246d9d700 1 -- 172.21.15.27:0/3230363224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22481027a0 msgr2=0x7f224819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2246d9d700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22481027a0 0x7f224819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2246d9d700 1 -- 172.21.15.27:0/3230363224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2248199130 con 0x7f2248107870 2024-06-06T08:41:53.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f2246d9d700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248107870 0x7f22481989d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2240007ee0 tx=0x7f224000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f22377fe700 1 -- 172.21.15.27:0/3230363224 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240004bb0 con 0x7f2248107870 2024-06-06T08:41:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f22377fe700 1 -- 172.21.15.27:0/3230363224 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2240004d50 con 0x7f2248107870 2024-06-06T08:41:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.935+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2248199420 con 0x7f2248107870 2024-06-06T08:41:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.936+0000 7f22377fe700 1 -- 172.21.15.27:0/3230363224 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240005960 con 0x7f2248107870 2024-06-06T08:41:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.936+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22481a56e0 con 0x7f2248107870 2024-06-06T08:41:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.937+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2228005330 con 0x7f2248107870 2024-06-06T08:41:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.938+0000 7f22377fe700 1 -- 172.21.15.27:0/3230363224 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f224001a020 con 0x7f2248107870 2024-06-06T08:41:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.938+0000 7f22377fe700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22300a2d90 0x7f22300a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.938+0000 7f224659c700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22300a2d90 0x7f22300a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.938+0000 7f22377fe700 1 -- 172.21.15.27:0/3230363224 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f22400cedb0 con 0x7f2248107870 2024-06-06T08:41:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.939+0000 7f224659c700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22300a2d90 0x7f22300a5250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2238000c00 tx=0x7f2238005fb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:53.943+0000 7f22377fe700 1 -- 172.21.15.27:0/3230363224 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2240010070 con 0x7f2248107870 2024-06-06T08:41:54.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.211+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22280058f0 con 0x7f2248107870 2024-06-06T08:41:54.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.211+0000 7f22377fe700 1 -- 172.21.15.27:0/3230363224 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7f224009b610 con 0x7f2248107870 2024-06-06T08:41:54.210 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:41:54.210 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:41:54.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22300a2d90 msgr2=0x7f22300a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22300a2d90 0x7f22300a5250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2238000c00 tx=0x7f2238005fb0 comp rx=0 tx=0).stop 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248107870 msgr2=0x7f22481989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248107870 0x7f22481989d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2240007ee0 tx=0x7f224000d3b0 comp rx=0 tx=0).stop 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 shutdown_connections 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248107870 0x7f22481989d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f22300a2d90 0x7f22300a5250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f22481027a0 0x7f224819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 --2- 172.21.15.27:0/3230363224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2248103150 0x7f224819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 >> 172.21.15.27:0/3230363224 conn(0x7f2248079520 msgr2=0x7f2248101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 shutdown_connections 2024-06-06T08:41:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:54.214+0000 7f224c992700 1 -- 172.21.15.27:0/3230363224 wait complete. 2024-06-06T08:41:54.214 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:41:54.526 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:54 smithi027 ceph-mon[32335]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-06-06T08:41:54.526 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:54 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3230363224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:54.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:54 smithi203 ceph-mon[36579]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-06-06T08:41:54.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:54 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3230363224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:54.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:54 smithi176 ceph-mon[36786]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-06-06T08:41:54.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:54 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3230363224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:41:54.991 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T08:41:55.255 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:56.672 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:56 smithi027 ceph-mon[32335]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-06T08:41:56.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:56 smithi203 ceph-mon[36579]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-06T08:41:56.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:56 smithi176 ceph-mon[36786]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-06T08:41:57.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:41:57.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.045+0000 7fafe67dc700 1 -- 172.21.15.27:0/1467398325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafe0100a80 msgr2=0x7fafe0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.045+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1467398325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafe0100a80 0x7fafe0100f00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009990 tx=0x7fafd4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.045+0000 7fafe67dc700 1 -- 172.21.15.27:0/1467398325 shutdown_connections 2024-06-06T08:41:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.045+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1467398325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe0101440 0x7fafe010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.045+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1467398325 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe0108510 0x7fafe01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.045+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1467398325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafe0100a80 0x7fafe0100f00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.045+0000 7fafe67dc700 1 -- 172.21.15.27:0/1467398325 >> 172.21.15.27:0/1467398325 conn(0x7fafe0077a00 msgr2=0x7fafe00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.045+0000 7fafe67dc700 1 -- 172.21.15.27:0/1467398325 shutdown_connections 2024-06-06T08:41:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.045+0000 7fafe67dc700 1 -- 172.21.15.27:0/1467398325 wait complete. 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.046+0000 7fafe67dc700 1 Processor -- start 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.046+0000 7fafe67dc700 1 -- start start 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.046+0000 7fafe67dc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe0100a80 0x7fafe019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.046+0000 7fafe67dc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe0101440 0x7fafe019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.046+0000 7fafe67dc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafe0108510 0x7fafe019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.046+0000 7fafe67dc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0112770 con 0x7fafe0108510 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.046+0000 7fafe67dc700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01125f0 con 0x7fafe0101440 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.046+0000 7fafe67dc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01128f0 con 0x7fafe0100a80 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafe4d79700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafe0108510 0x7fafe019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafe4d79700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafe0108510 0x7fafe019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42748/0 (socket says 172.21.15.27:42748) 2024-06-06T08:41:57.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafe4d79700 1 -- 172.21.15.27:0/1275656529 learned_addr learned my addr 172.21.15.27:0/1275656529 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:57.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafdffff700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe0100a80 0x7fafe019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe0101440 0x7fafe019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafdffff700 1 -- 172.21.15.27:0/1275656529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe0101440 msgr2=0x7fafe019b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafdffff700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe0101440 0x7fafe019b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafdffff700 1 -- 172.21.15.27:0/1275656529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafe0108510 msgr2=0x7fafe019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafdffff700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafe0108510 0x7fafe019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafdffff700 1 -- 172.21.15.27:0/1275656529 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe019feb0 con 0x7fafe0100a80 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.047+0000 7fafdffff700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe0100a80 0x7fafe019aea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d950 tx=0x7fafd000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.048+0000 7fafdd7fa700 1 -- 172.21.15.27:0/1275656529 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0016070 con 0x7fafe0100a80 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.048+0000 7fafdd7fa700 1 -- 172.21.15.27:0/1275656529 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd000fd70 con 0x7fafe0100a80 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.048+0000 7fafdd7fa700 1 -- 172.21.15.27:0/1275656529 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd000e3b0 con 0x7fafe0100a80 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.048+0000 7fafe67dc700 1 -- 172.21.15.27:0/1275656529 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe01a01a0 con 0x7fafe0100a80 2024-06-06T08:41:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.048+0000 7fafe67dc700 1 -- 172.21.15.27:0/1275656529 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01a7a20 con 0x7fafe0100a80 2024-06-06T08:41:57.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.051+0000 7fafdd7fa700 1 -- 172.21.15.27:0/1275656529 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fafd0019070 con 0x7fafe0100a80 2024-06-06T08:41:57.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7faff105fb80 1 Processor -- start 2024-06-06T08:41:57.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7fafdd7fa700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fafc80a2e10 0x7fafc80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7faff105fb80 1 -- start start 2024-06-06T08:41:57.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7fafdd7fa700 1 -- 172.21.15.27:0/1275656529 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fafd00cf110 con 0x7fafe0100a80 2024-06-06T08:41:57.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fafc80a2e10 0x7fafc80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7faff105fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998989340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7faff105fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998989880 0x556998983260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7faff105fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x556998983bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7faff105fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556998875f60 con 0x556998988f60 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7faff105fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5569988d4850 con 0x5569989837a0 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7faff105fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55699883c460 con 0x556998989880 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.052+0000 7fafdffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998989340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafdffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998989340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42752/0 (socket says 172.21.15.27:42752) 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafdffff700 1 -- 172.21.15.27:0/2031308768 learned_addr learned my addr 172.21.15.27:0/2031308768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafe4d79700 1 --2- 172.21.15.27:0/2031308768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x556998983bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafdffff700 1 -- 172.21.15.27:0/2031308768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998989880 msgr2=0x556998983260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafdffff700 1 --2- 172.21.15.27:0/2031308768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998989880 0x556998983260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafdffff700 1 -- 172.21.15.27:0/2031308768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 msgr2=0x556998983bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafdffff700 1 --2- 172.21.15.27:0/2031308768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x556998983bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafdffff700 1 -- 172.21.15.27:0/2031308768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569988238d0 con 0x556998988f60 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fafc80a2e10 0x7fafc80a52d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fafd400b5c0 tx=0x7fafd4005960 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:57.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafdffff700 1 --2- 172.21.15.27:0/2031308768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998989340 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fafd00976d0 tx=0x7fafd009ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a971e52c45d45d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:57.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafc67fc700 1 -- 172.21.15.27:0/2031308768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0020040 con 0x556998988f60 2024-06-06T08:41:57.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafc67fc700 1 -- 172.21.15.27:0/2031308768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd009b270 con 0x556998988f60 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.053+0000 7fafc67fc700 1 -- 172.21.15.27:0/2031308768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd00a3a50 con 0x556998988f60 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.054+0000 7faff105fb80 1 -- 172.21.15.27:0/2031308768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 msgr2=0x556998989340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.054+0000 7faff105fb80 1 --2- 172.21.15.27:0/2031308768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998989340 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fafd00976d0 tx=0x7fafd009ad00 comp rx=0 tx=0).stop 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.054+0000 7faff105fb80 1 -- 172.21.15.27:0/2031308768 shutdown_connections 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.054+0000 7faff105fb80 1 --2- 172.21.15.27:0/2031308768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x556998983bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.054+0000 7faff105fb80 1 --2- 172.21.15.27:0/2031308768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998989340 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.054+0000 7faff105fb80 1 --2- 172.21.15.27:0/2031308768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998989880 0x556998983260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.054+0000 7faff105fb80 1 -- 172.21.15.27:0/2031308768 >> 172.21.15.27:0/2031308768 conn(0x5569988b0080 msgr2=0x55699897e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.054+0000 7faff105fb80 1 -- 172.21.15.27:0/2031308768 shutdown_connections 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.054+0000 7faff105fb80 1 -- 172.21.15.27:0/2031308768 wait complete. 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.055+0000 7faff105fb80 1 -- start start 2024-06-06T08:41:57.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.055+0000 7faff105fb80 1 Processor -- start 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7faff105fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x55699898f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7faff105fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x55699898c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7faff105fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699898cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7faff105fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556998874fc0 con 0x556998989880 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7faff105fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5569988d0810 con 0x5569989837a0 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7faff105fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556998990260 con 0x556998988f60 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7faff105fb80 10 client.0 _ll_register_callbacks cb 0x55699886a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafe4d79700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699898cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafdf7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x55699898c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafdf7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x55699898c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:43524/0 (socket says 172.21.15.27:43524) 2024-06-06T08:41:57.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafdf7fe700 1 -- 172.21.15.27:0/3459650619 learned_addr learned my addr 172.21.15.27:0/3459650619 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafdffff700 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x55699898f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafdf7fe700 1 -- 172.21.15.27:0/3459650619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 msgr2=0x55699898f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x55699898f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafdf7fe700 1 -- 172.21.15.27:0/3459650619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 msgr2=0x55699898cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699898cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.056+0000 7fafdf7fe700 1 -- 172.21.15.27:0/3459650619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699889a750 con 0x556998988f60 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.057+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x55699898c9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009990 tx=0x7fafd4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d64be284c98c38 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.057+0000 7fafbf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.057+0000 7fafbf7fe700 1 -- 172.21.15.27:0/3459650619 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4034060 con 0x556998988f60 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.057+0000 7fafbf7fe700 1 -- 172.21.15.27:0/3459650619 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fafd400fd60 con 0x556998988f60 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.057+0000 7faff105fb80 1 -- 172.21.15.27:0/3459650619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd009b270 con 0x556998988f60 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.057+0000 7fafbf7fe700 1 -- 172.21.15.27:0/3459650619 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fafd403b050 con 0x556998988f60 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.057+0000 7faff105fb80 10 client.25420 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.057+0000 7faff105fb80 1 -- 172.21.15.27:0/3459650619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55699899aac0 con 0x556998988f60 2024-06-06T08:41:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.058+0000 7fafbf7fe700 1 -- 172.21.15.27:0/3459650619 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd009b270 con 0x556998988f60 2024-06-06T08:41:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.058+0000 7fafbf7fe700 1 -- 172.21.15.27:0/3459650619 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fafd4017480 con 0x556998988f60 2024-06-06T08:41:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.058+0000 7faff105fb80 10 client.25420 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.058+0000 7faff105fb80 10 client.25420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:41:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.058+0000 7faff105fb80 20 client.25420 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.058+0000 7faff105fb80 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55699899d100 0x55699899f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.058+0000 7faff105fb80 4 client.25420 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.058+0000 7faff105fb80 1 -- 172.21.15.27:0/3459650619 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569988e1d50 con 0x55699899d100 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.058+0000 7fafdffff700 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55699899d100 0x55699899f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.059+0000 7fafdffff700 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55699899d100 0x55699899f4f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=538d737488f9597a server_cookie=a47a4d3aa0e1f850 in_seq=0 out_seq=0 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.059+0000 7fafbf7fe700 10 client.25420 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7fafbf7fe700 1 -- 172.21.15.27:0/3459650619 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5569988e1d50 con 0x55699899d100 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7fafbf7fe700 10 client.25420 handle_command_reply: tid=0 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 client.25420 shutdown 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 -- 172.21.15.27:0/3459650619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 msgr2=0x55699898c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x55699898c9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009990 tx=0x7fafd4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 -- 172.21.15.27:0/3459650619 shutdown_connections 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55699899d100 0x55699899f4f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699898cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x55699898f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7fafbf7fe700 0 client.25420 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 --2- 172.21.15.27:0/3459650619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x55699898c9f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 -- 172.21.15.27:0/3459650619 >> 172.21.15.27:0/3459650619 conn(0x5569988b0080 msgr2=0x55699897d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 -- 172.21.15.27:0/3459650619 shutdown_connections 2024-06-06T08:41:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 1 -- 172.21.15.27:0/3459650619 wait complete. 2024-06-06T08:41:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.060+0000 7faff105fb80 20 client.25420 trim_cache size 0 max 16384 2024-06-06T08:41:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7faff105fb80 1 Processor -- start 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7faff105fb80 1 -- start start 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7faff105fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x556998983260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7faff105fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x556998999380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7faff105fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699897de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7faff105fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01b3110 con 0x556998989880 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7faff105fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01b09a0 con 0x5569989837a0 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7faff105fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01ae7f0 con 0x556998988f60 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7fafe4d79700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699897de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7fafe4d79700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699897de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42782/0 (socket says 172.21.15.27:42782) 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7fafe4d79700 1 -- 172.21.15.27:0/3402639057 learned_addr learned my addr 172.21.15.27:0/3402639057 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x556998999380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.079+0000 7fafdffff700 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x556998983260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafe4d79700 1 -- 172.21.15.27:0/3402639057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 msgr2=0x556998999380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafe4d79700 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x556998999380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafe4d79700 1 -- 172.21.15.27:0/3402639057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 msgr2=0x556998983260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafe4d79700 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x556998983260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafe4d79700 1 -- 172.21.15.27:0/3402639057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699897f9a0 con 0x556998989880 2024-06-06T08:41:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafdffff700 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x556998983260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:41:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafe4d79700 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699897de80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fafd8007510 tx=0x7fafd80091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54cad8a84fac048 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafc5ffb700 1 -- 172.21.15.27:0/3402639057 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8020470 con 0x556998989880 2024-06-06T08:41:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafc5ffb700 1 -- 172.21.15.27:0/3402639057 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd801b070 con 0x556998989880 2024-06-06T08:41:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.080+0000 7fafc5ffb700 1 -- 172.21.15.27:0/3402639057 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8016780 con 0x556998989880 2024-06-06T08:41:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 -- 172.21.15.27:0/3402639057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 msgr2=0x55699897de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699897de80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fafd8007510 tx=0x7fafd80091b0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 -- 172.21.15.27:0/3402639057 shutdown_connections 2024-06-06T08:41:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998989880 0x55699897de80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x556998983260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 --2- 172.21.15.27:0/3402639057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998988f60 0x556998999380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 -- 172.21.15.27:0/3402639057 >> 172.21.15.27:0/3402639057 conn(0x5569988b0080 msgr2=0x55699898b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 -- 172.21.15.27:0/3402639057 shutdown_connections 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 -- 172.21.15.27:0/3402639057 wait complete. 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.081+0000 7faff105fb80 1 -- start start 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7faff105fb80 1 Processor -- start 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7faff105fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x5569989954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7faff105fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998995a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7fafdf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998995a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7faff105fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998989880 0x5569989a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7faff105fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01aee60 con 0x556998988f60 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7fafdf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998995a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42786/0 (socket says 172.21.15.27:42786) 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7fafdf7fe700 1 -- 172.21.15.27:0/4054886952 learned_addr learned my addr 172.21.15.27:0/4054886952 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01b02d0 con 0x5569989837a0 2024-06-06T08:41:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01af530 con 0x556998989880 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7faff105fb80 10 client.0 _ll_register_callbacks cb 0x55699886a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.082+0000 7fafdffff700 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x5569989954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafdf7fe700 1 -- 172.21.15.27:0/4054886952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998989880 msgr2=0x5569989a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998989880 0x5569989a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafdf7fe700 1 -- 172.21.15.27:0/4054886952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 msgr2=0x5569989954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x5569989954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafdf7fe700 1 -- 172.21.15.27:0/4054886952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569989960f0 con 0x556998988f60 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafdf7fe700 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998995a30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fafd4017850 tx=0x7fafd4010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b668cf54dc6735de server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafbf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafbf7fe700 1 -- 172.21.15.27:0/4054886952 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4044060 con 0x556998988f60 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafbf7fe700 1 -- 172.21.15.27:0/4054886952 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fafd40173f0 con 0x556998988f60 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd801b070 con 0x556998988f60 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7fafbf7fe700 1 -- 172.21.15.27:0/4054886952 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fafd404b050 con 0x556998988f60 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7faff105fb80 10 client.15876 fetch_fsmap learned FSMap version 25 2024-06-06T08:41:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.083+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556998a0fb00 con 0x556998988f60 2024-06-06T08:41:57.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.084+0000 7fafbf7fe700 1 -- 172.21.15.27:0/4054886952 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd801b070 con 0x556998988f60 2024-06-06T08:41:57.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.084+0000 7fafbf7fe700 1 -- 172.21.15.27:0/4054886952 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fafd4030450 con 0x556998988f60 2024-06-06T08:41:57.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.084+0000 7faff105fb80 10 client.15876 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:41:57.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.084+0000 7faff105fb80 10 client.15876 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:41:57.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.084+0000 7faff105fb80 20 client.15876 populate_metadata read hostname 'smithi027' 2024-06-06T08:41:57.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.084+0000 7faff105fb80 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55699899d100 0x556998a12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:41:57.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.084+0000 7faff105fb80 4 client.15876 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:41:57.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.084+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569988dc2d0 con 0x55699899d100 2024-06-06T08:41:57.087 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.084+0000 7fafdffff700 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55699899d100 0x556998a12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7fafdffff700 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55699899d100 0x556998a12220 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37e7884523a9b13c server_cookie=bb86b6da24857835 in_seq=0 out_seq=0 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7fafbf7fe700 10 client.15876 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7fafbf7fe700 1 -- 172.21.15.27:0/4054886952 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569988dc2d0 con 0x55699899d100 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7fafbf7fe700 10 client.15876 handle_command_reply: tid=0 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 client.15876 shutdown 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 msgr2=0x556998995a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998995a30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fafd4017850 tx=0x7fafd4010fa0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 shutdown_connections 2024-06-06T08:41:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55699899d100 0x556998a12220 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556998989880 0x5569989a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569989837a0 0x5569989954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 --2- 172.21.15.27:0/4054886952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556998988f60 0x556998995a30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 >> 172.21.15.27:0/4054886952 conn(0x5569988b0080 msgr2=0x55699898eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7fafbf7fe700 0 client.15876 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 shutdown_connections 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 1 -- 172.21.15.27:0/4054886952 wait complete. 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.085+0000 7faff105fb80 20 client.15876 trim_cache size 0 max 16384 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 -- 172.21.15.27:0/1275656529 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fafc80a2e10 msgr2=0x7fafc80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fafc80a2e10 0x7fafc80a52d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fafd400b5c0 tx=0x7fafd4005960 comp rx=0 tx=0).stop 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 -- 172.21.15.27:0/1275656529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe0100a80 msgr2=0x7fafe019aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe0100a80 0x7fafe019aea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d950 tx=0x7fafd000dc60 comp rx=0 tx=0).stop 2024-06-06T08:41:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 -- 172.21.15.27:0/1275656529 shutdown_connections 2024-06-06T08:41:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafe0108510 0x7fafe019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe0100a80 0x7fafe019aea0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fafc80a2e10 0x7fafc80a52d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 --2- 172.21.15.27:0/1275656529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe0101440 0x7fafe019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:41:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 -- 172.21.15.27:0/1275656529 >> 172.21.15.27:0/1275656529 conn(0x7fafe0077a00 msgr2=0x7fafe00fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:41:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 -- 172.21.15.27:0/1275656529 shutdown_connections 2024-06-06T08:41:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:41:57.086+0000 7fafe67dc700 1 -- 172.21.15.27:0/1275656529 wait complete. 2024-06-06T08:41:57.777 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T08:41:57.850 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:41:58.672 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:41:58 smithi027 ceph-mon[32335]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-06T08:41:58.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:41:58 smithi203 ceph-mon[36579]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-06T08:41:58.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:41:58 smithi176 ceph-mon[36786]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-06T08:42:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.272+0000 7f92f687b700 1 -- 172.21.15.27:0/2944191566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 msgr2=0x7f92f0107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.272+0000 7f92f687b700 1 --2- 172.21.15.27:0/2944191566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 0x7f92f0107830 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009990 tx=0x7f92e0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.273+0000 7f92f687b700 1 -- 172.21.15.27:0/2944191566 shutdown_connections 2024-06-06T08:42:00.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.273+0000 7f92f687b700 1 --2- 172.21.15.27:0/2944191566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f0102ee0 0x7f92f010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.273+0000 7f92f687b700 1 --2- 172.21.15.27:0/2944191566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 0x7f92f0107830 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.273+0000 7f92f687b700 1 --2- 172.21.15.27:0/2944191566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f0102540 0x7f92f01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.273+0000 7f92f687b700 1 -- 172.21.15.27:0/2944191566 >> 172.21.15.27:0/2944191566 conn(0x7f92f00795f0 msgr2=0x7f92f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:00.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.273+0000 7f92f687b700 1 -- 172.21.15.27:0/2944191566 shutdown_connections 2024-06-06T08:42:00.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.273+0000 7f92f687b700 1 -- 172.21.15.27:0/2944191566 wait complete. 2024-06-06T08:42:00.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.273+0000 7f92f687b700 1 Processor -- start 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.273+0000 7f92f687b700 1 -- start start 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f687b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f0102540 0x7f92f0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f687b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f0102ee0 0x7f92f0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f687b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 0x7f92f0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f687b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01127c0 con 0x7f92f0107450 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f687b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0112640 con 0x7f92f0102ee0 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f687b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0112940 con 0x7f92f0102540 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f4e18700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 0x7f92f0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92effff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f0102540 0x7f92f0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92ef7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f0102ee0 0x7f92f0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92effff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f0102540 0x7f92f0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:37660/0 (socket says 172.21.15.27:37660) 2024-06-06T08:42:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92effff700 1 -- 172.21.15.27:0/2230204127 learned_addr learned my addr 172.21.15.27:0/2230204127 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f4e18700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 0x7f92f0075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41172/0 (socket says 172.21.15.27:41172) 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f4e18700 1 -- 172.21.15.27:0/2230204127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f0102540 msgr2=0x7f92f0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f4e18700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f0102540 0x7f92f0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f4e18700 1 -- 172.21.15.27:0/2230204127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f0102ee0 msgr2=0x7f92f0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.274+0000 7f92f4e18700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f0102ee0 0x7f92f0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.275+0000 7f92f4e18700 1 -- 172.21.15.27:0/2230204127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f01a7170 con 0x7f92f0107450 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.275+0000 7f92ef7fe700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f0102ee0 0x7f92f0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.275+0000 7f92f4e18700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 0x7f92f0075210 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f92e800aa90 tx=0x7f92e800ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.275+0000 7f92ed7fa700 1 -- 172.21.15.27:0/2230204127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8017020 con 0x7f92f0107450 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.275+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f01a7460 con 0x7f92f0107450 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.275+0000 7f92ed7fa700 1 -- 172.21.15.27:0/2230204127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e8004bb0 con 0x7f92f0107450 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.275+0000 7f92ed7fa700 1 -- 172.21.15.27:0/2230204127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8005280 con 0x7f92f0107450 2024-06-06T08:42:00.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.275+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a79f0 con 0x7f92f0107450 2024-06-06T08:42:00.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.276+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f0069180 con 0x7f92f0107450 2024-06-06T08:42:00.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.277+0000 7f92ed7fa700 1 -- 172.21.15.27:0/2230204127 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f92e800c580 con 0x7f92f0107450 2024-06-06T08:42:00.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.278+0000 7f92ed7fa700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92d80a2e10 0x7f92d80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:00.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.278+0000 7f92ed7fa700 1 -- 172.21.15.27:0/2230204127 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f92e80ceb90 con 0x7f92f0107450 2024-06-06T08:42:00.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.278+0000 7f92effff700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92d80a2e10 0x7f92d80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:00.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.279+0000 7f92effff700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92d80a2e10 0x7f92d80a52d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f92e0006010 tx=0x7f92e0005960 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:00.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.283+0000 7f92ed7fa700 1 -- 172.21.15.27:0/2230204127 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f92e8021020 con 0x7f92f0107450 2024-06-06T08:42:00.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:00 smithi027 ceph-mon[32335]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-06T08:42:00.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.578+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92f002dca0 con 0x7f92f0107450 2024-06-06T08:42:00.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.579+0000 7f92ed7fa700 1 -- 172.21.15.27:0/2230204127 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11882 (secure 0 0 0) 0x7f92e809b480 con 0x7f92f0107450 2024-06-06T08:42:00.577 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:42:00.578 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:40:15.140203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/266251643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":266251643},{"type":"v1","addr":"172.21.15.203:6837","nonce":266251643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92d80a2e10 msgr2=0x7f92d80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92d80a2e10 0x7f92d80a52d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f92e0006010 tx=0x7f92e0005960 comp rx=0 tx=0).stop 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 msgr2=0x7f92f0075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 0x7f92f0075210 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f92e800aa90 tx=0x7f92e800ada0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 shutdown_connections 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0107450 0x7f92f0075210 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f0102540 0x7f92f0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92d80a2e10 0x7f92d80a52d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 --2- 172.21.15.27:0/2230204127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f0102ee0 0x7f92f0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 >> 172.21.15.27:0/2230204127 conn(0x7f92f00795f0 msgr2=0x7f92f0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:00.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 shutdown_connections 2024-06-06T08:42:00.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:00.581+0000 7f92f687b700 1 -- 172.21.15.27:0/2230204127 wait complete. 2024-06-06T08:42:00.581 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-06-06T08:42:00.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:00 smithi203 ceph-mon[36579]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-06T08:42:00.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:00 smithi176 ceph-mon[36786]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-06T08:42:01.621 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id ef5f116f --release --await' 2024-06-06T08:42:01.621 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id ef5f116f --release --await 2024-06-06T08:42:01.685 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:01 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2230204127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:42:01.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:01 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2230204127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:42:01.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:01 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2230204127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:42:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.859+0000 7f36ba83f700 1 -- 172.21.15.27:0/2720465596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 msgr2=0x7f36b41077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.859+0000 7f36ba83f700 1 --2- 172.21.15.27:0/2720465596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 0x7f36b41077b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009990 tx=0x7f36a4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.859+0000 7f36ba83f700 1 -- 172.21.15.27:0/2720465596 shutdown_connections 2024-06-06T08:42:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.859+0000 7f36ba83f700 1 --2- 172.21.15.27:0/2720465596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4101d30 0x7f36b410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.859+0000 7f36ba83f700 1 --2- 172.21.15.27:0/2720465596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 0x7f36b41077b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.859+0000 7f36ba83f700 1 --2- 172.21.15.27:0/2720465596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b4101390 0x7f36b41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.859+0000 7f36ba83f700 1 -- 172.21.15.27:0/2720465596 >> 172.21.15.27:0/2720465596 conn(0x7f36b40782c0 msgr2=0x7f36b40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.859+0000 7f36ba83f700 1 -- 172.21.15.27:0/2720465596 shutdown_connections 2024-06-06T08:42:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.859+0000 7f36ba83f700 1 -- 172.21.15.27:0/2720465596 wait complete. 2024-06-06T08:42:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.860+0000 7f36ba83f700 1 Processor -- start 2024-06-06T08:42:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.860+0000 7f36ba83f700 1 -- start start 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36ba83f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4101390 0x7f36b419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36ba83f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b4101d30 0x7f36b419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36ba83f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 0x7f36b4196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36ba83f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41105d0 con 0x7f36b41073d0 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36ba83f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4110450 con 0x7f36b4101390 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36ba83f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4110750 con 0x7f36b4101d30 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b8ddc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 0x7f36b4196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b8ddc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 0x7f36b4196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41200/0 (socket says 172.21.15.27:41200) 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b8ddc700 1 -- 172.21.15.27:0/1974111138 learned_addr learned my addr 172.21.15.27:0/1974111138 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4101390 0x7f36b419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b37fe700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b4101d30 0x7f36b419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b8ddc700 1 -- 172.21.15.27:0/1974111138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b4101d30 msgr2=0x7f36b419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b8ddc700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b4101d30 0x7f36b419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b8ddc700 1 -- 172.21.15.27:0/1974111138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4101390 msgr2=0x7f36b419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b8ddc700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4101390 0x7f36b419c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.861+0000 7f36b8ddc700 1 -- 172.21.15.27:0/1974111138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b4197050 con 0x7f36b41073d0 2024-06-06T08:42:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.862+0000 7f36b8ddc700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 0x7f36b4196870 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f36ac012370 tx=0x7f36ac00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.862+0000 7f36b17fa700 1 -- 172.21.15.27:0/1974111138 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac012ea0 con 0x7f36b41073d0 2024-06-06T08:42:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.862+0000 7f36b17fa700 1 -- 172.21.15.27:0/1974111138 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ac004bb0 con 0x7f36b41073d0 2024-06-06T08:42:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.862+0000 7f36b17fa700 1 -- 172.21.15.27:0/1974111138 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac004440 con 0x7f36b41073d0 2024-06-06T08:42:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.862+0000 7f36ba83f700 1 -- 172.21.15.27:0/1974111138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b41972e0 con 0x7f36b41073d0 2024-06-06T08:42:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.862+0000 7f36ba83f700 1 -- 172.21.15.27:0/1974111138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b41a3500 con 0x7f36b41073d0 2024-06-06T08:42:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.864+0000 7f36b17fa700 1 -- 172.21.15.27:0/1974111138 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f36ac01f030 con 0x7f36b41073d0 2024-06-06T08:42:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.865+0000 7f36b17fa700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f369c0a2e10 0x7f369c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.865+0000 7f36b17fa700 1 -- 172.21.15.27:0/1974111138 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f36ac0d3160 con 0x7f36b41073d0 2024-06-06T08:42:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.865+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f369c0a2e10 0x7f369c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.866+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f369c0a2e10 0x7f369c0a52d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36a4006010 tx=0x7f36a400b260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:01.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.867+0000 7f36c50c2b80 1 Processor -- start 2024-06-06T08:42:01.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.867+0000 7f36c50c2b80 1 -- start start 2024-06-06T08:42:01.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.867+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b17a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.867+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 0x557613b119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.867+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b11f20 0x557613b12340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.867+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557613a63810 con 0x557613b17640 2024-06-06T08:42:01.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.867+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557613a0b720 con 0x557613b17f60 2024-06-06T08:42:01.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.867+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557613a5f7d0 con 0x557613b11f20 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b17a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b17a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41216/0 (socket says 172.21.15.27:41216) 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b3fff700 1 -- 172.21.15.27:0/1918083093 learned_addr learned my addr 172.21.15.27:0/1918083093 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b37fe700 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 0x557613b119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b8ddc700 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b11f20 0x557613b12340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b3fff700 1 -- 172.21.15.27:0/1918083093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b11f20 msgr2=0x557613b12340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b11f20 0x557613b12340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b3fff700 1 -- 172.21.15.27:0/1918083093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 msgr2=0x557613b119e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 0x557613b119e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b3fff700 1 -- 172.21.15.27:0/1918083093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557613a01f50 con 0x557613b17640 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b17a20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f36a400bef0 tx=0x7f36a401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6800f57b8de6db20 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.868+0000 7f36b8ddc700 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b11f20 0x557613b12340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:01.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.869+0000 7f369a7fc700 1 -- 172.21.15.27:0/1918083093 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4003680 con 0x557613b17640 2024-06-06T08:42:01.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.869+0000 7f369a7fc700 1 -- 172.21.15.27:0/1918083093 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a4003d00 con 0x557613b17640 2024-06-06T08:42:01.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.869+0000 7f369a7fc700 1 -- 172.21.15.27:0/1918083093 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a400fc80 con 0x557613b17640 2024-06-06T08:42:01.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1918083093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 msgr2=0x557613b17a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b17a20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f36a400bef0 tx=0x7f36a401f000 comp rx=0 tx=0).stop 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1918083093 shutdown_connections 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b11f20 0x557613b12340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b17a20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1918083093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 0x557613b119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1918083093 >> 172.21.15.27:0/1918083093 conn(0x557613a3ccc0 msgr2=0x557613b0c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1918083093 shutdown_connections 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1918083093 wait complete. 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.870+0000 7f36c50c2b80 1 -- start start 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36c50c2b80 1 Processor -- start 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b11f20 0x557613b1dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b1af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 0x557613b1b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36b37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b1af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557613b1e9b0 con 0x557613b17640 2024-06-06T08:42:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557613a03bf0 con 0x557613b11f20 2024-06-06T08:42:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36b37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b1af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41224/0 (socket says 172.21.15.27:41224) 2024-06-06T08:42:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557613b1eb30 con 0x557613b17f60 2024-06-06T08:42:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36b37fe700 1 -- 172.21.15.27:0/1990841174 learned_addr learned my addr 172.21.15.27:0/1990841174 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b11f20 0x557613b1dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.871+0000 7f36c50c2b80 10 client.0 _ll_register_callbacks cb 0x5576139fd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36b8ddc700 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 0x557613b1b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36b3fff700 1 -- 172.21.15.27:0/1990841174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 msgr2=0x557613b1b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 0x557613b1b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36b3fff700 1 -- 172.21.15.27:0/1990841174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 msgr2=0x557613b1af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b1af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36b3fff700 1 -- 172.21.15.27:0/1990841174 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576139b28d0 con 0x557613b11f20 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36b37fe700 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b1af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36b3fff700 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b11f20 0x557613b1dbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f36a402f040 tx=0x7f36a4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f186ddebd48fb41a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36937fe700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36937fe700 1 -- 172.21.15.27:0/1990841174 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4003890 con 0x557613b11f20 2024-06-06T08:42:01.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36937fe700 1 -- 172.21.15.27:0/1990841174 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f36a4003a30 con 0x557613b11f20 2024-06-06T08:42:01.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.872+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1990841174 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a4003d00 con 0x557613b11f20 2024-06-06T08:42:01.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36937fe700 1 -- 172.21.15.27:0/1990841174 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36a4022410 con 0x557613b11f20 2024-06-06T08:42:01.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36c50c2b80 10 client.25448 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:01.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1990841174 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557613b29230 con 0x557613b11f20 2024-06-06T08:42:01.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36937fe700 1 -- 172.21.15.27:0/1990841174 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a4003d00 con 0x557613b11f20 2024-06-06T08:42:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36937fe700 1 -- 172.21.15.27:0/1990841174 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36a4035400 con 0x557613b11f20 2024-06-06T08:42:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36c50c2b80 10 client.25448 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36c50c2b80 10 client.25448 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:42:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36c50c2b80 20 client.25448 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x557613b2b700 0x557613b2daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36c50c2b80 4 client.25448 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.873+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1990841174 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557613a70d10 con 0x557613b2b700 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.874+0000 7f36b37fe700 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x557613b2b700 0x557613b2daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.874+0000 7f36b37fe700 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x557613b2b700 0x557613b2daf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ca3c5ae95821dce server_cookie=a029118a4fdb615d in_seq=0 out_seq=0 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.874+0000 7f36937fe700 10 client.25448 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36937fe700 1 -- 172.21.15.27:0/1990841174 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557613a70d10 con 0x557613b2b700 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36937fe700 10 client.25448 handle_command_reply: tid=0 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 client.25448 shutdown 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1990841174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b11f20 msgr2=0x557613b1dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b11f20 0x557613b1dbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f36a402f040 tx=0x7f36a4005e80 comp rx=0 tx=0).stop 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1990841174 shutdown_connections 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 0x557613b1b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x557613b2b700 0x557613b2daf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b11f20 0x557613b1dbf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/1990841174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b17640 0x557613b1af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1990841174 >> 172.21.15.27:0/1990841174 conn(0x557613a3ccc0 msgr2=0x557613b109d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36937fe700 0 client.25448 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:42:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1990841174 shutdown_connections 2024-06-06T08:42:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 1 -- 172.21.15.27:0/1990841174 wait complete. 2024-06-06T08:42:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.875+0000 7f36c50c2b80 20 client.25448 trim_cache size 0 max 16384 2024-06-06T08:42:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36c50c2b80 1 Processor -- start 2024-06-06T08:42:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36c50c2b80 1 -- start start 2024-06-06T08:42:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17640 0x557613b1dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 0x557613b18bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41aec50 con 0x557613b11f20 2024-06-06T08:42:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41ac4e0 con 0x557613b17640 2024-06-06T08:42:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41aa330 con 0x557613b17f60 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b119e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41234/0 (socket says 172.21.15.27:41234) 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.898+0000 7f36b3fff700 1 -- 172.21.15.27:0/2498285813 learned_addr learned my addr 172.21.15.27:0/2498285813 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f36b8ddc700 1 --2- 172.21.15.27:0/2498285813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 0x557613b18bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f36b3fff700 1 -- 172.21.15.27:0/2498285813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 msgr2=0x557613b18bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f36b3fff700 1 --2- 172.21.15.27:0/2498285813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 0x557613b18bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f36b3fff700 1 -- 172.21.15.27:0/2498285813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17640 msgr2=0x557613b1dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f36b37fe700 1 --2- 172.21.15.27:0/2498285813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17640 0x557613b1dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f36b3fff700 1 --2- 172.21.15.27:0/2498285813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17640 0x557613b1dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f36b3fff700 1 -- 172.21.15.27:0/2498285813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557613b194f0 con 0x557613b11f20 2024-06-06T08:42:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f36b3fff700 1 --2- 172.21.15.27:0/2498285813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b119e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f36a400f790 tx=0x7f36a4035f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce87b57d2552286 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:01.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f3699ffb700 1 -- 172.21.15.27:0/2498285813 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4035c50 con 0x557613b11f20 2024-06-06T08:42:01.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f3699ffb700 1 -- 172.21.15.27:0/2498285813 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a4005bc0 con 0x557613b11f20 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.899+0000 7f3699ffb700 1 -- 172.21.15.27:0/2498285813 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4022810 con 0x557613b11f20 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.900+0000 7f36c50c2b80 1 -- 172.21.15.27:0/2498285813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 msgr2=0x557613b119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.900+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/2498285813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b119e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f36a400f790 tx=0x7f36a4035f60 comp rx=0 tx=0).stop 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.900+0000 7f36c50c2b80 1 -- 172.21.15.27:0/2498285813 shutdown_connections 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.900+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/2498285813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17f60 0x557613b18bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.900+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/2498285813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b119e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.900+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/2498285813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17640 0x557613b1dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.900+0000 7f36c50c2b80 1 -- 172.21.15.27:0/2498285813 >> 172.21.15.27:0/2498285813 conn(0x557613a3ccc0 msgr2=0x557613b05e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.901+0000 7f36c50c2b80 1 -- 172.21.15.27:0/2498285813 shutdown_connections 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.901+0000 7f36c50c2b80 1 -- 172.21.15.27:0/2498285813 wait complete. 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.901+0000 7f36c50c2b80 1 -- start start 2024-06-06T08:42:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.901+0000 7f36c50c2b80 1 Processor -- start 2024-06-06T08:42:01.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b2f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17640 0x557613b2fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36c50c2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 0x557613b2ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41aa9a0 con 0x557613b11f20 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41abe10 con 0x557613b17f60 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36c50c2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41ab070 con 0x557613b17640 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36c50c2b80 10 client.0 _ll_register_callbacks cb 0x5576139fd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b2f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b2f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41248/0 (socket says 172.21.15.27:41248) 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b3fff700 1 -- 172.21.15.27:0/811551852 learned_addr learned my addr 172.21.15.27:0/811551852 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b8ddc700 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 0x557613b2ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b37fe700 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17640 0x557613b2fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b3fff700 1 -- 172.21.15.27:0/811551852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17640 msgr2=0x557613b2fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b3fff700 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17640 0x557613b2fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b3fff700 1 -- 172.21.15.27:0/811551852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 msgr2=0x557613b2ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b3fff700 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 0x557613b2ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.902+0000 7f36b3fff700 1 -- 172.21.15.27:0/811551852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557613b940c0 con 0x557613b11f20 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.903+0000 7f36b3fff700 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b2f520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f36a4000c00 tx=0x7f36a400fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3a421b36ad4b43 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.903+0000 7f36937fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.903+0000 7f36937fe700 1 -- 172.21.15.27:0/811551852 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4047020 con 0x557613b11f20 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.903+0000 7f36937fe700 1 -- 172.21.15.27:0/811551852 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f36a4003aa0 con 0x557613b11f20 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.903+0000 7f36c50c2b80 1 -- 172.21.15.27:0/811551852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a4005bc0 con 0x557613b11f20 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.903+0000 7f36937fe700 1 -- 172.21.15.27:0/811551852 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36a4052050 con 0x557613b11f20 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.903+0000 7f36c50c2b80 10 client.15900 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.903+0000 7f36c50c2b80 1 -- 172.21.15.27:0/811551852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557613b953c0 con 0x557613b11f20 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.903+0000 7f36937fe700 1 -- 172.21.15.27:0/811551852 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a4005bc0 con 0x557613b11f20 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.904+0000 7f36937fe700 1 -- 172.21.15.27:0/811551852 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36a4041020 con 0x557613b11f20 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.904+0000 7f36c50c2b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.904+0000 7f36c50c2b80 10 client.15900 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:42:01.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.904+0000 7f36c50c2b80 20 client.15900 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:01.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.904+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x557613b95870 0x557613b97c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:01.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.904+0000 7f36c50c2b80 4 client.15900 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:42:01.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.904+0000 7f36c50c2b80 1 -- 172.21.15.27:0/811551852 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557613a6b290 con 0x557613b95870 2024-06-06T08:42:01.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.904+0000 7f36b37fe700 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x557613b95870 0x557613b97c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36937fe700 10 client.15900 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36b37fe700 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x557613b95870 0x557613b97c60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0c7ab152646ebc6 server_cookie=59e972c3733699d6 in_seq=0 out_seq=0 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36937fe700 1 -- 172.21.15.27:0/811551852 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557613a6b290 con 0x557613b95870 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36937fe700 10 client.15900 handle_command_reply: tid=0 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36c50c2b80 1 client.15900 shutdown 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36c50c2b80 1 -- 172.21.15.27:0/811551852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 msgr2=0x557613b2f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b2f520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f36a4000c00 tx=0x7f36a400fd40 comp rx=0 tx=0).stop 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36c50c2b80 1 -- 172.21.15.27:0/811551852 shutdown_connections 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x557613b95870 0x557613b97c60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557613b17f60 0x557613b2ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557613b11f20 0x557613b2f520 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36c50c2b80 1 --2- 172.21.15.27:0/811551852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557613b17640 0x557613b2fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36937fe700 0 client.15900 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:42:01.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.905+0000 7f36c50c2b80 1 -- 172.21.15.27:0/811551852 >> 172.21.15.27:0/811551852 conn(0x557613a3ccc0 msgr2=0x557613b1c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:01.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.906+0000 7f36c50c2b80 1 -- 172.21.15.27:0/811551852 shutdown_connections 2024-06-06T08:42:01.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.906+0000 7f36c50c2b80 1 -- 172.21.15.27:0/811551852 wait complete. 2024-06-06T08:42:01.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.906+0000 7f36c50c2b80 20 client.15900 trim_cache size 0 max 16384 2024-06-06T08:42:01.907 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:42:01.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.906+0000 7f36ba83f700 1 -- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f369c0a2e10 msgr2=0x7f369c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:01.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.906+0000 7f36ba83f700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f369c0a2e10 0x7f369c0a52d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36a4006010 tx=0x7f36a400b260 comp rx=0 tx=0).stop 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.906+0000 7f36ba83f700 1 -- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 msgr2=0x7f36b4196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.906+0000 7f36ba83f700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 0x7f36b4196870 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f36ac012370 tx=0x7f36ac00ef90 comp rx=0 tx=0).stop 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.907+0000 7f36ba83f700 1 -- 172.21.15.27:0/1974111138 shutdown_connections 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.907+0000 7f36ba83f700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36b41073d0 0x7f36b4196870 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.907+0000 7f36ba83f700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f369c0a2e10 0x7f369c0a52d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.907+0000 7f36ba83f700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f36b4101390 0x7f36b419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.907+0000 7f36ba83f700 1 --2- 172.21.15.27:0/1974111138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b4101d30 0x7f36b419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.907+0000 7f36ba83f700 1 -- 172.21.15.27:0/1974111138 >> 172.21.15.27:0/1974111138 conn(0x7f36b40782c0 msgr2=0x7f36b41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.907+0000 7f36ba83f700 1 -- 172.21.15.27:0/1974111138 shutdown_connections 2024-06-06T08:42:01.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:01.907+0000 7f36ba83f700 1 -- 172.21.15.27:0/1974111138 wait complete. 2024-06-06T08:42:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:42:02.674 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:02 smithi027 ceph-mon[32335]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:02.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T08:42:02.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:02 smithi203 ceph-mon[36579]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:02.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:02 smithi176 ceph-mon[36786]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:03.467 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:42:04.640 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:04 smithi027 ceph-mon[32335]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-06-06T08:42:04.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:04 smithi203 ceph-mon[36579]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-06-06T08:42:04.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:04 smithi176 ceph-mon[36786]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-06-06T08:42:04.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.861+0000 7f92f950d700 1 -- 172.21.15.27:0/1694887537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101d40 msgr2=0x7f92f410e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:04.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.861+0000 7f92f950d700 1 --2- 172.21.15.27:0/1694887537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101d40 0x7f92f410e200 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00b0a0 tx=0x7f92ec00b3b0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.861+0000 7f92f950d700 1 -- 172.21.15.27:0/1694887537 shutdown_connections 2024-06-06T08:42:04.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.861+0000 7f92f950d700 1 --2- 172.21.15.27:0/1694887537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101d40 0x7f92f410e200 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.861+0000 7f92f950d700 1 --2- 172.21.15.27:0/1694887537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f41073c0 0x7f92f41077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.861+0000 7f92f950d700 1 --2- 172.21.15.27:0/1694887537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f4101380 0x7f92f4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.861+0000 7f92f950d700 1 -- 172.21.15.27:0/1694887537 >> 172.21.15.27:0/1694887537 conn(0x7f92f4078330 msgr2=0x7f92f4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:04.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.862+0000 7f92f950d700 1 -- 172.21.15.27:0/1694887537 shutdown_connections 2024-06-06T08:42:04.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.862+0000 7f92f950d700 1 -- 172.21.15.27:0/1694887537 wait complete. 2024-06-06T08:42:04.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.863+0000 7f92f950d700 1 Processor -- start 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.863+0000 7f92f950d700 1 -- start start 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.863+0000 7f92f950d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101380 0x7f92f4103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.863+0000 7f92f950d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f4101d40 0x7f92f4103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.863+0000 7f92f950d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f41073c0 0x7f92f4103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.863+0000 7f92f950d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4111660 con 0x7f92f4101380 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.863+0000 7f92f950d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41114e0 con 0x7f92f41073c0 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.863+0000 7f92f950d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41117e0 con 0x7f92f4101d40 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101380 0x7f92f4103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101380 0x7f92f4103030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41270/0 (socket says 172.21.15.27:41270) 2024-06-06T08:42:04.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f2ffd700 1 -- 172.21.15.27:0/1761574792 learned_addr learned my addr 172.21.15.27:0/1761574792 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f27fc700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f4101d40 0x7f92f4103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f37fe700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f41073c0 0x7f92f4103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f2ffd700 1 -- 172.21.15.27:0/1761574792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f4101d40 msgr2=0x7f92f4103570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f4101d40 0x7f92f4103570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f2ffd700 1 -- 172.21.15.27:0/1761574792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f41073c0 msgr2=0x7f92f4103ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f41073c0 0x7f92f4103ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f2ffd700 1 -- 172.21.15.27:0/1761574792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f41a7100 con 0x7f92f4101380 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101380 0x7f92f4103030 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f92e400d950 tx=0x7f92e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92ebfff700 1 -- 172.21.15.27:0/1761574792 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4016070 con 0x7f92f4101380 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92ebfff700 1 -- 172.21.15.27:0/1761574792 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e400fd70 con 0x7f92f4101380 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f950d700 1 -- 172.21.15.27:0/1761574792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f41a73f0 con 0x7f92f4101380 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92ebfff700 1 -- 172.21.15.27:0/1761574792 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e400e610 con 0x7f92f4101380 2024-06-06T08:42:04.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.864+0000 7f92f950d700 1 -- 172.21.15.27:0/1761574792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f41a7930 con 0x7f92f4101380 2024-06-06T08:42:04.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.867+0000 7f92ebfff700 1 -- 172.21.15.27:0/1761574792 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f92e4019070 con 0x7f92f4101380 2024-06-06T08:42:04.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.868+0000 7f92ebfff700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92e00a2d90 0x7f92e00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.868+0000 7f92ebfff700 1 -- 172.21.15.27:0/1761574792 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f92e40cf280 con 0x7f92f4101380 2024-06-06T08:42:04.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.868+0000 7f92f27fc700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92e00a2d90 0x7f92e00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.868+0000 7f92f27fc700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92e00a2d90 0x7f92e00a5250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f92f40ffa50 tx=0x7f92dc00b3c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:04.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.868+0000 7f9303d90b80 1 Processor -- start 2024-06-06T08:42:04.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f9303d90b80 1 -- start start 2024-06-06T08:42:04.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9702020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 0x5556f96fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f96fc3d0 0x5556f96fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f9303d90b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5556f964d910 con 0x5556f9701c40 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f9303d90b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5556f95f5820 con 0x5556f9702560 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f9303d90b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5556f96498d0 con 0x5556f96fc3d0 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9702020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9702020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41278/0 (socket says 172.21.15.27:41278) 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f2ffd700 1 -- 172.21.15.27:0/2069756805 learned_addr learned my addr 172.21.15.27:0/2069756805 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f27fc700 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 0x5556f96fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f37fe700 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f96fc3d0 0x5556f96fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f2ffd700 1 -- 172.21.15.27:0/2069756805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f96fc3d0 msgr2=0x5556f96fc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:04.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f96fc3d0 0x5556f96fc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f2ffd700 1 -- 172.21.15.27:0/2069756805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 msgr2=0x5556f96fbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:04.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 0x5556f96fbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f2ffd700 1 -- 172.21.15.27:0/2069756805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f95ec050 con 0x5556f9701c40 2024-06-06T08:42:04.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f37fe700 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f96fc3d0 0x5556f96fc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:42:04.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.869+0000 7f92f27fc700 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 0x5556f96fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:04.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.870+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9702020 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f92e4097840 tx=0x7f92e40977b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513430252dbab6f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:04.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.870+0000 7f92e97fa700 1 -- 172.21.15.27:0/2069756805 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4020070 con 0x5556f9701c40 2024-06-06T08:42:04.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.870+0000 7f92e97fa700 1 -- 172.21.15.27:0/2069756805 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e409f790 con 0x5556f9701c40 2024-06-06T08:42:04.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.870+0000 7f92e97fa700 1 -- 172.21.15.27:0/2069756805 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e409b840 con 0x5556f9701c40 2024-06-06T08:42:04.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 -- 172.21.15.27:0/2069756805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 msgr2=0x5556f9702020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:04.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9702020 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f92e4097840 tx=0x7f92e40977b0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 -- 172.21.15.27:0/2069756805 shutdown_connections 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f96fc3d0 0x5556f96fc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9702020 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2069756805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 0x5556f96fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 -- 172.21.15.27:0/2069756805 >> 172.21.15.27:0/2069756805 conn(0x5556f9626dc0 msgr2=0x5556f96f7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 -- 172.21.15.27:0/2069756805 shutdown_connections 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 -- 172.21.15.27:0/2069756805 wait complete. 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 -- start start 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.871+0000 7f9303d90b80 1 Processor -- start 2024-06-06T08:42:04.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f96fc3d0 0x5556f97082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9705610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9702560 0x5556f9705b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f9303d90b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5556f9708f10 con 0x5556f9701c40 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f9303d90b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5556f95edcf0 con 0x5556f96fc3d0 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f9303d90b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5556f9709090 con 0x5556f9702560 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f9303d90b80 10 client.0 _ll_register_callbacks cb 0x5556f95e7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9705610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9705610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41286/0 (socket says 172.21.15.27:41286) 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f27fc700 1 -- 172.21.15.27:0/4142309244 learned_addr learned my addr 172.21.15.27:0/4142309244 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f96fc3d0 0x5556f97082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f37fe700 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9702560 0x5556f9705b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f27fc700 1 -- 172.21.15.27:0/4142309244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9702560 msgr2=0x5556f9705b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f27fc700 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9702560 0x5556f9705b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f27fc700 1 -- 172.21.15.27:0/4142309244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f96fc3d0 msgr2=0x5556f97082a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f27fc700 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f96fc3d0 0x5556f97082a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f27fc700 1 -- 172.21.15.27:0/4142309244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f959c9d0 con 0x5556f9701c40 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92f27fc700 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9705610 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f92dc004130 tx=0x7f92dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4e22a908553073 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.872+0000 7f92da7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f92da7fc700 1 -- 172.21.15.27:0/4142309244 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc011db0 con 0x5556f9701c40 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f92da7fc700 1 -- 172.21.15.27:0/4142309244 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f92dc028470 con 0x5556f9701c40 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f9303d90b80 1 -- 172.21.15.27:0/4142309244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e409f790 con 0x5556f9701c40 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f92da7fc700 1 -- 172.21.15.27:0/4142309244 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92dc031050 con 0x5556f9701c40 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f9303d90b80 10 client.15912 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:04.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f9303d90b80 1 -- 172.21.15.27:0/4142309244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5556f9713860 con 0x5556f9701c40 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f92da7fc700 1 -- 172.21.15.27:0/4142309244 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e409f790 con 0x5556f9701c40 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f92da7fc700 1 -- 172.21.15.27:0/4142309244 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f92dc01e690 con 0x5556f9701c40 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f9303d90b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f9303d90b80 10 client.15912 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f9303d90b80 20 client.15912 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f9303d90b80 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5556f9715ea0 0x5556f9718290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f9303d90b80 4 client.15912 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.873+0000 7f9303d90b80 1 -- 172.21.15.27:0/4142309244 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556f965ae10 con 0x5556f9715ea0 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.874+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5556f9715ea0 0x5556f9718290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.874+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5556f9715ea0 0x5556f9718290 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7747f72cce4b3c9b server_cookie=8f71f910c9681099 in_seq=0 out_seq=0 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.874+0000 7f92da7fc700 10 client.15912 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:04.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.875+0000 7f92da7fc700 1 -- 172.21.15.27:0/4142309244 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5556f965ae10 con 0x5556f9715ea0 2024-06-06T08:42:04.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.875+0000 7f92da7fc700 10 client.15912 handle_command_reply: tid=0 2024-06-06T08:42:04.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.875+0000 7f9303d90b80 1 client.15912 shutdown 2024-06-06T08:42:04.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.875+0000 7f9303d90b80 1 -- 172.21.15.27:0/4142309244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 msgr2=0x5556f9705610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:04.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.875+0000 7f9303d90b80 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9705610 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f92dc004130 tx=0x7f92dc0047c0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.875+0000 7f9303d90b80 1 -- 172.21.15.27:0/4142309244 shutdown_connections 2024-06-06T08:42:04.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.875+0000 7f9303d90b80 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9702560 0x5556f9705b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.875+0000 7f9303d90b80 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f96fc3d0 0x5556f97082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.876+0000 7f9303d90b80 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5556f9715ea0 0x5556f9718290 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.876+0000 7f9303d90b80 1 --2- 172.21.15.27:0/4142309244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9701c40 0x5556f9705610 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.876+0000 7f92da7fc700 0 client.15912 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:04.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.876+0000 7f9303d90b80 1 -- 172.21.15.27:0/4142309244 >> 172.21.15.27:0/4142309244 conn(0x5556f9626dc0 msgr2=0x5556f96f6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:04.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.876+0000 7f9303d90b80 1 -- 172.21.15.27:0/4142309244 shutdown_connections 2024-06-06T08:42:04.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.876+0000 7f9303d90b80 1 -- 172.21.15.27:0/4142309244 wait complete. 2024-06-06T08:42:04.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.876+0000 7f9303d90b80 20 client.15912 trim_cache size 0 max 16384 2024-06-06T08:42:04.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.910+0000 7f9303d90b80 1 Processor -- start 2024-06-06T08:42:04.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.910+0000 7f9303d90b80 1 -- start start 2024-06-06T08:42:04.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f96fc3d0 0x5556f96fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9701c40 0x5556f96f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 0x5556f971db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f9303d90b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41b3140 con 0x5556f96fc3d0 2024-06-06T08:42:04.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f9303d90b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41b09d0 con 0x5556f9702560 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f9303d90b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41ae820 con 0x5556f9701c40 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f96fc3d0 0x5556f96fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f92f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f96fc3d0 0x5556f96fbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41300/0 (socket says 172.21.15.27:41300) 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f92f2ffd700 1 -- 172.21.15.27:0/3982292297 learned_addr learned my addr 172.21.15.27:0/3982292297 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f92f37fe700 1 --2- 172.21.15.27:0/3982292297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 0x5556f971db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f92f2ffd700 1 -- 172.21.15.27:0/3982292297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9701c40 msgr2=0x5556f96f6ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/3982292297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9701c40 0x5556f96f6ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f92f2ffd700 1 -- 172.21.15.27:0/3982292297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 msgr2=0x5556f971db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/3982292297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 0x5556f971db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.911+0000 7f92f2ffd700 1 -- 172.21.15.27:0/3982292297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f96f7660 con 0x5556f96fc3d0 2024-06-06T08:42:04.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.912+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/3982292297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f96fc3d0 0x5556f96fbe90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f92e4099690 tx=0x7f92e4097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff2b1684cf32780 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:04.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.912+0000 7f92e8ff9700 1 -- 172.21.15.27:0/3982292297 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e40b4020 con 0x5556f96fc3d0 2024-06-06T08:42:04.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.912+0000 7f92e8ff9700 1 -- 172.21.15.27:0/3982292297 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e409f790 con 0x5556f96fc3d0 2024-06-06T08:42:04.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.912+0000 7f92e8ff9700 1 -- 172.21.15.27:0/3982292297 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4020040 con 0x5556f96fc3d0 2024-06-06T08:42:04.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.913+0000 7f9303d90b80 1 -- 172.21.15.27:0/3982292297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f96fc3d0 msgr2=0x5556f96fbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:04.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.913+0000 7f9303d90b80 1 --2- 172.21.15.27:0/3982292297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f96fc3d0 0x5556f96fbe90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f92e4099690 tx=0x7f92e4097040 comp rx=0 tx=0).stop 2024-06-06T08:42:04.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.913+0000 7f9303d90b80 1 -- 172.21.15.27:0/3982292297 shutdown_connections 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.913+0000 7f9303d90b80 1 --2- 172.21.15.27:0/3982292297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f9702560 0x5556f971db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.913+0000 7f9303d90b80 1 --2- 172.21.15.27:0/3982292297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f96fc3d0 0x5556f96fbe90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.913+0000 7f9303d90b80 1 --2- 172.21.15.27:0/3982292297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9701c40 0x5556f96f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.913+0000 7f9303d90b80 1 -- 172.21.15.27:0/3982292297 >> 172.21.15.27:0/3982292297 conn(0x5556f9626dc0 msgr2=0x5556f96f68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.913+0000 7f9303d90b80 1 -- 172.21.15.27:0/3982292297 shutdown_connections 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.913+0000 7f9303d90b80 1 -- 172.21.15.27:0/3982292297 wait complete. 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f9303d90b80 1 -- start start 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f9303d90b80 1 Processor -- start 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f96fc3d0 0x5556f970b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9701c40 0x5556f970b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f9303d90b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9702560 0x5556f970bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f92f37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9702560 0x5556f970bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f92f37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9702560 0x5556f970bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41308/0 (socket says 172.21.15.27:41308) 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f9303d90b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41aee90 con 0x5556f9702560 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f92f37fe700 1 -- 172.21.15.27:0/2324049939 learned_addr learned my addr 172.21.15.27:0/2324049939 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41b0300 con 0x5556f96fc3d0 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.914+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41af560 con 0x5556f9701c40 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.915+0000 7f92f27fc700 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9701c40 0x5556f970b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.915+0000 7f9303d90b80 10 client.0 _ll_register_callbacks cb 0x5556f96f1980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.915+0000 7f92f37fe700 1 -- 172.21.15.27:0/2324049939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9701c40 msgr2=0x5556f970b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.915+0000 7f92f37fe700 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9701c40 0x5556f970b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.915+0000 7f92f37fe700 1 -- 172.21.15.27:0/2324049939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f96fc3d0 msgr2=0x5556f970b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:04.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.915+0000 7f92f37fe700 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f96fc3d0 0x5556f970b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.915+0000 7f92f37fe700 1 -- 172.21.15.27:0/2324049939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f9780b40 con 0x5556f9702560 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.915+0000 7f92f37fe700 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9702560 0x5556f970bda0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d010 tx=0x7f92ec003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0b400a8906a15f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.915+0000 7f92da7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.916+0000 7f92da7fc700 1 -- 172.21.15.27:0/2324049939 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0076f0 con 0x5556f9702560 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.916+0000 7f92da7fc700 1 -- 172.21.15.27:0/2324049939 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f92ec007890 con 0x5556f9702560 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.916+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e409f790 con 0x5556f9702560 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.916+0000 7f92da7fc700 1 -- 172.21.15.27:0/2324049939 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92ec031050 con 0x5556f9702560 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.916+0000 7f9303d90b80 10 client.15918 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.916+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5556f9781ca0 con 0x5556f9702560 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.916+0000 7f92da7fc700 1 -- 172.21.15.27:0/2324049939 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92ec005040 con 0x5556f9702560 2024-06-06T08:42:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.917+0000 7f92da7fc700 1 -- 172.21.15.27:0/2324049939 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f92ec011070 con 0x5556f9702560 2024-06-06T08:42:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.917+0000 7f9303d90b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.917+0000 7f9303d90b80 10 client.15918 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:42:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.917+0000 7f9303d90b80 20 client.15918 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.917+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5556f9782400 0x5556f9784840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.917+0000 7f9303d90b80 4 client.15918 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ef5f116f", "release": true, "await": true}] 2024-06-06T08:42:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.917+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ef5f116f", "release": true, "await": true}) -- 0x7f92e409f790 con 0x5556f9782400 2024-06-06T08:42:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.917+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5556f9782400 0x5556f9784840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.918+0000 7f92da7fc700 10 client.15918 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.918+0000 7f92f2ffd700 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5556f9782400 0x5556f9784840 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1654e2ac648823ec server_cookie=d500b9d0714ed2bf in_seq=0 out_seq=0 2024-06-06T08:42:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.919+0000 7f92da7fc700 1 -- 172.21.15.27:0/2324049939 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f92e409f790 con 0x5556f9782400 2024-06-06T08:42:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.919+0000 7f92da7fc700 10 client.15918 handle_command_reply: tid=0 2024-06-06T08:42:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.919+0000 7f9303d90b80 1 client.15918 shutdown 2024-06-06T08:42:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9702560 msgr2=0x5556f970bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9702560 0x5556f970bda0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d010 tx=0x7f92ec003ca0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 shutdown_connections 2024-06-06T08:42:04.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5556f9782400 0x5556f9784840 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5556f9702560 0x5556f970bda0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556f96fc3d0 0x5556f970b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f9303d90b80 1 --2- 172.21.15.27:0/2324049939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5556f9701c40 0x5556f970b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 >> 172.21.15.27:0/2324049939 conn(0x5556f9626dc0 msgr2=0x5556f9702940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f92da7fc700 0 client.15918 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.920+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 shutdown_connections 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.921+0000 7f9303d90b80 1 -- 172.21.15.27:0/2324049939 wait complete. 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.921+0000 7f9303d90b80 20 client.15918 trim_cache size 0 max 16384 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:42:04.921 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 25, 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24482, 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 4, 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "ef5f116f": { 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 4, 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-06-06T08:42:04.922 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 154.4, 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:42:04.923 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:42:04.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 -- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92e00a2d90 msgr2=0x7f92e00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:04.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92e00a2d90 0x7f92e00a5250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f92f40ffa50 tx=0x7f92dc00b3c0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 -- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101380 msgr2=0x7f92f4103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:04.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101380 0x7f92f4103030 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f92e400d950 tx=0x7f92e400dc60 comp rx=0 tx=0).stop 2024-06-06T08:42:04.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 -- 172.21.15.27:0/1761574792 shutdown_connections 2024-06-06T08:42:04.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f92e00a2d90 0x7f92e00a5250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92f41073c0 0x7f92f4103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f4101380 0x7f92f4103030 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 --2- 172.21.15.27:0/1761574792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92f4101d40 0x7f92f4103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 -- 172.21.15.27:0/1761574792 >> 172.21.15.27:0/1761574792 conn(0x7f92f4078330 msgr2=0x7f92f4100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 -- 172.21.15.27:0/1761574792 shutdown_connections 2024-06-06T08:42:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:04.922+0000 7f92f950d700 1 -- 172.21.15.27:0/1761574792 wait complete. 2024-06-06T08:42:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 -- 172.21.15.27:0/2290168383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 msgr2=0x7fb5e810e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 --2- 172.21.15.27:0/2290168383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 0x7fb5e810e410 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009ca0 tx=0x7fb5e002a9e0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 -- 172.21.15.27:0/2290168383 shutdown_connections 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 --2- 172.21.15.27:0/2290168383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 0x7fb5e810e410 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 --2- 172.21.15.27:0/2290168383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e8103550 0x7fb5e810ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 --2- 172.21.15.27:0/2290168383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5e8102c30 0x7fb5e8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 -- 172.21.15.27:0/2290168383 >> 172.21.15.27:0/2290168383 conn(0x7fb5e8077a20 msgr2=0x7fb5e8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 -- 172.21.15.27:0/2290168383 shutdown_connections 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 -- 172.21.15.27:0/2290168383 wait complete. 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 Processor -- start 2024-06-06T08:42:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.077+0000 7fb5ee596640 1 -- start start 2024-06-06T08:42:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ee596640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5e8102c30 0x7fb5e81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ee596640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e8103550 0x7fb5e81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ee596640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 0x7fb5e819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ee596640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8111d70 con 0x7fb5e810bf70 2024-06-06T08:42:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ee596640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8111bf0 con 0x7fb5e8102c30 2024-06-06T08:42:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ee596640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8111ef0 con 0x7fb5e8103550 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ecb33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 0x7fb5e819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ecb33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 0x7fb5e819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41318/0 (socket says 172.21.15.27:41318) 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ecb33640 1 -- 172.21.15.27:0/1301234895 learned_addr learned my addr 172.21.15.27:0/1301234895 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5e8102c30 0x7fb5e81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e8103550 0x7fb5e81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ecb33640 1 -- 172.21.15.27:0/1301234895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e8103550 msgr2=0x7fb5e81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ecb33640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e8103550 0x7fb5e81986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ecb33640 1 -- 172.21.15.27:0/1301234895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5e8102c30 msgr2=0x7fb5e81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ecb33640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5e8102c30 0x7fb5e81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ecb33640 1 -- 172.21.15.27:0/1301234895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e819d1a0 con 0x7fb5e810bf70 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.078+0000 7fb5ecb33640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 0x7fb5e819ca70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006010 tx=0x7fb5e0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.079+0000 7fb5e57fa640 1 -- 172.21.15.27:0/1301234895 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0047070 con 0x7fb5e810bf70 2024-06-06T08:42:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.079+0000 7fb5e57fa640 1 -- 172.21.15.27:0/1301234895 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e002fb20 con 0x7fb5e810bf70 2024-06-06T08:42:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.079+0000 7fb5ee596640 1 -- 172.21.15.27:0/1301234895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e819d430 con 0x7fb5e810bf70 2024-06-06T08:42:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.079+0000 7fb5ee596640 1 -- 172.21.15.27:0/1301234895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81a4d20 con 0x7fb5e810bf70 2024-06-06T08:42:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.079+0000 7fb5e57fa640 1 -- 172.21.15.27:0/1301234895 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0033050 con 0x7fb5e810bf70 2024-06-06T08:42:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.082+0000 7fb5f1377740 1 Processor -- start 2024-06-06T08:42:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.082+0000 7fb5e57fa640 1 -- 172.21.15.27:0/1301234895 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb5e004a070 con 0x7fb5e810bf70 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.082+0000 7fb5f1377740 1 -- start start 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.082+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f36c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.082+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f376e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.082+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 0x55a31f379740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5f1377740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a31f2d04f0 con 0x55a31f36ca90 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5f1377740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a31f26e870 con 0x55a31f36c170 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5f1377740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a31f2cc4b0 con 0x55a31f377350 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f376e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f36c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f376e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41326/0 (socket says 172.21.15.27:41326) 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e7fff640 1 -- 172.21.15.27:0/613875320 learned_addr learned my addr 172.21.15.27:0/613875320 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/613875320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 0x55a31f379740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e7fff640 1 -- 172.21.15.27:0/613875320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 msgr2=0x55a31f379740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/613875320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 0x55a31f379740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e7fff640 1 -- 172.21.15.27:0/613875320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 msgr2=0x55a31f36c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/613875320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f36c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e7fff640 1 -- 172.21.15.27:0/613875320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31f28a9b0 con 0x55a31f36ca90 2024-06-06T08:42:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e57fa640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5d00a2ea0 0x7fb5d00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/613875320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f376e10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800bd10 tx=0x7fb5d80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708ff92d3621c9b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e57fa640 1 -- 172.21.15.27:0/1301234895 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb5e00f0320 con 0x7fb5e810bf70 2024-06-06T08:42:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.083+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5d00a2ea0 0x7fb5d00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.084+0000 7fb5ce7fc640 1 -- 172.21.15.27:0/613875320 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8004030 con 0x55a31f36ca90 2024-06-06T08:42:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.084+0000 7fb5ce7fc640 1 -- 172.21.15.27:0/613875320 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d80041d0 con 0x55a31f36ca90 2024-06-06T08:42:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.084+0000 7fb5ce7fc640 1 -- 172.21.15.27:0/613875320 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8011800 con 0x55a31f36ca90 2024-06-06T08:42:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5d00a2ea0 0x7fb5d00a5360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55a31f369aa0 tx=0x7fb5dc0046c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 -- 172.21.15.27:0/613875320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 msgr2=0x55a31f376e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 --2- 172.21.15.27:0/613875320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f376e10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800bd10 tx=0x7fb5d80107a0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 -- 172.21.15.27:0/613875320 shutdown_connections 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 --2- 172.21.15.27:0/613875320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 0x55a31f379740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 --2- 172.21.15.27:0/613875320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f376e10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 --2- 172.21.15.27:0/613875320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f36c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 -- 172.21.15.27:0/613875320 >> 172.21.15.27:0/613875320 conn(0x55a31f368680 msgr2=0x55a31f368a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 -- 172.21.15.27:0/613875320 shutdown_connections 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 -- 172.21.15.27:0/613875320 wait complete. 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 -- start start 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.085+0000 7fb5f1377740 1 Processor -- start 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f374320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f374860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 0x55a31f374da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.088 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5f1377740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a31f379f80 con 0x55a31f36ca90 2024-06-06T08:42:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5f1377740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55a31f259ae0 con 0x55a31f36c170 2024-06-06T08:42:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5f1377740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a31f37a100 con 0x55a31f377350 2024-06-06T08:42:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5f1377740 10 client.0 _ll_register_callbacks cb 0x55a31f2929b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:05.089 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f374860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f374860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41328/0 (socket says 172.21.15.27:41328) 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5e77fe640 1 -- 172.21.15.27:0/3006747495 learned_addr learned my addr 172.21.15.27:0/3006747495 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f374320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5ecb33640 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 0x55a31f374da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5e77fe640 1 -- 172.21.15.27:0/3006747495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 msgr2=0x55a31f374da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 0x55a31f374da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5e77fe640 1 -- 172.21.15.27:0/3006747495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 msgr2=0x55a31f374320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f374320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5e77fe640 1 -- 172.21.15.27:0/3006747495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31efe4210 con 0x55a31f36ca90 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.086+0000 7fb5ecb33640 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 0x55a31f374da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.087+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f374860 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc010fd0 tx=0x7fb5dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585bb50bf828dab3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:05.090 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.087+0000 7fb5c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.087+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3006747495 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc019040 con 0x55a31f36ca90 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.087+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3006747495 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb5dc004020 con 0x55a31f36ca90 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.087+0000 7fb5f1377740 1 -- 172.21.15.27:0/3006747495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5d80041d0 con 0x55a31f36ca90 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.087+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3006747495 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5dc036050 con 0x55a31f36ca90 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.087+0000 7fb5f1377740 10 client.15930 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.087+0000 7fb5f1377740 1 -- 172.21.15.27:0/3006747495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a31f0b0620 con 0x55a31f36ca90 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.087+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3006747495 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5d80041d0 con 0x55a31f36ca90 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3006747495 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb5dc003880 con 0x55a31f36ca90 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5f1377740 10 client.15930 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5f1377740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5f1377740 20 client.15930 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a31f387380 0x55a31f3897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5f1377740 4 client.15930 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5f1377740 1 -- 172.21.15.27:0/3006747495 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a31f2e3f10 con 0x55a31f387380 2024-06-06T08:42:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a31f387380 0x55a31f3897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5c77fe640 10 client.15930 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:05.092 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.088+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a31f387380 0x55a31f3897d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac6fa5bdfa18e0fd server_cookie=a88e9b46b2b0aff7 in_seq=0 out_seq=0 2024-06-06T08:42:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.089+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3006747495 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a31f2e3f10 con 0x55a31f387380 2024-06-06T08:42:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.089+0000 7fb5c77fe640 10 client.15930 handle_command_reply: tid=0 2024-06-06T08:42:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.089+0000 7fb5f1377740 1 client.15930 shutdown 2024-06-06T08:42:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.089+0000 7fb5f1377740 1 -- 172.21.15.27:0/3006747495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 msgr2=0x55a31f374860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.089+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f374860 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc010fd0 tx=0x7fb5dc004730 comp rx=0 tx=0).stop 2024-06-06T08:42:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.089+0000 7fb5f1377740 1 -- 172.21.15.27:0/3006747495 shutdown_connections 2024-06-06T08:42:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.089+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a31f387380 0x55a31f3897d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.090+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f377350 0x55a31f374da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.090+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f374860 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.090+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3006747495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f374320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.090+0000 7fb5f1377740 1 -- 172.21.15.27:0/3006747495 >> 172.21.15.27:0/3006747495 conn(0x55a31f368680 msgr2=0x55a31f36e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.090+0000 7fb5c77fe640 0 client.15930 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.090+0000 7fb5f1377740 1 -- 172.21.15.27:0/3006747495 shutdown_connections 2024-06-06T08:42:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.090+0000 7fb5f1377740 1 -- 172.21.15.27:0/3006747495 wait complete. 2024-06-06T08:42:05.094 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.090+0000 7fb5f1377740 20 client.15930 trim_cache size 0 max 16384 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.105+0000 7fb5f1377740 1 Processor -- start 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.105+0000 7fb5f1377740 1 -- start start 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.105+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f3749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.105+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36ca90 0x55a31f38f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.105+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f377350 0x55a31f38f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5f1377740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81af520 con 0x55a31f377350 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5f1377740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81acdf0 con 0x55a31f36c170 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5f1377740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81aac40 con 0x55a31f36ca90 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5ecb33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f377350 0x55a31f38f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5ecb33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f377350 0x55a31f38f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41330/0 (socket says 172.21.15.27:41330) 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5ecb33640 1 -- 172.21.15.27:0/2685370962 learned_addr learned my addr 172.21.15.27:0/2685370962 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f3749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36ca90 0x55a31f38f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5ecb33640 1 -- 172.21.15.27:0/2685370962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36ca90 msgr2=0x55a31f38f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5ecb33640 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36ca90 0x55a31f38f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5ecb33640 1 -- 172.21.15.27:0/2685370962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 msgr2=0x55a31f3749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5ecb33640 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f3749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5ecb33640 1 -- 172.21.15.27:0/2685370962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31f372870 con 0x55a31f377350 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f3749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5ecb33640 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f377350 0x55a31f38f620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00ed6b0 tx=0x7fb5e00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd6e34ee270763a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5cdffb640 1 -- 172.21.15.27:0/2685370962 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00c07d0 con 0x55a31f377350 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5cdffb640 1 -- 172.21.15.27:0/2685370962 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e00c5090 con 0x55a31f377350 2024-06-06T08:42:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.106+0000 7fb5cdffb640 1 -- 172.21.15.27:0/2685370962 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e002e5f0 con 0x55a31f377350 2024-06-06T08:42:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.107+0000 7fb5f1377740 1 -- 172.21.15.27:0/2685370962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f377350 msgr2=0x55a31f38f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.107+0000 7fb5f1377740 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f377350 0x55a31f38f620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00ed6b0 tx=0x7fb5e00095c0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.107+0000 7fb5f1377740 1 -- 172.21.15.27:0/2685370962 shutdown_connections 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.107+0000 7fb5f1377740 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f377350 0x55a31f38f620 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.107+0000 7fb5f1377740 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36ca90 0x55a31f38f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.107+0000 7fb5f1377740 1 --2- 172.21.15.27:0/2685370962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f36c170 0x55a31f3749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.107+0000 7fb5f1377740 1 -- 172.21.15.27:0/2685370962 >> 172.21.15.27:0/2685370962 conn(0x55a31f368680 msgr2=0x55a31f3742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.107+0000 7fb5f1377740 1 -- 172.21.15.27:0/2685370962 shutdown_connections 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.107+0000 7fb5f1377740 1 -- 172.21.15.27:0/2685370962 wait complete. 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5f1377740 1 -- start start 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5f1377740 1 Processor -- start 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36c170 0x55a31f37f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f37acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5f1377740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f377350 0x55a31f37b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5f1377740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ab2b0 con 0x55a31f36ca90 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5f1377740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ac720 con 0x55a31f377350 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5f1377740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81ab980 con 0x55a31f36c170 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f37acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5f1377740 10 client.0 _ll_register_callbacks cb 0x7fb5dc010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f37acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41340/0 (socket says 172.21.15.27:41340) 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5e77fe640 1 -- 172.21.15.27:0/3100935652 learned_addr learned my addr 172.21.15.27:0/3100935652 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5ecb33640 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f377350 0x55a31f37b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.108+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36c170 0x55a31f37f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5e77fe640 1 -- 172.21.15.27:0/3100935652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36c170 msgr2=0x55a31f37f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36c170 0x55a31f37f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5e77fe640 1 -- 172.21.15.27:0/3100935652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f377350 msgr2=0x55a31f37b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f377350 0x55a31f37b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5e77fe640 1 -- 172.21.15.27:0/3100935652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31f37b7e0 con 0x55a31f36ca90 2024-06-06T08:42:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5e77fe640 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f37acb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc003f90 tx=0x7fb5dc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dcda7edda77813 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3100935652 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc03e030 con 0x55a31f36ca90 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3100935652 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb5dc028070 con 0x55a31f36ca90 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5f1377740 1 -- 172.21.15.27:0/3100935652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e00c5090 con 0x55a31f36ca90 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3100935652 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb5dc04b050 con 0x55a31f36ca90 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5f1377740 10 client.15936 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.109+0000 7fb5f1377740 1 -- 172.21.15.27:0/3100935652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a31f3f1ec0 con 0x55a31f36ca90 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.110+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3100935652 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e00c5090 con 0x55a31f36ca90 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.110+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3100935652 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb5dc00fb80 con 0x55a31f36ca90 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.110+0000 7fb5f1377740 10 client.15936 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.110+0000 7fb5f1377740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.110+0000 7fb5f1377740 20 client.15936 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.110+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a31f3f25e0 0x55a31f3f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.110+0000 7fb5f1377740 4 client.15936 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:42:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.110+0000 7fb5f1377740 1 -- 172.21.15.27:0/3100935652 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a31f2ddc20 con 0x55a31f3f25e0 2024-06-06T08:42:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.110+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a31f3f25e0 0x55a31f3f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.111+0000 7fb5c77fe640 10 client.15936 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.111+0000 7fb5e7fff640 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a31f3f25e0 0x55a31f3f4a00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edfb0636bec1757f server_cookie=22d6041d95f8f00f in_seq=0 out_seq=0 2024-06-06T08:42:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.309+0000 7fb5c77fe640 1 -- 172.21.15.27:0/3100935652 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a31f2ddc20 con 0x55a31f3f25e0 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.309+0000 7fb5c77fe640 10 client.15936 handle_command_reply: tid=0 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.309+0000 7fb5f1377740 1 client.15936 shutdown 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 -- 172.21.15.27:0/3100935652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 msgr2=0x55a31f37acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f37acb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc003f90 tx=0x7fb5dc009c30 comp rx=0 tx=0).stop 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 -- 172.21.15.27:0/3100935652 shutdown_connections 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55a31f3f25e0 0x55a31f3f4a00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a31f377350 0x55a31f37b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a31f36ca90 0x55a31f37acb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 --2- 172.21.15.27:0/3100935652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a31f36c170 0x55a31f37f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 -- 172.21.15.27:0/3100935652 >> 172.21.15.27:0/3100935652 conn(0x55a31f368680 msgr2=0x55a31f36e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5c77fe640 0 client.15936 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 -- 172.21.15.27:0/3100935652 shutdown_connections 2024-06-06T08:42:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 1 -- 172.21.15.27:0/3100935652 wait complete. 2024-06-06T08:42:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.310+0000 7fb5f1377740 20 client.15936 trim_cache size 0 max 16384 2024-06-06T08:42:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:42:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:42:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:42:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 -- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5d00a2ea0 msgr2=0x7fb5d00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5d00a2ea0 0x7fb5d00a5360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55a31f369aa0 tx=0x7fb5dc0046c0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 -- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 msgr2=0x7fb5e819ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 0x7fb5e819ca70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006010 tx=0x7fb5e0003ba0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 -- 172.21.15.27:0/1301234895 shutdown_connections 2024-06-06T08:42:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb5d00a2ea0 0x7fb5d00a5360 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5e810bf70 0x7fb5e819ca70 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e8103550 0x7fb5e81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 --2- 172.21.15.27:0/1301234895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5e8102c30 0x7fb5e81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 -- 172.21.15.27:0/1301234895 >> 172.21.15.27:0/1301234895 conn(0x7fb5e8077a20 msgr2=0x7fb5e80fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 -- 172.21.15.27:0/1301234895 shutdown_connections 2024-06-06T08:42:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:05.311+0000 7fb5ee596640 1 -- 172.21.15.27:0/1301234895 wait complete. 2024-06-06T08:42:06.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:06 smithi027 ceph-mon[32335]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-06-06T08:42:06.599 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ef5f116f', total seconds elapsed: 43.9 2024-06-06T08:42:06.617 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-mon[36579]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-06-06T08:42:06.617 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: *** Caught signal (Segmentation fault) ** 2024-06-06T08:42:06.617 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: in thread 7f4a45b61700 thread_name:md_log_replay 2024-06-06T08:42:06.617 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4a56f23d20] 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2: /usr/bin/ceph-mds(+0x549a2f) [0x55cc27765a2f] 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55cc277691c3] 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 4: (MDLog::_replay_thread()+0x617) [0x55cc277c3ff7] 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55cc277cd861] 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f4a56f191ca] 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 7: clone() 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2024-06-06T08:42:06.251+0000 7f4a45b61700 -1 *** Caught signal (Segmentation fault) ** 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: in thread 7f4a45b61700 thread_name:md_log_replay 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4a56f23d20] 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2: /usr/bin/ceph-mds(+0x549a2f) [0x55cc27765a2f] 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55cc277691c3] 2024-06-06T08:42:06.618 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 4: (MDLog::_replay_thread()+0x617) [0x55cc277c3ff7] 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55cc277cd861] 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f4a56f191ca] 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 7: clone() 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 0> 2024-06-06T08:42:06.251+0000 7f4a45b61700 -1 *** Caught signal (Segmentation fault) ** 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: in thread 7f4a45b61700 thread_name:md_log_replay 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4a56f23d20] 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2: /usr/bin/ceph-mds(+0x549a2f) [0x55cc27765a2f] 2024-06-06T08:42:06.619 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55cc277691c3] 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 4: (MDLog::_replay_thread()+0x617) [0x55cc277c3ff7] 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55cc277cd861] 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f4a56f191ca] 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 7: clone() 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 0> 2024-06-06T08:42:06.251+0000 7f4a45b61700 -1 *** Caught signal (Segmentation fault) ** 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: in thread 7f4a45b61700 thread_name:md_log_replay 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4a56f23d20] 2024-06-06T08:42:06.620 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2: /usr/bin/ceph-mds(+0x549a2f) [0x55cc27765a2f] 2024-06-06T08:42:06.621 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55cc277691c3] 2024-06-06T08:42:06.621 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 4: (MDLog::_replay_thread()+0x617) [0x55cc277c3ff7] 2024-06-06T08:42:06.621 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55cc277cd861] 2024-06-06T08:42:06.621 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f4a56f191ca] 2024-06-06T08:42:06.621 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 7: clone() 2024-06-06T08:42:06.621 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-06T08:42:06.621 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[67199]: 2024-06-06T08:42:06.621 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 podman[71421]: 2024-06-06 08:42:06.311578223 +0000 UTC m=+0.020608432 container died 5c47540cd5cd4962bb2ad25d2202859c7502711d59863654cd93aaf93afbd017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-06T08:42:06.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:06 smithi176 ceph-mon[36786]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-06-06T08:42:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.050+0000 7fb93d6cc700 1 -- 172.21.15.27:0/287669046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93810a6d0 msgr2=0x7fb93810aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.050+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/287669046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93810a6d0 0x7fb93810aab0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb928009990 tx=0x7fb928009ca0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.050+0000 7fb93d6cc700 1 -- 172.21.15.27:0/287669046 shutdown_connections 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.050+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/287669046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb938105090 0x7fb9381115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.050+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/287669046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93810a6d0 0x7fb93810aab0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.050+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/287669046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9381046d0 0x7fb938104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.050+0000 7fb93d6cc700 1 -- 172.21.15.27:0/287669046 >> 172.21.15.27:0/287669046 conn(0x7fb9381001c0 msgr2=0x7fb9381025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.050+0000 7fb93d6cc700 1 -- 172.21.15.27:0/287669046 shutdown_connections 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.050+0000 7fb93d6cc700 1 -- 172.21.15.27:0/287669046 wait complete. 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.051+0000 7fb93d6cc700 1 Processor -- start 2024-06-06T08:42:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.051+0000 7fb93d6cc700 1 -- start start 2024-06-06T08:42:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb93d6cc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9381046d0 0x7fb93819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb93d6cc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb938105090 0x7fb93819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb93d6cc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb93810a6d0 0x7fb93819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb93d6cc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb938114a20 con 0x7fb938105090 2024-06-06T08:42:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb93d6cc700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381148a0 con 0x7fb93810a6d0 2024-06-06T08:42:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb93d6cc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb938114ba0 con 0x7fb9381046d0 2024-06-06T08:42:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb9367fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb938105090 0x7fb93819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb9367fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb938105090 0x7fb93819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41350/0 (socket says 172.21.15.27:41350) 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb9367fc700 1 -- 172.21.15.27:0/1834569523 learned_addr learned my addr 172.21.15.27:0/1834569523 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb936ffd700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9381046d0 0x7fb93819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb9377fe700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb93810a6d0 0x7fb93819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb9367fc700 1 -- 172.21.15.27:0/1834569523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9381046d0 msgr2=0x7fb93819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb9367fc700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9381046d0 0x7fb93819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb9367fc700 1 -- 172.21.15.27:0/1834569523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb93810a6d0 msgr2=0x7fb93819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb9367fc700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb93810a6d0 0x7fb93819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.052+0000 7fb9367fc700 1 -- 172.21.15.27:0/1834569523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93819fe30 con 0x7fb938105090 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.053+0000 7fb9367fc700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb938105090 0x7fb93819b3b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00d980 tx=0x7fb92c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.053+0000 7fb91ffff700 1 -- 172.21.15.27:0/1834569523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c017070 con 0x7fb938105090 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.053+0000 7fb91ffff700 1 -- 172.21.15.27:0/1834569523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb92c005020 con 0x7fb938105090 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.054+0000 7fb91ffff700 1 -- 172.21.15.27:0/1834569523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00fbf0 con 0x7fb938105090 2024-06-06T08:42:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.054+0000 7fb93d6cc700 1 -- 172.21.15.27:0/1834569523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9381a0120 con 0x7fb938105090 2024-06-06T08:42:07.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.054+0000 7fb93d6cc700 1 -- 172.21.15.27:0/1834569523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9381a79a0 con 0x7fb938105090 2024-06-06T08:42:07.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.058+0000 7fb91ffff700 1 -- 172.21.15.27:0/1834569523 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb92c0051d0 con 0x7fb938105090 2024-06-06T08:42:07.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb91ffff700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb9200a2e10 0x7fb9200a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb947f4fb80 1 Processor -- start 2024-06-06T08:42:07.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb936ffd700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb9200a2e10 0x7fb9200a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb91ffff700 1 -- 172.21.15.27:0/1834569523 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb92c0cf390 con 0x7fb938105090 2024-06-06T08:42:07.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb947f4fb80 1 -- start start 2024-06-06T08:42:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee8885e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88858260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee88858bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ee8874af60 con 0x55ee8885df60 2024-06-06T08:42:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ee887a9850 con 0x55ee888587a0 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.059+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ee88711460 con 0x55ee8885e880 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9367fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88858260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9367fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88858260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:37844/0 (socket says 172.21.15.27:37844) 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9367fc700 1 -- 172.21.15.27:0/299729398 learned_addr learned my addr 172.21.15.27:0/299729398 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb936ffd700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb9200a2e10 0x7fb9200a52d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9280098e0 tx=0x7fb928005960 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9377fe700 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee88858bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb936ffd700 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee8885e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9367fc700 1 -- 172.21.15.27:0/299729398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 msgr2=0x55ee88858bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9367fc700 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee88858bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9367fc700 1 -- 172.21.15.27:0/299729398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 msgr2=0x55ee8885e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9367fc700 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee8885e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9367fc700 1 -- 172.21.15.27:0/299729398 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee886f88d0 con 0x55ee8885e880 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb936ffd700 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee8885e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.060+0000 7fb9367fc700 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88858260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0978f0 tx=0x7fb92c09bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7138e422573155db server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb91d7fa700 1 -- 172.21.15.27:0/299729398 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0b4070 con 0x55ee8885e880 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb91d7fa700 1 -- 172.21.15.27:0/299729398 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb92c0cf160 con 0x55ee8885e880 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb91d7fa700 1 -- 172.21.15.27:0/299729398 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0b4470 con 0x55ee8885e880 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb947f4fb80 1 -- 172.21.15.27:0/299729398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 msgr2=0x55ee88858260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88858260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0978f0 tx=0x7fb92c09bc90 comp rx=0 tx=0).stop 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb947f4fb80 1 -- 172.21.15.27:0/299729398 shutdown_connections 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee88858bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee8885e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/299729398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88858260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.061+0000 7fb947f4fb80 1 -- 172.21.15.27:0/299729398 >> 172.21.15.27:0/299729398 conn(0x55ee88785080 msgr2=0x55ee88853220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 -- 172.21.15.27:0/299729398 shutdown_connections 2024-06-06T08:42:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 -- 172.21.15.27:0/299729398 wait complete. 2024-06-06T08:42:07.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 -- start start 2024-06-06T08:42:07.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 Processor -- start 2024-06-06T08:42:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee888646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee888619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88861f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ee88749fc0 con 0x55ee8885df60 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ee887a5810 con 0x55ee888587a0 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.062+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ee888652b0 con 0x55ee8885e880 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb947f4fb80 10 client.0 _ll_register_callbacks cb 0x55ee8873f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9377fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88861f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9367fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee888619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9377fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88861f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:37858/0 (socket says 172.21.15.27:37858) 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9377fe700 1 -- 172.21.15.27:0/1270253823 learned_addr learned my addr 172.21.15.27:0/1270253823 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9367fc700 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee888619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41360/0 (socket says 172.21.15.27:41360) 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb936ffd700 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee888646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9377fe700 1 -- 172.21.15.27:0/1270253823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 msgr2=0x55ee888646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9377fe700 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee888646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9377fe700 1 -- 172.21.15.27:0/1270253823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 msgr2=0x55ee888619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9377fe700 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee888619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9377fe700 1 -- 172.21.15.27:0/1270253823 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8876f750 con 0x55ee8885e880 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb9377fe700 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88861f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb930003fd0 tx=0x7fb930004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ab4631307160b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb91aff5700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.063+0000 7fb91aff5700 1 -- 172.21.15.27:0/1270253823 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93001a070 con 0x55ee8885e880 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb947f4fb80 1 -- 172.21.15.27:0/1270253823 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb92c0cf160 con 0x55ee8885e880 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb91aff5700 1 -- 172.21.15.27:0/1270253823 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb930007700 con 0x55ee8885e880 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb91aff5700 1 -- 172.21.15.27:0/1270253823 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb930026050 con 0x55ee8885e880 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb947f4fb80 10 client.25486 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb947f4fb80 1 -- 172.21.15.27:0/1270253823 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee8886fb90 con 0x55ee8885e880 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb91aff5700 1 -- 172.21.15.27:0/1270253823 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb92c0cf160 con 0x55ee8885e880 2024-06-06T08:42:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb91aff5700 1 -- 172.21.15.27:0/1270253823 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb930016ab0 con 0x55ee8885e880 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb947f4fb80 10 client.25486 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb947f4fb80 10 client.25486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb947f4fb80 20 client.25486 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ee88872120 0x55ee88874510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb947f4fb80 4 client.25486 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.064+0000 7fb947f4fb80 1 -- 172.21.15.27:0/1270253823 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee887b6d50 con 0x55ee88872120 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.065+0000 7fb936ffd700 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ee88872120 0x55ee88874510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.065+0000 7fb91aff5700 10 client.25486 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.065+0000 7fb936ffd700 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ee88872120 0x55ee88874510 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9875a479452e1f8d server_cookie=ce4dfbef8a79c5d6 in_seq=0 out_seq=0 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb91aff5700 1 -- 172.21.15.27:0/1270253823 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ee887b6d50 con 0x55ee88872120 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb91aff5700 10 client.25486 handle_command_reply: tid=0 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb947f4fb80 1 client.25486 shutdown 2024-06-06T08:42:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb947f4fb80 1 -- 172.21.15.27:0/1270253823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 msgr2=0x55ee88861f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88861f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb930003fd0 tx=0x7fb930004040 comp rx=0 tx=0).stop 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb947f4fb80 1 -- 172.21.15.27:0/1270253823 shutdown_connections 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885e880 0x55ee88861f00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee888646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ee88872120 0x55ee88874510 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/1270253823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885df60 0x55ee888619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb947f4fb80 1 -- 172.21.15.27:0/1270253823 >> 172.21.15.27:0/1270253823 conn(0x55ee88785080 msgr2=0x55ee88852700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.066+0000 7fb91aff5700 0 client.25486 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.067+0000 7fb947f4fb80 1 -- 172.21.15.27:0/1270253823 shutdown_connections 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.067+0000 7fb947f4fb80 1 -- 172.21.15.27:0/1270253823 wait complete. 2024-06-06T08:42:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.067+0000 7fb947f4fb80 20 client.25486 trim_cache size 0 max 16384 2024-06-06T08:42:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.096+0000 7fb947f4fb80 1 Processor -- start 2024-06-06T08:42:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.096+0000 7fb947f4fb80 1 -- start start 2024-06-06T08:42:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee88858260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8885f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 0x55ee88864200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b3150 con 0x55ee8885e880 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b09e0 con 0x55ee888587a0 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381ae830 con 0x55ee8885df60 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb9367fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8885f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb9367fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8885f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:37864/0 (socket says 172.21.15.27:37864) 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb9367fc700 1 -- 172.21.15.27:0/4049572546 learned_addr learned my addr 172.21.15.27:0/4049572546 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb9377fe700 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 0x55ee88864200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb936ffd700 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee88858260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb9367fc700 1 -- 172.21.15.27:0/4049572546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 msgr2=0x55ee88858260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb9367fc700 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee88858260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb9367fc700 1 -- 172.21.15.27:0/4049572546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 msgr2=0x55ee88864200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb9367fc700 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 0x55ee88864200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb9367fc700 1 -- 172.21.15.27:0/4049572546 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee888549a0 con 0x55ee8885df60 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.097+0000 7fb936ffd700 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee88858260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:42:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.098+0000 7fb9367fc700 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8885f430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb92c09bd40 tx=0x7fb92c0ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddced5c0f52f00cc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.098+0000 7fb91cff9700 1 -- 172.21.15.27:0/4049572546 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0b4040 con 0x55ee8885df60 2024-06-06T08:42:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.098+0000 7fb91cff9700 1 -- 172.21.15.27:0/4049572546 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb92c0a4100 con 0x55ee8885df60 2024-06-06T08:42:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.098+0000 7fb91cff9700 1 -- 172.21.15.27:0/4049572546 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c09b950 con 0x55ee8885df60 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 -- 172.21.15.27:0/4049572546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 msgr2=0x55ee8885f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8885f430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb92c09bd40 tx=0x7fb92c0ce040 comp rx=0 tx=0).stop 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 -- 172.21.15.27:0/4049572546 shutdown_connections 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 0x55ee88864200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee88858260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/4049572546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8885f430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 -- 172.21.15.27:0/4049572546 >> 172.21.15.27:0/4049572546 conn(0x55ee88785080 msgr2=0x55ee8885ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 -- 172.21.15.27:0/4049572546 shutdown_connections 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 -- 172.21.15.27:0/4049572546 wait complete. 2024-06-06T08:42:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 -- start start 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.099+0000 7fb947f4fb80 1 Processor -- start 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee8886a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8886ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb947f4fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 0x55ee88879540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381aeea0 con 0x55ee8885e880 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b0310 con 0x55ee888587a0 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb947f4fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381af570 con 0x55ee8885df60 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb947f4fb80 10 client.0 _ll_register_callbacks cb 0x55ee8873f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb936ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee8886a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb936ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee8886a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:48722/0 (socket says 172.21.15.27:48722) 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb936ffd700 1 -- 172.21.15.27:0/3080098975 learned_addr learned my addr 172.21.15.27:0/3080098975 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb9367fc700 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8886ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.100+0000 7fb9377fe700 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 0x55ee88879540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb936ffd700 1 -- 172.21.15.27:0/3080098975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 msgr2=0x55ee8886ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb936ffd700 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8886ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb936ffd700 1 -- 172.21.15.27:0/3080098975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 msgr2=0x55ee88879540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb936ffd700 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 0x55ee88879540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb936ffd700 1 -- 172.21.15.27:0/3080098975 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8884f810 con 0x55ee888587a0 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb9377fe700 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 0x55ee88879540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb936ffd700 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee8886a920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb928000c00 tx=0x7fb928009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324ad9bc570f1ce0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb91aff5700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb91aff5700 1 -- 172.21.15.27:0/3080098975 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92800fac0 con 0x55ee888587a0 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb947f4fb80 1 -- 172.21.15.27:0/3080098975 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb92c0a4100 con 0x55ee888587a0 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.101+0000 7fb91aff5700 1 -- 172.21.15.27:0/3080098975 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fb92800fc60 con 0x55ee888587a0 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb91aff5700 1 -- 172.21.15.27:0/3080098975 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb928032450 con 0x55ee888587a0 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb947f4fb80 10 client.25499 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb947f4fb80 1 -- 172.21.15.27:0/3080098975 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee888dc790 con 0x55ee888587a0 2024-06-06T08:42:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb91aff5700 1 -- 172.21.15.27:0/3080098975 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb92c0a4100 con 0x55ee888587a0 2024-06-06T08:42:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb91aff5700 1 -- 172.21.15.27:0/3080098975 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb9280246b0 con 0x55ee888587a0 2024-06-06T08:42:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb947f4fb80 10 client.25499 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb947f4fb80 10 client.25499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb947f4fb80 20 client.25499 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ee888dce30 0x55ee888df250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb947f4fb80 4 client.25499 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:42:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.102+0000 7fb947f4fb80 1 -- 172.21.15.27:0/3080098975 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee887b12d0 con 0x55ee888dce30 2024-06-06T08:42:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.103+0000 7fb9367fc700 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ee888dce30 0x55ee888df250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.103+0000 7fb9367fc700 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ee888dce30 0x55ee888df250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb7e7c1bcc207b6 server_cookie=ad0cdf489bfd6327 in_seq=0 out_seq=0 2024-06-06T08:42:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.103+0000 7fb91aff5700 10 client.25499 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.103+0000 7fb91aff5700 1 -- 172.21.15.27:0/3080098975 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee887b12d0 con 0x55ee888dce30 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.103+0000 7fb91aff5700 10 client.25499 handle_command_reply: tid=0 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.103+0000 7fb947f4fb80 1 client.25499 shutdown 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 -- 172.21.15.27:0/3080098975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 msgr2=0x55ee8886a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee8886a920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb928000c00 tx=0x7fb928009ca0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 -- 172.21.15.27:0/3080098975 shutdown_connections 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ee888dce30 0x55ee888df250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee8885e880 0x55ee88879540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ee888587a0 0x55ee8886a920 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 --2- 172.21.15.27:0/3080098975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee8885df60 0x55ee8886ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 -- 172.21.15.27:0/3080098975 >> 172.21.15.27:0/3080098975 conn(0x55ee88785080 msgr2=0x55ee8885f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb91aff5700 0 client.25499 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 -- 172.21.15.27:0/3080098975 shutdown_connections 2024-06-06T08:42:07.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 1 -- 172.21.15.27:0/3080098975 wait complete. 2024-06-06T08:42:07.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.104+0000 7fb947f4fb80 20 client.25499 trim_cache size 0 max 16384 2024-06-06T08:42:07.107 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:42:07.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 -- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb9200a2e10 msgr2=0x7fb9200a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:07.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb9200a2e10 0x7fb9200a52d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9280098e0 tx=0x7fb928005960 comp rx=0 tx=0).stop 2024-06-06T08:42:07.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 -- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb938105090 msgr2=0x7fb93819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb938105090 0x7fb93819b3b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00d980 tx=0x7fb92c00dc90 comp rx=0 tx=0).stop 2024-06-06T08:42:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 -- 172.21.15.27:0/1834569523 shutdown_connections 2024-06-06T08:42:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb9200a2e10 0x7fb9200a52d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb93810a6d0 0x7fb93819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9381046d0 0x7fb93819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 --2- 172.21.15.27:0/1834569523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb938105090 0x7fb93819b3b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 -- 172.21.15.27:0/1834569523 >> 172.21.15.27:0/1834569523 conn(0x7fb9381001c0 msgr2=0x7fb9381007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 -- 172.21.15.27:0/1834569523 shutdown_connections 2024-06-06T08:42:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:07.105+0000 7fb93d6cc700 1 -- 172.21.15.27:0/1834569523 wait complete. 2024-06-06T08:42:07.240 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 podman[71421]: 2024-06-06 08:42:06.987367872 +0000 UTC m=+0.696398063 container remove 5c47540cd5cd4962bb2ad25d2202859c7502711d59863654cd93aaf93afbd017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-06T08:42:07.240 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:06 smithi203 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-06-06T08:42:07.739 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:07 smithi203 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-06-06T08:42:07.740 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:07 smithi203 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.133s CPU time. 2024-06-06T08:42:08.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:08 smithi027 ceph-mon[32335]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-06-06T08:42:08.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:08 smithi203 ceph-mon[36579]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-06-06T08:42:08.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:08 smithi176 ceph-mon[36786]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-06-06T08:42:08.894 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T08:42:08.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T08:42:09.158 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:42:10.659 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:10 smithi027 ceph-mon[32335]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-06-06T08:42:10.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:10 smithi203 ceph-mon[36579]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-06-06T08:42:10.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:10 smithi176 ceph-mon[36786]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-06-06T08:42:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.120+0000 7fbdd852d700 1 -- 172.21.15.27:0/2882520692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdd0101d50 msgr2=0x7fbdd010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.120+0000 7fbdd852d700 1 --2- 172.21.15.27:0/2882520692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdd0101d50 0x7fbdd010e240 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800b0a0 tx=0x7fbdc800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.120+0000 7fbdd852d700 1 -- 172.21.15.27:0/2882520692 shutdown_connections 2024-06-06T08:42:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.120+0000 7fbdd852d700 1 --2- 172.21.15.27:0/2882520692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdd0101d50 0x7fbdd010e240 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.120+0000 7fbdd852d700 1 --2- 172.21.15.27:0/2882520692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbdd0107400 0x7fbdd01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.120+0000 7fbdd852d700 1 --2- 172.21.15.27:0/2882520692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0101390 0x7fbdd0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.120+0000 7fbdd852d700 1 -- 172.21.15.27:0/2882520692 >> 172.21.15.27:0/2882520692 conn(0x7fbdd0078340 msgr2=0x7fbdd0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.120+0000 7fbdd852d700 1 -- 172.21.15.27:0/2882520692 shutdown_connections 2024-06-06T08:42:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.120+0000 7fbdd852d700 1 -- 172.21.15.27:0/2882520692 wait complete. 2024-06-06T08:42:11.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.121+0000 7fbdd852d700 1 Processor -- start 2024-06-06T08:42:11.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.121+0000 7fbdd852d700 1 -- start start 2024-06-06T08:42:11.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.121+0000 7fbdd852d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbdd0101390 0x7fbdd019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd852d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdd0101d50 0x7fbdd019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd5ac8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdd0101d50 0x7fbdd019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd5ac8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdd0101d50 0x7fbdd019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49468/0 (socket says 172.21.15.27:49468) 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd5ac8700 1 -- 172.21.15.27:0/1865617302 learned_addr learned my addr 172.21.15.27:0/1865617302 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd852d700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0107400 0x7fbdd01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01116a0 con 0x7fbdd0101d50 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0111520 con 0x7fbdd0101390 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0111820 con 0x7fbdd0107400 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd62c9700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbdd0101390 0x7fbdd019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0107400 0x7fbdd01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd62c9700 1 -- 172.21.15.27:0/1865617302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0107400 msgr2=0x7fbdd01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd62c9700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0107400 0x7fbdd01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd62c9700 1 -- 172.21.15.27:0/1865617302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdd0101d50 msgr2=0x7fbdd019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd62c9700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdd0101d50 0x7fbdd019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.122+0000 7fbdd62c9700 1 -- 172.21.15.27:0/1865617302 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd0199130 con 0x7fbdd0101390 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.123+0000 7fbdd62c9700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbdd0101390 0x7fbdd019e810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00d950 tx=0x7fbdcc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.123+0000 7fbdc77fe700 1 -- 172.21.15.27:0/1865617302 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc016070 con 0x7fbdd0101390 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.123+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd0199420 con 0x7fbdd0101390 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.123+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd01a56e0 con 0x7fbdd0101390 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.123+0000 7fbdc77fe700 1 -- 172.21.15.27:0/1865617302 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdcc00fd70 con 0x7fbdd0101390 2024-06-06T08:42:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.124+0000 7fbdc77fe700 1 -- 172.21.15.27:0/1865617302 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00e3b0 con 0x7fbdd0101390 2024-06-06T08:42:11.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.127+0000 7fbdc77fe700 1 -- 172.21.15.27:0/1865617302 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbdcc019070 con 0x7fbdd0101390 2024-06-06T08:42:11.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.127+0000 7fbde2db0b80 1 Processor -- start 2024-06-06T08:42:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbde2db0b80 1 -- start start 2024-06-06T08:42:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 0x55c3a4d732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d73820 0x55c3a4d73c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbde2db0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c3a4c65fe0 con 0x55c3a4d73820 2024-06-06T08:42:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbde2db0b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c3a4cc48d0 con 0x55c3a4d78fe0 2024-06-06T08:42:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbde2db0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c3a4c2c4e0 con 0x55c3a4d79900 2024-06-06T08:42:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbdd6aca700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbdd6aca700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d793c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:51566/0 (socket says 172.21.15.27:51566) 2024-06-06T08:42:11.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbdd6aca700 1 -- 172.21.15.27:0/2814318896 learned_addr learned my addr 172.21.15.27:0/2814318896 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:11.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbdd5ac8700 1 --2- 172.21.15.27:0/2814318896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d73820 0x55c3a4d73c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.128+0000 7fbdd62c9700 1 --2- 172.21.15.27:0/2814318896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 0x55c3a4d732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.129+0000 7fbdd6aca700 1 -- 172.21.15.27:0/2814318896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 msgr2=0x55c3a4d732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:11.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.129+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/2814318896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 0x55c3a4d732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.129+0000 7fbdd6aca700 1 -- 172.21.15.27:0/2814318896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d73820 msgr2=0x55c3a4d73c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:11.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.129+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/2814318896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d73820 0x55c3a4d73c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.129+0000 7fbdd6aca700 1 -- 172.21.15.27:0/2814318896 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a4c13950 con 0x55c3a4d78fe0 2024-06-06T08:42:11.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.129+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/2814318896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d793c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009360 tx=0x7fbdc8009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6184a7c69234bf27 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:11.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.129+0000 7fbdc4ff9700 1 -- 172.21.15.27:0/2814318896 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8029070 con 0x55c3a4d78fe0 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.129+0000 7fbdc4ff9700 1 -- 172.21.15.27:0/2814318896 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc8006020 con 0x55c3a4d78fe0 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.129+0000 7fbdc4ff9700 1 -- 172.21.15.27:0/2814318896 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8007520 con 0x55c3a4d78fe0 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbdc77fe700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbdbc0a2e90 0x7fbdbc0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbdd62c9700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbdbc0a2e90 0x7fbdbc0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbdc77fe700 1 -- 172.21.15.27:0/1865617302 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fbdcc0cf120 con 0x7fbdd0101390 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbdd62c9700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbdbc0a2e90 0x7fbdbc0a5350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc01d000 tx=0x7fbdcc00bf90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbde2db0b80 1 -- 172.21.15.27:0/2814318896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 msgr2=0x55c3a4d793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/2814318896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d793c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009360 tx=0x7fbdc8009390 comp rx=0 tx=0).stop 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbde2db0b80 1 -- 172.21.15.27:0/2814318896 shutdown_connections 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/2814318896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d73820 0x55c3a4d73c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/2814318896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d793c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/2814318896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 0x55c3a4d732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.130+0000 7fbde2db0b80 1 -- 172.21.15.27:0/2814318896 >> 172.21.15.27:0/2814318896 conn(0x55c3a4ca0100 msgr2=0x55c3a4d6e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:11.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.131+0000 7fbde2db0b80 1 -- 172.21.15.27:0/2814318896 shutdown_connections 2024-06-06T08:42:11.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.131+0000 7fbde2db0b80 1 -- 172.21.15.27:0/2814318896 wait complete. 2024-06-06T08:42:11.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.131+0000 7fbde2db0b80 1 -- start start 2024-06-06T08:42:11.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.131+0000 7fbde2db0b80 1 Processor -- start 2024-06-06T08:42:11.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d7f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d7c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbde2db0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c3a4c65040 con 0x55c3a4d79900 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbdd6aca700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d7f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbdd6aca700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d7f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:51576/0 (socket says 172.21.15.27:51576) 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbdd6aca700 1 -- 172.21.15.27:0/1846818038 learned_addr learned my addr 172.21.15.27:0/1846818038 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c3a4cc0890 con 0x55c3a4d73820 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c3a4d80280 con 0x55c3a4d78fe0 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.132+0000 7fbdd5ac8700 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d7c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbde2db0b80 10 client.0 _ll_register_callbacks cb 0x55c3a4c5a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:11.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdd5ac8700 1 -- 172.21.15.27:0/1846818038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 msgr2=0x55c3a4d7f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdd5ac8700 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d7f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdd5ac8700 1 -- 172.21.15.27:0/1846818038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 msgr2=0x55c3a4d7ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdd5ac8700 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdd5ac8700 1 -- 172.21.15.27:0/1846818038 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a4c8a7d0 con 0x55c3a4d78fe0 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdd5ac8700 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d7c940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00096d0 tx=0x7fbdc0011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e31b93e5c2aaebee server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/1846818038 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc001e020 con 0x55c3a4d78fe0 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc8006020 con 0x55c3a4d78fe0 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.133+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/1846818038 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fbdc0004830 con 0x55c3a4d78fe0 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.134+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/1846818038 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbdc0026050 con 0x55c3a4d78fe0 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.134+0000 7fbde2db0b80 10 client.25504 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.134+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c3a4d8aa60 con 0x55c3a4d78fe0 2024-06-06T08:42:11.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.134+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/1846818038 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc8006020 con 0x55c3a4d78fe0 2024-06-06T08:42:11.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.135+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/1846818038 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbdc0019070 con 0x55c3a4d78fe0 2024-06-06T08:42:11.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.135+0000 7fbde2db0b80 10 client.25504 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:11.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.135+0000 7fbde2db0b80 10 client.25504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:11.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.135+0000 7fbde2db0b80 20 client.25504 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:11.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.135+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c3a4d8d0f0 0x55c3a4d8f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.135+0000 7fbde2db0b80 4 client.25504 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:11.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.135+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3a4cd1dd0 con 0x55c3a4d8d0f0 2024-06-06T08:42:11.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.135+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c3a4d8d0f0 0x55c3a4d8f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.136+0000 7fbdb9ffb700 10 client.25504 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.136+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c3a4d8d0f0 0x55c3a4d8f530 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f40ef699b487e2ff server_cookie=a8622763668a072c in_seq=0 out_seq=0 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.136+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/1846818038 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c3a4cd1dd0 con 0x55c3a4d8d0f0 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.136+0000 7fbdb9ffb700 10 client.25504 handle_command_reply: tid=0 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.136+0000 7fbde2db0b80 1 client.25504 shutdown 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d78fe0 msgr2=0x55c3a4d7c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d7c940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00096d0 tx=0x7fbdc0011a30 comp rx=0 tx=0).stop 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 shutdown_connections 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c3a4d8d0f0 0x55c3a4d8f530 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d7f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/1846818038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d7c940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 >> 172.21.15.27:0/1846818038 conn(0x55c3a4ca0100 msgr2=0x55c3a4d6d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbdb9ffb700 0 client.25504 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 shutdown_connections 2024-06-06T08:42:11.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 1 -- 172.21.15.27:0/1846818038 wait complete. 2024-06-06T08:42:11.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.137+0000 7fbde2db0b80 20 client.25504 trim_cache size 0 max 16384 2024-06-06T08:42:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbde2db0b80 1 Processor -- start 2024-06-06T08:42:11.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbde2db0b80 1 -- start start 2024-06-06T08:42:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d73820 0x55c3a4d732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d7cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbde2db0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01aebd0 con 0x55c3a4d79900 2024-06-06T08:42:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbde2db0b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01ac460 con 0x55c3a4d78fe0 2024-06-06T08:42:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbde2db0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01aa2b0 con 0x55c3a4d73820 2024-06-06T08:42:11.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbdd6aca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbdd6aca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49494/0 (socket says 172.21.15.27:49494) 2024-06-06T08:42:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbdd6aca700 1 -- 172.21.15.27:0/273543202 learned_addr learned my addr 172.21.15.27:0/273543202 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/273543202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d73820 0x55c3a4d732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.162+0000 7fbdd5ac8700 1 --2- 172.21.15.27:0/273543202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d7cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.163+0000 7fbdd6aca700 1 -- 172.21.15.27:0/273543202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d73820 msgr2=0x55c3a4d732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:11.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.163+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/273543202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d73820 0x55c3a4d732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.163+0000 7fbdd6aca700 1 -- 172.21.15.27:0/273543202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 msgr2=0x55c3a4d7cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:11.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.163+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/273543202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d7cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.163+0000 7fbdd6aca700 1 -- 172.21.15.27:0/273543202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a4d7fdd0 con 0x55c3a4d79900 2024-06-06T08:42:11.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.163+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/273543202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7c260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800b070 tx=0x7fbdc80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21ef29cb31bf0d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:11.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.163+0000 7fbdbbfff700 1 -- 172.21.15.27:0/273543202 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8041040 con 0x55c3a4d79900 2024-06-06T08:42:11.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.164+0000 7fbdbbfff700 1 -- 172.21.15.27:0/273543202 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc80060b0 con 0x55c3a4d79900 2024-06-06T08:42:11.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.164+0000 7fbdbbfff700 1 -- 172.21.15.27:0/273543202 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8029040 con 0x55c3a4d79900 2024-06-06T08:42:11.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.164+0000 7fbde2db0b80 1 -- 172.21.15.27:0/273543202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 msgr2=0x55c3a4d7c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:11.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.164+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/273543202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7c260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800b070 tx=0x7fbdc80099e0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.165+0000 7fbde2db0b80 1 -- 172.21.15.27:0/273543202 shutdown_connections 2024-06-06T08:42:11.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.165+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/273543202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d79900 0x55c3a4d7c260 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.165+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/273543202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d73820 0x55c3a4d732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.165+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/273543202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d7cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.165+0000 7fbde2db0b80 1 -- 172.21.15.27:0/273543202 >> 172.21.15.27:0/273543202 conn(0x55c3a4ca0100 msgr2=0x55c3a4d7e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:11.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.165+0000 7fbde2db0b80 1 -- 172.21.15.27:0/273543202 shutdown_connections 2024-06-06T08:42:11.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.165+0000 7fbde2db0b80 1 -- 172.21.15.27:0/273543202 wait complete. 2024-06-06T08:42:11.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.165+0000 7fbde2db0b80 1 -- start start 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.165+0000 7fbde2db0b80 1 Processor -- start 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d85950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d85e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbde2db0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 0x55c3a4d82740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbde2db0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01aa920 con 0x55c3a4d78fe0 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd5ac8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d85e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd5ac8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d85e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49506/0 (socket says 172.21.15.27:49506) 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd5ac8700 1 -- 172.21.15.27:0/984840028 learned_addr learned my addr 172.21.15.27:0/984840028 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01abd90 con 0x55c3a4d73820 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d85950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01aaff0 con 0x55c3a4d79900 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 0x55c3a4d82740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd5ac8700 1 -- 172.21.15.27:0/984840028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 msgr2=0x55c3a4d82740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd5ac8700 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 0x55c3a4d82740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd5ac8700 1 -- 172.21.15.27:0/984840028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 msgr2=0x55c3a4d85950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd5ac8700 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d85950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.166+0000 7fbdd5ac8700 1 -- 172.21.15.27:0/984840028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a4d82f10 con 0x55c3a4d78fe0 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d85950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbde2db0b80 10 client.0 _ll_register_callbacks cb 0x55c3a4c5a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbdd5ac8700 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d85e90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55c3a4d6b0c0 tx=0x7fbdc001ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407e01a21f43a794 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc80060b0 con 0x55c3a4d78fe0 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbdb9ffb700 10 client.15972 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/984840028 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc001e040 con 0x55c3a4d78fe0 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/984840028 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fbdc001a6c0 con 0x55c3a4d78fe0 2024-06-06T08:42:11.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/984840028 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbdc0005030 con 0x55c3a4d78fe0 2024-06-06T08:42:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbde2db0b80 10 client.15972 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.167+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55c3a4df8790 con 0x55c3a4d78fe0 2024-06-06T08:42:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.168+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/984840028 <== mon.0 v2:172.21.15.27:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbdc0019070 con 0x55c3a4d78fe0 2024-06-06T08:42:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.168+0000 7fbde2db0b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.168+0000 7fbde2db0b80 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.168+0000 7fbde2db0b80 20 client.15972 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.168+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c3a4df8e40 0x55c3a4dfb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.168+0000 7fbde2db0b80 4 client.15972 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T08:42:11.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.168+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fbdc80060b0 con 0x55c3a4df8e40 2024-06-06T08:42:11.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.168+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c3a4df8e40 0x55c3a4dfb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:11.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.169+0000 7fbdd6aca700 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c3a4df8e40 0x55c3a4dfb2d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4abc20c4df279b34 server_cookie=4e880af650f0a5e8 in_seq=0 out_seq=0 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.169+0000 7fbdb9ffb700 10 client.15972 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbdb9ffb700 1 -- 172.21.15.27:0/984840028 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fbdc80060b0 con 0x55c3a4df8e40 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbdb9ffb700 10 client.15972 handle_command_reply: tid=0 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbde2db0b80 1 client.15972 shutdown 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d78fe0 msgr2=0x55c3a4d85e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d85e90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55c3a4d6b0c0 tx=0x7fbdc001ae40 comp rx=0 tx=0).stop 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 shutdown_connections 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3a4d79900 0x55c3a4d82740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c3a4d73820 0x55c3a4d85950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c3a4df8e40 0x55c3a4dfb2d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbde2db0b80 1 --2- 172.21.15.27:0/984840028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c3a4d78fe0 0x55c3a4d85e90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 >> 172.21.15.27:0/984840028 conn(0x55c3a4ca0100 msgr2=0x55c3a4d77a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:11.173 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:42:11.173 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-06-06T08:42:11.173 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "76d000eb-0b36-4e9f-b8e7-93e384d9b73b", 2024-06-06T08:42:11.173 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-06-06T08:42:11.173 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:42:11.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.170+0000 7fbdb9ffb700 0 client.15972 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 shutdown_connections 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbde2db0b80 1 -- 172.21.15.27:0/984840028 wait complete. 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbde2db0b80 20 client.15972 trim_cache size 0 max 16384 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbdbc0a2e90 msgr2=0x7fbdbc0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbdbc0a2e90 0x7fbdbc0a5350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc01d000 tx=0x7fbdcc00bf90 comp rx=0 tx=0).stop 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbdd0101390 msgr2=0x7fbdd019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbdd0101390 0x7fbdd019e810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00d950 tx=0x7fbdcc00dc60 comp rx=0 tx=0).stop 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 shutdown_connections 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0107400 0x7fbdd01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbdbc0a2e90 0x7fbdbc0a5350 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbdd0101390 0x7fbdd019e810 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00d950 tx=0x7fbdcc00dc60 comp rx=0 tx=0).stop 2024-06-06T08:42:11.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 --2- 172.21.15.27:0/1865617302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdd0101d50 0x7fbdd019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:11.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.171+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 >> 172.21.15.27:0/1865617302 conn(0x7fbdd0078340 msgr2=0x7fbdd01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:11.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.172+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 shutdown_connections 2024-06-06T08:42:11.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:11.172+0000 7fbdd852d700 1 -- 172.21.15.27:0/1865617302 wait complete. 2024-06-06T08:42:11.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:11 smithi027 ceph-mon[32335]: scrub queued for path: / 2024-06-06T08:42:11.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:42:11.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:42:11.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:42:11.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:11 smithi203 ceph-mon[36579]: scrub queued for path: / 2024-06-06T08:42:11.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:42:11.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:42:11.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:42:11.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:11 smithi176 ceph-mon[36786]: scrub queued for path: / 2024-06-06T08:42:11.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:42:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:42:11.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:42:11.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T08:42:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:42:12.672 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:12 smithi027 ceph-mon[32335]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-06-06T08:42:12.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:12 smithi203 ceph-mon[36579]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-06-06T08:42:12.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:12 smithi176 ceph-mon[36786]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-06-06T08:42:14.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.203+0000 7fcd15aaa700 1 -- 172.21.15.27:0/3487786360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101380 msgr2=0x7fcd10101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:14.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.203+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/3487786360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101380 0x7fcd10101800 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009990 tx=0x7fcd04009ca0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.203+0000 7fcd15aaa700 1 -- 172.21.15.27:0/3487786360 shutdown_connections 2024-06-06T08:42:14.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.203+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/3487786360 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10101d40 0x7fcd1010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.203+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/3487786360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd101073f0 0x7fcd101077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.203+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/3487786360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101380 0x7fcd10101800 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.203+0000 7fcd15aaa700 1 -- 172.21.15.27:0/3487786360 >> 172.21.15.27:0/3487786360 conn(0x7fcd10078300 msgr2=0x7fcd10078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:14.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.203+0000 7fcd15aaa700 1 -- 172.21.15.27:0/3487786360 shutdown_connections 2024-06-06T08:42:14.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.203+0000 7fcd15aaa700 1 -- 172.21.15.27:0/3487786360 wait complete. 2024-06-06T08:42:14.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.204+0000 7fcd15aaa700 1 Processor -- start 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd15aaa700 1 -- start start 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd15aaa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101d40 0x7fcd10197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd15aaa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd101073f0 0x7fcd10198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd15aaa700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10198540 0x7fcd101a7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd15aaa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10111660 con 0x7fcd10101d40 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd15aaa700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101114e0 con 0x7fcd10198540 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd15aaa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101117e0 con 0x7fcd101073f0 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd0f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101d40 0x7fcd10197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd0f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101d40 0x7fcd10197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49530/0 (socket says 172.21.15.27:49530) 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd0f7fe700 1 -- 172.21.15.27:0/2160379066 learned_addr learned my addr 172.21.15.27:0/2160379066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.205+0000 7fcd0ffff700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10198540 0x7fcd101a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0effd700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd101073f0 0x7fcd10198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0f7fe700 1 -- 172.21.15.27:0/2160379066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd101073f0 msgr2=0x7fcd10198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:14.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd101073f0 0x7fcd10198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0f7fe700 1 -- 172.21.15.27:0/2160379066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10198540 msgr2=0x7fcd101a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:14.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10198540 0x7fcd101a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0f7fe700 1 -- 172.21.15.27:0/2160379066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd101a7730 con 0x7fcd10101d40 2024-06-06T08:42:14.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101d40 0x7fcd10197ac0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d980 tx=0x7fcd0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:14.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0cff9700 1 -- 172.21.15.27:0/2160379066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00016070 con 0x7fcd10101d40 2024-06-06T08:42:14.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0cff9700 1 -- 172.21.15.27:0/2160379066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd00004d10 con 0x7fcd10101d40 2024-06-06T08:42:14.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd0cff9700 1 -- 172.21.15.27:0/2160379066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000055f0 con 0x7fcd10101d40 2024-06-06T08:42:14.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.206+0000 7fcd15aaa700 1 -- 172.21.15.27:0/2160379066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd101a7a20 con 0x7fcd10101d40 2024-06-06T08:42:14.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.207+0000 7fcd15aaa700 1 -- 172.21.15.27:0/2160379066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd101a7e20 con 0x7fcd10101d40 2024-06-06T08:42:14.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.210+0000 7fcd0cff9700 1 -- 172.21.15.27:0/2160379066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcd00004750 con 0x7fcd10101d40 2024-06-06T08:42:14.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.212+0000 7fcd0cff9700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fccf80a2e10 0x7fccf80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.212+0000 7fcd0effd700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fccf80a2e10 0x7fccf80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.212+0000 7fcd2032db80 1 Processor -- start 2024-06-06T08:42:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.212+0000 7fcd0cff9700 1 -- 172.21.15.27:0/2160379066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fcd000cf530 con 0x7fcd10101d40 2024-06-06T08:42:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.212+0000 7fcd2032db80 1 -- start start 2024-06-06T08:42:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 0x5623b39e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39e1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd2032db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5623b38d3f60 con 0x5623b39e7880 2024-06-06T08:42:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd2032db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5623b3932850 con 0x5623b39e6f60 2024-06-06T08:42:14.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd2032db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5623b389a460 con 0x5623b39e17a0 2024-06-06T08:42:14.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd0effd700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fccf80a2e10 0x7fccf80a52d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd040098e0 tx=0x7fcd0400b540 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:14.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd0effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd0effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39e1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49538/0 (socket says 172.21.15.27:49538) 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd0effd700 1 -- 172.21.15.27:0/3734733755 learned_addr learned my addr 172.21.15.27:0/3734733755 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.213+0000 7fcd0ffff700 1 --2- 172.21.15.27:0/3734733755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.214+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/3734733755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 0x5623b39e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.214+0000 7fcd0ffff700 1 -- 172.21.15.27:0/3734733755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 msgr2=0x5623b39e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.214+0000 7fcd0ffff700 1 --2- 172.21.15.27:0/3734733755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 0x5623b39e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.214+0000 7fcd0ffff700 1 -- 172.21.15.27:0/3734733755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 msgr2=0x5623b39e1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.214+0000 7fcd0ffff700 1 --2- 172.21.15.27:0/3734733755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39e1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.214+0000 7fcd0ffff700 1 -- 172.21.15.27:0/3734733755 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b38818d0 con 0x5623b39e17a0 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.214+0000 7fcd0ffff700 1 --2- 172.21.15.27:0/3734733755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd08003e90 tx=0x7fcd08009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59bb19beae4390da server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.214+0000 7fccf5ffb700 1 -- 172.21.15.27:0/3734733755 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08004bb0 con 0x5623b39e17a0 2024-06-06T08:42:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.215+0000 7fccf5ffb700 1 -- 172.21.15.27:0/3734733755 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd080048d0 con 0x5623b39e17a0 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.215+0000 7fcd2032db80 1 -- 172.21.15.27:0/3734733755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 msgr2=0x5623b39e1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.215+0000 7fcd2032db80 1 --2- 172.21.15.27:0/3734733755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd08003e90 tx=0x7fcd08009c20 comp rx=0 tx=0).stop 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.216+0000 7fcd2032db80 1 -- 172.21.15.27:0/3734733755 shutdown_connections 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.216+0000 7fcd2032db80 1 --2- 172.21.15.27:0/3734733755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1bc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.216+0000 7fcd2032db80 1 --2- 172.21.15.27:0/3734733755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 0x5623b39e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.216+0000 7fcd2032db80 1 --2- 172.21.15.27:0/3734733755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39e1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.216+0000 7fcd2032db80 1 -- 172.21.15.27:0/3734733755 >> 172.21.15.27:0/3734733755 conn(0x5623b390e080 msgr2=0x5623b39dc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.216+0000 7fcd2032db80 1 -- 172.21.15.27:0/3734733755 shutdown_connections 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.216+0000 7fcd2032db80 1 -- 172.21.15.27:0/3734733755 wait complete. 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.216+0000 7fcd2032db80 1 -- start start 2024-06-06T08:42:14.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd2032db80 1 Processor -- start 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e17a0 0x5623b39ed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e6f60 0x5623b39ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd2032db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5623b38d2fc0 con 0x5623b39e7880 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd2032db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5623b392e810 con 0x5623b39e17a0 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd0f7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e17a0 0x5623b39ed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd0ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd0ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39ead00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49548/0 (socket says 172.21.15.27:49548) 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd0ffff700 1 -- 172.21.15.27:0/2560181529 learned_addr learned my addr 172.21.15.27:0/2560181529 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd2032db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5623b39ee2b0 con 0x5623b39e6f60 2024-06-06T08:42:14.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.217+0000 7fcd0effd700 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e6f60 0x5623b39ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcd2032db80 10 client.0 _ll_register_callbacks cb 0x5623b38c85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:14.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcd0ffff700 1 -- 172.21.15.27:0/2560181529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e6f60 msgr2=0x5623b39ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcd0ffff700 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e6f60 0x5623b39ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcd0ffff700 1 -- 172.21.15.27:0/2560181529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e17a0 msgr2=0x5623b39ed4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcd0ffff700 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e17a0 0x5623b39ed4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcd0ffff700 1 -- 172.21.15.27:0/2560181529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b38f8750 con 0x5623b39e7880 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e17a0 0x5623b39ed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcd2032db80 1 -- 172.21.15.27:0/2560181529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623b393fd50 con 0x5623b39e7880 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcd0ffff700 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39ead00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcd08007a80 tx=0x7fcd080095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8463486b13014d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.218+0000 7fcceeffd700 10 client.15984 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcceeffd700 1 -- 172.21.15.27:0/2560181529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0802c050 con 0x5623b39e7880 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcceeffd700 1 -- 172.21.15.27:0/2560181529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fcd08015690 con 0x5623b39e7880 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcceeffd700 1 -- 172.21.15.27:0/2560181529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd08022070 con 0x5623b39e7880 2024-06-06T08:42:14.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcd2032db80 10 client.15984 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcd2032db80 1 -- 172.21.15.27:0/2560181529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5623b3a00e60 con 0x5623b39e7880 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcceeffd700 1 -- 172.21.15.27:0/2560181529 <== mon.0 v2:172.21.15.27:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcd08015910 con 0x5623b39e7880 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcd2032db80 10 client.15984 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcd2032db80 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcd2032db80 20 client.15984 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcd2032db80 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5623b3a03330 0x5623b3a05720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcd2032db80 4 client.15984 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.219+0000 7fcd2032db80 1 -- 172.21.15.27:0/2560181529 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623b393a2d0 con 0x5623b3a03330 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.220+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5623b3a03330 0x5623b3a05720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.220+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5623b3a03330 0x5623b3a05720 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb96d5fec91edadb server_cookie=474300bde2c07ef8 in_seq=0 out_seq=0 2024-06-06T08:42:14.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.220+0000 7fcceeffd700 10 client.15984 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:14.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcceeffd700 1 -- 172.21.15.27:0/2560181529 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5623b393a2d0 con 0x5623b3a03330 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcceeffd700 10 client.15984 handle_command_reply: tid=0 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcd2032db80 1 client.15984 shutdown 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcd2032db80 1 -- 172.21.15.27:0/2560181529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 msgr2=0x5623b39ead00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcd2032db80 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39ead00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcd08007a80 tx=0x7fcd080095e0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcd2032db80 1 -- 172.21.15.27:0/2560181529 shutdown_connections 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcd2032db80 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5623b3a03330 0x5623b3a05720 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcceeffd700 0 client.15984 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcd2032db80 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e7880 0x5623b39ead00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcd2032db80 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e17a0 0x5623b39ed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcd2032db80 1 --2- 172.21.15.27:0/2560181529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e6f60 0x5623b39ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.221+0000 7fcd2032db80 1 -- 172.21.15.27:0/2560181529 >> 172.21.15.27:0/2560181529 conn(0x5623b390e080 msgr2=0x5623b39db700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.222+0000 7fcd2032db80 1 -- 172.21.15.27:0/2560181529 shutdown_connections 2024-06-06T08:42:14.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.222+0000 7fcd2032db80 1 -- 172.21.15.27:0/2560181529 wait complete. 2024-06-06T08:42:14.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.222+0000 7fcd2032db80 20 client.15984 trim_cache size 0 max 16384 2024-06-06T08:42:14.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.263+0000 7fcd2032db80 1 Processor -- start 2024-06-06T08:42:14.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.263+0000 7fcd2032db80 1 -- start start 2024-06-06T08:42:14.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.263+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.263+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e6f60 0x5623b39e9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.263+0000 7fcd0f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.263+0000 7fcd0f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:35452/0 (socket says 172.21.15.27:35452) 2024-06-06T08:42:14.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.263+0000 7fcd0effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e6f60 0x5623b39e9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd0effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e6f60 0x5623b39e9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49552/0 (socket says 172.21.15.27:49552) 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd0f7fe700 1 -- 172.21.15.27:0/3262764890 learned_addr learned my addr 172.21.15.27:0/3262764890 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd2032db80 1 --2- 172.21.15.27:0/3262764890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e7880 0x5623b39e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd2032db80 1 -- 172.21.15.27:0/3262764890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101af2f0 con 0x5623b39e6f60 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd2032db80 1 -- 172.21.15.27:0/3262764890 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101acb80 con 0x5623b39e7880 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd2032db80 1 -- 172.21.15.27:0/3262764890 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101aa9f0 con 0x5623b39e17a0 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd0ffff700 1 --2- 172.21.15.27:0/3262764890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e7880 0x5623b39e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd0effd700 1 -- 172.21.15.27:0/3262764890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 msgr2=0x5623b39e1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd0effd700 1 --2- 172.21.15.27:0/3262764890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd0effd700 1 -- 172.21.15.27:0/3262764890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e7880 msgr2=0x5623b39e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd0effd700 1 --2- 172.21.15.27:0/3262764890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e7880 0x5623b39e82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.264+0000 7fcd0effd700 1 -- 172.21.15.27:0/3262764890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b39f7480 con 0x5623b39e6f60 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.265+0000 7fcd0effd700 1 --2- 172.21.15.27:0/3262764890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e6f60 0x5623b39e9960 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd04006010 tx=0x7fcd0401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbe0e94866a08c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.265+0000 7fccf57fa700 1 -- 172.21.15.27:0/3262764890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04037070 con 0x5623b39e6f60 2024-06-06T08:42:14.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.265+0000 7fccf57fa700 1 -- 172.21.15.27:0/3262764890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0400f940 con 0x5623b39e6f60 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.265+0000 7fccf57fa700 1 -- 172.21.15.27:0/3262764890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd040328d0 con 0x5623b39e6f60 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.266+0000 7fcd2032db80 1 -- 172.21.15.27:0/3262764890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e6f60 msgr2=0x5623b39e9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.266+0000 7fcd2032db80 1 --2- 172.21.15.27:0/3262764890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e6f60 0x5623b39e9960 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd04006010 tx=0x7fcd0401f000 comp rx=0 tx=0).stop 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.266+0000 7fcd2032db80 1 -- 172.21.15.27:0/3262764890 shutdown_connections 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.266+0000 7fcd2032db80 1 --2- 172.21.15.27:0/3262764890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e7880 0x5623b39e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.266+0000 7fcd2032db80 1 --2- 172.21.15.27:0/3262764890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e17a0 0x5623b39e1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.266+0000 7fcd2032db80 1 --2- 172.21.15.27:0/3262764890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e6f60 0x5623b39e9960 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.266+0000 7fcd2032db80 1 -- 172.21.15.27:0/3262764890 >> 172.21.15.27:0/3262764890 conn(0x5623b390e080 msgr2=0x5623b39ebff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.267+0000 7fcd2032db80 1 -- 172.21.15.27:0/3262764890 shutdown_connections 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.267+0000 7fcd2032db80 1 -- 172.21.15.27:0/3262764890 wait complete. 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.267+0000 7fcd2032db80 1 -- start start 2024-06-06T08:42:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.267+0000 7fcd2032db80 1 Processor -- start 2024-06-06T08:42:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e17a0 0x5623b3a0ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 0x5623b3a0b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd2032db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e7880 0x5623b3a0b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd2032db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101ab040 con 0x5623b39e17a0 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd2032db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101ac4b0 con 0x5623b39e6f60 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd2032db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101ab710 con 0x5623b39e7880 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd2032db80 10 client.0 _ll_register_callbacks cb 0x5623b38c85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e17a0 0x5623b3a0ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e17a0 0x5623b3a0ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49568/0 (socket says 172.21.15.27:49568) 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0f7fe700 1 -- 172.21.15.27:0/808780227 learned_addr learned my addr 172.21.15.27:0/808780227 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0effd700 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 0x5623b3a0b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0effd700 1 -- 172.21.15.27:0/808780227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e7880 msgr2=0x5623b3a0b7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0effd700 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e7880 0x5623b3a0b7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0effd700 1 -- 172.21.15.27:0/808780227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e17a0 msgr2=0x5623b3a0ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0effd700 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e17a0 0x5623b3a0ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0effd700 1 -- 172.21.15.27:0/808780227 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b3a6d0a0 con 0x5623b39e6f60 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e17a0 0x5623b3a0ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcd0effd700 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 0x5623b3a0b260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009750 tx=0x7fcd0400fd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eba59aac5c08bf37 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.268+0000 7fcceeffd700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.269+0000 7fcceeffd700 1 -- 172.21.15.27:0/808780227 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04037070 con 0x5623b39e6f60 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.269+0000 7fcceeffd700 1 -- 172.21.15.27:0/808780227 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7fcd040327a0 con 0x5623b39e6f60 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.269+0000 7fcd2032db80 1 -- 172.21.15.27:0/808780227 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd0400f940 con 0x5623b39e6f60 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.269+0000 7fcceeffd700 1 -- 172.21.15.27:0/808780227 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd040059d0 con 0x5623b39e6f60 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.269+0000 7fcd2032db80 10 client.25535 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.269+0000 7fcd2032db80 1 -- 172.21.15.27:0/808780227 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5623b3a6e370 con 0x5623b39e6f60 2024-06-06T08:42:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.269+0000 7fcceeffd700 1 -- 172.21.15.27:0/808780227 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0400f940 con 0x5623b39e6f60 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.269+0000 7fcceeffd700 1 -- 172.21.15.27:0/808780227 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcd04024980 con 0x5623b39e6f60 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcd2032db80 10 client.25535 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcd2032db80 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcd2032db80 20 client.25535 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcd2032db80 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5623b3a03330 0x5623b3a70920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcd2032db80 4 client.25535 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcd2032db80 1 -- 172.21.15.27:0/808780227 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5623b38edf50 con 0x5623b3a03330 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5623b3a03330 0x5623b3a70920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcceeffd700 10 client.25535 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcd0f7fe700 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5623b3a03330 0x5623b3a70920 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3352c714b1cdc15e server_cookie=bb094ee3d41246f3 in_seq=0 out_seq=0 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcceeffd700 1 -- 172.21.15.27:0/808780227 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5623b38edf50 con 0x5623b3a03330 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.270+0000 7fcceeffd700 10 client.25535 handle_command_reply: tid=0 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 client.25535 shutdown 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 -- 172.21.15.27:0/808780227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 msgr2=0x5623b3a0b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 0x5623b3a0b260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009750 tx=0x7fcd0400fd20 comp rx=0 tx=0).stop 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 -- 172.21.15.27:0/808780227 shutdown_connections 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5623b39e7880 0x5623b3a0b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5623b3a03330 0x5623b3a70920 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5623b39e17a0 0x5623b3a0ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 --2- 172.21.15.27:0/808780227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623b39e6f60 0x5623b3a0b260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 -- 172.21.15.27:0/808780227 >> 172.21.15.27:0/808780227 conn(0x5623b390e080 msgr2=0x5623b39f6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcceeffd700 0 client.25535 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 -- 172.21.15.27:0/808780227 shutdown_connections 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 1 -- 172.21.15.27:0/808780227 wait complete. 2024-06-06T08:42:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.271+0000 7fcd2032db80 20 client.25535 trim_cache size 0 max 16384 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "76d000eb-0b36-4e9f-b8e7-93e384d9b73b": { 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "76d000eb-0b36-4e9f-b8e7-93e384d9b73b", 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:42:14.273 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:42:14.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 -- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fccf80a2e10 msgr2=0x7fccf80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:14.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fccf80a2e10 0x7fccf80a52d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd040098e0 tx=0x7fcd0400b540 comp rx=0 tx=0).stop 2024-06-06T08:42:14.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 -- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101d40 msgr2=0x7fcd10197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:14.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101d40 0x7fcd10197ac0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d980 tx=0x7fcd0000dc90 comp rx=0 tx=0).stop 2024-06-06T08:42:14.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 -- 172.21.15.27:0/2160379066 shutdown_connections 2024-06-06T08:42:14.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fccf80a2e10 0x7fccf80a52d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10198540 0x7fcd101a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd10101d40 0x7fcd10197ac0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 --2- 172.21.15.27:0/2160379066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd101073f0 0x7fcd10198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:14.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.272+0000 7fcd15aaa700 1 -- 172.21.15.27:0/2160379066 >> 172.21.15.27:0/2160379066 conn(0x7fcd10078300 msgr2=0x7fcd101001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:14.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.273+0000 7fcd15aaa700 1 -- 172.21.15.27:0/2160379066 shutdown_connections 2024-06-06T08:42:14.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:14.273+0000 7fcd15aaa700 1 -- 172.21.15.27:0/2160379066 wait complete. 2024-06-06T08:42:14.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:14 smithi027 ceph-mon[32335]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T08:42:14.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:14 smithi203 ceph-mon[36579]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T08:42:14.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:14 smithi176 ceph-mon[36786]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T08:42:15.066 INFO:tasks.cephfs.filesystem:scrub status for tag:76d000eb-0b36-4e9f-b8e7-93e384d9b73b - {'path': '/', 'tag': '76d000eb-0b36-4e9f-b8e7-93e384d9b73b', 'options': 'recursive,force'} 2024-06-06T08:42:15.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:15 smithi027 ceph-mon[32335]: scrub summary: idle 2024-06-06T08:42:15.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:15 smithi203 ceph-mon[36579]: scrub summary: idle 2024-06-06T08:42:15.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:15 smithi176 ceph-mon[36786]: scrub summary: idle 2024-06-06T08:42:16.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:16 smithi027 ceph-mon[32335]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-06-06T08:42:16.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:16 smithi203 ceph-mon[36579]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-06-06T08:42:16.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:16 smithi176 ceph-mon[36786]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-06-06T08:42:17.753 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:17 smithi203 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-06-06T08:42:17.753 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:17 smithi203 systemd[1]: Stopped Ceph mds.c for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:42:17.753 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:17 smithi203 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.133s CPU time. 2024-06-06T08:42:17.753 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:17 smithi203 systemd[1]: Starting Ceph mds.c for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:42:17.753 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:17 smithi203 podman[71514]: 2024-06-06 08:42:17.631003619 +0000 UTC m=+0.018244078 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:42:18.064 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:17 smithi203 podman[71514]: 2024-06-06 08:42:17.755178775 +0000 UTC m=+0.142419221 container create 1faa63afe90c8d67e78f665f766374a336597d59d3cb6deb71d433a0150f0214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-06T08:42:18.065 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:17 smithi203 podman[71514]: 2024-06-06 08:42:17.975099517 +0000 UTC m=+0.362339956 container init 1faa63afe90c8d67e78f665f766374a336597d59d3cb6deb71d433a0150f0214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T08:42:18.065 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:17 smithi203 podman[71514]: 2024-06-06 08:42:17.977955323 +0000 UTC m=+0.365195769 container start 1faa63afe90c8d67e78f665f766374a336597d59d3cb6deb71d433a0150f0214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:42:18.065 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:18 smithi203 ceph-mds[71533]: -- 172.21.15.203:0/907319242 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aeda6f5a40 con 0x55aeda6c9000 2024-06-06T08:42:18.065 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:18 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[71528]: starting mds.c at 2024-06-06T08:42:18.065 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:18 smithi203 bash[71514]: 1faa63afe90c8d67e78f665f766374a336597d59d3cb6deb71d433a0150f0214 2024-06-06T08:42:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.405+0000 7f9c4b363640 1 -- 172.21.15.27:0/338964180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4410bfe0 msgr2=0x7f9c4410e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.405+0000 7f9c4b363640 1 --2- 172.21.15.27:0/338964180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4410bfe0 0x7f9c4410e430 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009ca0 tx=0x7f9c3c02a950 comp rx=0 tx=0).stop 2024-06-06T08:42:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.405+0000 7f9c4b363640 1 -- 172.21.15.27:0/338964180 shutdown_connections 2024-06-06T08:42:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.405+0000 7f9c4b363640 1 --2- 172.21.15.27:0/338964180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4410bfe0 0x7f9c4410e430 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.405+0000 7f9c4b363640 1 --2- 172.21.15.27:0/338964180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c44103530 0x7f9c4410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.405+0000 7f9c4b363640 1 --2- 172.21.15.27:0/338964180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c44102c10 0x7f9c44102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.405+0000 7f9c4b363640 1 -- 172.21.15.27:0/338964180 >> 172.21.15.27:0/338964180 conn(0x7f9c44077ab0 msgr2=0x7f9c44077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.405+0000 7f9c4b363640 1 -- 172.21.15.27:0/338964180 shutdown_connections 2024-06-06T08:42:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.405+0000 7f9c4b363640 1 -- 172.21.15.27:0/338964180 wait complete. 2024-06-06T08:42:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c4b363640 1 Processor -- start 2024-06-06T08:42:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c4b363640 1 -- start start 2024-06-06T08:42:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c4b363640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c44102c10 0x7f9c44100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c4b363640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c44103530 0x7f9c44100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c4b363640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c4410bfe0 0x7f9c441012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c4b363640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44111d90 con 0x7f9c44102c10 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c4b363640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44111c10 con 0x7f9c4410bfe0 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c4b363640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44111f10 con 0x7f9c44103530 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c490ff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c44102c10 0x7f9c44100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c490ff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c44102c10 0x7f9c44100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49578/0 (socket says 172.21.15.27:49578) 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c490ff640 1 -- 172.21.15.27:0/3620378081 learned_addr learned my addr 172.21.15.27:0/3620378081 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.406+0000 7f9c488fe640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c44103530 0x7f9c44100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c49900640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c4410bfe0 0x7f9c441012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c490ff640 1 -- 172.21.15.27:0/3620378081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c44103530 msgr2=0x7f9c44100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c490ff640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c44103530 0x7f9c44100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c490ff640 1 -- 172.21.15.27:0/3620378081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c4410bfe0 msgr2=0x7f9c441012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c490ff640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c4410bfe0 0x7f9c441012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c490ff640 1 -- 172.21.15.27:0/3620378081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c441a6670 con 0x7f9c44102c10 2024-06-06T08:42:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c488fe640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c44103530 0x7f9c44100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c49900640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c4410bfe0 0x7f9c441012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:42:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c490ff640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c44102c10 0x7f9c44100860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000d950 tx=0x7f9c4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c3a7fc640 1 -- 172.21.15.27:0/3620378081 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40017070 con 0x7f9c44102c10 2024-06-06T08:42:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c4b363640 1 -- 172.21.15.27:0/3620378081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c441a6960 con 0x7f9c44102c10 2024-06-06T08:42:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c3a7fc640 1 -- 172.21.15.27:0/3620378081 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c40009c30 con 0x7f9c44102c10 2024-06-06T08:42:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c3a7fc640 1 -- 172.21.15.27:0/3620378081 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4000f6b0 con 0x7f9c44102c10 2024-06-06T08:42:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.407+0000 7f9c4b363640 1 -- 172.21.15.27:0/3620378081 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c441a6f00 con 0x7f9c44102c10 2024-06-06T08:42:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.409+0000 7f9c3a7fc640 1 -- 172.21.15.27:0/3620378081 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9c400109d0 con 0x7f9c44102c10 2024-06-06T08:42:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.410+0000 7f9c4e07c740 1 Processor -- start 2024-06-06T08:42:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c4e07c740 1 -- start start 2024-06-06T08:42:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c3a7fc640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9c300a2da0 0x7f9c300a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90170 0x564934d90550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934d9ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d9b350 0x564934d9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564934cf44f0 con 0x564934d90a90 2024-06-06T08:42:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564934c92870 con 0x564934d9b350 2024-06-06T08:42:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564934cf04b0 con 0x564934d90170 2024-06-06T08:42:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c49900640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9c300a2da0 0x7f9c300a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c490ff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90170 0x564934d90550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c490ff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90170 0x564934d90550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:35486/0 (socket says 172.21.15.27:35486) 2024-06-06T08:42:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c490ff640 1 -- 172.21.15.27:0/2689358980 learned_addr learned my addr 172.21.15.27:0/2689358980 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c488fe640 1 --2- 172.21.15.27:0/2689358980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934d9ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c490ff640 1 -- 172.21.15.27:0/2689358980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d9b350 msgr2=0x564934d9d740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c490ff640 1 --2- 172.21.15.27:0/2689358980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d9b350 0x564934d9d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c490ff640 1 -- 172.21.15.27:0/2689358980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 msgr2=0x564934d9ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c490ff640 1 --2- 172.21.15.27:0/2689358980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934d9ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c490ff640 1 -- 172.21.15.27:0/2689358980 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564934cae9b0 con 0x564934d90170 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.411+0000 7f9c488fe640 1 --2- 172.21.15.27:0/2689358980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934d9ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.412+0000 7f9c490ff640 1 --2- 172.21.15.27:0/2689358980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90170 0x564934d90550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c40097f50 tx=0x7f9c400661b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caadc5168218dc64 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.412+0000 7f9c3a7fc640 1 -- 172.21.15.27:0/3620378081 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9c400cf420 con 0x7f9c44102c10 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.412+0000 7f9c49900640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9c300a2da0 0x7f9c300a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c005e30 tx=0x7f9c3c005d80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.412+0000 7f9c2f7fe640 1 -- 172.21.15.27:0/2689358980 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40009de0 con 0x564934d90170 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.412+0000 7f9c2f7fe640 1 -- 172.21.15.27:0/2689358980 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c4009b580 con 0x564934d90170 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.412+0000 7f9c2f7fe640 1 -- 172.21.15.27:0/2689358980 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40097bf0 con 0x564934d90170 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2689358980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90170 msgr2=0x564934d90550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2689358980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90170 0x564934d90550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c40097f50 tx=0x7f9c400661b0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2689358980 shutdown_connections 2024-06-06T08:42:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2689358980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d9b350 0x564934d9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2689358980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934d9ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2689358980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90170 0x564934d90550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2689358980 >> 172.21.15.27:0/2689358980 conn(0x564934d8c680 msgr2=0x564934d8ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2689358980 shutdown_connections 2024-06-06T08:42:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2689358980 wait complete. 2024-06-06T08:42:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 -- start start 2024-06-06T08:42:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.413+0000 7f9c4e07c740 1 Processor -- start 2024-06-06T08:42:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90a90 0x564934d98850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 0x564934d98d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564934daef90 con 0x564934d90170 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564934c7dae0 con 0x564934d90a90 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564934daf110 con 0x564934d9b350 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c490ff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c490ff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49608/0 (socket says 172.21.15.27:49608) 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c490ff640 1 -- 172.21.15.27:0/2725902740 learned_addr learned my addr 172.21.15.27:0/2725902740 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c488fe640 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90a90 0x564934d98850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c4e07c740 10 client.0 _ll_register_callbacks cb 0x564934cb69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c49900640 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 0x564934d98d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c490ff640 1 -- 172.21.15.27:0/2725902740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 msgr2=0x564934d98d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c490ff640 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 0x564934d98d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c490ff640 1 -- 172.21.15.27:0/2725902740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90a90 msgr2=0x564934d98850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c490ff640 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90a90 0x564934d98850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c490ff640 1 -- 172.21.15.27:0/2725902740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564934a08210 con 0x564934d90170 2024-06-06T08:42:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c49900640 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 0x564934d98d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.414+0000 7f9c488fe640 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90a90 0x564934d98850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c490ff640 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98310 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9c400a3cc0 tx=0x7f9c40010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea146296186ec29 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c2cff9640 1 -- 172.21.15.27:0/2725902740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40020070 con 0x564934d90170 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c2cff9640 1 -- 172.21.15.27:0/2725902740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9c400cf170 con 0x564934d90170 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2725902740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c4009b580 con 0x564934d90170 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c2cff9640 1 -- 172.21.15.27:0/2725902740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c4006b030 con 0x564934d90170 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c4e07c740 10 client.16005 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2725902740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564934ad4620 con 0x564934d90170 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c2cff9640 1 -- 172.21.15.27:0/2725902740 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c4009b580 con 0x564934d90170 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.415+0000 7f9c2cff9640 1 -- 172.21.15.27:0/2725902740 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9c400a3d60 con 0x564934d90170 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.416+0000 7f9c4e07c740 10 client.16005 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.416+0000 7f9c4e07c740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.416+0000 7f9c4e07c740 20 client.16005 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.416+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564934dbc350 0x564934dbe740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.416+0000 7f9c4e07c740 4 client.16005 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.416+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2725902740 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564934d07f10 con 0x564934dbc350 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.416+0000 7f9c488fe640 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564934dbc350 0x564934dbe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.416+0000 7f9c488fe640 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564934dbc350 0x564934dbe740 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a36d9d065526d60 server_cookie=402f816f5cf87f85 in_seq=0 out_seq=0 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.416+0000 7f9c2cff9640 10 client.16005 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c2cff9640 1 -- 172.21.15.27:0/2725902740 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564934d07f10 con 0x564934dbc350 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c2cff9640 10 client.16005 handle_command_reply: tid=0 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c4e07c740 1 client.16005 shutdown 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2725902740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 msgr2=0x564934d98310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98310 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9c400a3cc0 tx=0x7f9c40010cc0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2725902740 shutdown_connections 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564934dbc350 0x564934dbe740 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c2cff9640 0 client.16005 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 0x564934d98d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90a90 0x564934d98850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/2725902740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98310 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.417+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2725902740 >> 172.21.15.27:0/2725902740 conn(0x564934d8c680 msgr2=0x564934d92400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.418+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2725902740 shutdown_connections 2024-06-06T08:42:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.418+0000 7f9c4e07c740 1 -- 172.21.15.27:0/2725902740 wait complete. 2024-06-06T08:42:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.418+0000 7f9c4e07c740 20 client.16005 trim_cache size 0 max 16384 2024-06-06T08:42:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.433+0000 7f9c4e07c740 1 Processor -- start 2024-06-06T08:42:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.433+0000 7f9c4e07c740 1 -- start start 2024-06-06T08:42:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.433+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.433+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90a90 0x564934dc1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.433+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d9b350 0x564934dc1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.433+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441ab210 con 0x564934d90170 2024-06-06T08:42:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.433+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441a8ae0 con 0x564934d9b350 2024-06-06T08:42:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.433+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4404f760 con 0x564934d90a90 2024-06-06T08:42:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c490ff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c490ff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49612/0 (socket says 172.21.15.27:49612) 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c490ff640 1 -- 172.21.15.27:0/3428225629 learned_addr learned my addr 172.21.15.27:0/3428225629 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c488fe640 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90a90 0x564934dc1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c49900640 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d9b350 0x564934dc1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c490ff640 1 -- 172.21.15.27:0/3428225629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90a90 msgr2=0x564934dc1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c490ff640 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90a90 0x564934dc1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c490ff640 1 -- 172.21.15.27:0/3428225629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d9b350 msgr2=0x564934dc1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c490ff640 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d9b350 0x564934dc1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c490ff640 1 -- 172.21.15.27:0/3428225629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564934d90880 con 0x564934d90170 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c488fe640 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90a90 0x564934dc1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c490ff640 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9c40097f50 tx=0x7f9c4000df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7d35a1eca42b40 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c2effd640 1 -- 172.21.15.27:0/3428225629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40020070 con 0x564934d90170 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c2effd640 1 -- 172.21.15.27:0/3428225629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c40097d60 con 0x564934d90170 2024-06-06T08:42:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.434+0000 7f9c2effd640 1 -- 172.21.15.27:0/3428225629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40099880 con 0x564934d90170 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.435+0000 7f9c4e07c740 1 -- 172.21.15.27:0/3428225629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 msgr2=0x564934d98bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.435+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9c40097f50 tx=0x7f9c4000df40 comp rx=0 tx=0).stop 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.435+0000 7f9c4e07c740 1 -- 172.21.15.27:0/3428225629 shutdown_connections 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.435+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d9b350 0x564934dc1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.435+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d90a90 0x564934dc1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.435+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/3428225629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90170 0x564934d98bb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.435+0000 7f9c4e07c740 1 -- 172.21.15.27:0/3428225629 >> 172.21.15.27:0/3428225629 conn(0x564934d8c680 msgr2=0x564934d9a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.435+0000 7f9c4e07c740 1 -- 172.21.15.27:0/3428225629 shutdown_connections 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.435+0000 7f9c4e07c740 1 -- 172.21.15.27:0/3428225629 wait complete. 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c4e07c740 1 -- start start 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c4e07c740 1 Processor -- start 2024-06-06T08:42:18.438 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90170 0x564934dc1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934dbf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c4e07c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 0x564934dbf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c440739b0 con 0x564934d90a90 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441a8410 con 0x564934d90170 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c4e07c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441a7670 con 0x564934d9b350 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c4e07c740 10 client.0 _ll_register_callbacks cb 0x7f9c4006bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c488fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934dbf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c488fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934dbf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49628/0 (socket says 172.21.15.27:49628) 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c488fe640 1 -- 172.21.15.27:0/1387611394 learned_addr learned my addr 172.21.15.27:0/1387611394 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.436+0000 7f9c490ff640 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90170 0x564934dc1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c49900640 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 0x564934dbf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c488fe640 1 -- 172.21.15.27:0/1387611394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 msgr2=0x564934dbf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c488fe640 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 0x564934dbf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c488fe640 1 -- 172.21.15.27:0/1387611394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90170 msgr2=0x564934dc1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c488fe640 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90170 0x564934dc1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c488fe640 1 -- 172.21.15.27:0/1387611394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564934dbfad0 con 0x564934d90a90 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c488fe640 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934dbf020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9c34005e00 tx=0x7f9c34013a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe72fc37e489d20 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c2cff9640 1 -- 172.21.15.27:0/1387611394 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34005220 con 0x564934d90a90 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c2cff9640 1 -- 172.21.15.27:0/1387611394 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9474+0+0 (secure 0 0 0) 0x7f9c340053c0 con 0x564934d90a90 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c4e07c740 1 -- 172.21.15.27:0/1387611394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c40097d60 con 0x564934d90a90 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.437+0000 7f9c2cff9640 1 -- 172.21.15.27:0/1387611394 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c3402e050 con 0x564934d90a90 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.438+0000 7f9c4e07c740 10 client.16011 fetch_fsmap learned FSMap version 25 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.438+0000 7f9c4e07c740 1 -- 172.21.15.27:0/1387611394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564934e26a60 con 0x564934d90a90 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.438+0000 7f9c2cff9640 1 -- 172.21.15.27:0/1387611394 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c340055c0 con 0x564934d90a90 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.438+0000 7f9c2cff9640 1 -- 172.21.15.27:0/1387611394 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9c34004cf0 con 0x564934d90a90 2024-06-06T08:42:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.438+0000 7f9c4e07c740 10 client.16011 fetch_fsmap finished waiting for FSMap version 25 2024-06-06T08:42:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.438+0000 7f9c4e07c740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.439+0000 7f9c4e07c740 20 client.16011 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.439+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564934e271d0 0x564934e295f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.439+0000 7f9c4e07c740 4 client.16011 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:42:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.439+0000 7f9c4e07c740 1 -- 172.21.15.27:0/1387611394 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9c40097d60 con 0x564934e271d0 2024-06-06T08:42:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.439+0000 7f9c490ff640 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564934e271d0 0x564934e295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.439+0000 7f9c490ff640 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564934e271d0 0x564934e295f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd377f6fd84499c7 server_cookie=321be8e3c43eb49b in_seq=0 out_seq=0 2024-06-06T08:42:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.439+0000 7f9c2cff9640 10 client.16011 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c2cff9640 1 -- 172.21.15.27:0/1387611394 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f9c40097d60 con 0x564934e271d0 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c2cff9640 10 client.16011 handle_command_reply: tid=0 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c4e07c740 1 client.16011 shutdown 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c4e07c740 1 -- 172.21.15.27:0/1387611394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 msgr2=0x564934dbf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934dbf020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9c34005e00 tx=0x7f9c34013a30 comp rx=0 tx=0).stop 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c4e07c740 1 -- 172.21.15.27:0/1387611394 shutdown_connections 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564934e271d0 0x564934e295f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564934d9b350 0x564934dbf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c2cff9640 0 client.16011 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564934d90a90 0x564934dbf020 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c4e07c740 1 --2- 172.21.15.27:0/1387611394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564934d90170 0x564934dc1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.443+0000 7f9c4e07c740 1 -- 172.21.15.27:0/1387611394 >> 172.21.15.27:0/1387611394 conn(0x564934d8c680 msgr2=0x564934d91a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4e07c740 1 -- 172.21.15.27:0/1387611394 shutdown_connections 2024-06-06T08:42:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4e07c740 1 -- 172.21.15.27:0/1387611394 wait complete. 2024-06-06T08:42:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4e07c740 20 client.16011 trim_cache size 0 max 16384 2024-06-06T08:42:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:42:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:42:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 -- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9c300a2da0 msgr2=0x7f9c300a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9c300a2da0 0x7f9c300a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c005e30 tx=0x7f9c3c005d80 comp rx=0 tx=0).stop 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 -- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c44102c10 msgr2=0x7f9c44100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c44102c10 0x7f9c44100860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000d950 tx=0x7f9c4000de10 comp rx=0 tx=0).stop 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 -- 172.21.15.27:0/3620378081 shutdown_connections 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f9c300a2da0 0x7f9c300a5260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c4410bfe0 0x7f9c441012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c44103530 0x7f9c44100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 --2- 172.21.15.27:0/3620378081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c44102c10 0x7f9c44100860 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 -- 172.21.15.27:0/3620378081 >> 172.21.15.27:0/3620378081 conn(0x7f9c44077ab0 msgr2=0x7f9c440fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.444+0000 7f9c4b363640 1 -- 172.21.15.27:0/3620378081 shutdown_connections 2024-06-06T08:42:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:18.445+0000 7f9c4b363640 1 -- 172.21.15.27:0/3620378081 wait complete. 2024-06-06T08:42:18.450 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:42:18 smithi203 systemd[1]: Started Ceph mds.c for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:42:18.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:18 smithi027 ceph-mon[32335]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-06-06T08:42:18.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:18 smithi027 ceph-mon[32335]: daemon mds.c restarted 2024-06-06T08:42:18.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:18 smithi203 ceph-mon[36579]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-06-06T08:42:18.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:18 smithi203 ceph-mon[36579]: daemon mds.c restarted 2024-06-06T08:42:18.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:18 smithi176 ceph-mon[36786]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-06-06T08:42:18.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:18 smithi176 ceph-mon[36786]: daemon mds.c restarted 2024-06-06T08:42:19.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:19 smithi027 ceph-mon[32335]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T08:42:19.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:19 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6836/3498487306,v1:172.21.15.203:6837/3498487306] up:boot 2024-06-06T08:42:19.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:19 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:42:19.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:19 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:42:19.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:19 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:42:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:19 smithi203 ceph-mon[36579]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T08:42:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:19 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6836/3498487306,v1:172.21.15.203:6837/3498487306] up:boot 2024-06-06T08:42:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:19 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:42:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:19 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:42:19.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:19 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:42:19.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:19 smithi176 ceph-mon[36786]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T08:42:19.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:19 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6836/3498487306,v1:172.21.15.203:6837/3498487306] up:boot 2024-06-06T08:42:19.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:19 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-06-06T08:42:19.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:19 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:42:19.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:19 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:42:20.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:20 smithi203 ceph-mon[36579]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:42:20.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:20 smithi176 ceph-mon[36786]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:42:20.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:20 smithi027 ceph-mon[32335]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:42:22.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:22 smithi027 ceph-mon[32335]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:42:22.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:22 smithi203 ceph-mon[36579]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:42:22.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:22 smithi176 ceph-mon[36786]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:42:24.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:24 smithi203 ceph-mon[36579]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-06-06T08:42:24.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:24 smithi176 ceph-mon[36786]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-06-06T08:42:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:24 smithi027 ceph-mon[32335]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-06-06T08:42:26.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:26 smithi203 ceph-mon[36579]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-06-06T08:42:26.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:26 smithi176 ceph-mon[36786]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-06-06T08:42:26.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:26 smithi027 ceph-mon[32335]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-06-06T08:42:28.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:28 smithi203 ceph-mon[36579]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-06-06T08:42:28.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:28 smithi176 ceph-mon[36786]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-06-06T08:42:28.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:28 smithi027 ceph-mon[32335]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 307 B/s wr, 6 op/s 2024-06-06T08:42:30.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:30 smithi203 ceph-mon[36579]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 282 B/s wr, 6 op/s 2024-06-06T08:42:30.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:30 smithi176 ceph-mon[36786]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 282 B/s wr, 6 op/s 2024-06-06T08:42:30.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:30 smithi027 ceph-mon[32335]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 282 B/s wr, 6 op/s 2024-06-06T08:42:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.541+0000 7f781bf22640 1 -- 172.21.15.27:0/3673021395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814106890 msgr2=0x7f7814106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.541+0000 7f781bf22640 1 --2- 172.21.15.27:0/3673021395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814106890 0x7f7814106c70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7810009930 tx=0x7f781002a420 comp rx=0 tx=0).stop 2024-06-06T08:42:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.541+0000 7f781bf22640 1 -- 172.21.15.27:0/3673021395 shutdown_connections 2024-06-06T08:42:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.541+0000 7f781bf22640 1 --2- 172.21.15.27:0/3673021395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7814101290 0x7f781410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.541+0000 7f781bf22640 1 --2- 172.21.15.27:0/3673021395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78141008d0 0x7f7814100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.541+0000 7f781bf22640 1 --2- 172.21.15.27:0/3673021395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814106890 0x7f7814106c70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.541+0000 7f781bf22640 1 -- 172.21.15.27:0/3673021395 >> 172.21.15.27:0/3673021395 conn(0x7f7814077a20 msgr2=0x7f7814077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 -- 172.21.15.27:0/3673021395 shutdown_connections 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 -- 172.21.15.27:0/3673021395 wait complete. 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 Processor -- start 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 -- start start 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78141008d0 0x7f78141981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814101290 0x7f7814198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7814106890 0x7f781419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7814111da0 con 0x7f7814101290 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7814111c20 con 0x7f78141008d0 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f781bf22640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7814111f20 con 0x7f7814106890 2024-06-06T08:42:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.542+0000 7f78194bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814101290 0x7f7814198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f78194bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814101290 0x7f7814198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56534/0 (socket says 172.21.15.27:56534) 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f78194bd640 1 -- 172.21.15.27:0/3671128393 learned_addr learned my addr 172.21.15.27:0/3671128393 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f781a4bf640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7814106890 0x7f781419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f7819cbe640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78141008d0 0x7f78141981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f78194bd640 1 -- 172.21.15.27:0/3671128393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7814106890 msgr2=0x7f781419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f78194bd640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7814106890 0x7f781419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f78194bd640 1 -- 172.21.15.27:0/3671128393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78141008d0 msgr2=0x7f78141981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f78194bd640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78141008d0 0x7f78141981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f78194bd640 1 -- 172.21.15.27:0/3671128393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781419d1d0 con 0x7f7814101290 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f7819cbe640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78141008d0 0x7f78141981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f78194bd640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814101290 0x7f7814198720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f780400d950 tx=0x7f780400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f780affd640 1 -- 172.21.15.27:0/3671128393 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804017030 con 0x7f7814101290 2024-06-06T08:42:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f780affd640 1 -- 172.21.15.27:0/3671128393 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7804004020 con 0x7f7814101290 2024-06-06T08:42:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f781bf22640 1 -- 172.21.15.27:0/3671128393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781419d4c0 con 0x7f7814101290 2024-06-06T08:42:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f780affd640 1 -- 172.21.15.27:0/3671128393 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78040047a0 con 0x7f7814101290 2024-06-06T08:42:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.543+0000 7f781bf22640 1 -- 172.21.15.27:0/3671128393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78141a4d50 con 0x7f7814101290 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.545+0000 7f780affd640 1 -- 172.21.15.27:0/3671128393 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f78040041d0 con 0x7f7814101290 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f781ec35740 1 Processor -- start 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f781ec35740 1 -- start start 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f781ec35740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e248450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f781ec35740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e257220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f781ec35740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e259b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f781ec35740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561f5e1ac4f0 con 0x561f5e257760 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f781ec35740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561f5e14a870 con 0x561f5e248990 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f781ec35740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561f5e1a84b0 con 0x561f5e248070 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f78194bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e259b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f78194bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e259b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56536/0 (socket says 172.21.15.27:56536) 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f78194bd640 1 -- 172.21.15.27:0/3492104156 learned_addr learned my addr 172.21.15.27:0/3492104156 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f7819cbe640 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e257220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f781a4bf640 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e248450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.546+0000 7f780affd640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f78000a2da0 0x7f78000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f78194bd640 1 -- 172.21.15.27:0/3492104156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 msgr2=0x561f5e248450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f78194bd640 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e248450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f78194bd640 1 -- 172.21.15.27:0/3492104156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 msgr2=0x561f5e257220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f78194bd640 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e257220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f78194bd640 1 -- 172.21.15.27:0/3492104156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5e1669b0 con 0x561f5e257760 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f781a4bf640 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e248450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f781a4bf640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f78000a2da0 0x7f78000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f780affd640 1 -- 172.21.15.27:0/3671128393 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f78040cf2b0 con 0x7f7814101290 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f78194bd640 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e259b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f78040cc5b0 tx=0x7f7804021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c82e79d9a5ee7b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f77fffff640 1 -- 172.21.15.27:0/3492104156 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780409b1e0 con 0x561f5e257760 2024-06-06T08:42:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f781a4bf640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f78000a2da0 0x7f78000a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f780c005fd0 tx=0x7f780c004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f77fffff640 1 -- 172.21.15.27:0/3492104156 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780409bb10 con 0x561f5e257760 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.547+0000 7f77fffff640 1 -- 172.21.15.27:0/3492104156 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780409ad70 con 0x561f5e257760 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 -- 172.21.15.27:0/3492104156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 msgr2=0x561f5e259b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e259b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f78040cc5b0 tx=0x7f7804021000 comp rx=0 tx=0).stop 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 -- 172.21.15.27:0/3492104156 shutdown_connections 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e259b50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e257220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 --2- 172.21.15.27:0/3492104156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e248450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 -- 172.21.15.27:0/3492104156 >> 172.21.15.27:0/3492104156 conn(0x561f5e244680 msgr2=0x561f5e244a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 -- 172.21.15.27:0/3492104156 shutdown_connections 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 -- 172.21.15.27:0/3492104156 wait complete. 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 -- start start 2024-06-06T08:42:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.548+0000 7f781ec35740 1 Processor -- start 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781ec35740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e254580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781ec35740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e254ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781ec35740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e255000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781ec35740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561f5e25a330 con 0x561f5e257760 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781ec35740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x561f5e135ae0 con 0x561f5e248990 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781ec35740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561f5e25a4b0 con 0x561f5e248070 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781ec35740 10 client.0 _ll_register_callbacks cb 0x561f5e16e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781a4bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e255000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781a4bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e255000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56550/0 (socket says 172.21.15.27:56550) 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781a4bf640 1 -- 172.21.15.27:0/3433643274 learned_addr learned my addr 172.21.15.27:0/3433643274 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f78194bd640 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e254ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f7819cbe640 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e254580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781a4bf640 1 -- 172.21.15.27:0/3433643274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 msgr2=0x561f5e254580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781a4bf640 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e254580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781a4bf640 1 -- 172.21.15.27:0/3433643274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 msgr2=0x561f5e254ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781a4bf640 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e254ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781a4bf640 1 -- 172.21.15.27:0/3433643274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5dec0210 con 0x561f5e257760 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f7819cbe640 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e254580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f781a4bf640 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e255000 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f780c00ec10 tx=0x7f780c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201c6c95d3c887f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.549+0000 7f77fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f77fd7fa640 1 -- 172.21.15.27:0/3433643274 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c00fea0 con 0x561f5e257760 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f77fd7fa640 1 -- 172.21.15.27:0/3433643274 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f780c01e420 con 0x561f5e257760 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f781ec35740 1 -- 172.21.15.27:0/3433643274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780409bb10 con 0x561f5e257760 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f77fd7fa640 1 -- 172.21.15.27:0/3433643274 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f780c01e620 con 0x561f5e257760 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f781ec35740 10 client.16023 fetch_fsmap learned FSMap version 27 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f781ec35740 1 -- 172.21.15.27:0/3433643274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f5df8c620 con 0x561f5e257760 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f77fd7fa640 1 -- 172.21.15.27:0/3433643274 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780409bb10 con 0x561f5e257760 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f77fd7fa640 1 -- 172.21.15.27:0/3433643274 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f780c026af0 con 0x561f5e257760 2024-06-06T08:42:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f781ec35740 10 client.16023 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f781ec35740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f781ec35740 20 client.16023 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f781ec35740 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x561f5e267710 0x561f5e269b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f781ec35740 4 client.16023 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.550+0000 7f781ec35740 1 -- 172.21.15.27:0/3433643274 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f5e1bff10 con 0x561f5e267710 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.551+0000 7f7819cbe640 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x561f5e267710 0x561f5e269b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.551+0000 7f77fd7fa640 10 client.16023 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.551+0000 7f7819cbe640 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x561f5e267710 0x561f5e269b00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f23c9d3309fa545a server_cookie=3bb21c58e5fe7d42 in_seq=0 out_seq=0 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f77fd7fa640 1 -- 172.21.15.27:0/3433643274 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561f5e1bff10 con 0x561f5e267710 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f77fd7fa640 10 client.16023 handle_command_reply: tid=0 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 client.16023 shutdown 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 -- 172.21.15.27:0/3433643274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 msgr2=0x561f5e255000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e255000 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f780c00ec10 tx=0x7f780c004ab0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 -- 172.21.15.27:0/3433643274 shutdown_connections 2024-06-06T08:42:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x561f5e267710 0x561f5e269b00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e255000 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e254ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 --2- 172.21.15.27:0/3433643274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e254580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f77fd7fa640 0 client.16023 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 -- 172.21.15.27:0/3433643274 >> 172.21.15.27:0/3433643274 conn(0x561f5e244680 msgr2=0x561f5e24a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 -- 172.21.15.27:0/3433643274 shutdown_connections 2024-06-06T08:42:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 1 -- 172.21.15.27:0/3433643274 wait complete. 2024-06-06T08:42:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.552+0000 7f781ec35740 20 client.16023 trim_cache size 0 max 16384 2024-06-06T08:42:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.567+0000 7f781ec35740 1 Processor -- start 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.567+0000 7f781ec35740 1 -- start start 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781ec35740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e26f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781ec35740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e26f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781ec35740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e254540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781ec35740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f78141af660 con 0x561f5e257760 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781ec35740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f78141acf30 con 0x561f5e248990 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781ec35740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f78141aad80 con 0x561f5e248070 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781a4bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e254540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781a4bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e254540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56560/0 (socket says 172.21.15.27:56560) 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781a4bf640 1 -- 172.21.15.27:0/2762340194 learned_addr learned my addr 172.21.15.27:0/2762340194 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f7819cbe640 1 --2- 172.21.15.27:0/2762340194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e26f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781a4bf640 1 -- 172.21.15.27:0/2762340194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 msgr2=0x561f5e26f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781a4bf640 1 --2- 172.21.15.27:0/2762340194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e26f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781a4bf640 1 -- 172.21.15.27:0/2762340194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 msgr2=0x561f5e26f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781a4bf640 1 --2- 172.21.15.27:0/2762340194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e26f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781a4bf640 1 -- 172.21.15.27:0/2762340194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5e2440a0 con 0x561f5e257760 2024-06-06T08:42:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f781a4bf640 1 --2- 172.21.15.27:0/2762340194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e254540 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f780c026f40 tx=0x7f780c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23721623204ef6c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f77ff7fe640 1 -- 172.21.15.27:0/2762340194 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c00e7c0 con 0x561f5e257760 2024-06-06T08:42:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f77ff7fe640 1 -- 172.21.15.27:0/2762340194 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c00fca0 con 0x561f5e257760 2024-06-06T08:42:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.568+0000 7f77ff7fe640 1 -- 172.21.15.27:0/2762340194 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c026660 con 0x561f5e257760 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 -- 172.21.15.27:0/2762340194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 msgr2=0x561f5e254540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 --2- 172.21.15.27:0/2762340194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e254540 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f780c026f40 tx=0x7f780c009ea0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 -- 172.21.15.27:0/2762340194 shutdown_connections 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 --2- 172.21.15.27:0/2762340194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e257760 0x561f5e254540 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 --2- 172.21.15.27:0/2762340194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e248990 0x561f5e26f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 --2- 172.21.15.27:0/2762340194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e26f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 -- 172.21.15.27:0/2762340194 >> 172.21.15.27:0/2762340194 conn(0x561f5e244680 msgr2=0x561f5e24f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 -- 172.21.15.27:0/2762340194 shutdown_connections 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 -- 172.21.15.27:0/2762340194 wait complete. 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 -- start start 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.569+0000 7f781ec35740 1 Processor -- start 2024-06-06T08:42:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f781ec35740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e25fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f781ec35740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e248990 0x561f5e251000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f781ec35740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e257760 0x561f5e251540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f781ec35740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f78141ab3f0 con 0x561f5e248990 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f781ec35740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f78141ac860 con 0x561f5e257760 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f781ec35740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f78141abac0 con 0x561f5e248070 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f781ec35740 10 client.0 _ll_register_callbacks cb 0x7f780c009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f78194bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e248990 0x561f5e251000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f78194bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e248990 0x561f5e251000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56572/0 (socket says 172.21.15.27:56572) 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f78194bd640 1 -- 172.21.15.27:0/1693183373 learned_addr learned my addr 172.21.15.27:0/1693183373 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f781a4bf640 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e257760 0x561f5e251540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f7819cbe640 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e25fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f78194bd640 1 -- 172.21.15.27:0/1693183373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 msgr2=0x561f5e25fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f78194bd640 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e25fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f78194bd640 1 -- 172.21.15.27:0/1693183373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e257760 msgr2=0x561f5e251540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f78194bd640 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e257760 0x561f5e251540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f78194bd640 1 -- 172.21.15.27:0/1693183373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5e2444b0 con 0x561f5e248990 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f78194bd640 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e248990 0x561f5e251000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7804020000 tx=0x7f7804005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20f162210b385d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f77fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f77fd7fa640 1 -- 172.21.15.27:0/1693183373 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780409af50 con 0x561f5e248990 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f77fd7fa640 1 -- 172.21.15.27:0/1693183373 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f78040663e0 con 0x561f5e248990 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.570+0000 7f781ec35740 1 -- 172.21.15.27:0/1693183373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780c00fca0 con 0x561f5e248990 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f77fd7fa640 1 -- 172.21.15.27:0/1693183373 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f780409ad70 con 0x561f5e248990 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f781ec35740 10 client.16029 fetch_fsmap learned FSMap version 27 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f781ec35740 1 -- 172.21.15.27:0/1693183373 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f5e25e240 con 0x561f5e248990 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f77fd7fa640 1 -- 172.21.15.27:0/1693183373 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c00fca0 con 0x561f5e248990 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f77fd7fa640 1 -- 172.21.15.27:0/1693183373 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f78040cc8c0 con 0x561f5e248990 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f781ec35740 10 client.16029 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f781ec35740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f781ec35740 20 client.16029 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f781ec35740 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x561f5e2d23a0 0x561f5e2d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f781ec35740 4 client.16029 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:42:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f781ec35740 1 -- 172.21.15.27:0/1693183373 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f5e1b9c20 con 0x561f5e2d23a0 2024-06-06T08:42:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.571+0000 7f7819cbe640 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x561f5e2d23a0 0x561f5e2d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.572+0000 7f7819cbe640 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x561f5e2d23a0 0x561f5e2d4790 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996fca908efae833 server_cookie=f677bec785410b1e in_seq=0 out_seq=0 2024-06-06T08:42:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.572+0000 7f77fd7fa640 10 client.16029 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.575+0000 7f77fd7fa640 1 -- 172.21.15.27:0/1693183373 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561f5e1b9c20 con 0x561f5e2d23a0 2024-06-06T08:42:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.575+0000 7f77fd7fa640 10 client.16029 handle_command_reply: tid=0 2024-06-06T08:42:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.575+0000 7f781ec35740 1 client.16029 shutdown 2024-06-06T08:42:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 -- 172.21.15.27:0/1693183373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e248990 msgr2=0x561f5e251000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e248990 0x561f5e251000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7804020000 tx=0x7f7804005db0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 -- 172.21.15.27:0/1693183373 shutdown_connections 2024-06-06T08:42:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x561f5e2d23a0 0x561f5e2d4790 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561f5e257760 0x561f5e251540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561f5e248990 0x561f5e251000 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 --2- 172.21.15.27:0/1693183373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561f5e248070 0x561f5e25fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f77fd7fa640 0 client.16029 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 -- 172.21.15.27:0/1693183373 >> 172.21.15.27:0/1693183373 conn(0x561f5e244680 msgr2=0x561f5e242f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 -- 172.21.15.27:0/1693183373 shutdown_connections 2024-06-06T08:42:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 1 -- 172.21.15.27:0/1693183373 wait complete. 2024-06-06T08:42:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781ec35740 20 client.16029 trim_cache size 0 max 16384 2024-06-06T08:42:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:42:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:42:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:42:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 -- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f78000a2da0 msgr2=0x7f78000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f78000a2da0 0x7f78000a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f780c005fd0 tx=0x7f780c004a40 comp rx=0 tx=0).stop 2024-06-06T08:42:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 -- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814101290 msgr2=0x7f7814198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814101290 0x7f7814198720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f780400d950 tx=0x7f780400de10 comp rx=0 tx=0).stop 2024-06-06T08:42:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 -- 172.21.15.27:0/3671128393 shutdown_connections 2024-06-06T08:42:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f78000a2da0 0x7f78000a5260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7814106890 0x7f781419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7814101290 0x7f7814198720 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 --2- 172.21.15.27:0/3671128393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78141008d0 0x7f78141981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.576+0000 7f781bf22640 1 -- 172.21.15.27:0/3671128393 >> 172.21.15.27:0/3671128393 conn(0x7f7814077a20 msgr2=0x7f78141033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.577+0000 7f781bf22640 1 -- 172.21.15.27:0/3671128393 shutdown_connections 2024-06-06T08:42:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:31.577+0000 7f781bf22640 1 -- 172.21.15.27:0/3671128393 wait complete. 2024-06-06T08:42:32.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:32 smithi027 ceph-mon[32335]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:32.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:32 smithi203 ceph-mon[36579]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:32.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:32 smithi176 ceph-mon[36786]: pgmap v71: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:33.344 INFO:tasks.workunit.client.0.smithi027.stderr:Updating files: 93% (11787/12568) Updating files: 94% (11814/12568) Updating files: 95% (11940/12568) Updating files: 96% (12066/12568) Updating files: 97% (12191/12568) Updating files: 98% (12317/12568) Updating files: 99% (12443/12568) Updating files: 100% (12568/12568) Updating files: 100% (12568/12568), done. 2024-06-06T08:42:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:33 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:42:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:33 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:42:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:33 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:42:33.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:33 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:42:33.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:33 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:42:33.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:33 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:42:33.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:33 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:42:33.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:33 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:42:33.848 INFO:tasks.workunit.client.0.smithi027.stderr:Note: switching to '3d746ca5346f34eef7e826a1eb71e59e02b682ae'. 2024-06-06T08:42:33.848 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by switching back to a branch. 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr: git switch -c 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr:Or undo this operation with: 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr: git switch - 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-06-06T08:42:33.849 INFO:tasks.workunit.client.0.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-06T08:42:33.850 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-06-06T08:42:33.850 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at 3d746ca5346 Merge pull request #57870 from zdover23/wip-doc-2024-06-05-backport-57867-to-squid 2024-06-06T08:42:33.853 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-06T08:42:33.878 INFO:tasks.workunit.client.0.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-06T08:42:33.880 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T08:42:33.881 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-06T08:42:33.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:33 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:42:33.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:33 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:42:33.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:33 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:42:33.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:33 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:42:34.627 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-06T08:42:34.688 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-06T08:42:34.732 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T08:42:34.734 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T08:42:34.734 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-06T08:42:34.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:34 smithi203 ceph-mon[36579]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-06T08:42:34.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:34 smithi176 ceph-mon[36786]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-06T08:42:34.774 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T08:42:34.778 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:42:34.778 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-06T08:42:34.836 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-06-06T08:42:34.837 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-06-06T08:42:34.837 DEBUG:teuthology.orchestra.run.smithi027:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-06-06T08:42:34.901 INFO:tasks.workunit.client.0.smithi027.stderr:+ mkdir -p fsstress 2024-06-06T08:42:34.903 INFO:tasks.workunit.client.0.smithi027.stderr:+ pushd fsstress 2024-06-06T08:42:34.903 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-06-06T08:42:34.903 INFO:tasks.workunit.client.0.smithi027.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-06-06T08:42:34.955 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:34 smithi027 ceph-mon[32335]: pgmap v72: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-06T08:42:35.412 INFO:tasks.workunit.client.0.smithi027.stderr:+ tar xzf ltp-full.tgz 2024-06-06T08:42:36.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:36 smithi203 ceph-mon[36579]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:36.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:36 smithi176 ceph-mon[36786]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:36.955 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:36 smithi027 ceph-mon[32335]: pgmap v73: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:38.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:38 smithi176 ceph-mon[36786]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:38.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:38 smithi203 ceph-mon[36579]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:38.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:38 smithi027 ceph-mon[32335]: pgmap v74: 129 pgs: 129 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T08:42:40.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:40 smithi176 ceph-mon[36786]: pgmap v75: 129 pgs: 129 active+clean; 5.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 361 KiB/s wr, 11 op/s 2024-06-06T08:42:40.891 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:40 smithi203 ceph-mon[36579]: pgmap v75: 129 pgs: 129 active+clean; 5.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 361 KiB/s wr, 11 op/s 2024-06-06T08:42:40.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:40 smithi027 ceph-mon[32335]: pgmap v75: 129 pgs: 129 active+clean; 5.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 361 KiB/s wr, 11 op/s 2024-06-06T08:42:41.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:41 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:42:41.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:41 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:42:41.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:41 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:42:42.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:42 smithi176 ceph-mon[36786]: pgmap v76: 129 pgs: 129 active+clean; 5.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 361 KiB/s wr, 8 op/s 2024-06-06T08:42:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:42 smithi027 ceph-mon[32335]: pgmap v76: 129 pgs: 129 active+clean; 5.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 361 KiB/s wr, 8 op/s 2024-06-06T08:42:42.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:42 smithi203 ceph-mon[36579]: pgmap v76: 129 pgs: 129 active+clean; 5.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 361 KiB/s wr, 8 op/s 2024-06-06T08:42:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.667+0000 7f61fa929640 1 -- 172.21.15.27:0/3676035671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 msgr2=0x7f61f4103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.667+0000 7f61fa929640 1 --2- 172.21.15.27:0/3676035671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 0x7f61f4103010 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009900 tx=0x7f61e402a510 comp rx=0 tx=0).stop 2024-06-06T08:42:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.667+0000 7f61fa929640 1 -- 172.21.15.27:0/3676035671 shutdown_connections 2024-06-06T08:42:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.667+0000 7f61fa929640 1 --2- 172.21.15.27:0/3676035671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61f410bf70 0x7f61f410e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.667+0000 7f61fa929640 1 --2- 172.21.15.27:0/3676035671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f4103550 0x7f61f410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.667+0000 7f61fa929640 1 --2- 172.21.15.27:0/3676035671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 0x7f61f4103010 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.667+0000 7f61fa929640 1 -- 172.21.15.27:0/3676035671 >> 172.21.15.27:0/3676035671 conn(0x7f61f4077a20 msgr2=0x7f61f4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.667+0000 7f61fa929640 1 -- 172.21.15.27:0/3676035671 shutdown_connections 2024-06-06T08:42:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.667+0000 7f61fa929640 1 -- 172.21.15.27:0/3676035671 wait complete. 2024-06-06T08:42:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.668+0000 7f61fa929640 1 Processor -- start 2024-06-06T08:42:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.668+0000 7f61fa929640 1 -- start start 2024-06-06T08:42:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.668+0000 7f61fa929640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 0x7f61f4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.668+0000 7f61fa929640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f4103550 0x7f61f41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.668+0000 7f61fa929640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61f410bf70 0x7f61f419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.668+0000 7f61fa929640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f61f4111d70 con 0x7f61f4102c30 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.668+0000 7f61fa929640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f61f4111bf0 con 0x7f61f410bf70 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.668+0000 7f61fa929640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f61f4111ef0 con 0x7f61f4103550 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 0x7f61f4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 0x7f61f4198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45740/0 (socket says 172.21.15.27:45740) 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f3fff640 1 -- 172.21.15.27:0/162674518 learned_addr learned my addr 172.21.15.27:0/162674518 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f8ec6640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61f410bf70 0x7f61f419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f37fe640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f4103550 0x7f61f41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f3fff640 1 -- 172.21.15.27:0/162674518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f4103550 msgr2=0x7f61f41987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f3fff640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f4103550 0x7f61f41987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f3fff640 1 -- 172.21.15.27:0/162674518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61f410bf70 msgr2=0x7f61f419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f3fff640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61f410bf70 0x7f61f419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f3fff640 1 -- 172.21.15.27:0/162674518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f419d230 con 0x7f61f4102c30 2024-06-06T08:42:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f3fff640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 0x7f61f4198290 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f61e4006010 tx=0x7f61e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f17fa640 1 -- 172.21.15.27:0/162674518 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e403f030 con 0x7f61f4102c30 2024-06-06T08:42:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61fa929640 1 -- 172.21.15.27:0/162674518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f419d4c0 con 0x7f61f4102c30 2024-06-06T08:42:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f17fa640 1 -- 172.21.15.27:0/162674518 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e4003ce0 con 0x7f61f4102c30 2024-06-06T08:42:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61f17fa640 1 -- 172.21.15.27:0/162674518 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4035070 con 0x7f61f4102c30 2024-06-06T08:42:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.669+0000 7f61fa929640 1 -- 172.21.15.27:0/162674518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f41a4db0 con 0x7f61f4102c30 2024-06-06T08:42:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.672+0000 7f61f17fa640 1 -- 172.21.15.27:0/162674518 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f61e4042030 con 0x7f61f4102c30 2024-06-06T08:42:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.672+0000 7f61fd63c740 1 Processor -- start 2024-06-06T08:42:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.672+0000 7f61fd63c740 1 -- start start 2024-06-06T08:42:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.672+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138070 0x556fff138450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.672+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff147220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.672+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff147760 0x556fff149b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.672+0000 7f61fd63c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556fff09c4f0 con 0x556fff138990 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.672+0000 7f61fd63c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556fff03a870 con 0x556fff147760 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.672+0000 7f61fd63c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556fff0984b0 con 0x556fff138070 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff147220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45752/0 (socket says 172.21.15.27:45752) 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f3fff640 1 -- 172.21.15.27:0/2995922916 learned_addr learned my addr 172.21.15.27:0/2995922916 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f37fe640 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff147760 0x556fff149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f8ec6640 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138070 0x556fff138450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f3fff640 1 -- 172.21.15.27:0/2995922916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138070 msgr2=0x556fff138450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f3fff640 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138070 0x556fff138450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f3fff640 1 -- 172.21.15.27:0/2995922916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff147760 msgr2=0x556fff149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f3fff640 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff147760 0x556fff149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f3fff640 1 -- 172.21.15.27:0/2995922916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fff0569b0 con 0x556fff138990 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f37fe640 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff147760 0x556fff149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f17fa640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f61e00a2da0 0x7f61e00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f3fff640 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff147220 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f61e40ecff0 tx=0x7f61e40f8460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393d32125c50aedc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f8ec6640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f61e00a2da0 0x7f61e00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61ea7fc640 1 -- 172.21.15.27:0/2995922916 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40b80f0 con 0x556fff138990 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61f17fa640 1 -- 172.21.15.27:0/162674518 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f61e40efcf0 con 0x7f61f4102c30 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61ea7fc640 1 -- 172.21.15.27:0/2995922916 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e40be070 con 0x556fff138990 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.673+0000 7f61ea7fc640 1 -- 172.21.15.27:0/2995922916 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40d86f0 con 0x556fff138990 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61fd63c740 1 -- 172.21.15.27:0/2995922916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 msgr2=0x556fff147220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61f8ec6640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f61e00a2da0 0x7f61e00a5260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f61ec005fd0 tx=0x7f61ec009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff147220 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f61e40ecff0 tx=0x7f61e40f8460 comp rx=0 tx=0).stop 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61fd63c740 1 -- 172.21.15.27:0/2995922916 shutdown_connections 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff147760 0x556fff149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff147220 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2995922916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138070 0x556fff138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61fd63c740 1 -- 172.21.15.27:0/2995922916 >> 172.21.15.27:0/2995922916 conn(0x556fff134680 msgr2=0x556fff134a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61fd63c740 1 -- 172.21.15.27:0/2995922916 shutdown_connections 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.674+0000 7f61fd63c740 1 -- 172.21.15.27:0/2995922916 wait complete. 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61fd63c740 1 -- start start 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61fd63c740 1 Processor -- start 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff1446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff144c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff147760 0x556fff145140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61fd63c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556fff14a150 con 0x556fff138990 2024-06-06T08:42:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61fd63c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556fff025ae0 con 0x556fff138070 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61fd63c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556fff14a2d0 con 0x556fff147760 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61fd63c740 10 client.0 _ll_register_callbacks cb 0x556fff05e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff1446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61f37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff144c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45754/0 (socket says 172.21.15.27:45754) 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61f37fe640 1 -- 172.21.15.27:0/2562468070 learned_addr learned my addr 172.21.15.27:0/2562468070 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.675+0000 7f61f8ec6640 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff147760 0x556fff145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61f37fe640 1 -- 172.21.15.27:0/2562468070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff147760 msgr2=0x556fff145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61f37fe640 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff147760 0x556fff145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61f37fe640 1 -- 172.21.15.27:0/2562468070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 msgr2=0x556fff1446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61f37fe640 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff1446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61f37fe640 1 -- 172.21.15.27:0/2562468070 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffedb0210 con 0x556fff138990 2024-06-06T08:42:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61f3fff640 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff1446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61f37fe640 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff144c00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00bf00 tx=0x7f61dc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6aa91a155ed795 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61d37fe640 1 -- 172.21.15.27:0/2562468070 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc005220 con 0x556fff138990 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61d37fe640 1 -- 172.21.15.27:0/2562468070 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f61dc0053c0 con 0x556fff138990 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61fd63c740 1 -- 172.21.15.27:0/2562468070 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e40be070 con 0x556fff138990 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61d37fe640 1 -- 172.21.15.27:0/2562468070 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61dc0055c0 con 0x556fff138990 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61fd63c740 10 client.16041 fetch_fsmap learned FSMap version 28 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61fd63c740 1 -- 172.21.15.27:0/2562468070 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556ffee7c620 con 0x556fff138990 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.676+0000 7f61d37fe640 1 -- 172.21.15.27:0/2562468070 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e40be070 con 0x556fff138990 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61d37fe640 1 -- 172.21.15.27:0/2562468070 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f61dc019070 con 0x556fff138990 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61fd63c740 10 client.16041 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61fd63c740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61fd63c740 20 client.16041 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556fff1576f0 0x556fff159b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61fd63c740 4 client.16041 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61fd63c740 1 -- 172.21.15.27:0/2562468070 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fff0aff10 con 0x556fff1576f0 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61f3fff640 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556fff1576f0 0x556fff159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61d37fe640 10 client.16041 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.677+0000 7f61f3fff640 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556fff1576f0 0x556fff159b20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef712ddd213320a0 server_cookie=5e78f2e2b341c61f in_seq=0 out_seq=0 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61d37fe640 1 -- 172.21.15.27:0/2562468070 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556fff0aff10 con 0x556fff1576f0 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61d37fe640 10 client.16041 handle_command_reply: tid=0 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 client.16041 shutdown 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 -- 172.21.15.27:0/2562468070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 msgr2=0x556fff144c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff144c00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00bf00 tx=0x7f61dc012820 comp rx=0 tx=0).stop 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 -- 172.21.15.27:0/2562468070 shutdown_connections 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556fff1576f0 0x556fff159b20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff147760 0x556fff145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff138990 0x556fff144c00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61d37fe640 0 client.16041 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2562468070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff1446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 -- 172.21.15.27:0/2562468070 >> 172.21.15.27:0/2562468070 conn(0x556fff134680 msgr2=0x556fff13a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 -- 172.21.15.27:0/2562468070 shutdown_connections 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 1 -- 172.21.15.27:0/2562468070 wait complete. 2024-06-06T08:42:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.678+0000 7f61fd63c740 20 client.16041 trim_cache size 0 max 16384 2024-06-06T08:42:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61fd63c740 1 Processor -- start 2024-06-06T08:42:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61fd63c740 1 -- start start 2024-06-06T08:42:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff15f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff15f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61fd63c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41af540 con 0x556fff147760 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61fd63c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41acdb0 con 0x556fff138070 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61fd63c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41aac00 con 0x556fff138990 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f8ec6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff15f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff15f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff15f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:33646/0 (socket says 172.21.15.27:33646) 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f37fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff15f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:43102/0 (socket says 172.21.15.27:43102) 2024-06-06T08:42:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f3fff640 1 -- 172.21.15.27:0/2390585297 learned_addr learned my addr 172.21.15.27:0/2390585297 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f3fff640 1 -- 172.21.15.27:0/2390585297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 msgr2=0x556fff15f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f3fff640 1 --2- 172.21.15.27:0/2390585297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff15f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f3fff640 1 -- 172.21.15.27:0/2390585297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 msgr2=0x556fff145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f3fff640 1 --2- 172.21.15.27:0/2390585297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f3fff640 1 -- 172.21.15.27:0/2390585297 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fff12ff70 con 0x556fff138070 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.694+0000 7f61f37fe640 1 --2- 172.21.15.27:0/2390585297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff15f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.695+0000 7f61f3fff640 1 --2- 172.21.15.27:0/2390585297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff15f480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f61e40ed680 tx=0x7f61e4005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3b4e872057e19c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.695+0000 7f61e9ffb640 1 -- 172.21.15.27:0/2390585297 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4049030 con 0x556fff138070 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.695+0000 7f61e9ffb640 1 -- 172.21.15.27:0/2390585297 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e40be070 con 0x556fff138070 2024-06-06T08:42:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.695+0000 7f61e9ffb640 1 -- 172.21.15.27:0/2390585297 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40d8a70 con 0x556fff138070 2024-06-06T08:42:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 -- 172.21.15.27:0/2390585297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 msgr2=0x556fff15f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2390585297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff15f480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f61e40ed680 tx=0x7f61e4005c90 comp rx=0 tx=0).stop 2024-06-06T08:42:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 -- 172.21.15.27:0/2390585297 shutdown_connections 2024-06-06T08:42:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2390585297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2390585297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff15f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 --2- 172.21.15.27:0/2390585297 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff15f480 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 -- 172.21.15.27:0/2390585297 >> 172.21.15.27:0/2390585297 conn(0x556fff134680 msgr2=0x556fff13f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 -- 172.21.15.27:0/2390585297 shutdown_connections 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 -- 172.21.15.27:0/2390585297 wait complete. 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 -- start start 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 Processor -- start 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff14fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff14aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff14b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41ab270 con 0x556fff147760 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.696+0000 7f61fd63c740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41ac6e0 con 0x556fff138070 2024-06-06T08:42:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61fd63c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41ab940 con 0x556fff138990 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61fd63c740 10 client.0 _ll_register_callbacks cb 0x7f61dc00b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f8ec6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff14b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f8ec6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff14b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45760/0 (socket says 172.21.15.27:45760) 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f8ec6640 1 -- 172.21.15.27:0/1065250717 learned_addr learned my addr 172.21.15.27:0/1065250717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f37fe640 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff14aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f3fff640 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff14fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f37fe640 1 -- 172.21.15.27:0/1065250717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 msgr2=0x556fff14fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f37fe640 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff14fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f37fe640 1 -- 172.21.15.27:0/1065250717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 msgr2=0x556fff14b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f37fe640 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff14b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f37fe640 1 -- 172.21.15.27:0/1065250717 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fff14bb10 con 0x556fff138990 2024-06-06T08:42:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f8ec6640 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff14b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61f37fe640 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff14aee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f61dc011010 tx=0x7f61dc00b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2079228f5fb3cef1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61d37fe640 1 -- 172.21.15.27:0/1065250717 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc010b70 con 0x556fff138990 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61d37fe640 1 -- 172.21.15.27:0/1065250717 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f61dc01a840 con 0x556fff138990 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.697+0000 7f61fd63c740 1 -- 172.21.15.27:0/1065250717 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e40be070 con 0x556fff138990 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61d37fe640 1 -- 172.21.15.27:0/1065250717 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61dc01aaa0 con 0x556fff138990 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61fd63c740 10 client.25576 fetch_fsmap learned FSMap version 28 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61fd63c740 1 -- 172.21.15.27:0/1065250717 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fff1c2360 con 0x556fff138990 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61d37fe640 1 -- 172.21.15.27:0/1065250717 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e40be070 con 0x556fff138990 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61d37fe640 1 -- 172.21.15.27:0/1065250717 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f61dc019040 con 0x556fff138990 2024-06-06T08:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61fd63c740 10 client.25576 fetch_fsmap finished waiting for FSMap version 28 2024-06-06T08:42:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61fd63c740 10 client.25576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61fd63c740 20 client.25576 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61fd63c740 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556fff1c2b40 0x556fff1c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61fd63c740 4 client.25576 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:42:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61fd63c740 1 -- 172.21.15.27:0/1065250717 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fff0a9c20 con 0x556fff1c2b40 2024-06-06T08:42:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.698+0000 7f61f3fff640 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556fff1c2b40 0x556fff1c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.699+0000 7f61f3fff640 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556fff1c2b40 0x556fff1c4f60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=226a145dbb166220 server_cookie=adb2bcbcac41c359 in_seq=0 out_seq=0 2024-06-06T08:42:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:44.699+0000 7f61d37fe640 10 client.25576 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:44.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:44 smithi027 ceph-mon[32335]: pgmap v77: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 265 op/s 2024-06-06T08:42:44.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:44 smithi203 ceph-mon[36579]: pgmap v77: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 265 op/s 2024-06-06T08:42:45.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:44 smithi176 ceph-mon[36786]: pgmap v77: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 265 op/s 2024-06-06T08:42:45.067 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T08:42:45.323 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:42:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61d37fe640 1 -- 172.21.15.27:0/1065250717 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556fff0a9c20 con 0x556fff1c2b40 2024-06-06T08:42:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61d37fe640 10 client.25576 handle_command_reply: tid=0 2024-06-06T08:42:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 client.25576 shutdown 2024-06-06T08:42:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 -- 172.21.15.27:0/1065250717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 msgr2=0x556fff14aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff14aee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f61dc011010 tx=0x7f61dc00b760 comp rx=0 tx=0).stop 2024-06-06T08:42:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 -- 172.21.15.27:0/1065250717 shutdown_connections 2024-06-06T08:42:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556fff1c2b40 0x556fff1c4f60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fff147760 0x556fff14b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556fff138990 0x556fff14aee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 --2- 172.21.15.27:0/1065250717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fff138070 0x556fff14fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61d37fe640 0 client.25576 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 -- 172.21.15.27:0/1065250717 >> 172.21.15.27:0/1065250717 conn(0x556fff134680 msgr2=0x556fff144660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 -- 172.21.15.27:0/1065250717 shutdown_connections 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 1 -- 172.21.15.27:0/1065250717 wait complete. 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.849+0000 7f61fd63c740 20 client.25576 trim_cache size 0 max 16384 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:42:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:42:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:42:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 -- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f61e00a2da0 msgr2=0x7f61e00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f61e00a2da0 0x7f61e00a5260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f61ec005fd0 tx=0x7f61ec009500 comp rx=0 tx=0).stop 2024-06-06T08:42:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 -- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 msgr2=0x7f61f4198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 0x7f61f4198290 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f61e4006010 tx=0x7f61e40042f0 comp rx=0 tx=0).stop 2024-06-06T08:42:46.851 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:46 smithi027 ceph-mon[32335]: pgmap v78: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-06-06T08:42:46.851 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:46 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:42:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 -- 172.21.15.27:0/162674518 shutdown_connections 2024-06-06T08:42:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f61e00a2da0 0x7f61e00a5260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f61f410bf70 0x7f61f419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61f4103550 0x7f61f41987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 --2- 172.21.15.27:0/162674518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61f4102c30 0x7f61f4198290 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 -- 172.21.15.27:0/162674518 >> 172.21.15.27:0/162674518 conn(0x7f61f4077a20 msgr2=0x7f61f40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 -- 172.21.15.27:0/162674518 shutdown_connections 2024-06-06T08:42:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:46.850+0000 7f61fa929640 1 -- 172.21.15.27:0/162674518 wait complete. 2024-06-06T08:42:46.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:46 smithi203 ceph-mon[36579]: pgmap v78: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-06-06T08:42:46.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:46 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:42:47.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:46 smithi176 ceph-mon[36786]: pgmap v78: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-06-06T08:42:47.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:46 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:42:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.227+0000 7f6370a96700 1 -- 172.21.15.27:0/3198404585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c1071a0 msgr2=0x7f636c107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.227+0000 7f6370a96700 1 --2- 172.21.15.27:0/3198404585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c1071a0 0x7f636c107620 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f63600099c0 tx=0x7f6360009cd0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.227+0000 7f6370a96700 1 -- 172.21.15.27:0/3198404585 shutdown_connections 2024-06-06T08:42:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.227+0000 7f6370a96700 1 --2- 172.21.15.27:0/3198404585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f636c107b60 0x7f636c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.227+0000 7f6370a96700 1 --2- 172.21.15.27:0/3198404585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f636c105330 0x7f636c105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.227+0000 7f6370a96700 1 --2- 172.21.15.27:0/3198404585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c1071a0 0x7f636c107620 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.227+0000 7f6370a96700 1 -- 172.21.15.27:0/3198404585 >> 172.21.15.27:0/3198404585 conn(0x7f636c0766b0 msgr2=0x7f636c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.227+0000 7f6370a96700 1 -- 172.21.15.27:0/3198404585 shutdown_connections 2024-06-06T08:42:47.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.227+0000 7f6370a96700 1 -- 172.21.15.27:0/3198404585 wait complete. 2024-06-06T08:42:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f6370a96700 1 Processor -- start 2024-06-06T08:42:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f6370a96700 1 -- start start 2024-06-06T08:42:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f6370a96700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f636c105330 0x7f636c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f6370a96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f636c1071a0 0x7f636c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f6370a96700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c107b60 0x7f636c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f6370a96700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f636c111780 con 0x7f636c107b60 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f6370a96700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f636c111600 con 0x7f636c105330 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f6370a96700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f636c111900 con 0x7f636c1071a0 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f636ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c107b60 0x7f636c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f636ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c107b60 0x7f636c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45764/0 (socket says 172.21.15.27:45764) 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f636ad9d700 1 -- 172.21.15.27:0/3537631413 learned_addr learned my addr 172.21.15.27:0/3537631413 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f6369d9b700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f636c1071a0 0x7f636c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.228+0000 7f636a59c700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f636c105330 0x7f636c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f636ad9d700 1 -- 172.21.15.27:0/3537631413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f636c1071a0 msgr2=0x7f636c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f636ad9d700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f636c1071a0 0x7f636c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f636ad9d700 1 -- 172.21.15.27:0/3537631413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f636c105330 msgr2=0x7f636c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f636ad9d700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f636c105330 0x7f636c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f636ad9d700 1 -- 172.21.15.27:0/3537631413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c1992f0 con 0x7f636c107b60 2024-06-06T08:42:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f636ad9d700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c107b60 0x7f636c198b50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6364007d70 tx=0x7f636400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f635b7fe700 1 -- 172.21.15.27:0/3537631413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364016020 con 0x7f636c107b60 2024-06-06T08:42:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f635b7fe700 1 -- 172.21.15.27:0/3537631413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636400ac90 con 0x7f636c107b60 2024-06-06T08:42:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f6370a96700 1 -- 172.21.15.27:0/3537631413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c1995e0 con 0x7f636c107b60 2024-06-06T08:42:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f635b7fe700 1 -- 172.21.15.27:0/3537631413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640115c0 con 0x7f636c107b60 2024-06-06T08:42:47.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.229+0000 7f6370a96700 1 -- 172.21.15.27:0/3537631413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c1a5810 con 0x7f636c107b60 2024-06-06T08:42:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.231+0000 7f635b7fe700 1 -- 172.21.15.27:0/3537631413 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6364019020 con 0x7f636c107b60 2024-06-06T08:42:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.232+0000 7f635b7fe700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63540a2d90 0x7f63540a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.232+0000 7f635b7fe700 1 -- 172.21.15.27:0/3537631413 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f63640cf610 con 0x7f636c107b60 2024-06-06T08:42:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.232+0000 7f636a59c700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63540a2d90 0x7f63540a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f636a59c700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63540a2d90 0x7f63540a5250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f635c007be0 tx=0x7f635c0073d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f637b319b80 1 Processor -- start 2024-06-06T08:42:47.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f637b319b80 1 -- start start 2024-06-06T08:42:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f637b319b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f637b319b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15cf60 0x564dfa1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f637b319b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 0x564dfa157340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f637b319b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564dfa0a8810 con 0x564dfa15c640 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f637b319b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564dfa050720 con 0x564dfa156f20 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f637b319b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564dfa0a47d0 con 0x564dfa15cf60 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f636a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f636a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45772/0 (socket says 172.21.15.27:45772) 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.233+0000 7f636a59c700 1 -- 172.21.15.27:0/1352647845 learned_addr learned my addr 172.21.15.27:0/1352647845 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f6369d9b700 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15cf60 0x564dfa1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f636ad9d700 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 0x564dfa157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f636a59c700 1 -- 172.21.15.27:0/1352647845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15cf60 msgr2=0x564dfa1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f636a59c700 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15cf60 0x564dfa1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f636a59c700 1 -- 172.21.15.27:0/1352647845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 msgr2=0x564dfa157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f636a59c700 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 0x564dfa157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f636a59c700 1 -- 172.21.15.27:0/1352647845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dfa046f50 con 0x564dfa15c640 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f6369d9b700 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15cf60 0x564dfa1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f636ad9d700 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 0x564dfa157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f636a59c700 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ca20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f635c010e80 tx=0x7f635c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba7a4fa47e23dee server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f6358ff9700 1 -- 172.21.15.27:0/1352647845 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c024070 con 0x564dfa15c640 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f6358ff9700 1 -- 172.21.15.27:0/1352647845 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f635c015070 con 0x564dfa15c640 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.234+0000 7f6358ff9700 1 -- 172.21.15.27:0/1352647845 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c011a60 con 0x564dfa15c640 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 -- 172.21.15.27:0/1352647845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 msgr2=0x564dfa15ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ca20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f635c010e80 tx=0x7f635c013000 comp rx=0 tx=0).stop 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 -- 172.21.15.27:0/1352647845 shutdown_connections 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 0x564dfa157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ca20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 --2- 172.21.15.27:0/1352647845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15cf60 0x564dfa1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 -- 172.21.15.27:0/1352647845 >> 172.21.15.27:0/1352647845 conn(0x564dfa081cc0 msgr2=0x564dfa151850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:47.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 -- 172.21.15.27:0/1352647845 shutdown_connections 2024-06-06T08:42:47.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 -- 172.21.15.27:0/1352647845 wait complete. 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 -- start start 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.235+0000 7f637b319b80 1 Processor -- start 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f637b319b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 0x564dfa162d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f637b319b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 0x564dfa160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f637b319b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15cf60 0x564dfa160550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f637b319b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564dfa1639d0 con 0x564dfa15cf60 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f637b319b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564dfa048bf0 con 0x564dfa156f20 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f637b319b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564dfa163b50 con 0x564dfa15c640 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f637b319b80 10 client.0 _ll_register_callbacks cb 0x564dfa042920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15cf60 0x564dfa160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15cf60 0x564dfa160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45780/0 (socket says 172.21.15.27:45780) 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636ad9d700 1 -- 172.21.15.27:0/3816655985 learned_addr learned my addr 172.21.15.27:0/3816655985 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636a59c700 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 0x564dfa162d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f6369d9b700 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 0x564dfa160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636ad9d700 1 -- 172.21.15.27:0/3816655985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 msgr2=0x564dfa160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636ad9d700 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 0x564dfa160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636ad9d700 1 -- 172.21.15.27:0/3816655985 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 msgr2=0x564dfa162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636ad9d700 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 0x564dfa162d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636ad9d700 1 -- 172.21.15.27:0/3816655985 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df9ff78d0 con 0x564dfa15cf60 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f6369d9b700 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 0x564dfa160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f636ad9d700 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15cf60 0x564dfa160550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6364097ce0 tx=0x7f636409b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a319b06d4d2287 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.236+0000 7f6351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f6351ffb700 1 -- 172.21.15.27:0/3816655985 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636409fc30 con 0x564dfa15cf60 2024-06-06T08:42:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f6351ffb700 1 -- 172.21.15.27:0/3816655985 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f63640cf200 con 0x564dfa15cf60 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f637b319b80 1 -- 172.21.15.27:0/3816655985 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f635c015070 con 0x564dfa15cf60 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f6351ffb700 1 -- 172.21.15.27:0/3816655985 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63640a44f0 con 0x564dfa15cf60 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f637b319b80 10 client.16059 fetch_fsmap learned FSMap version 29 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f637b319b80 1 -- 172.21.15.27:0/3816655985 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564dfa16e370 con 0x564dfa15cf60 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f6351ffb700 1 -- 172.21.15.27:0/3816655985 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f635c015070 con 0x564dfa15cf60 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f6351ffb700 1 -- 172.21.15.27:0/3816655985 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f636409bae0 con 0x564dfa15cf60 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f637b319b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f637b319b80 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f637b319b80 20 client.16059 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f637b319b80 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564dfa1709b0 0x564dfa172da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f637b319b80 4 client.16059 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.237+0000 7f637b319b80 1 -- 172.21.15.27:0/3816655985 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dfa0b5d10 con 0x564dfa1709b0 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.238+0000 7f636a59c700 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564dfa1709b0 0x564dfa172da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.238+0000 7f6351ffb700 10 client.16059 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.238+0000 7f636a59c700 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564dfa1709b0 0x564dfa172da0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f566c94808b9d4ab server_cookie=f42e22ceb129c971 in_seq=0 out_seq=0 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f6351ffb700 1 -- 172.21.15.27:0/3816655985 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564dfa0b5d10 con 0x564dfa1709b0 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f6351ffb700 10 client.16059 handle_command_reply: tid=0 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 client.16059 shutdown 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 -- 172.21.15.27:0/3816655985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15cf60 msgr2=0x564dfa160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15cf60 0x564dfa160550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6364097ce0 tx=0x7f636409b2d0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 -- 172.21.15.27:0/3816655985 shutdown_connections 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564dfa1709b0 0x564dfa172da0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15cf60 0x564dfa160550 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa156f20 0x564dfa162d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f6351ffb700 0 client.16059 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 --2- 172.21.15.27:0/3816655985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 0x564dfa160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 -- 172.21.15.27:0/3816655985 >> 172.21.15.27:0/3816655985 conn(0x564dfa081cc0 msgr2=0x564dfa1559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:47.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 -- 172.21.15.27:0/3816655985 shutdown_connections 2024-06-06T08:42:47.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 1 -- 172.21.15.27:0/3816655985 wait complete. 2024-06-06T08:42:47.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.239+0000 7f637b319b80 20 client.16059 trim_cache size 0 max 16384 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.260+0000 7f637b319b80 1 Processor -- start 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.260+0000 7f637b319b80 1 -- start start 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f637b319b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa156f20 0x564dfa1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f637b319b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f637b319b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 0x564dfa14b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f637b319b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1b0f50 con 0x564dfa15c640 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f637b319b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ae750 con 0x564dfa15cf60 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f637b319b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ac5a0 con 0x564dfa156f20 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f6369d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f6369d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45796/0 (socket says 172.21.15.27:45796) 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f6369d9b700 1 -- 172.21.15.27:0/2188143396 learned_addr learned my addr 172.21.15.27:0/2188143396 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f636ad9d700 1 --2- 172.21.15.27:0/2188143396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 0x564dfa14b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f636a59c700 1 --2- 172.21.15.27:0/2188143396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa156f20 0x564dfa1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f6369d9b700 1 -- 172.21.15.27:0/2188143396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa156f20 msgr2=0x564dfa1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f6369d9b700 1 --2- 172.21.15.27:0/2188143396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa156f20 0x564dfa1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f6369d9b700 1 -- 172.21.15.27:0/2188143396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 msgr2=0x564dfa14b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f6369d9b700 1 --2- 172.21.15.27:0/2188143396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 0x564dfa14b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f6369d9b700 1 -- 172.21.15.27:0/2188143396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dfa1733d0 con 0x564dfa15c640 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.261+0000 7f6369d9b700 1 --2- 172.21.15.27:0/2188143396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ef70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f63600099c0 tx=0x7f6360005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42634051fa9bd866 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f6353fff700 1 -- 172.21.15.27:0/2188143396 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360028070 con 0x564dfa15c640 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f6353fff700 1 -- 172.21.15.27:0/2188143396 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6360005590 con 0x564dfa15c640 2024-06-06T08:42:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f6353fff700 1 -- 172.21.15.27:0/2188143396 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636000bb90 con 0x564dfa15c640 2024-06-06T08:42:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f637b319b80 1 -- 172.21.15.27:0/2188143396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 msgr2=0x564dfa15ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f637b319b80 1 --2- 172.21.15.27:0/2188143396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ef70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f63600099c0 tx=0x7f6360005fb0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f637b319b80 1 -- 172.21.15.27:0/2188143396 shutdown_connections 2024-06-06T08:42:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f637b319b80 1 --2- 172.21.15.27:0/2188143396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 0x564dfa14b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f637b319b80 1 --2- 172.21.15.27:0/2188143396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa156f20 0x564dfa1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f637b319b80 1 --2- 172.21.15.27:0/2188143396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa15c640 0x564dfa15ef70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f637b319b80 1 -- 172.21.15.27:0/2188143396 >> 172.21.15.27:0/2188143396 conn(0x564dfa081cc0 msgr2=0x564dfa15d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f637b319b80 1 -- 172.21.15.27:0/2188143396 shutdown_connections 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.262+0000 7f637b319b80 1 -- 172.21.15.27:0/2188143396 wait complete. 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f637b319b80 1 -- start start 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f637b319b80 1 Processor -- start 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f637b319b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa156f20 0x564dfa178410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f637b319b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 0x564dfa178950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f637b319b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 0x564dfa178e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f637b319b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1acc10 con 0x564dfa156f20 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f637b319b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ae080 con 0x564dfa15cf60 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f637b319b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ad2e0 con 0x564dfa15c640 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f637b319b80 10 client.0 _ll_register_callbacks cb 0x564dfa042920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f636a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa156f20 0x564dfa178410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f636a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa156f20 0x564dfa178410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45806/0 (socket says 172.21.15.27:45806) 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f636a59c700 1 -- 172.21.15.27:0/2598202346 learned_addr learned my addr 172.21.15.27:0/2598202346 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f6369d9b700 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 0x564dfa178950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f636ad9d700 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 0x564dfa178e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f636a59c700 1 -- 172.21.15.27:0/2598202346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 msgr2=0x564dfa178950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.263+0000 7f636a59c700 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 0x564dfa178950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f636a59c700 1 -- 172.21.15.27:0/2598202346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 msgr2=0x564dfa178e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f636a59c700 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 0x564dfa178e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f636a59c700 1 -- 172.21.15.27:0/2598202346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dfa1da7c0 con 0x564dfa156f20 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f636ad9d700 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 0x564dfa178e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f636a59c700 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa156f20 0x564dfa178410 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f635c003fa0 tx=0x7f635c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afb20adce52e6d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f6351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f6351ffb700 1 -- 172.21.15.27:0/2598202346 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c03a060 con 0x564dfa156f20 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f6351ffb700 1 -- 172.21.15.27:0/2598202346 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f635c010910 con 0x564dfa156f20 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f637b319b80 1 -- 172.21.15.27:0/2598202346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6360005590 con 0x564dfa156f20 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f6351ffb700 1 -- 172.21.15.27:0/2598202346 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f635c041050 con 0x564dfa156f20 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f637b319b80 10 client.16065 fetch_fsmap learned FSMap version 29 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f637b319b80 1 -- 172.21.15.27:0/2598202346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564dfa1dbad0 con 0x564dfa156f20 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f6351ffb700 1 -- 172.21.15.27:0/2598202346 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6360005590 con 0x564dfa156f20 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.264+0000 7f6351ffb700 1 -- 172.21.15.27:0/2598202346 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f635c02a040 con 0x564dfa156f20 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f637b319b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f637b319b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f637b319b80 20 client.16065 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f637b319b80 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564dfa1dbed0 0x564dfa1de2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f637b319b80 4 client.16065 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T08:42:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f637b319b80 1 -- 172.21.15.27:0/2598202346 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564dfa0b0290 con 0x564dfa1dbed0 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f6369d9b700 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564dfa1dbed0 0x564dfa1de2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f6369d9b700 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564dfa1dbed0 0x564dfa1de2f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d30dcb1114987b server_cookie=48eac9f0b935c269 in_seq=0 out_seq=0 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f6351ffb700 10 client.16065 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f6351ffb700 1 -- 172.21.15.27:0/2598202346 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564dfa0b0290 con 0x564dfa1dbed0 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f6351ffb700 10 client.16065 handle_command_reply: tid=0 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.265+0000 7f637b319b80 1 client.16065 shutdown 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 -- 172.21.15.27:0/2598202346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa156f20 msgr2=0x564dfa178410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa156f20 0x564dfa178410 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f635c003fa0 tx=0x7f635c005b00 comp rx=0 tx=0).stop 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 -- 172.21.15.27:0/2598202346 shutdown_connections 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564dfa15cf60 0x564dfa178e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564dfa1dbed0 0x564dfa1de2f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564dfa156f20 0x564dfa178410 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 --2- 172.21.15.27:0/2598202346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564dfa15c640 0x564dfa178950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 -- 172.21.15.27:0/2598202346 >> 172.21.15.27:0/2598202346 conn(0x564dfa081cc0 msgr2=0x564dfa14b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6351ffb700 0 client.16065 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 -- 172.21.15.27:0/2598202346 shutdown_connections 2024-06-06T08:42:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 1 -- 172.21.15.27:0/2598202346 wait complete. 2024-06-06T08:42:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f637b319b80 20 client.16065 trim_cache size 0 max 16384 2024-06-06T08:42:47.270 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:42:47.270 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-06-06T08:42:47.270 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {} 2024-06-06T08:42:47.270 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 -- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63540a2d90 msgr2=0x7f63540a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63540a2d90 0x7f63540a5250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f635c007be0 tx=0x7f635c0073d0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 -- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c107b60 msgr2=0x7f636c198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c107b60 0x7f636c198b50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6364007d70 tx=0x7f636400a340 comp rx=0 tx=0).stop 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 -- 172.21.15.27:0/3537631413 shutdown_connections 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c107b60 0x7f636c198b50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f63540a2d90 0x7f63540a5250 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f636c105330 0x7f636c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 --2- 172.21.15.27:0/3537631413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f636c1071a0 0x7f636c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.266+0000 7f6370a96700 1 -- 172.21.15.27:0/3537631413 >> 172.21.15.27:0/3537631413 conn(0x7f636c0766b0 msgr2=0x7f636c100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:47.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.267+0000 7f6370a96700 1 -- 172.21.15.27:0/3537631413 shutdown_connections 2024-06-06T08:42:47.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:47.267+0000 7f6370a96700 1 -- 172.21.15.27:0/3537631413 wait complete. 2024-06-06T08:42:48.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:48 smithi027 ceph-mon[32335]: pgmap v79: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-06-06T08:42:48.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:48 smithi203 ceph-mon[36579]: pgmap v79: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-06-06T08:42:49.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:48 smithi176 ceph-mon[36786]: pgmap v79: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-06-06T08:42:49.544 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T08:42:49.544 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:42:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:42:50.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:50 smithi027 ceph-mon[32335]: pgmap v80: 129 pgs: 129 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 1.04k op/s 2024-06-06T08:42:50.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:50 smithi203 ceph-mon[36579]: pgmap v80: 129 pgs: 129 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 1.04k op/s 2024-06-06T08:42:51.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:50 smithi176 ceph-mon[36786]: pgmap v80: 129 pgs: 129 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 1.04k op/s 2024-06-06T08:42:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.560+0000 7f43f75b5700 1 -- 172.21.15.27:0/3998463206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0101d50 msgr2=0x7f43f010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.560+0000 7f43f75b5700 1 --2- 172.21.15.27:0/3998463206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0101d50 0x7f43f010e240 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f43e800b0a0 tx=0x7f43e800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.560+0000 7f43f75b5700 1 -- 172.21.15.27:0/3998463206 shutdown_connections 2024-06-06T08:42:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.560+0000 7f43f75b5700 1 --2- 172.21.15.27:0/3998463206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0101d50 0x7f43f010e240 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.560+0000 7f43f75b5700 1 --2- 172.21.15.27:0/3998463206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f0107400 0x7f43f01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.560+0000 7f43f75b5700 1 --2- 172.21.15.27:0/3998463206 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43f0101390 0x7f43f0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.560+0000 7f43f75b5700 1 -- 172.21.15.27:0/3998463206 >> 172.21.15.27:0/3998463206 conn(0x7f43f0078340 msgr2=0x7f43f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.560+0000 7f43f75b5700 1 -- 172.21.15.27:0/3998463206 shutdown_connections 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.560+0000 7f43f75b5700 1 -- 172.21.15.27:0/3998463206 wait complete. 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f75b5700 1 Processor -- start 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f75b5700 1 -- start start 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f75b5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43f0101390 0x7f43f019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f75b5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f0101d50 0x7f43f019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f75b5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0107400 0x7f43f0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f75b5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43f0111690 con 0x7f43f0107400 2024-06-06T08:42:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f75b5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f43f0111510 con 0x7f43f0101390 2024-06-06T08:42:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f75b5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f43f0111810 con 0x7f43f0101d50 2024-06-06T08:42:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f5b52700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0107400 0x7f43f0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f5b52700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0107400 0x7f43f0198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39148/0 (socket says 172.21.15.27:39148) 2024-06-06T08:42:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f5b52700 1 -- 172.21.15.27:0/69136587 learned_addr learned my addr 172.21.15.27:0/69136587 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f5351700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43f0101390 0x7f43f019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.561+0000 7f43f4b50700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f0101d50 0x7f43f019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43f5b52700 1 -- 172.21.15.27:0/69136587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f0101d50 msgr2=0x7f43f019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43f5b52700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f0101d50 0x7f43f019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43f5b52700 1 -- 172.21.15.27:0/69136587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43f0101390 msgr2=0x7f43f019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43f5b52700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43f0101390 0x7f43f019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43f5b52700 1 -- 172.21.15.27:0/69136587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f0199160 con 0x7f43f0107400 2024-06-06T08:42:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43f5351700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43f0101390 0x7f43f019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43f5b52700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0107400 0x7f43f0198a00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f43e800b780 tx=0x7f43e8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43e67fc700 1 -- 172.21.15.27:0/69136587 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8021070 con 0x7f43f0107400 2024-06-06T08:42:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43e67fc700 1 -- 172.21.15.27:0/69136587 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e8007b80 con 0x7f43f0107400 2024-06-06T08:42:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f01993f0 con 0x7f43f0107400 2024-06-06T08:42:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43e67fc700 1 -- 172.21.15.27:0/69136587 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e80052a0 con 0x7f43f0107400 2024-06-06T08:42:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.562+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f01a56c0 con 0x7f43f0107400 2024-06-06T08:42:51.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.563+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f0102ab0 con 0x7f43f0107400 2024-06-06T08:42:51.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.565+0000 7f43e67fc700 1 -- 172.21.15.27:0/69136587 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f43e8024070 con 0x7f43f0107400 2024-06-06T08:42:51.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.566+0000 7f43e67fc700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f43dc0a2d90 0x7f43dc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:51.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.566+0000 7f43f5351700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f43dc0a2d90 0x7f43dc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:51.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.566+0000 7f43e67fc700 1 -- 172.21.15.27:0/69136587 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f43e800f020 con 0x7f43f0107400 2024-06-06T08:42:51.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.566+0000 7f43f5351700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f43dc0a2d90 0x7f43dc0a5250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f43f00ffb40 tx=0x7f43ec00b3c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:51.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.570+0000 7f43e67fc700 1 -- 172.21.15.27:0/69136587 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f43e802b020 con 0x7f43f0107400 2024-06-06T08:42:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.855+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43f01a5890 con 0x7f43f0107400 2024-06-06T08:42:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.856+0000 7f43e67fc700 1 -- 172.21.15.27:0/69136587 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11889 (secure 0 0 0) 0x7f43e809d990 con 0x7f43f0107400 2024-06-06T08:42:51.856 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:42:51.856 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:42:45.177186+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24458,24460,24482]},"id":1}]} 2024-06-06T08:42:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f43dc0a2d90 msgr2=0x7f43dc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f43dc0a2d90 0x7f43dc0a5250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f43f00ffb40 tx=0x7f43ec00b3c0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0107400 msgr2=0x7f43f0198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0107400 0x7f43f0198a00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f43e800b780 tx=0x7f43e8004d00 comp rx=0 tx=0).stop 2024-06-06T08:42:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 shutdown_connections 2024-06-06T08:42:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43f0107400 0x7f43f0198a00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f43dc0a2d90 0x7f43dc0a5250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43f0101390 0x7f43f019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 --2- 172.21.15.27:0/69136587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f0101d50 0x7f43f019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:51.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 >> 172.21.15.27:0/69136587 conn(0x7f43f0078340 msgr2=0x7f43f01005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:51.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 shutdown_connections 2024-06-06T08:42:51.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:51.858+0000 7f43f75b5700 1 -- 172.21.15.27:0/69136587 wait complete. 2024-06-06T08:42:51.859 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-06-06T08:42:52.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T08:42:52.878 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:42:52.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:52 smithi027 ceph-mon[32335]: pgmap v81: 129 pgs: 129 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 1.03k op/s 2024-06-06T08:42:52.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:52 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/69136587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:42:52.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:52 smithi203 ceph-mon[36579]: pgmap v81: 129 pgs: 129 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 1.03k op/s 2024-06-06T08:42:52.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:52 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/69136587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:42:53.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:52 smithi176 ceph-mon[36786]: pgmap v81: 129 pgs: 129 active+clean; 27 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 1.03k op/s 2024-06-06T08:42:53.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:52 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/69136587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:42:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:54 smithi027 ceph-mon[32335]: pgmap v82: 129 pgs: 129 active+clean; 32 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 1.04k op/s 2024-06-06T08:42:54.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:54 smithi203 ceph-mon[36579]: pgmap v82: 129 pgs: 129 active+clean; 32 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 1.04k op/s 2024-06-06T08:42:55.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:54 smithi176 ceph-mon[36786]: pgmap v82: 129 pgs: 129 active+clean; 32 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 1.04k op/s 2024-06-06T08:42:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.609+0000 7f1757ccf700 1 -- 172.21.15.27:0/1995892823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 msgr2=0x7f1750101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.609+0000 7f1757ccf700 1 --2- 172.21.15.27:0/1995892823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 0x7f1750101810 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1740009990 tx=0x7f1740009ca0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.609+0000 7f1757ccf700 1 -- 172.21.15.27:0/1995892823 shutdown_connections 2024-06-06T08:42:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.609+0000 7f1757ccf700 1 --2- 172.21.15.27:0/1995892823 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1750101d50 0x7f175010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.609+0000 7f1757ccf700 1 --2- 172.21.15.27:0/1995892823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1750107400 0x7f17501077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.609+0000 7f1757ccf700 1 --2- 172.21.15.27:0/1995892823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 0x7f1750101810 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.609+0000 7f1757ccf700 1 -- 172.21.15.27:0/1995892823 >> 172.21.15.27:0/1995892823 conn(0x7f1750078340 msgr2=0x7f1750078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.609+0000 7f1757ccf700 1 -- 172.21.15.27:0/1995892823 shutdown_connections 2024-06-06T08:42:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.609+0000 7f1757ccf700 1 -- 172.21.15.27:0/1995892823 wait complete. 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1757ccf700 1 Processor -- start 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1757ccf700 1 -- start start 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1757ccf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 0x7f175019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1757ccf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1750101d50 0x7f175019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1757ccf700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1750107400 0x7f17501989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1757ccf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f17501116c0 con 0x7f1750101390 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1757ccf700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1750111540 con 0x7f1750107400 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1757ccf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1750111840 con 0x7f1750101d50 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1755a6b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 0x7f175019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1755a6b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 0x7f175019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39160/0 (socket says 172.21.15.27:39160) 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f1755a6b700 1 -- 172.21.15.27:0/2315972609 learned_addr learned my addr 172.21.15.27:0/2315972609 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f175626c700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1750107400 0x7f17501989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.610+0000 7f175526a700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1750101d50 0x7f175019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1755a6b700 1 -- 172.21.15.27:0/2315972609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1750101d50 msgr2=0x7f175019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1755a6b700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1750101d50 0x7f175019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1755a6b700 1 -- 172.21.15.27:0/2315972609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1750107400 msgr2=0x7f17501989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1755a6b700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1750107400 0x7f17501989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1755a6b700 1 -- 172.21.15.27:0/2315972609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1750199150 con 0x7f1750101390 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f175626c700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1750107400 0x7f17501989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1755a6b700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 0x7f175019e830 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f174c00c9b0 tx=0x7f174c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1746ffd700 1 -- 172.21.15.27:0/2315972609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c007b00 con 0x7f1750101390 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1746ffd700 1 -- 172.21.15.27:0/2315972609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174c007ca0 con 0x7f1750101390 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1757ccf700 1 -- 172.21.15.27:0/2315972609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1750199440 con 0x7f1750101390 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1746ffd700 1 -- 172.21.15.27:0/2315972609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c018730 con 0x7f1750101390 2024-06-06T08:42:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.611+0000 7f1757ccf700 1 -- 172.21.15.27:0/2315972609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17501a5700 con 0x7f1750101390 2024-06-06T08:42:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.613+0000 7f1746ffd700 1 -- 172.21.15.27:0/2315972609 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f174c0188d0 con 0x7f1750101390 2024-06-06T08:42:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.614+0000 7f1746ffd700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f173c0a2d90 0x7f173c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.614+0000 7f175526a700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f173c0a2d90 0x7f173c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.614+0000 7f1746ffd700 1 -- 172.21.15.27:0/2315972609 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f174c0cf7a0 con 0x7f1750101390 2024-06-06T08:42:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f175526a700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f173c0a2d90 0x7f173c0a5250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f174000b2e0 tx=0x7f174001a040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f1762552b80 1 Processor -- start 2024-06-06T08:42:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f1762552b80 1 -- start start 2024-06-06T08:42:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f1762552b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c4f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f1762552b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f1762552b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c54090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f1762552b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f760b9c810 con 0x55f760c4f8f0 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f1762552b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f760b44720 con 0x55f760c53c30 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f1762552b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f760b987d0 con 0x55f760c4efd0 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.615+0000 7f175526a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175626c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c54090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175626c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c54090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:51810/0 (socket says 172.21.15.27:51810) 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175526a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39170/0 (socket says 172.21.15.27:39170) 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175626c700 1 -- 172.21.15.27:0/3341653481 learned_addr learned my addr 172.21.15.27:0/3341653481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f1755a6b700 1 --2- 172.21.15.27:0/3341653481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c4f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175626c700 1 -- 172.21.15.27:0/3341653481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 msgr2=0x55f760c4f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175626c700 1 --2- 172.21.15.27:0/3341653481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c4f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175626c700 1 -- 172.21.15.27:0/3341653481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 msgr2=0x55f760c536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175626c700 1 --2- 172.21.15.27:0/3341653481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175626c700 1 -- 172.21.15.27:0/3341653481 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f760b3af50 con 0x55f760c53c30 2024-06-06T08:42:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175526a700 1 --2- 172.21.15.27:0/3341653481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f175626c700 1 --2- 172.21.15.27:0/3341653481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c54090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1748003eb0 tx=0x7f1748009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6080e387d9b1c6e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f173bfff700 1 -- 172.21.15.27:0/3341653481 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748019020 con 0x55f760c53c30 2024-06-06T08:42:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f173bfff700 1 -- 172.21.15.27:0/3341653481 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17480092d0 con 0x55f760c53c30 2024-06-06T08:42:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.616+0000 7f173bfff700 1 -- 172.21.15.27:0/3341653481 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174800e910 con 0x55f760c53c30 2024-06-06T08:42:56.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.617+0000 7f1762552b80 1 -- 172.21.15.27:0/3341653481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 msgr2=0x55f760c54090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:56.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.617+0000 7f1762552b80 1 --2- 172.21.15.27:0/3341653481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c54090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1748003eb0 tx=0x7f1748009920 comp rx=0 tx=0).stop 2024-06-06T08:42:56.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.617+0000 7f1762552b80 1 -- 172.21.15.27:0/3341653481 shutdown_connections 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.617+0000 7f1762552b80 1 --2- 172.21.15.27:0/3341653481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c54090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.617+0000 7f1762552b80 1 --2- 172.21.15.27:0/3341653481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c4f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.617+0000 7f1762552b80 1 --2- 172.21.15.27:0/3341653481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.617+0000 7f1762552b80 1 -- 172.21.15.27:0/3341653481 >> 172.21.15.27:0/3341653481 conn(0x55f760b75cc0 msgr2=0x55f760c45850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.617+0000 7f1762552b80 1 -- 172.21.15.27:0/3341653481 shutdown_connections 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.617+0000 7f1762552b80 1 -- 172.21.15.27:0/3341653481 wait complete. 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1762552b80 1 -- start start 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1762552b80 1 Processor -- start 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1762552b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c5b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1762552b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4f8f0 0x55f760c58490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1762552b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1762552b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f760c5bd10 con 0x55f760c53c30 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1762552b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f760b3cbf0 con 0x55f760c4f8f0 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1762552b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f760c5be90 con 0x55f760c4efd0 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1762552b80 10 client.0 _ll_register_callbacks cb 0x55f760b36920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1755a6b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c5b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f175526a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4f8f0 0x55f760c58490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1755a6b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c5b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:33604/0 (socket says 172.21.15.27:33604) 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.618+0000 7f1755a6b700 1 -- 172.21.15.27:0/2997015800 learned_addr learned my addr 172.21.15.27:0/2997015800 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f175626c700 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f175526a700 1 -- 172.21.15.27:0/2997015800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 msgr2=0x55f760c5b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f175526a700 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c5b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f175526a700 1 -- 172.21.15.27:0/2997015800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 msgr2=0x55f760c589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f175526a700 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f175526a700 1 -- 172.21.15.27:0/2997015800 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f760aeb8d0 con 0x55f760c4f8f0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f175526a700 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4f8f0 0x55f760c58490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17500788b0 tx=0x7f1740009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d95149fb6e2aae server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f17397fa700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f17397fa700 1 -- 172.21.15.27:0/2997015800 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174002f650 con 0x55f760c4f8f0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f1762552b80 1 -- 172.21.15.27:0/2997015800 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17480092d0 con 0x55f760c4f8f0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f17397fa700 1 -- 172.21.15.27:0/2997015800 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f174002f7f0 con 0x55f760c4f8f0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f17397fa700 1 -- 172.21.15.27:0/2997015800 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f174003c050 con 0x55f760c4f8f0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f1762552b80 10 client.25610 fetch_fsmap learned FSMap version 29 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.619+0000 7f1762552b80 1 -- 172.21.15.27:0/2997015800 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f760c66670 con 0x55f760c4f8f0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f17397fa700 1 -- 172.21.15.27:0/2997015800 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17480092d0 con 0x55f760c4f8f0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f17397fa700 1 -- 172.21.15.27:0/2997015800 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f1740023c70 con 0x55f760c4f8f0 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f1762552b80 10 client.25610 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f1762552b80 10 client.25610 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f1762552b80 20 client.25610 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f1762552b80 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55f760c68c50 0x55f760c6b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f1762552b80 4 client.25610 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f1762552b80 1 -- 172.21.15.27:0/2997015800 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f760ba9d10 con 0x55f760c68c50 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f1755a6b700 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55f760c68c50 0x55f760c6b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f17397fa700 10 client.25610 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.620+0000 7f1755a6b700 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55f760c68c50 0x55f760c6b040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd7ec097a3344381 server_cookie=60e3aaca02366805 in_seq=0 out_seq=0 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f17397fa700 1 -- 172.21.15.27:0/2997015800 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f760ba9d10 con 0x55f760c68c50 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f17397fa700 10 client.25610 handle_command_reply: tid=0 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f1762552b80 1 client.25610 shutdown 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f1762552b80 1 -- 172.21.15.27:0/2997015800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4f8f0 msgr2=0x55f760c58490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f1762552b80 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4f8f0 0x55f760c58490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17500788b0 tx=0x7f1740009310 comp rx=0 tx=0).stop 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f1762552b80 1 -- 172.21.15.27:0/2997015800 shutdown_connections 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f1762552b80 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55f760c68c50 0x55f760c6b040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f1762552b80 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f17397fa700 0 client.25610 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:42:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f1762552b80 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c5b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f1762552b80 1 --2- 172.21.15.27:0/2997015800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4f8f0 0x55f760c58490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.621+0000 7f1762552b80 1 -- 172.21.15.27:0/2997015800 >> 172.21.15.27:0/2997015800 conn(0x55f760b75cc0 msgr2=0x55f760c499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.622+0000 7f1762552b80 1 -- 172.21.15.27:0/2997015800 shutdown_connections 2024-06-06T08:42:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.622+0000 7f1762552b80 1 -- 172.21.15.27:0/2997015800 wait complete. 2024-06-06T08:42:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.622+0000 7f1762552b80 20 client.25610 trim_cache size 0 max 16384 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.637+0000 7f1762552b80 1 Processor -- start 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.637+0000 7f1762552b80 1 -- start start 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f1762552b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4efd0 0x55f760c459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f1762552b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4f8f0 0x55f760c46ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f1762552b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c3f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f1762552b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f17501aec80 con 0x55f760c53c30 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f1762552b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f17501ac480 con 0x55f760c4efd0 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f1762552b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17501aa2d0 con 0x55f760c4f8f0 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175626c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c3f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f1755a6b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4efd0 0x55f760c459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175626c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c3f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39180/0 (socket says 172.21.15.27:39180) 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175626c700 1 -- 172.21.15.27:0/1918493198 learned_addr learned my addr 172.21.15.27:0/1918493198 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175526a700 1 --2- 172.21.15.27:0/1918493198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4f8f0 0x55f760c46ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175626c700 1 -- 172.21.15.27:0/1918493198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4f8f0 msgr2=0x55f760c46ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175626c700 1 --2- 172.21.15.27:0/1918493198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4f8f0 0x55f760c46ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175626c700 1 -- 172.21.15.27:0/1918493198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4efd0 msgr2=0x55f760c459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175626c700 1 --2- 172.21.15.27:0/1918493198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4efd0 0x55f760c459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175626c700 1 -- 172.21.15.27:0/1918493198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f760c6b6a0 con 0x55f760c53c30 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f1755a6b700 1 --2- 172.21.15.27:0/1918493198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4efd0 0x55f760c459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f175626c700 1 --2- 172.21.15.27:0/1918493198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c3f510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1748003e80 tx=0x7f1748003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50dd44a89a4d7a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f173b7fe700 1 -- 172.21.15.27:0/1918493198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748019040 con 0x55f760c53c30 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f173b7fe700 1 -- 172.21.15.27:0/1918493198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1748009290 con 0x55f760c53c30 2024-06-06T08:42:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.638+0000 7f173b7fe700 1 -- 172.21.15.27:0/1918493198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174800e7e0 con 0x55f760c53c30 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 -- 172.21.15.27:0/1918493198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 msgr2=0x55f760c3f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 --2- 172.21.15.27:0/1918493198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c3f510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1748003e80 tx=0x7f1748003f40 comp rx=0 tx=0).stop 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 -- 172.21.15.27:0/1918493198 shutdown_connections 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 --2- 172.21.15.27:0/1918493198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c53c30 0x55f760c3f510 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 --2- 172.21.15.27:0/1918493198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c4efd0 0x55f760c459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 --2- 172.21.15.27:0/1918493198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4f8f0 0x55f760c46ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 -- 172.21.15.27:0/1918493198 >> 172.21.15.27:0/1918493198 conn(0x55f760b75cc0 msgr2=0x55f760c3ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 -- 172.21.15.27:0/1918493198 shutdown_connections 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 -- 172.21.15.27:0/1918493198 wait complete. 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.639+0000 7f1762552b80 1 -- start start 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f1762552b80 1 Processor -- start 2024-06-06T08:42:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f1762552b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c61650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f1762552b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c5c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f1762552b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c5cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f1762552b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f17501aa940 con 0x55f760c4f8f0 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f1762552b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f17501abdb0 con 0x55f760c53c30 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f1762552b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17501ab010 con 0x55f760c4efd0 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f175526a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c5c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f1762552b80 10 client.0 _ll_register_callbacks cb 0x55f760b36920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f175526a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c5c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39196/0 (socket says 172.21.15.27:39196) 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f175526a700 1 -- 172.21.15.27:0/961463768 learned_addr learned my addr 172.21.15.27:0/961463768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f175626c700 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c5cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.640+0000 7f1755a6b700 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c61650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f175526a700 1 -- 172.21.15.27:0/961463768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 msgr2=0x55f760c61650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:56.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f175526a700 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c61650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f175526a700 1 -- 172.21.15.27:0/961463768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 msgr2=0x55f760c5cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f175526a700 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c5cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f175526a700 1 -- 172.21.15.27:0/961463768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f760c5d500 con 0x55f760c4f8f0 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f175626c700 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c5cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f175526a700 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c5c9d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55f760c3fc50 tx=0x7f17400178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87edf9121168cf0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f17397fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f17397fa700 1 -- 172.21.15.27:0/961463768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740044070 con 0x55f760c4f8f0 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f17397fa700 1 -- 172.21.15.27:0/961463768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f1740017bf0 con 0x55f760c4f8f0 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f1762552b80 1 -- 172.21.15.27:0/961463768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1748009290 con 0x55f760c4f8f0 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f17397fa700 1 -- 172.21.15.27:0/961463768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1740050050 con 0x55f760c4f8f0 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f1762552b80 10 client.16089 fetch_fsmap learned FSMap version 29 2024-06-06T08:42:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f1762552b80 1 -- 172.21.15.27:0/961463768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f760cd36c0 con 0x55f760c4f8f0 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f17397fa700 1 -- 172.21.15.27:0/961463768 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1748009290 con 0x55f760c4f8f0 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.641+0000 7f17397fa700 1 -- 172.21.15.27:0/961463768 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f1740023510 con 0x55f760c4f8f0 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1762552b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1762552b80 10 client.16089 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1762552b80 20 client.16089 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1762552b80 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55f760cd3d10 0x55f760cd6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1762552b80 4 client.16089 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1762552b80 1 -- 172.21.15.27:0/961463768 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f760ba4290 con 0x55f760cd3d10 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1755a6b700 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55f760cd3d10 0x55f760cd6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1755a6b700 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55f760cd3d10 0x55f760cd6130 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acb8d3bc01c926b4 server_cookie=e92275bf863ac921 in_seq=0 out_seq=0 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f17397fa700 10 client.16089 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f17397fa700 1 -- 172.21.15.27:0/961463768 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f760ba4290 con 0x55f760cd3d10 2024-06-06T08:42:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f17397fa700 10 client.16089 handle_command_reply: tid=0 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1762552b80 1 client.16089 shutdown 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1762552b80 1 -- 172.21.15.27:0/961463768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 msgr2=0x55f760c5c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.642+0000 7f1762552b80 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c5c9d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55f760c3fc50 tx=0x7f17400178e0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1762552b80 1 -- 172.21.15.27:0/961463768 shutdown_connections 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1762552b80 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x55f760cd3d10 0x55f760cd6130 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1762552b80 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f760c53c30 0x55f760c5cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1762552b80 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f760c4efd0 0x55f760c61650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1762552b80 1 --2- 172.21.15.27:0/961463768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f760c4f8f0 0x55f760c5c9d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f17397fa700 0 client.16089 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1762552b80 1 -- 172.21.15.27:0/961463768 >> 172.21.15.27:0/961463768 conn(0x55f760b75cc0 msgr2=0x55f760c525f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1762552b80 1 -- 172.21.15.27:0/961463768 shutdown_connections 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1762552b80 1 -- 172.21.15.27:0/961463768 wait complete. 2024-06-06T08:42:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1762552b80 20 client.16089 trim_cache size 0 max 16384 2024-06-06T08:42:56.647 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 -- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f173c0a2d90 msgr2=0x7f173c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f173c0a2d90 0x7f173c0a5250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f174000b2e0 tx=0x7f174001a040 comp rx=0 tx=0).stop 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 -- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 msgr2=0x7f175019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 0x7f175019e830 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f174c00c9b0 tx=0x7f174c00ccc0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 -- 172.21.15.27:0/2315972609 shutdown_connections 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f173c0a2d90 0x7f173c0a5250 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1750107400 0x7f17501989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1750101390 0x7f175019e830 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 --2- 172.21.15.27:0/2315972609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1750101d50 0x7f175019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.643+0000 7f1757ccf700 1 -- 172.21.15.27:0/2315972609 >> 172.21.15.27:0/2315972609 conn(0x7f1750078340 msgr2=0x7f1750100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.644+0000 7f1757ccf700 1 -- 172.21.15.27:0/2315972609 shutdown_connections 2024-06-06T08:42:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:56.644+0000 7f1757ccf700 1 -- 172.21.15.27:0/2315972609 wait complete. 2024-06-06T08:42:56.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:56 smithi027 ceph-mon[32335]: pgmap v83: 129 pgs: 129 active+clean; 32 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 787 op/s 2024-06-06T08:42:56.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:56 smithi203 ceph-mon[36579]: pgmap v83: 129 pgs: 129 active+clean; 32 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 787 op/s 2024-06-06T08:42:57.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:56 smithi176 ceph-mon[36786]: pgmap v83: 129 pgs: 129 active+clean; 32 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 787 op/s 2024-06-06T08:42:57.411 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T08:42:57.845 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:42:58.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:42:58 smithi027 ceph-mon[32335]: pgmap v84: 129 pgs: 129 active+clean; 32 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 787 op/s 2024-06-06T08:42:58.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:42:58 smithi203 ceph-mon[36579]: pgmap v84: 129 pgs: 129 active+clean; 32 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 787 op/s 2024-06-06T08:42:59.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:42:58 smithi176 ceph-mon[36786]: pgmap v84: 129 pgs: 129 active+clean; 32 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 787 op/s 2024-06-06T08:42:59.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.794+0000 7f1cdc585700 1 -- 172.21.15.27:0/875591602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4102490 msgr2=0x7f1cd4102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.794+0000 7f1cdc585700 1 --2- 172.21.15.27:0/875591602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4102490 0x7f1cd4102910 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40099c0 tx=0x7f1cc4009cd0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.794+0000 7f1cdc585700 1 -- 172.21.15.27:0/875591602 shutdown_connections 2024-06-06T08:42:59.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.794+0000 7f1cdc585700 1 --2- 172.21.15.27:0/875591602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd4102e50 0x7f1cd410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.794+0000 7f1cdc585700 1 --2- 172.21.15.27:0/875591602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cd4108540 0x7f1cd4108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.794+0000 7f1cdc585700 1 --2- 172.21.15.27:0/875591602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4102490 0x7f1cd4102910 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.794+0000 7f1cdc585700 1 -- 172.21.15.27:0/875591602 >> 172.21.15.27:0/875591602 conn(0x7f1cd40fdf40 msgr2=0x7f1cd4100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.794+0000 7f1cdc585700 1 -- 172.21.15.27:0/875591602 shutdown_connections 2024-06-06T08:42:59.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.794+0000 7f1cdc585700 1 -- 172.21.15.27:0/875591602 wait complete. 2024-06-06T08:42:59.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdc585700 1 Processor -- start 2024-06-06T08:42:59.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdc585700 1 -- start start 2024-06-06T08:42:59.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdc585700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd4102490 0x7f1cd419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdc585700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cd4102e50 0x7f1cd419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdc585700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4108540 0x7f1cd41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdc585700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd41127a0 con 0x7f1cd4108540 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdc585700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4112620 con 0x7f1cd4102e50 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdc585700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4112920 con 0x7f1cd4102490 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdab22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4108540 0x7f1cd41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdab22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4108540 0x7f1cd41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39206/0 (socket says 172.21.15.27:39206) 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cdab22700 1 -- 172.21.15.27:0/443230896 learned_addr learned my addr 172.21.15.27:0/443230896 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cd9b20700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cd4102e50 0x7f1cd419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.795+0000 7f1cda321700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd4102490 0x7f1cd419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1cdab22700 1 -- 172.21.15.27:0/443230896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd4102490 msgr2=0x7f1cd419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1cdab22700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd4102490 0x7f1cd419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1cdab22700 1 -- 172.21.15.27:0/443230896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cd4102e50 msgr2=0x7f1cd419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1cdab22700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cd4102e50 0x7f1cd419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1cdab22700 1 -- 172.21.15.27:0/443230896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd4199120 con 0x7f1cd4108540 2024-06-06T08:42:59.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1cdab22700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4108540 0x7f1cd41989c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc007ee0 tx=0x7f1ccc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1ccb7fe700 1 -- 172.21.15.27:0/443230896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc00dbd0 con 0x7f1cd4108540 2024-06-06T08:42:59.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1cdc585700 1 -- 172.21.15.27:0/443230896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd4199410 con 0x7f1cd4108540 2024-06-06T08:42:59.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1ccb7fe700 1 -- 172.21.15.27:0/443230896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ccc00dd70 con 0x7f1cd4108540 2024-06-06T08:42:59.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1ccb7fe700 1 -- 172.21.15.27:0/443230896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc005a60 con 0x7f1cd4108540 2024-06-06T08:42:59.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.796+0000 7f1cdc585700 1 -- 172.21.15.27:0/443230896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd41a5680 con 0x7f1cd4108540 2024-06-06T08:42:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.798+0000 7f1ccb7fe700 1 -- 172.21.15.27:0/443230896 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1ccc01a020 con 0x7f1cd4108540 2024-06-06T08:42:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.799+0000 7f1ccb7fe700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1cc00a2d90 0x7f1cc00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.799+0000 7f1cda321700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1cc00a2d90 0x7f1cc00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.799+0000 7f1ccb7fe700 1 -- 172.21.15.27:0/443230896 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f1ccc0cef30 con 0x7f1cd4108540 2024-06-06T08:42:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.799+0000 7f1cda321700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1cc00a2d90 0x7f1cc00a5250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4107560 tx=0x7f1cd0009260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1ce6e08b80 1 Processor -- start 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1ce6e08b80 1 -- start start 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225640 0x55d422225a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d42221f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d42221ff20 0x55d422220340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d422171810 con 0x55d42221ff20 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d422119720 con 0x55d422225640 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d42216d7d0 con 0x55d422225f60 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1cdab22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d42221ff20 0x55d422220340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1cda321700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225640 0x55d422225a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1cdab22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d42221ff20 0x55d422220340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39212/0 (socket says 172.21.15.27:39212) 2024-06-06T08:42:59.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1cda321700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225640 0x55d422225a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:51864/0 (socket says 172.21.15.27:51864) 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.800+0000 7f1cdab22700 1 -- 172.21.15.27:0/3224998451 learned_addr learned my addr 172.21.15.27:0/3224998451 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cd9b20700 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d42221f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cda321700 1 -- 172.21.15.27:0/3224998451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 msgr2=0x55d42221f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cda321700 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d42221f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cda321700 1 -- 172.21.15.27:0/3224998451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d42221ff20 msgr2=0x55d422220340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cda321700 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d42221ff20 0x55d422220340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cda321700 1 -- 172.21.15.27:0/3224998451 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42210ff50 con 0x55d422225640 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cd9b20700 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d42221f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cdab22700 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d42221ff20 0x55d422220340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cda321700 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225640 0x55d422225a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0011770 tx=0x7f1cd0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6499a89635b12ccb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cc8ff9700 1 -- 172.21.15.27:0/3224998451 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0012ae0 con 0x55d422225640 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cc8ff9700 1 -- 172.21.15.27:0/3224998451 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd0012c80 con 0x55d422225640 2024-06-06T08:42:59.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.801+0000 7f1cc8ff9700 1 -- 172.21.15.27:0/3224998451 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd001d680 con 0x55d422225640 2024-06-06T08:42:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3224998451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225640 msgr2=0x55d422225a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225640 0x55d422225a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0011770 tx=0x7f1cd0014000 comp rx=0 tx=0).stop 2024-06-06T08:42:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3224998451 shutdown_connections 2024-06-06T08:42:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d42221ff20 0x55d422220340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225640 0x55d422225a20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3224998451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d42221f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3224998451 >> 172.21.15.27:0/3224998451 conn(0x55d42214acc0 msgr2=0x55d42221a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3224998451 shutdown_connections 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3224998451 wait complete. 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.802+0000 7f1ce6e08b80 1 -- start start 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1ce6e08b80 1 Processor -- start 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d42221ff20 0x55d42222bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422228fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225f60 0x55d422229500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d42222c9d0 con 0x55d422225640 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d422111bf0 con 0x55d422225f60 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d42222cb50 con 0x55d42221ff20 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1ce6e08b80 10 client.0 _ll_register_callbacks cb 0x55d42210b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cd9b20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422228fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cdab22700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225f60 0x55d422229500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cd9b20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422228fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39218/0 (socket says 172.21.15.27:39218) 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cd9b20700 1 -- 172.21.15.27:0/815017532 learned_addr learned my addr 172.21.15.27:0/815017532 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cda321700 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d42221ff20 0x55d42222bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cd9b20700 1 -- 172.21.15.27:0/815017532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d42221ff20 msgr2=0x55d42222bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cd9b20700 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d42221ff20 0x55d42222bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cd9b20700 1 -- 172.21.15.27:0/815017532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225f60 msgr2=0x55d422229500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cd9b20700 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225f60 0x55d422229500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cd9b20700 1 -- 172.21.15.27:0/815017532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4220c08d0 con 0x55d422225640 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cda321700 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d42221ff20 0x55d42222bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.803+0000 7f1cdab22700 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225f60 0x55d422229500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1cd9b20700 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422228fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40099c0 tx=0x7f1cc400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73b3da1a7ac2de9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1cbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1cbdffb700 1 -- 172.21.15.27:0/815017532 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4003a60 con 0x55d422225640 2024-06-06T08:42:59.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1cbdffb700 1 -- 172.21.15.27:0/815017532 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f1cc4003c00 con 0x55d422225640 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/815017532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd0012c80 con 0x55d422225640 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1cbdffb700 1 -- 172.21.15.27:0/815017532 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1cc4030050 con 0x55d422225640 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1ce6e08b80 10 client.16101 fetch_fsmap learned FSMap version 29 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/815017532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d422237380 con 0x55d422225640 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1cbdffb700 1 -- 172.21.15.27:0/815017532 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd0012c80 con 0x55d422225640 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.804+0000 7f1cbdffb700 1 -- 172.21.15.27:0/815017532 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f1cc402b030 con 0x55d422225640 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.805+0000 7f1ce6e08b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.805+0000 7f1ce6e08b80 10 client.16101 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.805+0000 7f1ce6e08b80 20 client.16101 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.805+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55d422239a60 0x55d42223be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.805+0000 7f1ce6e08b80 4 client.16101 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.805+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/815017532 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d42217ed10 con 0x55d422239a60 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.805+0000 7f1cda321700 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55d422239a60 0x55d42223be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.805+0000 7f1cda321700 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55d422239a60 0x55d42223be50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48d793f35f08f88 server_cookie=de339dd1a4b1891e in_seq=0 out_seq=0 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.805+0000 7f1cbdffb700 10 client.16101 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1cbdffb700 1 -- 172.21.15.27:0/815017532 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d42217ed10 con 0x55d422239a60 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1cbdffb700 10 client.16101 handle_command_reply: tid=0 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 client.16101 shutdown 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/815017532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 msgr2=0x55d422228fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422228fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40099c0 tx=0x7f1cc400bee0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/815017532 shutdown_connections 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55d422239a60 0x55d42223be50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d422225f60 0x55d422229500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1cbdffb700 0 client.16101 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d42221ff20 0x55d42222bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/815017532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422228fc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/815017532 >> 172.21.15.27:0/815017532 conn(0x55d42214acc0 msgr2=0x55d42221e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/815017532 shutdown_connections 2024-06-06T08:42:59.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/815017532 wait complete. 2024-06-06T08:42:59.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.806+0000 7f1ce6e08b80 20 client.16101 trim_cache size 0 max 16384 2024-06-06T08:42:59.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1ce6e08b80 1 Processor -- start 2024-06-06T08:42:59.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1ce6e08b80 1 -- start start 2024-06-06T08:42:59.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d42221f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422226a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d422241700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd41b0e20 con 0x55d422225640 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd41ae620 con 0x55d42221ff20 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd41ac470 con 0x55d422225f60 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1cd9b20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422226a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1cd9b20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422226a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39234/0 (socket says 172.21.15.27:39234) 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1cd9b20700 1 -- 172.21.15.27:0/179173574 learned_addr learned my addr 172.21.15.27:0/179173574 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1cda321700 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d42221f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.822+0000 7f1cdab22700 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d422241700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cd9b20700 1 -- 172.21.15.27:0/179173574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 msgr2=0x55d422241700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cd9b20700 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d422241700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cd9b20700 1 -- 172.21.15.27:0/179173574 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 msgr2=0x55d42221f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cd9b20700 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d42221f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cd9b20700 1 -- 172.21.15.27:0/179173574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d422226f90 con 0x55d422225640 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cda321700 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d42221f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cdab22700 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d422241700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cd9b20700 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422226a50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4000c00 tx=0x7f1cc400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894602610176ab64 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cbffff700 1 -- 172.21.15.27:0/179173574 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4038020 con 0x55d422225640 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cbffff700 1 -- 172.21.15.27:0/179173574 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc4003680 con 0x55d422225640 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1cbffff700 1 -- 172.21.15.27:0/179173574 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4005410 con 0x55d422225640 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/179173574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 msgr2=0x55d422226a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422226a50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4000c00 tx=0x7f1cc400ef50 comp rx=0 tx=0).stop 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.823+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/179173574 shutdown_connections 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225f60 0x55d422241700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d42221f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/179173574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225640 0x55d422226a50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/179173574 >> 172.21.15.27:0/179173574 conn(0x55d42214acc0 msgr2=0x55d422226340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/179173574 shutdown_connections 2024-06-06T08:42:59.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/179173574 wait complete. 2024-06-06T08:42:59.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 -- start start 2024-06-06T08:42:59.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 Processor -- start 2024-06-06T08:42:59.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d422231e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225640 0x55d4222323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225f60 0x55d4222406f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd41acae0 con 0x55d422225f60 2024-06-06T08:42:59.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd41adf50 con 0x55d42221ff20 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd41ad1b0 con 0x55d422225640 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1ce6e08b80 10 client.0 _ll_register_callbacks cb 0x55d42210b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1cdab22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225f60 0x55d4222406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1cda321700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d422231e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1cda321700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d422231e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:51892/0 (socket says 172.21.15.27:51892) 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1cdab22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225f60 0x55d4222406f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39246/0 (socket says 172.21.15.27:39246) 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1cda321700 1 -- 172.21.15.27:0/3339449621 learned_addr learned my addr 172.21.15.27:0/3339449621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.824+0000 7f1cd9b20700 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225640 0x55d4222323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cdab22700 1 -- 172.21.15.27:0/3339449621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225640 msgr2=0x55d4222323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cdab22700 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225640 0x55d4222323b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cdab22700 1 -- 172.21.15.27:0/3339449621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 msgr2=0x55d422231e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cdab22700 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d422231e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cdab22700 1 -- 172.21.15.27:0/3339449621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d422240c30 con 0x55d422225f60 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cda321700 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d422231e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cdab22700 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225f60 0x55d4222406f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc097a60 tx=0x7f1ccc09aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f24c0cf2f87440 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cbdffb700 1 -- 172.21.15.27:0/3339449621 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0a3d00 con 0x55d422225f60 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cbdffb700 1 -- 172.21.15.27:0/3339449621 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f1ccc09f0e0 con 0x55d422225f60 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3339449621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc4003680 con 0x55d422225f60 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cbdffb700 1 -- 172.21.15.27:0/3339449621 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ccc065050 con 0x55d422225f60 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1ce6e08b80 10 client.16107 fetch_fsmap learned FSMap version 29 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3339449621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d4222ac500 con 0x55d422225f60 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.825+0000 7f1cbdffb700 1 -- 172.21.15.27:0/3339449621 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc4003680 con 0x55d422225f60 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1cbdffb700 1 -- 172.21.15.27:0/3339449621 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f1ccc0a3870 con 0x55d422225f60 2024-06-06T08:42:59.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1ce6e08b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1ce6e08b80 10 client.16107 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1ce6e08b80 20 client.16107 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55d422239a60 0x55d4222aec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1ce6e08b80 4 client.16107 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3339449621 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d422179290 con 0x55d422239a60 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1cda321700 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55d422239a60 0x55d4222aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1cbdffb700 10 client.16107 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.826+0000 7f1cda321700 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55d422239a60 0x55d4222aec20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19a797d6ad25451e server_cookie=a463cf7618060fec in_seq=0 out_seq=0 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1cbdffb700 1 -- 172.21.15.27:0/3339449621 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d422179290 con 0x55d422239a60 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1cbdffb700 10 client.16107 handle_command_reply: tid=0 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 client.16107 shutdown 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3339449621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225f60 msgr2=0x55d4222406f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225f60 0x55d4222406f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc097a60 tx=0x7f1ccc09aee0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3339449621 shutdown_connections 2024-06-06T08:42:59.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d422225f60 0x55d4222406f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x55d422239a60 0x55d4222aec20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d42221ff20 0x55d422231e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 --2- 172.21.15.27:0/3339449621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d422225640 0x55d4222323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1cbdffb700 0 client.16107 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:42:59.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3339449621 >> 172.21.15.27:0/3339449621 conn(0x55d42214acc0 msgr2=0x55d422213e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3339449621 shutdown_connections 2024-06-06T08:42:59.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 1 -- 172.21.15.27:0/3339449621 wait complete. 2024-06-06T08:42:59.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1ce6e08b80 20 client.16107 trim_cache size 0 max 16384 2024-06-06T08:42:59.834 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:42:59.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1cdc585700 1 -- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1cc00a2d90 msgr2=0x7f1cc00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1cdc585700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1cc00a2d90 0x7f1cc00a5250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4107560 tx=0x7f1cd0009260 comp rx=0 tx=0).stop 2024-06-06T08:42:59.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1cdc585700 1 -- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4108540 msgr2=0x7f1cd41989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.827+0000 7f1cdc585700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4108540 0x7f1cd41989c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc007ee0 tx=0x7f1ccc00d3b0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.828+0000 7f1cdc585700 1 -- 172.21.15.27:0/443230896 shutdown_connections 2024-06-06T08:42:59.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.828+0000 7f1cdc585700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cd4108540 0x7f1cd41989c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.828+0000 7f1cdc585700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd4102490 0x7f1cd419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.828+0000 7f1cdc585700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f1cc00a2d90 0x7f1cc00a5250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.828+0000 7f1cdc585700 1 --2- 172.21.15.27:0/443230896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cd4102e50 0x7f1cd419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.828+0000 7f1cdc585700 1 -- 172.21.15.27:0/443230896 >> 172.21.15.27:0/443230896 conn(0x7f1cd40fdf40 msgr2=0x7f1cd40ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.828+0000 7f1cdc585700 1 -- 172.21.15.27:0/443230896 shutdown_connections 2024-06-06T08:42:59.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:42:59.828+0000 7f1cdc585700 1 -- 172.21.15.27:0/443230896 wait complete. 2024-06-06T08:42:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 -- 172.21.15.27:0/2681074028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d6006ab90 msgr2=0x7f6d6010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 --2- 172.21.15.27:0/2681074028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d6006ab90 0x7f6d6010e840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009c10 tx=0x7f6d5802a770 comp rx=0 tx=0).stop 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 -- 172.21.15.27:0/2681074028 shutdown_connections 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 --2- 172.21.15.27:0/2681074028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d6006ab90 0x7f6d6010e840 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 --2- 172.21.15.27:0/2681074028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6006a1d0 0x7f6d6006a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 --2- 172.21.15.27:0/2681074028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d60107ba0 0x7f6d60107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 -- 172.21.15.27:0/2681074028 >> 172.21.15.27:0/2681074028 conn(0x7f6d600766a0 msgr2=0x7f6d60076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 -- 172.21.15.27:0/2681074028 shutdown_connections 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 -- 172.21.15.27:0/2681074028 wait complete. 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 Processor -- start 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.943+0000 7f6d656e1640 1 -- start start 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d656e1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d6006a1d0 0x7f6d60198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d656e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6006ab90 0x7f6d60198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d656e1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d60107ba0 0x7f6d6019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d656e1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60111d70 con 0x7f6d60107ba0 2024-06-06T08:42:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d656e1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60111bf0 con 0x7f6d6006a1d0 2024-06-06T08:42:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d656e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60111ef0 con 0x7f6d6006ab90 2024-06-06T08:42:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d60107ba0 0x7f6d6019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d60107ba0 0x7f6d6019c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56610/0 (socket says 172.21.15.27:56610) 2024-06-06T08:42:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5f7fe640 1 -- 172.21.15.27:0/428685664 learned_addr learned my addr 172.21.15.27:0/428685664 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5effd640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d6006a1d0 0x7f6d60198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d57fff640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6006ab90 0x7f6d60198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5f7fe640 1 -- 172.21.15.27:0/428685664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6006ab90 msgr2=0x7f6d60198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5f7fe640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6006ab90 0x7f6d60198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5f7fe640 1 -- 172.21.15.27:0/428685664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d6006a1d0 msgr2=0x7f6d60198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5f7fe640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d6006a1d0 0x7f6d60198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5f7fe640 1 -- 172.21.15.27:0/428685664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6019d120 con 0x7f6d60107ba0 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5effd640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d6006a1d0 0x7f6d60198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5f7fe640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d60107ba0 0x7f6d6019c9f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6d580305d0 tx=0x7f6d58030600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.944+0000 7f6d5cff9640 1 -- 172.21.15.27:0/428685664 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58041020 con 0x7f6d60107ba0 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.945+0000 7f6d656e1640 1 -- 172.21.15.27:0/428685664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6019d3b0 con 0x7f6d60107ba0 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.945+0000 7f6d5cff9640 1 -- 172.21.15.27:0/428685664 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d58037040 con 0x7f6d60107ba0 2024-06-06T08:42:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.945+0000 7f6d5cff9640 1 -- 172.21.15.27:0/428685664 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58036070 con 0x7f6d60107ba0 2024-06-06T08:42:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.945+0000 7f6d656e1640 1 -- 172.21.15.27:0/428685664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d601a4ca0 con 0x7f6d60107ba0 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.947+0000 7f6d5cff9640 1 -- 172.21.15.27:0/428685664 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6d58044020 con 0x7f6d60107ba0 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.947+0000 7f6d683f5740 1 Processor -- start 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.947+0000 7f6d683f5740 1 -- start start 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 0x55f897101450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897110220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 0x55f897112b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d683f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f8970654f0 con 0x55f897101990 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d683f5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f897003870 con 0x55f897101070 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d683f5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f8970614b0 con 0x55f897110760 2024-06-06T08:42:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897110220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56624/0 (socket says 172.21.15.27:56624) 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5effd640 1 -- 172.21.15.27:0/4292143606 learned_addr learned my addr 172.21.15.27:0/4292143606 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d57fff640 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 0x55f897101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d57fff640 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 0x55f897112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5effd640 1 -- 172.21.15.27:0/4292143606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 msgr2=0x55f897112b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5effd640 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 0x55f897112b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5effd640 1 -- 172.21.15.27:0/4292143606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 msgr2=0x55f897101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5effd640 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 0x55f897101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5effd640 1 -- 172.21.15.27:0/4292143606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89701f9b0 con 0x55f897101990 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d57fff640 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 0x55f897112b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5cff9640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d4c0a2da0 0x7f6d4c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5effd640 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897110220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6d500114f0 tx=0x7f6d500119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c412447206688cf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d57fff640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d4c0a2da0 0x7f6d4c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d5cff9640 1 -- 172.21.15.27:0/428685664 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6d580f0210 con 0x7f6d60107ba0 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d557fa640 1 -- 172.21.15.27:0/4292143606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50005020 con 0x55f897101990 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d557fa640 1 -- 172.21.15.27:0/4292143606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d500047a0 con 0x55f897101990 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.948+0000 7f6d557fa640 1 -- 172.21.15.27:0/4292143606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000f960 con 0x55f897101990 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d683f5740 1 -- 172.21.15.27:0/4292143606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 msgr2=0x55f897110220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d57fff640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d4c0a2da0 0x7f6d4c0a5260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55f897102c30 tx=0x7f6d48004730 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897110220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6d500114f0 tx=0x7f6d500119b0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d683f5740 1 -- 172.21.15.27:0/4292143606 shutdown_connections 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 0x55f897112b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897110220 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4292143606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 0x55f897101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d683f5740 1 -- 172.21.15.27:0/4292143606 >> 172.21.15.27:0/4292143606 conn(0x55f8970fd680 msgr2=0x55f8970fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d683f5740 1 -- 172.21.15.27:0/4292143606 shutdown_connections 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.949+0000 7f6d683f5740 1 -- 172.21.15.27:0/4292143606 wait complete. 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d683f5740 1 -- start start 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d683f5740 1 Processor -- start 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f8971100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 0x55f89710d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 0x55f89710db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d683f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f897113390 con 0x55f897101070 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d683f5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55f896feeae0 con 0x55f897110760 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d683f5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f897113510 con 0x55f897101990 2024-06-06T08:42:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d683f5740 10 client.0 _ll_register_callbacks cb 0x55f8970279b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f8971100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f8971100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56628/0 (socket says 172.21.15.27:56628) 2024-06-06T08:42:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d5effd640 1 -- 172.21.15.27:0/4180344066 learned_addr learned my addr 172.21.15.27:0/4180344066 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d5f7fe640 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 0x55f89710db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d57fff640 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 0x55f89710d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d5effd640 1 -- 172.21.15.27:0/4180344066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 msgr2=0x55f89710d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d5effd640 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 0x55f89710d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d5effd640 1 -- 172.21.15.27:0/4180344066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 msgr2=0x55f89710db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d5effd640 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 0x55f89710db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.950+0000 7f6d5effd640 1 -- 172.21.15.27:0/4180344066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f896d79210 con 0x55f897101070 2024-06-06T08:42:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d5effd640 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f8971100d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6d50000c00 tx=0x7f6d50005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3258fc661719c5d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d467fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d467fc640 1 -- 172.21.15.27:0/4180344066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50028070 con 0x55f897101070 2024-06-06T08:42:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d683f5740 1 -- 172.21.15.27:0/4180344066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d500047a0 con 0x55f897101070 2024-06-06T08:42:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d467fc640 1 -- 172.21.15.27:0/4180344066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6d5000f6c0 con 0x55f897101070 2024-06-06T08:42:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d467fc640 1 -- 172.21.15.27:0/4180344066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d50034050 con 0x55f897101070 2024-06-06T08:42:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d683f5740 10 client.16119 fetch_fsmap learned FSMap version 29 2024-06-06T08:42:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d683f5740 1 -- 172.21.15.27:0/4180344066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f896e45620 con 0x55f897101070 2024-06-06T08:42:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d467fc640 1 -- 172.21.15.27:0/4180344066 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d500047a0 con 0x55f897101070 2024-06-06T08:42:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d467fc640 1 -- 172.21.15.27:0/4180344066 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f6d500246a0 con 0x55f897101070 2024-06-06T08:42:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d683f5740 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:42:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d683f5740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d683f5740 20 client.16119 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f897124c60 0x55f897127050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d683f5740 4 client.16119 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:42:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.951+0000 7f6d683f5740 1 -- 172.21.15.27:0/4180344066 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f897078f10 con 0x55f897124c60 2024-06-06T08:42:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.952+0000 7f6d57fff640 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f897124c60 0x55f897127050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.952+0000 7f6d467fc640 10 client.16119 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.952+0000 7f6d57fff640 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f897124c60 0x55f897127050 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a2d9a44872053dd server_cookie=52265625dd2bb3d9 in_seq=0 out_seq=0 2024-06-06T08:42:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d467fc640 1 -- 172.21.15.27:0/4180344066 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f897078f10 con 0x55f897124c60 2024-06-06T08:42:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d467fc640 10 client.16119 handle_command_reply: tid=0 2024-06-06T08:42:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 client.16119 shutdown 2024-06-06T08:42:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 -- 172.21.15.27:0/4180344066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 msgr2=0x55f8971100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f8971100d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6d50000c00 tx=0x7f6d50005250 comp rx=0 tx=0).stop 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 -- 172.21.15.27:0/4180344066 shutdown_connections 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f897124c60 0x55f897127050 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 0x55f89710db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 0x55f89710d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d467fc640 0 client.16119 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 --2- 172.21.15.27:0/4180344066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f8971100d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 -- 172.21.15.27:0/4180344066 >> 172.21.15.27:0/4180344066 conn(0x55f8970fd680 msgr2=0x55f8971034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 -- 172.21.15.27:0/4180344066 shutdown_connections 2024-06-06T08:42:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 1 -- 172.21.15.27:0/4180344066 wait complete. 2024-06-06T08:42:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.954+0000 7f6d683f5740 20 client.16119 trim_cache size 0 max 16384 2024-06-06T08:42:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.970+0000 7f6d683f5740 1 Processor -- start 2024-06-06T08:42:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.970+0000 7f6d683f5740 1 -- start start 2024-06-06T08:42:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f89710fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 0x55f89712c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 0x55f89712cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d683f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601af5b0 con 0x55f897101070 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d683f5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601acdf0 con 0x55f897110760 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d683f5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601aac40 con 0x55f897101990 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f89710fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f89710fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56640/0 (socket says 172.21.15.27:56640) 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5f7fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 0x55f89712cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5effd640 1 -- 172.21.15.27:0/1736801848 learned_addr learned my addr 172.21.15.27:0/1736801848 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d57fff640 1 --2- 172.21.15.27:0/1736801848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 0x55f89712c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5effd640 1 -- 172.21.15.27:0/1736801848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 msgr2=0x55f89712c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5effd640 1 --2- 172.21.15.27:0/1736801848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 0x55f89712c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5effd640 1 -- 172.21.15.27:0/1736801848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 msgr2=0x55f89712cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5effd640 1 --2- 172.21.15.27:0/1736801848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 0x55f89712cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5effd640 1 -- 172.21.15.27:0/1736801848 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8971017c0 con 0x55f897101070 2024-06-06T08:42:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5f7fe640 1 --2- 172.21.15.27:0/1736801848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 0x55f89712cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:42:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.971+0000 7f6d5effd640 1 --2- 172.21.15.27:0/1736801848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f89710fe60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6d50006010 tx=0x7f6d500119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899002f0f5248550 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d54ff9640 1 -- 172.21.15.27:0/1736801848 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50028040 con 0x55f897101070 2024-06-06T08:42:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d54ff9640 1 -- 172.21.15.27:0/1736801848 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d50004780 con 0x55f897101070 2024-06-06T08:42:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d54ff9640 1 -- 172.21.15.27:0/1736801848 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50022930 con 0x55f897101070 2024-06-06T08:42:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d683f5740 1 -- 172.21.15.27:0/1736801848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 msgr2=0x55f89710fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:42:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d683f5740 1 --2- 172.21.15.27:0/1736801848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f89710fe60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6d50006010 tx=0x7f6d500119b0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d683f5740 1 -- 172.21.15.27:0/1736801848 shutdown_connections 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d683f5740 1 --2- 172.21.15.27:0/1736801848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897110760 0x55f89712cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d683f5740 1 --2- 172.21.15.27:0/1736801848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897101990 0x55f89712c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d683f5740 1 --2- 172.21.15.27:0/1736801848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101070 0x55f89710fe60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d683f5740 1 -- 172.21.15.27:0/1736801848 >> 172.21.15.27:0/1736801848 conn(0x55f8970fd680 msgr2=0x55f897108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d683f5740 1 -- 172.21.15.27:0/1736801848 shutdown_connections 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.972+0000 7f6d683f5740 1 -- 172.21.15.27:0/1736801848 wait complete. 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d683f5740 1 -- start start 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d683f5740 1 Processor -- start 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 0x55f897121220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897121760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d683f5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 0x55f897109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d683f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601ab2b0 con 0x55f897101990 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d683f5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601ac720 con 0x55f897101070 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d683f5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601ab980 con 0x55f897110760 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d683f5740 10 client.0 _ll_register_callbacks cb 0x7f6d48009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 0x55f897121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 0x55f897121220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:43244/0 (socket says 172.21.15.27:43244) 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d5effd640 1 -- 172.21.15.27:0/3502268021 learned_addr learned my addr 172.21.15.27:0/3502268021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897121760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56652/0 (socket says 172.21.15.27:56652) 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d5f7fe640 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 0x55f897109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d57fff640 1 -- 172.21.15.27:0/3502268021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 msgr2=0x55f897109ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.973+0000 7f6d57fff640 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 0x55f897109ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d57fff640 1 -- 172.21.15.27:0/3502268021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 msgr2=0x55f897121220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:42:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d57fff640 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 0x55f897121220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d57fff640 1 -- 172.21.15.27:0/3502268021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89710a430 con 0x55f897101990 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d57fff640 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897121760 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55f8971277b0 tx=0x7f6d48005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee16876a5a03a73 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d467fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d467fc640 1 -- 172.21.15.27:0/3502268021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48010430 con 0x55f897101990 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d467fc640 1 -- 172.21.15.27:0/3502268021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6d480105d0 con 0x55f897101990 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d683f5740 1 -- 172.21.15.27:0/3502268021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d50004780 con 0x55f897101990 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d467fc640 1 -- 172.21.15.27:0/3502268021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d4803b050 con 0x55f897101990 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d683f5740 10 client.16125 fetch_fsmap learned FSMap version 29 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d683f5740 1 -- 172.21.15.27:0/3502268021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f89718efe0 con 0x55f897101990 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d467fc640 1 -- 172.21.15.27:0/3502268021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d50004780 con 0x55f897101990 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d467fc640 1 -- 172.21.15.27:0/3502268021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f6d480218b0 con 0x55f897101990 2024-06-06T08:42:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d683f5740 10 client.16125 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:42:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d683f5740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:42:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d683f5740 20 client.16125 populate_metadata read hostname 'smithi027' 2024-06-06T08:42:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d683f5740 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f89718f760 0x55f897191b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:42:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d683f5740 4 client.16125 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:42:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.974+0000 7f6d683f5740 1 -- 172.21.15.27:0/3502268021 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f897072c20 con 0x55f89718f760 2024-06-06T08:42:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.975+0000 7f6d5effd640 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f89718f760 0x55f897191b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:42:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.975+0000 7f6d5effd640 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f89718f760 0x55f897191b50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3216ef75372acd7f server_cookie=790c8a40e860fe33 in_seq=0 out_seq=0 2024-06-06T08:42:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:42:59.975+0000 7f6d467fc640 10 client.16125 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:00.684 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T08:43:00.705 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:00 smithi027 ceph-mon[32335]: pgmap v85: 129 pgs: 129 active+clean; 58 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 1.11k op/s 2024-06-06T08:43:00.941 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:43:00.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:00 smithi203 ceph-mon[36579]: pgmap v85: 129 pgs: 129 active+clean; 58 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 1.11k op/s 2024-06-06T08:43:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:00 smithi176 ceph-mon[36786]: pgmap v85: 129 pgs: 129 active+clean; 58 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 1.11k op/s 2024-06-06T08:43:02.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.787+0000 7f69b06f6700 1 -- 172.21.15.27:0/2214702038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a8104ec0 msgr2=0x7f69a8105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:02.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.787+0000 7f69b06f6700 1 --2- 172.21.15.27:0/2214702038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a8104ec0 0x7f69a8105350 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f699801a9a0 tx=0x7f699801acb0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.787+0000 7f69b06f6700 1 -- 172.21.15.27:0/2214702038 shutdown_connections 2024-06-06T08:43:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.787+0000 7f69b06f6700 1 --2- 172.21.15.27:0/2214702038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a8104ec0 0x7f69a8105350 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.787+0000 7f69b06f6700 1 --2- 172.21.15.27:0/2214702038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a81073b0 0x7f69a806aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.787+0000 7f69b06f6700 1 --2- 172.21.15.27:0/2214702038 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69a806af70 0x7f69a806b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.787+0000 7f69b06f6700 1 -- 172.21.15.27:0/2214702038 >> 172.21.15.27:0/2214702038 conn(0x7f69a8078300 msgr2=0x7f69a8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:02.787 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:02 smithi027 ceph-mon[32335]: pgmap v86: 129 pgs: 129 active+clean; 58 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-06-06T08:43:02.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.790+0000 7f69b06f6700 1 -- 172.21.15.27:0/2214702038 shutdown_connections 2024-06-06T08:43:02.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.790+0000 7f69b06f6700 1 -- 172.21.15.27:0/2214702038 wait complete. 2024-06-06T08:43:02.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.792+0000 7f69b06f6700 1 Processor -- start 2024-06-06T08:43:02.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.792+0000 7f69b06f6700 1 -- start start 2024-06-06T08:43:02.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.792+0000 7f69b06f6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69a806af70 0x7f69a8111710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.792+0000 7f69b06f6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a81073b0 0x7f69a8111c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.792+0000 7f69b06f6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a819c2c0 0x7f69a819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.792+0000 7f69b06f6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69a810a310 con 0x7f69a81073b0 2024-06-06T08:43:02.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.792+0000 7f69b06f6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f69a810a190 con 0x7f69a806af70 2024-06-06T08:43:02.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.792+0000 7f69b06f6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69a810a490 con 0x7f69a819c2c0 2024-06-06T08:43:02.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.795+0000 7f69adc91700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a81073b0 0x7f69a8111c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.795+0000 7f69adc91700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a81073b0 0x7f69a8111c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56668/0 (socket says 172.21.15.27:56668) 2024-06-06T08:43:02.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.795+0000 7f69adc91700 1 -- 172.21.15.27:0/3125350616 learned_addr learned my addr 172.21.15.27:0/3125350616 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:02.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.796+0000 7f69adc91700 1 -- 172.21.15.27:0/3125350616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a819c2c0 msgr2=0x7f69a819c740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:43:02.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.796+0000 7f69adc91700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a819c2c0 0x7f69a819c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.796+0000 7f69adc91700 1 -- 172.21.15.27:0/3125350616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69a806af70 msgr2=0x7f69a8111710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:43:02.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.796+0000 7f69adc91700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69a806af70 0x7f69a8111710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.796+0000 7f69adc91700 1 -- 172.21.15.27:0/3125350616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a819ce70 con 0x7f69a81073b0 2024-06-06T08:43:02.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.796+0000 7f69adc91700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a81073b0 0x7f69a8111c50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f69a000ada0 tx=0x7f69a000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:02.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.797+0000 7f699f7fe700 1 -- 172.21.15.27:0/3125350616 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0017020 con 0x7f69a81073b0 2024-06-06T08:43:02.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.797+0000 7f69b06f6700 1 -- 172.21.15.27:0/3125350616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a806beb0 con 0x7f69a81073b0 2024-06-06T08:43:02.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.797+0000 7f699f7fe700 1 -- 172.21.15.27:0/3125350616 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69a0004bb0 con 0x7f69a81073b0 2024-06-06T08:43:02.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.797+0000 7f699f7fe700 1 -- 172.21.15.27:0/3125350616 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0005610 con 0x7f69a81073b0 2024-06-06T08:43:02.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.797+0000 7f69b06f6700 1 -- 172.21.15.27:0/3125350616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a806c3f0 con 0x7f69a81073b0 2024-06-06T08:43:02.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.803+0000 7f699f7fe700 1 -- 172.21.15.27:0/3125350616 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f69a00057b0 con 0x7f69a81073b0 2024-06-06T08:43:02.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.805+0000 7f699f7fe700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f69940a2fa0 0x7f69940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.805+0000 7f699f7fe700 1 -- 172.21.15.27:0/3125350616 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f69a00cef60 con 0x7f69a81073b0 2024-06-06T08:43:02.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.813+0000 7f69baf79b80 1 Processor -- start 2024-06-06T08:43:02.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.815+0000 7f69ae492700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f69940a2fa0 0x7f69940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.816+0000 7f69baf79b80 1 -- start start 2024-06-06T08:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.817+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 0x55d3e2793340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.817+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e278d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.817+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e278dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.817+0000 7f69baf79b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d3e267ff60 con 0x55d3e2792f60 2024-06-06T08:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.817+0000 7f69baf79b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d3e26de850 con 0x55d3e278d7a0 2024-06-06T08:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.817+0000 7f69baf79b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d3e2646460 con 0x55d3e2793880 2024-06-06T08:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.817+0000 7f69adc91700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e278d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.819+0000 7f69aec93700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e278dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.819+0000 7f69adc91700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e278d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:47974/0 (socket says 172.21.15.27:47974) 2024-06-06T08:43:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.819+0000 7f69aec93700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e278dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:43264/0 (socket says 172.21.15.27:43264) 2024-06-06T08:43:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.819+0000 7f69aec93700 1 -- 172.21.15.27:0/4226735669 learned_addr learned my addr 172.21.15.27:0/4226735669 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.820+0000 7f69aec93700 1 -- 172.21.15.27:0/4226735669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 msgr2=0x55d3e278d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.820+0000 7f69aec93700 1 --2- 172.21.15.27:0/4226735669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e278d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.820+0000 7f69aec93700 1 -- 172.21.15.27:0/4226735669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 msgr2=0x55d3e2793340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.820+0000 7f69aec93700 1 --2- 172.21.15.27:0/4226735669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 0x55d3e2793340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.820+0000 7f69aec93700 1 -- 172.21.15.27:0/4226735669 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e262d8d0 con 0x55d3e278d7a0 2024-06-06T08:43:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.820+0000 7f69aec93700 1 --2- 172.21.15.27:0/4226735669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e278dbc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f699801a6b0 tx=0x7f699801a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=235574abf168239d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.821+0000 7f69ae492700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f69940a2fa0 0x7f69940a5460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f69a400a940 tx=0x7f69a400ad00 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:02.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.824+0000 7f699cff9700 1 -- 172.21.15.27:0/4226735669 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699801ca70 con 0x55d3e278d7a0 2024-06-06T08:43:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.824+0000 7f699cff9700 1 -- 172.21.15.27:0/4226735669 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6998005550 con 0x55d3e278d7a0 2024-06-06T08:43:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.824+0000 7f699cff9700 1 -- 172.21.15.27:0/4226735669 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998003a00 con 0x55d3e278d7a0 2024-06-06T08:43:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.826+0000 7f69baf79b80 1 -- 172.21.15.27:0/4226735669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 msgr2=0x55d3e278dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.826+0000 7f69baf79b80 1 --2- 172.21.15.27:0/4226735669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e278dbc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f699801a6b0 tx=0x7f699801a6e0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.826+0000 7f69baf79b80 1 -- 172.21.15.27:0/4226735669 shutdown_connections 2024-06-06T08:43:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.826+0000 7f69baf79b80 1 --2- 172.21.15.27:0/4226735669 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e278dbc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.826+0000 7f69baf79b80 1 --2- 172.21.15.27:0/4226735669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 0x55d3e2793340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.826+0000 7f69baf79b80 1 --2- 172.21.15.27:0/4226735669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e278d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.826+0000 7f69baf79b80 1 -- 172.21.15.27:0/4226735669 >> 172.21.15.27:0/4226735669 conn(0x55d3e26ba080 msgr2=0x55d3e2788220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.828+0000 7f69baf79b80 1 -- 172.21.15.27:0/4226735669 shutdown_connections 2024-06-06T08:43:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.829+0000 7f69baf79b80 1 -- 172.21.15.27:0/4226735669 wait complete. 2024-06-06T08:43:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.829+0000 7f69baf79b80 1 -- start start 2024-06-06T08:43:02.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.829+0000 7f69baf79b80 1 Processor -- start 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.829+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e2799740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.829+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2792f60 0x55d3e2796a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.829+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2793880 0x55d3e2796f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.829+0000 7f69baf79b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d3e267efc0 con 0x55d3e2793880 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.829+0000 7f69baf79b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d3e26da810 con 0x55d3e278d7a0 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.829+0000 7f69baf79b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d3e279a200 con 0x55d3e2792f60 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.830+0000 7f69baf79b80 10 client.0 _ll_register_callbacks cb 0x55d3e26745b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.830+0000 7f69ae492700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e2799740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.830+0000 7f69ae492700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e2799740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:43280/0 (socket says 172.21.15.27:43280) 2024-06-06T08:43:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.830+0000 7f69ae492700 1 -- 172.21.15.27:0/2116480487 learned_addr learned my addr 172.21.15.27:0/2116480487 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.830+0000 7f69aec93700 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2793880 0x55d3e2796f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.831+0000 7f69adc91700 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2792f60 0x55d3e2796a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.831+0000 7f69aec93700 1 -- 172.21.15.27:0/2116480487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2792f60 msgr2=0x55d3e2796a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.831+0000 7f69aec93700 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2792f60 0x55d3e2796a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.831+0000 7f69aec93700 1 -- 172.21.15.27:0/2116480487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 msgr2=0x55d3e2799740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.831+0000 7f69aec93700 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e2799740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.831+0000 7f69aec93700 1 -- 172.21.15.27:0/2116480487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e26a4750 con 0x55d3e2793880 2024-06-06T08:43:02.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.833+0000 7f69aec93700 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2793880 0x55d3e2796f80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f699801a9a0 tx=0x7f6998005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392ef7ae9052a1e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:02.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.834+0000 7f6991ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:02.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.834+0000 7f6991ffb700 1 -- 172.21.15.27:0/2116480487 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699801cc80 con 0x55d3e2793880 2024-06-06T08:43:02.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.834+0000 7f6991ffb700 1 -- 172.21.15.27:0/2116480487 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f699802e050 con 0x55d3e2793880 2024-06-06T08:43:02.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.834+0000 7f69baf79b80 1 -- 172.21.15.27:0/2116480487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6998005550 con 0x55d3e2793880 2024-06-06T08:43:02.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.834+0000 7f6991ffb700 1 -- 172.21.15.27:0/2116480487 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f699801ce20 con 0x55d3e2793880 2024-06-06T08:43:02.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.835+0000 7f69baf79b80 10 client.16134 fetch_fsmap learned FSMap version 29 2024-06-06T08:43:02.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.835+0000 7f69baf79b80 1 -- 172.21.15.27:0/2116480487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3e27a4ae0 con 0x55d3e2793880 2024-06-06T08:43:02.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.835+0000 7f6991ffb700 1 -- 172.21.15.27:0/2116480487 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6998005550 con 0x55d3e2793880 2024-06-06T08:43:02.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.835+0000 7f6991ffb700 1 -- 172.21.15.27:0/2116480487 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f6998032790 con 0x55d3e2793880 2024-06-06T08:43:02.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.835+0000 7f69baf79b80 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:43:02.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.835+0000 7f69baf79b80 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.835+0000 7f69baf79b80 20 client.16134 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.836+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d3e27a7000 0x55d3e27a9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.836+0000 7f69baf79b80 4 client.16134 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:43:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.836+0000 7f69baf79b80 1 -- 172.21.15.27:0/2116480487 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3e26ebd50 con 0x55d3e27a7000 2024-06-06T08:43:02.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.840+0000 7f69ae492700 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d3e27a7000 0x55d3e27a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.853+0000 7f69ae492700 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d3e27a7000 0x55d3e27a9420 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7439655af7769cc4 server_cookie=31698a98ca6f1136 in_seq=0 out_seq=0 2024-06-06T08:43:02.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.853+0000 7f6991ffb700 10 client.16134 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:02.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.885+0000 7f6991ffb700 1 -- 172.21.15.27:0/2116480487 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d3e26ebd50 con 0x55d3e27a7000 2024-06-06T08:43:02.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.885+0000 7f6991ffb700 10 client.16134 handle_command_reply: tid=0 2024-06-06T08:43:02.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.885+0000 7f69baf79b80 1 client.16134 shutdown 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 -- 172.21.15.27:0/2116480487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2793880 msgr2=0x55d3e2796f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2793880 0x55d3e2796f80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f699801a9a0 tx=0x7f6998005260 comp rx=0 tx=0).stop 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 -- 172.21.15.27:0/2116480487 shutdown_connections 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d3e27a7000 0x55d3e27a9420 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2793880 0x55d3e2796f80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e2799740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2116480487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2792f60 0x55d3e2796a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 -- 172.21.15.27:0/2116480487 >> 172.21.15.27:0/2116480487 conn(0x55d3e26ba080 msgr2=0x55d3e2787700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f6991ffb700 0 client.16134 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 -- 172.21.15.27:0/2116480487 shutdown_connections 2024-06-06T08:43:02.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 1 -- 172.21.15.27:0/2116480487 wait complete. 2024-06-06T08:43:02.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.886+0000 7f69baf79b80 20 client.16134 trim_cache size 0 max 16384 2024-06-06T08:43:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.919+0000 7f69baf79b80 1 Processor -- start 2024-06-06T08:43:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.920+0000 7f69baf79b80 1 -- start start 2024-06-06T08:43:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.920+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e278d7a0 0x55d3e278d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.921+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e2792f60 0x55d3e27a3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.921+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e2794370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.921+0000 7f69baf79b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81b7690 con 0x55d3e278d7a0 2024-06-06T08:43:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.921+0000 7f69baf79b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81b4e90 con 0x55d3e2792f60 2024-06-06T08:43:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.921+0000 7f69baf79b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8073020 con 0x55d3e2793880 2024-06-06T08:43:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.922+0000 7f69ae492700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e278d7a0 0x55d3e278d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.922+0000 7f69ae492700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e278d7a0 0x55d3e278d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56692/0 (socket says 172.21.15.27:56692) 2024-06-06T08:43:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.922+0000 7f69ae492700 1 -- 172.21.15.27:0/2216890036 learned_addr learned my addr 172.21.15.27:0/2216890036 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:02.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.922+0000 7f69adc91700 1 --2- 172.21.15.27:0/2216890036 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e2792f60 0x55d3e27a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.922+0000 7f69aec93700 1 --2- 172.21.15.27:0/2216890036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e2794370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.923+0000 7f69ae492700 1 -- 172.21.15.27:0/2216890036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 msgr2=0x55d3e2794370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.923+0000 7f69ae492700 1 --2- 172.21.15.27:0/2216890036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e2794370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.923+0000 7f69ae492700 1 -- 172.21.15.27:0/2216890036 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e2792f60 msgr2=0x55d3e27a3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.923+0000 7f69ae492700 1 --2- 172.21.15.27:0/2216890036 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e2792f60 0x55d3e27a3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.923+0000 7f69ae492700 1 -- 172.21.15.27:0/2216890036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e2797560 con 0x55d3e278d7a0 2024-06-06T08:43:02.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.925+0000 7f69ae492700 1 --2- 172.21.15.27:0/2216890036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e278d7a0 0x55d3e278d260 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f69a400cc70 tx=0x7f69a4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cc978facb2e2ec server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:02.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.928+0000 7f6993fff700 1 -- 172.21.15.27:0/2216890036 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a402a030 con 0x55d3e278d7a0 2024-06-06T08:43:02.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.929+0000 7f6993fff700 1 -- 172.21.15.27:0/2216890036 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69a4018040 con 0x55d3e278d7a0 2024-06-06T08:43:02.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.929+0000 7f6993fff700 1 -- 172.21.15.27:0/2216890036 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4004de0 con 0x55d3e278d7a0 2024-06-06T08:43:02.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.930+0000 7f69baf79b80 1 -- 172.21.15.27:0/2216890036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e278d7a0 msgr2=0x55d3e278d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:02.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.930+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2216890036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e278d7a0 0x55d3e278d260 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f69a400cc70 tx=0x7f69a4011000 comp rx=0 tx=0).stop 2024-06-06T08:43:02.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.930+0000 7f69baf79b80 1 -- 172.21.15.27:0/2216890036 shutdown_connections 2024-06-06T08:43:02.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.930+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2216890036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e2794370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.930+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2216890036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e278d7a0 0x55d3e278d260 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.930+0000 7f69baf79b80 1 --2- 172.21.15.27:0/2216890036 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e2792f60 0x55d3e27a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.930+0000 7f69baf79b80 1 -- 172.21.15.27:0/2216890036 >> 172.21.15.27:0/2216890036 conn(0x55d3e26ba080 msgr2=0x55d3e2793c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:02.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.932+0000 7f69baf79b80 1 -- 172.21.15.27:0/2216890036 shutdown_connections 2024-06-06T08:43:02.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.932+0000 7f69baf79b80 1 -- 172.21.15.27:0/2216890036 wait complete. 2024-06-06T08:43:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.933+0000 7f69baf79b80 1 -- start start 2024-06-06T08:43:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.933+0000 7f69baf79b80 1 Processor -- start 2024-06-06T08:43:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.933+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e27af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.934+0000 7f69baf79b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 0x55d3e279f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.934+0000 7f69ae492700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e27af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.934+0000 7f69ae492700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e27af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:43296/0 (socket says 172.21.15.27:43296) 2024-06-06T08:43:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.934+0000 7f69ae492700 1 -- 172.21.15.27:0/1408583591 learned_addr learned my addr 172.21.15.27:0/1408583591 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.934+0000 7f69baf79b80 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e279fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.934+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8073690 con 0x55d3e2792f60 2024-06-06T08:43:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.934+0000 7f69adc91700 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 0x55d3e279f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.934+0000 7f69aec93700 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e279fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.935+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81b47c0 con 0x55d3e278d7a0 2024-06-06T08:43:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.935+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8073d60 con 0x55d3e2793880 2024-06-06T08:43:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.935+0000 7f69baf79b80 10 client.0 _ll_register_callbacks cb 0x55d3e26745b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.935+0000 7f69adc91700 1 -- 172.21.15.27:0/1408583591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 msgr2=0x55d3e279fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.935+0000 7f69adc91700 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e279fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.935+0000 7f69adc91700 1 -- 172.21.15.27:0/1408583591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 msgr2=0x55d3e27af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.935+0000 7f69adc91700 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e27af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.935+0000 7f69adc91700 1 -- 172.21.15.27:0/1408583591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e27adde0 con 0x55d3e2792f60 2024-06-06T08:43:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.936+0000 7f69adc91700 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 0x55d3e279f4c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f69a00cc290 tx=0x7f69a009aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79497188a455b142 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.936+0000 7f6991ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.936+0000 7f6991ffb700 1 -- 172.21.15.27:0/1408583591 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a009b7f0 con 0x55d3e2792f60 2024-06-06T08:43:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.936+0000 7f6991ffb700 1 -- 172.21.15.27:0/1408583591 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f69a009b350 con 0x55d3e2792f60 2024-06-06T08:43:02.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.937+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a4018040 con 0x55d3e2792f60 2024-06-06T08:43:02.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.937+0000 7f6991ffb700 1 -- 172.21.15.27:0/1408583591 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69a0004d60 con 0x55d3e2792f60 2024-06-06T08:43:02.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.937+0000 7f69baf79b80 10 client.16140 fetch_fsmap learned FSMap version 29 2024-06-06T08:43:02.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.937+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3e28110a0 con 0x55d3e2792f60 2024-06-06T08:43:02.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.938+0000 7f6991ffb700 1 -- 172.21.15.27:0/1408583591 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69a009b0f0 con 0x55d3e2792f60 2024-06-06T08:43:02.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.938+0000 7f6991ffb700 1 -- 172.21.15.27:0/1408583591 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f69a00b6020 con 0x55d3e2792f60 2024-06-06T08:43:02.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.938+0000 7f69baf79b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T08:43:02.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.938+0000 7f69baf79b80 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:02.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.938+0000 7f69baf79b80 20 client.16140 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:02.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.938+0000 7f69baf79b80 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d3e28117c0 0x55d3e2813be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:02.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.938+0000 7f69baf79b80 4 client.16140 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:43:02.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.938+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f69a4018040 con 0x55d3e28117c0 2024-06-06T08:43:02.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.943+0000 7f69ae492700 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d3e28117c0 0x55d3e2813be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:02.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.955+0000 7f6991ffb700 10 client.16140 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:02.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.955+0000 7f69ae492700 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d3e28117c0 0x55d3e2813be0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31d91c49823c802d server_cookie=c5020414faaf47a4 in_seq=0 out_seq=0 2024-06-06T08:43:02.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:02 smithi203 ceph-mon[36579]: pgmap v86: 129 pgs: 129 active+clean; 58 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-06-06T08:43:02.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.997+0000 7f6991ffb700 1 -- 172.21.15.27:0/1408583591 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f69a4018040 con 0x55d3e28117c0 2024-06-06T08:43:02.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.997+0000 7f6991ffb700 10 client.16140 handle_command_reply: tid=0 2024-06-06T08:43:02.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 client.16140 shutdown 2024-06-06T08:43:02.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 msgr2=0x55d3e279f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:02.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 0x55d3e279f4c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f69a00cc290 tx=0x7f69a009aad0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 shutdown_connections 2024-06-06T08:43:02.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d3e2793880 0x55d3e279fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d3e278d7a0 0x55d3e27af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d3e28117c0 0x55d3e2813be0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 --2- 172.21.15.27:0/1408583591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d3e2792f60 0x55d3e279f4c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:02.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f6991ffb700 0 client.16140 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:02.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 >> 172.21.15.27:0/1408583591 conn(0x55d3e26ba080 msgr2=0x55d3e2795bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:02.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 shutdown_connections 2024-06-06T08:43:02.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 1 -- 172.21.15.27:0/1408583591 wait complete. 2024-06-06T08:43:02.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.998+0000 7f69baf79b80 20 client.16140 trim_cache size 0 max 16384 2024-06-06T08:43:02.998 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:43:02.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 -- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f69940a2fa0 msgr2=0x7f69940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:02.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f69940a2fa0 0x7f69940a5460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f69a400a940 tx=0x7f69a400ad00 comp rx=0 tx=0).stop 2024-06-06T08:43:02.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 -- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a81073b0 msgr2=0x7f69a8111c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:02.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a81073b0 0x7f69a8111c50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f69a000ada0 tx=0x7f69a000c330 comp rx=0 tx=0).stop 2024-06-06T08:43:03.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 -- 172.21.15.27:0/3125350616 shutdown_connections 2024-06-06T08:43:03.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a819c2c0 0x7f69a819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:03.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f69940a2fa0 0x7f69940a5460 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:03.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69a806af70 0x7f69a8111710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:03.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 --2- 172.21.15.27:0/3125350616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69a81073b0 0x7f69a8111c50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:03.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 -- 172.21.15.27:0/3125350616 >> 172.21.15.27:0/3125350616 conn(0x7f69a8078300 msgr2=0x7f69a81029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:03.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:02.999+0000 7f69b06f6700 1 -- 172.21.15.27:0/3125350616 shutdown_connections 2024-06-06T08:43:03.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:03.000+0000 7f69b06f6700 1 -- 172.21.15.27:0/3125350616 wait complete. 2024-06-06T08:43:03.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:02 smithi176 ceph-mon[36786]: pgmap v86: 129 pgs: 129 active+clean; 58 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-06-06T08:43:04.877 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:04 smithi027 ceph-mon[32335]: pgmap v87: 129 pgs: 129 active+clean; 65 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 552 op/s 2024-06-06T08:43:04.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:04 smithi203 ceph-mon[36579]: pgmap v87: 129 pgs: 129 active+clean; 65 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 552 op/s 2024-06-06T08:43:05.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:04 smithi176 ceph-mon[36786]: pgmap v87: 129 pgs: 129 active+clean; 65 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 552 op/s 2024-06-06T08:43:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.302+0000 7f6d467fc640 1 -- 172.21.15.27:0/3502268021 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f897072c20 con 0x55f89718f760 2024-06-06T08:43:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d467fc640 10 client.16125 handle_command_reply: tid=0 2024-06-06T08:43:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 client.16125 shutdown 2024-06-06T08:43:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 -- 172.21.15.27:0/3502268021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 msgr2=0x55f897121760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897121760 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55f8971277b0 tx=0x7f6d48005f90 comp rx=0 tx=0).stop 2024-06-06T08:43:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 -- 172.21.15.27:0/3502268021 shutdown_connections 2024-06-06T08:43:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55f89718f760 0x55f897191b50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f897110760 0x55f897109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d467fc640 0 client.16125 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f897101990 0x55f897121760 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 --2- 172.21.15.27:0/3502268021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f897101070 0x55f897121220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 -- 172.21.15.27:0/3502268021 >> 172.21.15.27:0/3502268021 conn(0x55f8970fd680 msgr2=0x55f8970fbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 -- 172.21.15.27:0/3502268021 shutdown_connections 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 1 -- 172.21.15.27:0/3502268021 wait complete. 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.303+0000 7f6d683f5740 20 client.16125 trim_cache size 0 max 16384 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:43:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:43:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:43:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 -- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d4c0a2da0 msgr2=0x7f6d4c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d4c0a2da0 0x7f6d4c0a5260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55f897102c30 tx=0x7f6d48004730 comp rx=0 tx=0).stop 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 -- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d60107ba0 msgr2=0x7f6d6019c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d60107ba0 0x7f6d6019c9f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6d580305d0 tx=0x7f6d58030600 comp rx=0 tx=0).stop 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 -- 172.21.15.27:0/428685664 shutdown_connections 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d4c0a2da0 0x7f6d4c0a5260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d60107ba0 0x7f6d6019c9f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6006ab90 0x7f6d60198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 --2- 172.21.15.27:0/428685664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d6006a1d0 0x7f6d60198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 -- 172.21.15.27:0/428685664 >> 172.21.15.27:0/428685664 conn(0x7f6d600766a0 msgr2=0x7f6d600ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 -- 172.21.15.27:0/428685664 shutdown_connections 2024-06-06T08:43:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:06.304+0000 7f6d656e1640 1 -- 172.21.15.27:0/428685664 wait complete. 2024-06-06T08:43:06.318 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T08:43:06.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T08:43:06.578 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:43:06.700 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:06 smithi027 ceph-mon[32335]: pgmap v88: 129 pgs: 129 active+clean; 85 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 1.28k op/s 2024-06-06T08:43:06.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:06 smithi203 ceph-mon[36579]: pgmap v88: 129 pgs: 129 active+clean; 85 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 1.28k op/s 2024-06-06T08:43:07.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:06 smithi176 ceph-mon[36786]: pgmap v88: 129 pgs: 129 active+clean; 85 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 1.28k op/s 2024-06-06T08:43:07.757 INFO:tasks.workunit.client.0.smithi027.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-06-06T08:43:07.758 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-06-06T08:43:07.758 INFO:tasks.workunit.client.0.smithi027.stderr:+ make 2024-06-06T08:43:07.860 INFO:tasks.workunit.client.0.smithi027.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-06-06T08:43:08.285 INFO:tasks.workunit.client.0.smithi027.stderr:++ readlink -f fsstress 2024-06-06T08:43:08.288 INFO:tasks.workunit.client.0.smithi027.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-06-06T08:43:08.288 INFO:tasks.workunit.client.0.smithi027.stderr:+ popd 2024-06-06T08:43:08.289 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-06-06T08:43:08.289 INFO:tasks.workunit.client.0.smithi027.stderr:+ popd 2024-06-06T08:43:08.289 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp 2024-06-06T08:43:08.290 INFO:tasks.workunit.client.0.smithi027.stderr:++ mktemp -d -p . 2024-06-06T08:43:08.290 INFO:tasks.workunit.client.0.smithi027.stderr:+ T=./tmp.Es9eopsTLb 2024-06-06T08:43:08.291 INFO:tasks.workunit.client.0.smithi027.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.Es9eopsTLb -l 1 -n 1000 -p 10 -v 2024-06-06T08:43:08.292 INFO:tasks.workunit.client.0.smithi027.stdout:seed = 1717958293 2024-06-06T08:43:08.294 INFO:tasks.workunit.client.0.smithi027.stdout:0/0: link - no file 2024-06-06T08:43:08.295 INFO:tasks.workunit.client.0.smithi027.stdout:1/0: getdents . 0 2024-06-06T08:43:08.295 INFO:tasks.workunit.client.0.smithi027.stdout:1/1: dread - no filename 2024-06-06T08:43:08.295 INFO:tasks.workunit.client.0.smithi027.stdout:1/2: write - no filename 2024-06-06T08:43:08.295 INFO:tasks.workunit.client.0.smithi027.stdout:1/3: dwrite - no filename 2024-06-06T08:43:08.295 INFO:tasks.workunit.client.0.smithi027.stdout:0/1: creat f0 x:0 0 0 2024-06-06T08:43:08.295 INFO:tasks.workunit.client.0.smithi027.stdout:0/2: write f0 [972188,68258] 0 2024-06-06T08:43:08.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/0: link - no file 2024-06-06T08:43:08.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/1: rename - no filename 2024-06-06T08:43:08.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/2: truncate - no filename 2024-06-06T08:43:08.296 INFO:tasks.workunit.client.0.smithi027.stdout:1/4: symlink l0 0 2024-06-06T08:43:08.296 INFO:tasks.workunit.client.0.smithi027.stdout:1/5: read - no filename 2024-06-06T08:43:08.296 INFO:tasks.workunit.client.0.smithi027.stdout:1/6: dwrite - no filename 2024-06-06T08:43:08.297 INFO:tasks.workunit.client.0.smithi027.stdout:3/0: write - no filename 2024-06-06T08:43:08.298 INFO:tasks.workunit.client.0.smithi027.stdout:2/3: creat f0 x:0 0 0 2024-06-06T08:43:08.299 INFO:tasks.workunit.client.0.smithi027.stdout:4/0: readlink - no filename 2024-06-06T08:43:08.299 INFO:tasks.workunit.client.0.smithi027.stdout:4/1: write - no filename 2024-06-06T08:43:08.299 INFO:tasks.workunit.client.0.smithi027.stdout:4/2: getdents . 0 2024-06-06T08:43:08.299 INFO:tasks.workunit.client.0.smithi027.stdout:4/3: write - no filename 2024-06-06T08:43:08.299 INFO:tasks.workunit.client.0.smithi027.stdout:4/4: rmdir - no directory 2024-06-06T08:43:08.299 INFO:tasks.workunit.client.0.smithi027.stdout:4/5: write - no filename 2024-06-06T08:43:08.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/6: rename - no filename 2024-06-06T08:43:08.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/7: rename - no filename 2024-06-06T08:43:08.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/8: write - no filename 2024-06-06T08:43:08.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/9: getdents . 0 2024-06-06T08:43:08.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/10: dwrite - no filename 2024-06-06T08:43:08.300 INFO:tasks.workunit.client.0.smithi027.stdout:3/1: symlink l0 0 2024-06-06T08:43:08.301 INFO:tasks.workunit.client.0.smithi027.stdout:2/4: creat f1 x:0 0 0 2024-06-06T08:43:08.301 INFO:tasks.workunit.client.0.smithi027.stdout:2/5: chown f1 16133 1 2024-06-06T08:43:08.302 INFO:tasks.workunit.client.0.smithi027.stdout:5/0: write - no filename 2024-06-06T08:43:08.303 INFO:tasks.workunit.client.0.smithi027.stdout:4/11: mkdir d0 0 2024-06-06T08:43:08.305 INFO:tasks.workunit.client.0.smithi027.stdout:3/2: mkdir d1 0 2024-06-06T08:43:08.305 INFO:tasks.workunit.client.0.smithi027.stdout:3/3: dread - no filename 2024-06-06T08:43:08.306 INFO:tasks.workunit.client.0.smithi027.stdout:2/6: mknod c2 0 2024-06-06T08:43:08.307 INFO:tasks.workunit.client.0.smithi027.stdout:0/3: dread f0 [0,4194304] 0 2024-06-06T08:43:08.308 INFO:tasks.workunit.client.0.smithi027.stdout:5/1: mkdir d0 0 2024-06-06T08:43:08.308 INFO:tasks.workunit.client.0.smithi027.stdout:5/2: dwrite - no filename 2024-06-06T08:43:08.308 INFO:tasks.workunit.client.0.smithi027.stdout:5/3: chown d0 0 1 2024-06-06T08:43:08.309 INFO:tasks.workunit.client.0.smithi027.stdout:4/12: mkdir d0/d1 0 2024-06-06T08:43:08.309 INFO:tasks.workunit.client.0.smithi027.stdout:4/13: fsync - no filename 2024-06-06T08:43:08.309 INFO:tasks.workunit.client.0.smithi027.stdout:4/14: chown d0 5189144 1 2024-06-06T08:43:08.309 INFO:tasks.workunit.client.0.smithi027.stdout:4/15: dread - no filename 2024-06-06T08:43:08.309 INFO:tasks.workunit.client.0.smithi027.stdout:1/7: getdents . 0 2024-06-06T08:43:08.310 INFO:tasks.workunit.client.0.smithi027.stdout:1/8: dwrite - no filename 2024-06-06T08:43:08.310 INFO:tasks.workunit.client.0.smithi027.stdout:3/4: creat d1/f2 x:0 0 0 2024-06-06T08:43:08.310 INFO:tasks.workunit.client.0.smithi027.stdout:3/5: readlink l0 0 2024-06-06T08:43:08.315 INFO:tasks.workunit.client.0.smithi027.stdout:7/0: getdents . 0 2024-06-06T08:43:08.315 INFO:tasks.workunit.client.0.smithi027.stdout:7/1: dwrite - no filename 2024-06-06T08:43:08.315 INFO:tasks.workunit.client.0.smithi027.stdout:7/2: dwrite - no filename 2024-06-06T08:43:08.315 INFO:tasks.workunit.client.0.smithi027.stdout:7/3: dwrite - no filename 2024-06-06T08:43:08.315 INFO:tasks.workunit.client.0.smithi027.stdout:7/4: rename - no filename 2024-06-06T08:43:08.316 INFO:tasks.workunit.client.0.smithi027.stdout:7/5: unlink - no file 2024-06-06T08:43:08.316 INFO:tasks.workunit.client.0.smithi027.stdout:7/6: chown . 770478757 1 2024-06-06T08:43:08.326 INFO:tasks.workunit.client.0.smithi027.stdout:3/6: mknod d1/c3 0 2024-06-06T08:43:08.327 INFO:tasks.workunit.client.0.smithi027.stdout:3/7: dread - d1/f2 zero size 2024-06-06T08:43:08.327 INFO:tasks.workunit.client.0.smithi027.stdout:6/0: creat f0 x:0 0 0 2024-06-06T08:43:08.327 INFO:tasks.workunit.client.0.smithi027.stdout:6/1: read - f0 zero size 2024-06-06T08:43:08.327 INFO:tasks.workunit.client.0.smithi027.stdout:5/4: creat d0/f1 x:0 0 0 2024-06-06T08:43:08.327 INFO:tasks.workunit.client.0.smithi027.stdout:4/16: creat d0/d1/f2 x:0 0 0 2024-06-06T08:43:08.327 INFO:tasks.workunit.client.0.smithi027.stdout:4/17: stat d0 0 2024-06-06T08:43:08.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/9: link l0 l1 0 2024-06-06T08:43:08.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/10: write - no filename 2024-06-06T08:43:08.327 INFO:tasks.workunit.client.0.smithi027.stdout:7/7: mknod c0 0 2024-06-06T08:43:08.329 INFO:tasks.workunit.client.0.smithi027.stdout:6/2: symlink l1 0 2024-06-06T08:43:08.329 INFO:tasks.workunit.client.0.smithi027.stdout:4/18: rmdir d0/d1 39 2024-06-06T08:43:08.334 INFO:tasks.workunit.client.0.smithi027.stdout:6/3: rename l1 to l2 0 2024-06-06T08:43:08.334 INFO:tasks.workunit.client.0.smithi027.stdout:7/8: creat f1 x:0 0 0 2024-06-06T08:43:08.334 INFO:tasks.workunit.client.0.smithi027.stdout:7/9: dread - f1 zero size 2024-06-06T08:43:08.334 INFO:tasks.workunit.client.0.smithi027.stdout:7/10: write f1 [421572,37580] 0 2024-06-06T08:43:08.336 INFO:tasks.workunit.client.0.smithi027.stdout:4/19: creat d0/d1/f3 x:0 0 0 2024-06-06T08:43:08.337 INFO:tasks.workunit.client.0.smithi027.stdout:9/0: symlink l0 0 2024-06-06T08:43:08.339 INFO:tasks.workunit.client.0.smithi027.stdout:7/11: rename f1 to f2 0 2024-06-06T08:43:08.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/11: link l0 l2 0 2024-06-06T08:43:08.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/12: chown l1 35231505 1 2024-06-06T08:43:08.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/13: write - no filename 2024-06-06T08:43:08.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/14: chown l1 10154 1 2024-06-06T08:43:08.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/15: readlink l1 0 2024-06-06T08:43:08.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/16: dread - no filename 2024-06-06T08:43:08.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/17: write - no filename 2024-06-06T08:43:08.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/18: truncate - no filename 2024-06-06T08:43:08.341 INFO:tasks.workunit.client.0.smithi027.stdout:1/19: dread - no filename 2024-06-06T08:43:08.342 INFO:tasks.workunit.client.0.smithi027.stdout:4/20: mknod d0/d1/c4 0 2024-06-06T08:43:08.342 INFO:tasks.workunit.client.0.smithi027.stdout:4/21: write d0/d1/f3 [703496,110207] 0 2024-06-06T08:43:08.342 INFO:tasks.workunit.client.0.smithi027.stdout:4/22: dread - d0/d1/f2 zero size 2024-06-06T08:43:08.346 INFO:tasks.workunit.client.0.smithi027.stdout:9/1: link l0 l1 0 2024-06-06T08:43:08.350 INFO:tasks.workunit.client.0.smithi027.stdout:1/20: creat f3 x:0 0 0 2024-06-06T08:43:08.350 INFO:tasks.workunit.client.0.smithi027.stdout:4/23: creat d0/d1/f5 x:0 0 0 2024-06-06T08:43:08.350 INFO:tasks.workunit.client.0.smithi027.stdout:4/24: chown d0/d1/c4 2814 1 2024-06-06T08:43:08.350 INFO:tasks.workunit.client.0.smithi027.stdout:4/25: truncate d0/d1/f2 775906 0 2024-06-06T08:43:08.351 INFO:tasks.workunit.client.0.smithi027.stdout:9/2: mknod c2 0 2024-06-06T08:43:08.351 INFO:tasks.workunit.client.0.smithi027.stdout:9/3: chown l0 25829 1 2024-06-06T08:43:08.352 INFO:tasks.workunit.client.0.smithi027.stdout:4/26: mkdir d0/d1/d6 0 2024-06-06T08:43:08.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/27: unlink d0/d1/f5 0 2024-06-06T08:43:08.355 INFO:tasks.workunit.client.0.smithi027.stdout:1/21: symlink l4 0 2024-06-06T08:43:08.356 INFO:tasks.workunit.client.0.smithi027.stdout:1/22: readlink l4 0 2024-06-06T08:43:08.424 INFO:tasks.workunit.client.0.smithi027.stdout:2/7: dwrite f0 [0,4194304] 0 2024-06-06T08:43:08.628 INFO:tasks.workunit.client.0.smithi027.stdout:2/8: dwrite f1 [0,4194304] 0 2024-06-06T08:43:08.637 INFO:tasks.workunit.client.0.smithi027.stdout:5/5: dwrite d0/f1 [0,4194304] 0 2024-06-06T08:43:08.637 INFO:tasks.workunit.client.0.smithi027.stdout:5/6: stat d0/f1 0 2024-06-06T08:43:08.642 INFO:tasks.workunit.client.0.smithi027.stdout:2/9: write f1 [3828040,114964] 0 2024-06-06T08:43:08.643 INFO:tasks.workunit.client.0.smithi027.stdout:6/4: getdents . 0 2024-06-06T08:43:08.644 INFO:tasks.workunit.client.0.smithi027.stdout:6/5: truncate f0 251645 0 2024-06-06T08:43:08.648 INFO:tasks.workunit.client.0.smithi027.stdout:4/28: write d0/d1/f3 [363867,91400] 0 2024-06-06T08:43:08.660 INFO:tasks.workunit.client.0.smithi027.stdout:1/23: getdents . 0 2024-06-06T08:43:08.660 INFO:tasks.workunit.client.0.smithi027.stdout:1/24: fdatasync f3 0 2024-06-06T08:43:08.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/12: dread f2 [0,4194304] 0 2024-06-06T08:43:08.669 INFO:tasks.workunit.client.0.smithi027.stdout:4/29: dread d0/d1/f3 [0,4194304] 0 2024-06-06T08:43:08.669 INFO:tasks.workunit.client.0.smithi027.stdout:4/30: write d0/d1/f3 [706094,16618] 0 2024-06-06T08:43:08.670 INFO:tasks.workunit.client.0.smithi027.stdout:4/31: dread d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:08.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/10: write f0 [4881765,33459] 0 2024-06-06T08:43:08.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:08 smithi027 ceph-mon[32335]: pgmap v89: 129 pgs: 129 active+clean; 85 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 1.28k op/s 2024-06-06T08:43:08.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:08 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:08.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:08 smithi203 ceph-mon[36579]: pgmap v89: 129 pgs: 129 active+clean; 85 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 1.28k op/s 2024-06-06T08:43:08.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:08 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:09.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:08 smithi176 ceph-mon[36786]: pgmap v89: 129 pgs: 129 active+clean; 85 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 1.28k op/s 2024-06-06T08:43:09.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:08 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:09.209 INFO:tasks.workunit.client.0.smithi027.stdout:7/13: rename f2 to f3 0 2024-06-06T08:43:09.222 INFO:tasks.workunit.client.0.smithi027.stdout:4/32: link d0/d1/c4 d0/c7 0 2024-06-06T08:43:09.222 INFO:tasks.workunit.client.0.smithi027.stdout:4/33: chown d0/d1/c4 127 1 2024-06-06T08:43:09.225 INFO:tasks.workunit.client.0.smithi027.stdout:7/14: dwrite f3 [0,4194304] 0 2024-06-06T08:43:09.242 INFO:tasks.workunit.client.0.smithi027.stdout:7/15: write f3 [747672,126959] 0 2024-06-06T08:43:09.328 INFO:tasks.workunit.client.0.smithi027.stdout:4/34: mkdir d0/d1/d6/d8 0 2024-06-06T08:43:09.329 INFO:tasks.workunit.client.0.smithi027.stdout:7/16: rename c0 to c4 0 2024-06-06T08:43:09.330 INFO:tasks.workunit.client.0.smithi027.stdout:7/17: mknod c5 0 2024-06-06T08:43:09.330 INFO:tasks.workunit.client.0.smithi027.stdout:7/18: chown c5 4 1 2024-06-06T08:43:09.330 INFO:tasks.workunit.client.0.smithi027.stdout:7/19: write f3 [2889857,46075] 0 2024-06-06T08:43:09.341 INFO:tasks.workunit.client.0.smithi027.stdout:4/35: dwrite d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:09.359 INFO:tasks.workunit.client.0.smithi027.stdout:4/36: dwrite d0/d1/f3 [0,4194304] 0 2024-06-06T08:43:09.361 INFO:tasks.workunit.client.0.smithi027.stdout:4/37: symlink d0/l9 0 2024-06-06T08:43:09.376 INFO:tasks.workunit.client.0.smithi027.stdout:4/38: dread d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:09.387 INFO:tasks.workunit.client.0.smithi027.stdout:4/39: read d0/d1/f3 [294550,68036] 0 2024-06-06T08:43:09.389 INFO:tasks.workunit.client.0.smithi027.stdout:4/40: creat d0/d1/fa x:0 0 0 2024-06-06T08:43:09.390 INFO:tasks.workunit.client.0.smithi027.stdout:4/41: creat d0/d1/fb x:0 0 0 2024-06-06T08:43:09.391 INFO:tasks.workunit.client.0.smithi027.stdout:4/42: mknod d0/d1/d6/d8/cc 0 2024-06-06T08:43:09.406 INFO:tasks.workunit.client.0.smithi027.stdout:4/43: dwrite d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:09.407 INFO:tasks.workunit.client.0.smithi027.stdout:4/44: creat d0/fd x:0 0 0 2024-06-06T08:43:09.408 INFO:tasks.workunit.client.0.smithi027.stdout:4/45: link d0/d1/d6/d8/cc d0/d1/ce 0 2024-06-06T08:43:09.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/7: write d0/f1 [4569922,59363] 0 2024-06-06T08:43:09.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/8: write d0/f1 [5158814,4745] 0 2024-06-06T08:43:09.424 INFO:tasks.workunit.client.0.smithi027.stdout:4/46: dwrite d0/fd [0,4194304] 0 2024-06-06T08:43:09.424 INFO:tasks.workunit.client.0.smithi027.stdout:4/47: truncate d0/d1/fa 611163 0 2024-06-06T08:43:09.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/48: dwrite d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:09.442 INFO:tasks.workunit.client.0.smithi027.stdout:4/49: rename d0/d1/f3 to d0/d1/ff 0 2024-06-06T08:43:09.442 INFO:tasks.workunit.client.0.smithi027.stdout:5/9: dwrite d0/f1 [4194304,4194304] 0 2024-06-06T08:43:09.443 INFO:tasks.workunit.client.0.smithi027.stdout:4/50: symlink d0/d1/d6/l10 0 2024-06-06T08:43:09.458 INFO:tasks.workunit.client.0.smithi027.stdout:5/10: write d0/f1 [7326161,27761] 0 2024-06-06T08:43:09.483 INFO:tasks.workunit.client.0.smithi027.stdout:7/20: getdents . 0 2024-06-06T08:43:09.491 INFO:tasks.workunit.client.0.smithi027.stdout:5/11: truncate d0/f1 1063605 0 2024-06-06T08:43:09.491 INFO:tasks.workunit.client.0.smithi027.stdout:5/12: chown d0/f1 27282229 1 2024-06-06T08:43:09.498 INFO:tasks.workunit.client.0.smithi027.stdout:7/21: dread f3 [0,4194304] 0 2024-06-06T08:43:09.500 INFO:tasks.workunit.client.0.smithi027.stdout:7/22: unlink f3 0 2024-06-06T08:43:09.501 INFO:tasks.workunit.client.0.smithi027.stdout:5/13: write d0/f1 [191343,106243] 0 2024-06-06T08:43:09.501 INFO:tasks.workunit.client.0.smithi027.stdout:5/14: chown d0/f1 1764 1 2024-06-06T08:43:09.502 INFO:tasks.workunit.client.0.smithi027.stdout:7/23: mknod c6 0 2024-06-06T08:43:09.504 INFO:tasks.workunit.client.0.smithi027.stdout:5/15: creat d0/f2 x:0 0 0 2024-06-06T08:43:09.504 INFO:tasks.workunit.client.0.smithi027.stdout:5/16: write d0/f1 [1662154,79393] 0 2024-06-06T08:43:09.505 INFO:tasks.workunit.client.0.smithi027.stdout:7/24: unlink c5 0 2024-06-06T08:43:09.506 INFO:tasks.workunit.client.0.smithi027.stdout:5/17: rename d0/f2 to d0/f3 0 2024-06-06T08:43:09.507 INFO:tasks.workunit.client.0.smithi027.stdout:4/51: dwrite d0/fd [4194304,4194304] 0 2024-06-06T08:43:09.508 INFO:tasks.workunit.client.0.smithi027.stdout:7/25: creat f7 x:0 0 0 2024-06-06T08:43:09.510 INFO:tasks.workunit.client.0.smithi027.stdout:7/26: mknod c8 0 2024-06-06T08:43:09.510 INFO:tasks.workunit.client.0.smithi027.stdout:7/27: write f7 [525870,77004] 0 2024-06-06T08:43:09.511 INFO:tasks.workunit.client.0.smithi027.stdout:7/28: symlink l9 0 2024-06-06T08:43:09.512 INFO:tasks.workunit.client.0.smithi027.stdout:7/29: link f7 fa 0 2024-06-06T08:43:09.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/30: creat fb x:0 0 0 2024-06-06T08:43:09.517 INFO:tasks.workunit.client.0.smithi027.stdout:5/18: link d0/f3 d0/f4 0 2024-06-06T08:43:09.517 INFO:tasks.workunit.client.0.smithi027.stdout:5/19: dread - d0/f3 zero size 2024-06-06T08:43:09.518 INFO:tasks.workunit.client.0.smithi027.stdout:5/20: mknod d0/c5 0 2024-06-06T08:43:09.518 INFO:tasks.workunit.client.0.smithi027.stdout:5/21: stat d0/f4 0 2024-06-06T08:43:09.518 INFO:tasks.workunit.client.0.smithi027.stdout:5/22: read - d0/f4 zero size 2024-06-06T08:43:09.518 INFO:tasks.workunit.client.0.smithi027.stdout:5/23: dread - d0/f3 zero size 2024-06-06T08:43:09.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/24: rename d0/c5 to d0/c6 0 2024-06-06T08:43:09.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/25: rename d0 to d0/d7 22 2024-06-06T08:43:09.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/26: dread - d0/f4 zero size 2024-06-06T08:43:09.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/27: read - d0/f4 zero size 2024-06-06T08:43:09.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/28: write d0/f1 [381823,103865] 0 2024-06-06T08:43:09.521 INFO:tasks.workunit.client.0.smithi027.stdout:4/52: write d0/fd [5967902,91791] 0 2024-06-06T08:43:09.526 INFO:tasks.workunit.client.0.smithi027.stdout:7/31: dread fa [0,4194304] 0 2024-06-06T08:43:09.528 INFO:tasks.workunit.client.0.smithi027.stdout:7/32: mknod cc 0 2024-06-06T08:43:09.530 INFO:tasks.workunit.client.0.smithi027.stdout:7/33: creat fd x:0 0 0 2024-06-06T08:43:09.531 INFO:tasks.workunit.client.0.smithi027.stdout:7/34: mkdir de 0 2024-06-06T08:43:09.531 INFO:tasks.workunit.client.0.smithi027.stdout:7/35: write f7 [1384732,34633] 0 2024-06-06T08:43:09.531 INFO:tasks.workunit.client.0.smithi027.stdout:7/36: getdents de 0 2024-06-06T08:43:09.531 INFO:tasks.workunit.client.0.smithi027.stdout:7/37: fdatasync fd 0 2024-06-06T08:43:09.545 INFO:tasks.workunit.client.0.smithi027.stdout:5/29: dwrite d0/f3 [0,4194304] 0 2024-06-06T08:43:09.549 INFO:tasks.workunit.client.0.smithi027.stdout:5/30: creat d0/f8 x:0 0 0 2024-06-06T08:43:09.550 INFO:tasks.workunit.client.0.smithi027.stdout:5/31: fdatasync d0/f8 0 2024-06-06T08:43:09.550 INFO:tasks.workunit.client.0.smithi027.stdout:7/38: dwrite fb [0,4194304] 0 2024-06-06T08:43:09.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/32: creat d0/f9 x:0 0 0 2024-06-06T08:43:09.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/33: write d0/f8 [822247,9207] 0 2024-06-06T08:43:09.556 INFO:tasks.workunit.client.0.smithi027.stdout:5/34: mkdir d0/da 0 2024-06-06T08:43:09.558 INFO:tasks.workunit.client.0.smithi027.stdout:7/39: rmdir de 0 2024-06-06T08:43:09.558 INFO:tasks.workunit.client.0.smithi027.stdout:7/40: write fd [121708,39078] 0 2024-06-06T08:43:09.570 INFO:tasks.workunit.client.0.smithi027.stdout:7/41: dread fa [0,4194304] 0 2024-06-06T08:43:09.571 INFO:tasks.workunit.client.0.smithi027.stdout:7/42: rmdir - no directory 2024-06-06T08:43:09.571 INFO:tasks.workunit.client.0.smithi027.stdout:7/43: creat ff x:0 0 0 2024-06-06T08:43:09.572 INFO:tasks.workunit.client.0.smithi027.stdout:7/44: write f7 [1298769,109192] 0 2024-06-06T08:43:09.574 INFO:tasks.workunit.client.0.smithi027.stdout:7/45: symlink l10 0 2024-06-06T08:43:09.576 INFO:tasks.workunit.client.0.smithi027.stdout:7/46: unlink f7 0 2024-06-06T08:43:09.576 INFO:tasks.workunit.client.0.smithi027.stdout:7/47: rmdir - no directory 2024-06-06T08:43:09.577 INFO:tasks.workunit.client.0.smithi027.stdout:5/35: dwrite d0/f8 [0,4194304] 0 2024-06-06T08:43:09.577 INFO:tasks.workunit.client.0.smithi027.stdout:5/36: fsync d0/f8 0 2024-06-06T08:43:09.577 INFO:tasks.workunit.client.0.smithi027.stdout:7/48: creat f11 x:0 0 0 2024-06-06T08:43:09.580 INFO:tasks.workunit.client.0.smithi027.stdout:5/37: truncate d0/f4 2420009 0 2024-06-06T08:43:09.581 INFO:tasks.workunit.client.0.smithi027.stdout:7/49: mknod c12 0 2024-06-06T08:43:09.582 INFO:tasks.workunit.client.0.smithi027.stdout:5/38: mkdir d0/da/db 0 2024-06-06T08:43:09.583 INFO:tasks.workunit.client.0.smithi027.stdout:7/50: link c4 c13 0 2024-06-06T08:43:09.584 INFO:tasks.workunit.client.0.smithi027.stdout:5/39: symlink d0/da/db/lc 0 2024-06-06T08:43:09.584 INFO:tasks.workunit.client.0.smithi027.stdout:5/40: dread - d0/f9 zero size 2024-06-06T08:43:09.586 INFO:tasks.workunit.client.0.smithi027.stdout:5/41: link d0/c6 d0/da/cd 0 2024-06-06T08:43:09.587 INFO:tasks.workunit.client.0.smithi027.stdout:5/42: creat d0/fe x:0 0 0 2024-06-06T08:43:09.587 INFO:tasks.workunit.client.0.smithi027.stdout:5/43: dread - d0/fe zero size 2024-06-06T08:43:09.606 INFO:tasks.workunit.client.0.smithi027.stdout:5/44: dwrite d0/f8 [0,4194304] 0 2024-06-06T08:43:09.608 INFO:tasks.workunit.client.0.smithi027.stdout:5/45: rename d0/da/db/lc to d0/da/db/lf 0 2024-06-06T08:43:09.608 INFO:tasks.workunit.client.0.smithi027.stdout:5/46: truncate d0/f9 238255 0 2024-06-06T08:43:09.609 INFO:tasks.workunit.client.0.smithi027.stdout:5/47: write d0/fe [933301,44375] 0 2024-06-06T08:43:09.609 INFO:tasks.workunit.client.0.smithi027.stdout:7/51: dwrite fa [0,4194304] 0 2024-06-06T08:43:09.611 INFO:tasks.workunit.client.0.smithi027.stdout:7/52: dread - ff zero size 2024-06-06T08:43:09.616 INFO:tasks.workunit.client.0.smithi027.stdout:7/53: fdatasync fd 0 2024-06-06T08:43:09.616 INFO:tasks.workunit.client.0.smithi027.stdout:5/48: dread d0/f4 [0,4194304] 0 2024-06-06T08:43:09.616 INFO:tasks.workunit.client.0.smithi027.stdout:5/49: chown d0 209 1 2024-06-06T08:43:09.618 INFO:tasks.workunit.client.0.smithi027.stdout:7/54: write fb [4922680,467] 0 2024-06-06T08:43:09.620 INFO:tasks.workunit.client.0.smithi027.stdout:5/50: dread d0/f1 [0,4194304] 0 2024-06-06T08:43:09.624 INFO:tasks.workunit.client.0.smithi027.stdout:5/51: unlink d0/f8 0 2024-06-06T08:43:09.630 INFO:tasks.workunit.client.0.smithi027.stdout:5/52: link d0/f4 d0/da/db/f10 0 2024-06-06T08:43:09.630 INFO:tasks.workunit.client.0.smithi027.stdout:5/53: truncate d0/da/db/f10 3096952 0 2024-06-06T08:43:09.631 INFO:tasks.workunit.client.0.smithi027.stdout:7/55: rename fa to f14 0 2024-06-06T08:43:09.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/56: creat f15 x:0 0 0 2024-06-06T08:43:09.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/57: write f15 [184697,63083] 0 2024-06-06T08:43:09.635 INFO:tasks.workunit.client.0.smithi027.stdout:5/54: write d0/da/db/f10 [1139526,129841] 0 2024-06-06T08:43:09.636 INFO:tasks.workunit.client.0.smithi027.stdout:5/55: creat d0/da/db/f11 x:0 0 0 2024-06-06T08:43:09.637 INFO:tasks.workunit.client.0.smithi027.stdout:5/56: unlink d0/f9 0 2024-06-06T08:43:09.639 INFO:tasks.workunit.client.0.smithi027.stdout:5/57: read d0/f1 [1328844,29058] 0 2024-06-06T08:43:09.639 INFO:tasks.workunit.client.0.smithi027.stdout:5/58: dread - d0/da/db/f11 zero size 2024-06-06T08:43:09.644 INFO:tasks.workunit.client.0.smithi027.stdout:7/58: symlink l16 0 2024-06-06T08:43:09.660 INFO:tasks.workunit.client.0.smithi027.stdout:5/59: dwrite d0/f4 [0,4194304] 0 2024-06-06T08:43:09.660 INFO:tasks.workunit.client.0.smithi027.stdout:5/60: write d0/fe [1825017,85656] 0 2024-06-06T08:43:09.662 INFO:tasks.workunit.client.0.smithi027.stdout:5/61: creat d0/da/f12 x:0 0 0 2024-06-06T08:43:09.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/59: dwrite f11 [0,4194304] 0 2024-06-06T08:43:09.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.460+0000 7fe40511b700 1 -- 172.21.15.27:0/3468245249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe400102fc0 msgr2=0x7fe400107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:09.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.460+0000 7fe40511b700 1 --2- 172.21.15.27:0/3468245249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe400102fc0 0x7fe400107010 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40099c0 tx=0x7fe3f4009cd0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.461+0000 7fe40511b700 1 -- 172.21.15.27:0/3468245249 shutdown_connections 2024-06-06T08:43:09.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.461+0000 7fe40511b700 1 --2- 172.21.15.27:0/3468245249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4001076f0 0x7fe40010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.461+0000 7fe40511b700 1 --2- 172.21.15.27:0/3468245249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe400102610 0x7fe4001029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.461+0000 7fe40511b700 1 --2- 172.21.15.27:0/3468245249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe400102fc0 0x7fe400107010 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.461+0000 7fe40511b700 1 -- 172.21.15.27:0/3468245249 >> 172.21.15.27:0/3468245249 conn(0x7fe4000fdee0 msgr2=0x7fe400100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:09.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.464+0000 7fe40511b700 1 -- 172.21.15.27:0/3468245249 shutdown_connections 2024-06-06T08:43:09.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.464+0000 7fe40511b700 1 -- 172.21.15.27:0/3468245249 wait complete. 2024-06-06T08:43:09.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.465+0000 7fe40511b700 1 Processor -- start 2024-06-06T08:43:09.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.466+0000 7fe40511b700 1 -- start start 2024-06-06T08:43:09.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.467+0000 7fe40511b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe400102610 0x7fe40019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.467+0000 7fe40511b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe400102fc0 0x7fe40019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.467+0000 7fe40511b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4001076f0 0x7fe400198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.467+0000 7fe40511b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001127d0 con 0x7fe400102610 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.467+0000 7fe40511b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe400112650 con 0x7fe4001076f0 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.467+0000 7fe40511b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe400112950 con 0x7fe400102fc0 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.468+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe400102fc0 0x7fe40019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.469+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe400102fc0 0x7fe40019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:48018/0 (socket says 172.21.15.27:48018) 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.469+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/1572716280 learned_addr learned my addr 172.21.15.27:0/1572716280 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.469+0000 7fe40491a700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4001076f0 0x7fe400198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.470+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/1572716280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4001076f0 msgr2=0x7fe400198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.470+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4001076f0 0x7fe400198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.470+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/1572716280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe400102610 msgr2=0x7fe40019e8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:43:09.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.470+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe400102610 0x7fe40019e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.470+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/1572716280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4001991d0 con 0x7fe400102fc0 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.474+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe400102fc0 0x7fe40019edf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009990 tx=0x7fe3f4004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.476+0000 7fe3fd7fa700 1 -- 172.21.15.27:0/1572716280 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4026070 con 0x7fe400102fc0 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.476+0000 7fe3fd7fa700 1 -- 172.21.15.27:0/1572716280 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f4004050 con 0x7fe400102fc0 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.476+0000 7fe40511b700 1 -- 172.21.15.27:0/1572716280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe400199460 con 0x7fe400102fc0 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.477+0000 7fe40511b700 1 -- 172.21.15.27:0/1572716280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe400199850 con 0x7fe400102fc0 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.478+0000 7fe3fd7fa700 1 -- 172.21.15.27:0/1572716280 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4021410 con 0x7fe400102fc0 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.484+0000 7fe410c00b80 1 Processor -- start 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.487+0000 7fe3fd7fa700 1 -- 172.21.15.27:0/1572716280 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe3f4029070 con 0x7fe400102fc0 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.488+0000 7fe3fd7fa700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3e80a2e70 0x7fe3e80a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.488+0000 7fe410c00b80 1 -- start start 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.488+0000 7fe3fd7fa700 1 -- 172.21.15.27:0/1572716280 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fe3f40d0000 con 0x7fe400102fc0 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.488+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419556cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.488+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 0x564195567d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.488+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564195568280 0x5641955686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.488+0000 7fe410c00b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564195451fe0 con 0x56419556d250 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.488+0000 7fe410c00b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5641954b08d0 con 0x56419556c930 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.488+0000 7fe410c00b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5641954184e0 con 0x564195568280 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.489+0000 7fe40491a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564195568280 0x5641955686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.489+0000 7fe40491a700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3e80a2e70 0x7fe3e80a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.489+0000 7fe40491a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564195568280 0x5641955686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:48028/0 (socket says 172.21.15.27:48028) 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.489+0000 7fe40491a700 1 -- 172.21.15.27:0/477651080 learned_addr learned my addr 172.21.15.27:0/477651080 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.489+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/477651080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 0x564195567d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.490+0000 7fe40491a700 1 -- 172.21.15.27:0/477651080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 msgr2=0x56419556cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.490+0000 7fe40491a700 1 --2- 172.21.15.27:0/477651080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419556cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.490+0000 7fe40491a700 1 -- 172.21.15.27:0/477651080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 msgr2=0x564195567d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.490+0000 7fe40491a700 1 --2- 172.21.15.27:0/477651080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 0x564195567d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.490+0000 7fe40491a700 1 -- 172.21.15.27:0/477651080 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641953ff950 con 0x564195568280 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.490+0000 7fe40491a700 1 --2- 172.21.15.27:0/477651080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564195568280 0x5641955686e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009230 tx=0x7fe3f801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b73f3ae6536dc349 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.491+0000 7fe40491a700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3e80a2e70 0x7fe3e80a5330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009fd0 tx=0x7fe3f80091c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.491+0000 7fe3e67fc700 1 -- 172.21.15.27:0/477651080 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f801a960 con 0x564195568280 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.491+0000 7fe3e67fc700 1 -- 172.21.15.27:0/477651080 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f801e070 con 0x564195568280 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe3e67fc700 1 -- 172.21.15.27:0/477651080 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8019c50 con 0x564195568280 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe410c00b80 1 -- 172.21.15.27:0/477651080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564195568280 msgr2=0x5641955686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe410c00b80 1 --2- 172.21.15.27:0/477651080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564195568280 0x5641955686e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009230 tx=0x7fe3f801b000 comp rx=0 tx=0).stop 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe410c00b80 1 -- 172.21.15.27:0/477651080 shutdown_connections 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe410c00b80 1 --2- 172.21.15.27:0/477651080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564195568280 0x5641955686e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe410c00b80 1 --2- 172.21.15.27:0/477651080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419556cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe410c00b80 1 --2- 172.21.15.27:0/477651080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 0x564195567d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe410c00b80 1 -- 172.21.15.27:0/477651080 >> 172.21.15.27:0/477651080 conn(0x56419548c100 msgr2=0x56419555a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe410c00b80 1 -- 172.21.15.27:0/477651080 shutdown_connections 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.492+0000 7fe410c00b80 1 -- 172.21.15.27:0/477651080 wait complete. 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.493+0000 7fe410c00b80 1 -- start start 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.493+0000 7fe410c00b80 1 Processor -- start 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.493+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564195568280 0x564195573eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.493+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556c930 0x5641955711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.493+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 0x5641955716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.493+0000 7fe410c00b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564195451040 con 0x56419556d250 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.493+0000 7fe410c00b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5641954ac890 con 0x564195568280 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.493+0000 7fe410c00b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564195574cc0 con 0x56419556c930 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.493+0000 7fe410c00b80 10 client.0 _ll_register_callbacks cb 0x564195446630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.494+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556c930 0x5641955711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.494+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556c930 0x5641955711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:48034/0 (socket says 172.21.15.27:48034) 2024-06-06T08:43:09.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.494+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/3877865406 learned_addr learned my addr 172.21.15.27:0/3877865406 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.494+0000 7fe40491a700 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 0x5641955716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.495+0000 7fe3fffff700 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564195568280 0x564195573eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.496+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/3877865406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564195568280 msgr2=0x564195573eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.496+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564195568280 0x564195573eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.496+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/3877865406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 msgr2=0x5641955716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.496+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 0x5641955716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.496+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/3877865406 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641954767d0 con 0x56419556c930 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.496+0000 7fe3fffff700 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564195568280 0x564195573eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.498+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556c930 0x5641955711b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4003900 tx=0x7fe3f4003870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f29420cb6b9d6849 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.498+0000 7fe3df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.498+0000 7fe3df7fe700 1 -- 172.21.15.27:0/3877865406 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40b5060 con 0x56419556c930 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.498+0000 7fe410c00b80 1 -- 172.21.15.27:0/3877865406 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3f801e070 con 0x56419556c930 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.498+0000 7fe3df7fe700 1 -- 172.21.15.27:0/3877865406 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fe3f400f460 con 0x56419556c930 2024-06-06T08:43:09.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.499+0000 7fe3df7fe700 1 -- 172.21.15.27:0/3877865406 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3f40cd7c0 con 0x56419556c930 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.500+0000 7fe410c00b80 10 client.25657 fetch_fsmap learned FSMap version 30 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.500+0000 7fe410c00b80 1 -- 172.21.15.27:0/3877865406 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56419555d780 con 0x56419556c930 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.501+0000 7fe3df7fe700 1 -- 172.21.15.27:0/3877865406 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f801e070 con 0x56419556c930 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.501+0000 7fe3df7fe700 1 -- 172.21.15.27:0/3877865406 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe3f409c3f0 con 0x56419556c930 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.501+0000 7fe410c00b80 10 client.25657 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.501+0000 7fe410c00b80 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.501+0000 7fe410c00b80 20 client.25657 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.502+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56419555fdc0 0x564195562220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.502+0000 7fe410c00b80 4 client.25657 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.502+0000 7fe410c00b80 1 -- 172.21.15.27:0/3877865406 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641954bddd0 con 0x56419555fdc0 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.502+0000 7fe3fffff700 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56419555fdc0 0x564195562220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.507+0000 7fe3fffff700 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56419555fdc0 0x564195562220 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880a54ff9c7da771 server_cookie=ca3e28484497dcc0 in_seq=0 out_seq=0 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.507+0000 7fe3df7fe700 10 client.25657 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:09.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.510+0000 7fe3df7fe700 1 -- 172.21.15.27:0/3877865406 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5641954bddd0 con 0x56419555fdc0 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.510+0000 7fe3df7fe700 10 client.25657 handle_command_reply: tid=0 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.510+0000 7fe410c00b80 1 client.25657 shutdown 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.513+0000 7fe410c00b80 1 -- 172.21.15.27:0/3877865406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556c930 msgr2=0x5641955711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.513+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556c930 0x5641955711b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4003900 tx=0x7fe3f4003870 comp rx=0 tx=0).stop 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.513+0000 7fe410c00b80 1 -- 172.21.15.27:0/3877865406 shutdown_connections 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.513+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56419555fdc0 0x564195562220 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.513+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56419556d250 0x5641955716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.513+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564195568280 0x564195573eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.513+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3877865406 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556c930 0x5641955711b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.513+0000 7fe410c00b80 1 -- 172.21.15.27:0/3877865406 >> 172.21.15.27:0/3877865406 conn(0x56419548c100 msgr2=0x564195559140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.513+0000 7fe3df7fe700 0 client.25657 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:09.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.514+0000 7fe410c00b80 1 -- 172.21.15.27:0/3877865406 shutdown_connections 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.514+0000 7fe410c00b80 1 -- 172.21.15.27:0/3877865406 wait complete. 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.514+0000 7fe410c00b80 20 client.25657 trim_cache size 0 max 16384 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.560+0000 7fe410c00b80 1 Processor -- start 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.560+0000 7fe410c00b80 1 -- start start 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.560+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419555a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.560+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419556bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.560+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 0x56419556c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe410c00b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec011820 con 0x564195568280 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe410c00b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec00f020 con 0x56419556c930 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe410c00b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec00ce70 con 0x56419556d250 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419555a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419556bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419555a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56760/0 (socket says 172.21.15.27:56760) 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe3fffff700 1 -- 172.21.15.27:0/3236648807 learned_addr learned my addr 172.21.15.27:0/3236648807 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe40491a700 1 --2- 172.21.15.27:0/3236648807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 0x56419556c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/3236648807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 msgr2=0x56419556c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/3236648807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 0x56419556c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.561+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/3236648807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 msgr2=0x56419555a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.562+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/3236648807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419555a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.562+0000 7fe3ff7fe700 1 -- 172.21.15.27:0/3236648807 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564195570f80 con 0x56419556c930 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.562+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/3236648807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419556bb50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4098b60 tx=0x7fe3f4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdaf020162a028b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.564+0000 7fe3e5ffb700 1 -- 172.21.15.27:0/3236648807 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40a0540 con 0x56419556c930 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.564+0000 7fe3e5ffb700 1 -- 172.21.15.27:0/3236648807 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f40a4d70 con 0x56419556c930 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.564+0000 7fe3e5ffb700 1 -- 172.21.15.27:0/3236648807 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f409b660 con 0x56419556c930 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.565+0000 7fe410c00b80 1 -- 172.21.15.27:0/3236648807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 msgr2=0x56419556bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:09.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.565+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3236648807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419556bb50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4098b60 tx=0x7fe3f4009310 comp rx=0 tx=0).stop 2024-06-06T08:43:09.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.567+0000 7fe410c00b80 1 -- 172.21.15.27:0/3236648807 shutdown_connections 2024-06-06T08:43:09.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.567+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3236648807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 0x56419556c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.567+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3236648807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419555a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.567+0000 7fe410c00b80 1 --2- 172.21.15.27:0/3236648807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419556bb50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.567+0000 7fe410c00b80 1 -- 172.21.15.27:0/3236648807 >> 172.21.15.27:0/3236648807 conn(0x56419548c100 msgr2=0x56419556b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:09.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.570+0000 7fe410c00b80 1 -- 172.21.15.27:0/3236648807 shutdown_connections 2024-06-06T08:43:09.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.570+0000 7fe410c00b80 1 -- 172.21.15.27:0/3236648807 wait complete. 2024-06-06T08:43:09.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.571+0000 7fe410c00b80 1 -- start start 2024-06-06T08:43:09.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.571+0000 7fe410c00b80 1 Processor -- start 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.572+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419557a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.572+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419557a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.572+0000 7fe410c00b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 0x564195574ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.572+0000 7fe410c00b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec00d4e0 con 0x564195568280 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.572+0000 7fe410c00b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec00e950 con 0x56419556c930 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.572+0000 7fe410c00b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec00dbb0 con 0x56419556d250 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.573+0000 7fe410c00b80 10 client.0 _ll_register_callbacks cb 0x564195446630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.574+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419557a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.574+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419557a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56764/0 (socket says 172.21.15.27:56764) 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.574+0000 7fe3fffff700 1 -- 172.21.15.27:0/1426190450 learned_addr learned my addr 172.21.15.27:0/1426190450 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.575+0000 7fe40491a700 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 0x564195574ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.575+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419557a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.576+0000 7fe3fffff700 1 -- 172.21.15.27:0/1426190450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 msgr2=0x564195574ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.576+0000 7fe3fffff700 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 0x564195574ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.576+0000 7fe3fffff700 1 -- 172.21.15.27:0/1426190450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 msgr2=0x56419557a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.576+0000 7fe3fffff700 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419557a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.576+0000 7fe3fffff700 1 -- 172.21.15.27:0/1426190450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641955755b0 con 0x564195568280 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.578+0000 7fe3fffff700 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419557a040 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000a5f0 tx=0x7fe3f0015700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146d2331c091eb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.578+0000 7fe3df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.578+0000 7fe3df7fe700 1 -- 172.21.15.27:0/1426190450 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0013900 con 0x564195568280 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.579+0000 7fe3df7fe700 1 -- 172.21.15.27:0/1426190450 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fe3f0013aa0 con 0x564195568280 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.579+0000 7fe410c00b80 1 -- 172.21.15.27:0/1426190450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3f40a4d70 con 0x564195568280 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.579+0000 7fe3df7fe700 1 -- 172.21.15.27:0/1426190450 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3f0013ca0 con 0x564195568280 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.580+0000 7fe410c00b80 10 client.16155 fetch_fsmap learned FSMap version 30 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.580+0000 7fe410c00b80 1 -- 172.21.15.27:0/1426190450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641955f5a30 con 0x564195568280 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.581+0000 7fe3df7fe700 1 -- 172.21.15.27:0/1426190450 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f40a4d70 con 0x564195568280 2024-06-06T08:43:09.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.581+0000 7fe3df7fe700 1 -- 172.21.15.27:0/1426190450 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe3f001b4f0 con 0x564195568280 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.583+0000 7fe410c00b80 10 client.16155 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.583+0000 7fe410c00b80 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.583+0000 7fe410c00b80 20 client.16155 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.583+0000 7fe410c00b80 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5641955f61d0 0x5641955f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.583+0000 7fe410c00b80 4 client.16155 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.583+0000 7fe410c00b80 1 -- 172.21.15.27:0/1426190450 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5641954b8350 con 0x5641955f61d0 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.584+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5641955f61d0 0x5641955f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.590+0000 7fe3ff7fe700 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5641955f61d0 0x5641955f85f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff65a82ed89f48d server_cookie=799fb9746413fe2d in_seq=0 out_seq=0 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.590+0000 7fe3df7fe700 10 client.16155 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.593+0000 7fe3df7fe700 1 -- 172.21.15.27:0/1426190450 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641954b8350 con 0x5641955f61d0 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.593+0000 7fe3df7fe700 10 client.16155 handle_command_reply: tid=0 2024-06-06T08:43:09.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.593+0000 7fe410c00b80 1 client.16155 shutdown 2024-06-06T08:43:09.681 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:43:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "return_code": 0, 2024-06-06T08:43:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_tag": "5ccc9a12-fe19-4f51-aecc-bda187b537e4", 2024-06-06T08:43:09.681 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": "asynchronous" 2024-06-06T08:43:09.681 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:43:09.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe410c00b80 1 -- 172.21.15.27:0/1426190450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 msgr2=0x56419557a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe410c00b80 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419557a040 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000a5f0 tx=0x7fe3f0015700 comp rx=0 tx=0).stop 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe410c00b80 1 -- 172.21.15.27:0/1426190450 shutdown_connections 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe410c00b80 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56419556d250 0x564195574ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe410c00b80 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5641955f61d0 0x5641955f85f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe410c00b80 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564195568280 0x56419557a040 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe410c00b80 1 --2- 172.21.15.27:0/1426190450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56419556c930 0x56419557a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe410c00b80 1 -- 172.21.15.27:0/1426190450 >> 172.21.15.27:0/1426190450 conn(0x56419548c100 msgr2=0x56419556bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe3df7fe700 0 client.16155 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.594+0000 7fe410c00b80 1 -- 172.21.15.27:0/1426190450 shutdown_connections 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.595+0000 7fe410c00b80 1 -- 172.21.15.27:0/1426190450 wait complete. 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.595+0000 7fe410c00b80 20 client.16155 trim_cache size 0 max 16384 2024-06-06T08:43:09.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.596+0000 7fe40511b700 1 -- 172.21.15.27:0/1572716280 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3e80a2e70 msgr2=0x7fe3e80a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:09.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.596+0000 7fe40511b700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3e80a2e70 0x7fe3e80a5330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009fd0 tx=0x7fe3f80091c0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.683 INFO:tasks.workunit.client.0.smithi027.stdout:5/62: creat d0/da/f13 x:0 0 0 2024-06-06T08:43:09.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.596+0000 7fe40511b700 1 -- 172.21.15.27:0/1572716280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe400102fc0 msgr2=0x7fe40019edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:09.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.596+0000 7fe40511b700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe400102fc0 0x7fe40019edf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009990 tx=0x7fe3f4004790 comp rx=0 tx=0).stop 2024-06-06T08:43:09.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.597+0000 7fe40511b700 1 -- 172.21.15.27:0/1572716280 shutdown_connections 2024-06-06T08:43:09.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.597+0000 7fe40511b700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3e80a2e70 0x7fe3e80a5330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.597+0000 7fe40511b700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4001076f0 0x7fe400198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.597+0000 7fe40511b700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe400102610 0x7fe40019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.597+0000 7fe40511b700 1 --2- 172.21.15.27:0/1572716280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe400102fc0 0x7fe40019edf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:09.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.597+0000 7fe40511b700 1 -- 172.21.15.27:0/1572716280 >> 172.21.15.27:0/1572716280 conn(0x7fe4000fdee0 msgr2=0x7fe400100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:09.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.597+0000 7fe40511b700 1 -- 172.21.15.27:0/1572716280 shutdown_connections 2024-06-06T08:43:09.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:08.597+0000 7fe40511b700 1 -- 172.21.15.27:0/1572716280 wait complete. 2024-06-06T08:43:09.684 INFO:tasks.workunit.client.0.smithi027.stdout:7/60: mkdir d17 0 2024-06-06T08:43:09.688 INFO:tasks.workunit.client.0.smithi027.stdout:5/63: dwrite d0/da/f13 [0,4194304] 0 2024-06-06T08:43:09.688 INFO:tasks.workunit.client.0.smithi027.stdout:5/64: stat d0/da/db/f10 0 2024-06-06T08:43:09.690 INFO:tasks.workunit.client.0.smithi027.stdout:7/61: dwrite f14 [0,4194304] 0 2024-06-06T08:43:09.694 INFO:tasks.workunit.client.0.smithi027.stdout:7/62: mknod d17/c18 0 2024-06-06T08:43:09.700 INFO:tasks.workunit.client.0.smithi027.stdout:7/63: truncate f11 1626420 0 2024-06-06T08:43:09.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/65: dread d0/f4 [0,4194304] 0 2024-06-06T08:43:09.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/66: chown d0/da/f12 410069747 1 2024-06-06T08:43:09.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/64: mknod d17/c19 0 2024-06-06T08:43:09.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/67: stat d0/da/db/lf 0 2024-06-06T08:43:09.704 INFO:tasks.workunit.client.0.smithi027.stdout:7/65: symlink d17/l1a 0 2024-06-06T08:43:09.711 INFO:tasks.workunit.client.0.smithi027.stdout:7/66: getdents d17 0 2024-06-06T08:43:09.712 INFO:tasks.workunit.client.0.smithi027.stdout:5/68: truncate d0/da/db/f10 2533549 0 2024-06-06T08:43:09.712 INFO:tasks.workunit.client.0.smithi027.stdout:5/69: chown d0/fe 794658 1 2024-06-06T08:43:09.715 INFO:tasks.workunit.client.0.smithi027.stdout:5/70: symlink d0/l14 0 2024-06-06T08:43:09.715 INFO:tasks.workunit.client.0.smithi027.stdout:5/71: readlink d0/l14 0 2024-06-06T08:43:09.716 INFO:tasks.workunit.client.0.smithi027.stdout:5/72: creat d0/da/db/f15 x:0 0 0 2024-06-06T08:43:09.722 INFO:tasks.workunit.client.0.smithi027.stdout:7/67: dread f14 [0,4194304] 0 2024-06-06T08:43:09.723 INFO:tasks.workunit.client.0.smithi027.stdout:7/68: rename fb to d17/f1b 0 2024-06-06T08:43:09.723 INFO:tasks.workunit.client.0.smithi027.stdout:7/69: write f14 [4547833,92222] 0 2024-06-06T08:43:09.727 INFO:tasks.workunit.client.0.smithi027.stdout:5/73: dread d0/da/f13 [0,4194304] 0 2024-06-06T08:43:09.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/70: dread fd [0,4194304] 0 2024-06-06T08:43:09.729 INFO:tasks.workunit.client.0.smithi027.stdout:5/74: creat d0/da/db/f16 x:0 0 0 2024-06-06T08:43:09.729 INFO:tasks.workunit.client.0.smithi027.stdout:7/71: readlink l9 0 2024-06-06T08:43:09.730 INFO:tasks.workunit.client.0.smithi027.stdout:5/75: creat d0/da/db/f17 x:0 0 0 2024-06-06T08:43:09.731 INFO:tasks.workunit.client.0.smithi027.stdout:5/76: creat d0/da/f18 x:0 0 0 2024-06-06T08:43:09.731 INFO:tasks.workunit.client.0.smithi027.stdout:5/77: chown d0/fe 256119 1 2024-06-06T08:43:09.731 INFO:tasks.workunit.client.0.smithi027.stdout:5/78: write d0/f1 [2132798,78702] 0 2024-06-06T08:43:09.733 INFO:tasks.workunit.client.0.smithi027.stdout:5/79: rename d0/da/db/f16 to d0/da/db/f19 0 2024-06-06T08:43:09.734 INFO:tasks.workunit.client.0.smithi027.stdout:5/80: mkdir d0/da/db/d1a 0 2024-06-06T08:43:09.737 INFO:tasks.workunit.client.0.smithi027.stdout:7/72: truncate d17/f1b 1464660 0 2024-06-06T08:43:09.738 INFO:tasks.workunit.client.0.smithi027.stdout:7/73: symlink d17/l1c 0 2024-06-06T08:43:09.741 INFO:tasks.workunit.client.0.smithi027.stdout:5/81: write d0/da/f13 [1911190,111046] 0 2024-06-06T08:43:09.743 INFO:tasks.workunit.client.0.smithi027.stdout:5/82: unlink d0/da/db/f15 0 2024-06-06T08:43:09.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/74: getdents d17 0 2024-06-06T08:43:09.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/75: write f11 [2569624,33468] 0 2024-06-06T08:43:09.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/76: truncate f15 450876 0 2024-06-06T08:43:09.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/77: write f11 [3269900,74131] 0 2024-06-06T08:43:09.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/78: fsync ff 0 2024-06-06T08:43:09.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/79: creat d17/f1d x:0 0 0 2024-06-06T08:43:09.749 INFO:tasks.workunit.client.0.smithi027.stdout:7/80: truncate f14 2396073 0 2024-06-06T08:43:09.749 INFO:tasks.workunit.client.0.smithi027.stdout:7/81: write f15 [968468,10523] 0 2024-06-06T08:43:09.750 INFO:tasks.workunit.client.0.smithi027.stdout:7/82: creat d17/f1e x:0 0 0 2024-06-06T08:43:09.750 INFO:tasks.workunit.client.0.smithi027.stdout:7/83: chown d17/c19 8541742 1 2024-06-06T08:43:09.753 INFO:tasks.workunit.client.0.smithi027.stdout:7/84: getdents d17 0 2024-06-06T08:43:09.753 INFO:tasks.workunit.client.0.smithi027.stdout:7/85: write ff [154514,42385] 0 2024-06-06T08:43:09.754 INFO:tasks.workunit.client.0.smithi027.stdout:7/86: fdatasync f14 0 2024-06-06T08:43:09.755 INFO:tasks.workunit.client.0.smithi027.stdout:7/87: creat d17/f1f x:0 0 0 2024-06-06T08:43:09.756 INFO:tasks.workunit.client.0.smithi027.stdout:7/88: rmdir d17 39 2024-06-06T08:43:09.814 INFO:tasks.workunit.client.0.smithi027.stdout:7/89: dread f15 [0,4194304] 0 2024-06-06T08:43:09.814 INFO:tasks.workunit.client.0.smithi027.stdout:7/90: write d17/f1d [504777,50095] 0 2024-06-06T08:43:09.816 INFO:tasks.workunit.client.0.smithi027.stdout:7/91: truncate f14 460571 0 2024-06-06T08:43:09.816 INFO:tasks.workunit.client.0.smithi027.stdout:5/83: dwrite d0/da/db/f10 [0,4194304] 0 2024-06-06T08:43:09.816 INFO:tasks.workunit.client.0.smithi027.stdout:7/92: chown c6 175823308 1 2024-06-06T08:43:09.818 INFO:tasks.workunit.client.0.smithi027.stdout:5/84: creat d0/da/db/f1b x:0 0 0 2024-06-06T08:43:09.819 INFO:tasks.workunit.client.0.smithi027.stdout:7/93: dread - d17/f1e zero size 2024-06-06T08:43:09.822 INFO:tasks.workunit.client.0.smithi027.stdout:7/94: link c13 d17/c20 0 2024-06-06T08:43:09.829 INFO:tasks.workunit.client.0.smithi027.stdout:7/95: fdatasync f11 0 2024-06-06T08:43:09.829 INFO:tasks.workunit.client.0.smithi027.stdout:7/96: write ff [277076,44879] 0 2024-06-06T08:43:09.832 INFO:tasks.workunit.client.0.smithi027.stdout:7/97: unlink d17/l1a 0 2024-06-06T08:43:09.832 INFO:tasks.workunit.client.0.smithi027.stdout:7/98: write d17/f1f [114296,76909] 0 2024-06-06T08:43:09.833 INFO:tasks.workunit.client.0.smithi027.stdout:5/85: dread d0/fe [0,4194304] 0 2024-06-06T08:43:09.837 INFO:tasks.workunit.client.0.smithi027.stdout:5/86: link d0/l14 d0/da/l1c 0 2024-06-06T08:43:09.842 INFO:tasks.workunit.client.0.smithi027.stdout:5/87: symlink d0/l1d 0 2024-06-06T08:43:09.842 INFO:tasks.workunit.client.0.smithi027.stdout:5/88: truncate d0/f3 4930824 0 2024-06-06T08:43:09.842 INFO:tasks.workunit.client.0.smithi027.stdout:5/89: write d0/da/db/f19 [638867,53071] 0 2024-06-06T08:43:09.842 INFO:tasks.workunit.client.0.smithi027.stdout:5/90: chown d0/f1 420421 1 2024-06-06T08:43:09.863 INFO:tasks.workunit.client.0.smithi027.stdout:5/91: dwrite d0/da/db/f19 [0,4194304] 0 2024-06-06T08:43:09.865 INFO:tasks.workunit.client.0.smithi027.stdout:5/92: creat d0/da/f1e x:0 0 0 2024-06-06T08:43:09.865 INFO:tasks.workunit.client.0.smithi027.stdout:5/93: chown d0/da/db/lf 173684 1 2024-06-06T08:43:09.866 INFO:tasks.workunit.client.0.smithi027.stdout:5/94: creat d0/da/db/d1a/f1f x:0 0 0 2024-06-06T08:43:09.869 INFO:tasks.workunit.client.0.smithi027.stdout:5/95: dread d0/fe [0,4194304] 0 2024-06-06T08:43:09.888 INFO:tasks.workunit.client.0.smithi027.stdout:5/96: dwrite d0/da/f1e [0,4194304] 0 2024-06-06T08:43:09.888 INFO:tasks.workunit.client.0.smithi027.stdout:5/97: chown d0/da 23933 1 2024-06-06T08:43:09.890 INFO:tasks.workunit.client.0.smithi027.stdout:5/98: stat d0/c6 0 2024-06-06T08:43:09.892 INFO:tasks.workunit.client.0.smithi027.stdout:5/99: unlink d0/da/db/d1a/f1f 0 2024-06-06T08:43:09.902 INFO:tasks.workunit.client.0.smithi027.stdout:5/100: write d0/f4 [523972,42768] 0 2024-06-06T08:43:09.904 INFO:tasks.workunit.client.0.smithi027.stdout:5/101: mkdir d0/da/d20 0 2024-06-06T08:43:09.922 INFO:tasks.workunit.client.0.smithi027.stdout:5/102: dwrite d0/da/f1e [0,4194304] 0 2024-06-06T08:43:09.924 INFO:tasks.workunit.client.0.smithi027.stdout:5/103: rmdir d0 39 2024-06-06T08:43:09.942 INFO:tasks.workunit.client.0.smithi027.stdout:5/104: dwrite d0/da/f12 [0,4194304] 0 2024-06-06T08:43:09.946 INFO:tasks.workunit.client.0.smithi027.stdout:5/105: truncate d0/da/f13 2707281 0 2024-06-06T08:43:09.946 INFO:tasks.workunit.client.0.smithi027.stdout:5/106: write d0/da/db/f17 [156000,742] 0 2024-06-06T08:43:09.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:09 smithi027 ceph-mon[32335]: scrub queued for path: / 2024-06-06T08:43:09.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:09 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:09.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:09 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:09.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:09 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:09.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:09 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:09.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:09 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:09.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:09 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:09.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:09 smithi203 ceph-mon[36579]: scrub queued for path: / 2024-06-06T08:43:09.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:09 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:09.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:09 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:09.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:09 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:09.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:09 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:09.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:09 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:09.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:09 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:09 smithi176 ceph-mon[36786]: scrub queued for path: / 2024-06-06T08:43:10.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:09 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:09 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:09 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:09 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:09 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:09 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.486 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-06-06T08:43:10.486 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:43:10.492 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.028+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002819 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir01.c auth v38 ap=1 s=8175 n(v0 rc2024-06-06T08:42:56.555804+0000 b8175 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279275b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002819:[<0x10000002815/mkdir01.c v8>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002819:[<0x10000002815/mkdir01.c v38>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.492 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.028+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000281b [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir05.c auth v36 ap=1 s=5174 n(v0 rc2024-06-06T08:42:56.557804+0000 b5174 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279275c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000281b:[<0x10000002815/mkdir05.c v12>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000281b:[<0x10000002815/mkdir05.c v36>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.492 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.028+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000281a [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir04.c auth v24 ap=1 s=6645 n(v0 rc2024-06-06T08:42:56.556804+0000 b6645 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279275bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000281a:[<0x10000002815/mkdir04.c v24>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000281a:[<0x10000002815/mkdir04.c v24>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.492 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.029+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000281d [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir03.c auth v18 ap=1 s=7269 n(v0 rc2024-06-06T08:42:56.559804+0000 b7269 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279275c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000281d:[<0x10000002815/mkdir03.c v18>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000281d:[<0x10000002815/mkdir03.c v18>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.493 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.029+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002818 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/Makefile auth v22 ap=1 s=915 n(v0 rc2024-06-06T08:42:56.553804+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279275cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002818:[<0x10000002815/Makefile v22>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002818:[<0x10000002815/Makefile v22>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.493 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.029+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002817 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir08.c auth v40 ap=1 s=6533 n(v0 rc2024-06-06T08:42:56.552804+0000 b6533 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279275d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002817:[<0x10000002815/mkdir08.c v4>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002817:[<0x10000002815/mkdir08.c v40>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.493 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.029+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002816 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir09.c auth v20 ap=1 s=13380 n(v0 rc2024-06-06T08:42:56.551804+0000 b13380 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279275d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002816:[<0x10000002815/mkdir09.c v20>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002816:[<0x10000002815/mkdir09.c v20>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.493 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.029+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000281c [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir02.c auth v34 ap=1 s=8971 n(v0 rc2024-06-06T08:42:56.558804+0000 b8971 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792760000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000281c:[<0x10000002815/mkdir02.c v14>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000281c:[<0x10000002815/mkdir02.c v34>,<0x1000000261a/mkdir v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.493 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.036+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000029fe [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/Makefile auth v16 ap=1 s=915 n(v0 rc2009-10-09T17:56:02.000000+0000 b915 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793061700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000029fe:[<0x100000029fd/Makefile v2>,<0x1000000261a/timer_getoverrun v1512>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000029fe:[<0x100000029fd/Makefile v16>,<0x1000000261a/timer_getoverrun v1512>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.493 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.036+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000029ff [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/timer_getoverrun01.c auth v14 ap=1 s=10484 n(v0 rc2024-06-06T08:42:57.249786+0000 b10484 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279277fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000029ff:[<0x100000029fd/timer_getoverrun01.c v4>,<0x1000000261a/timer_getoverrun v1512>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000029ff:[<0x100000029fd/timer_getoverrun01.c v14>,<0x1000000261a/timer_getoverrun v1512>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.493 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.040+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000266c [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_02.c auth v14 ap=1 s=9597 n(v0 rc2009-03-23T13:36:06.000000+0000 b9597 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5627917c4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000266c:[<0x10000002669/signalfd4_02.c v6>,<0x1000000261a/signalfd4 v1594>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000266c:[<0x10000002669/signalfd4_02.c v14>,<0x1000000261a/signalfd4 v1594>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.493 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.041+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000266b [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_01.c auth v16 ap=1 s=9981 n(v0 rc2024-06-06T08:42:55.920820+0000 b9981 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792763700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000266b:[<0x10000002669/signalfd4_01.c v4>,<0x1000000261a/signalfd4 v1594>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000266b:[<0x10000002669/signalfd4_01.c v16>,<0x1000000261a/signalfd4 v1594>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.493 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.041+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000266a [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/Makefile auth v8 ap=1 s=915 n(v0 rc2024-06-06T08:42:55.918820+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x5627917c4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000266a:[<0x10000002669/Makefile v8>,<0x1000000261a/signalfd4 v1594>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000266a:[<0x10000002669/Makefile v8>,<0x1000000261a/signalfd4 v1594>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.494 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.044+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000274c [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/Makefile auth v8 ap=1 s=915 n(v0 rc2024-06-06T08:42:56.225812+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562792761080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000274c:[<0x1000000274a/Makefile v8>,<0x1000000261a/vfork v1598>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000274c:[<0x1000000274a/Makefile v8>,<0x1000000261a/vfork v1598>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.494 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.044+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000274b [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork02.c auth v16 ap=1 s=7230 n(v0 rc2024-06-06T08:42:56.224812+0000 b7230 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792761600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000274b:[<0x1000000274a/vfork02.c v2>,<0x1000000261a/vfork v1598>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000274b:[<0x1000000274a/vfork02.c v16>,<0x1000000261a/vfork v1598>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.494 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.045+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000274d [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork01.c auth v14 ap=1 s=10419 n(v0 rc2024-06-06T08:42:56.227812+0000 b10419 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792761b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000274d:[<0x1000000274a/vfork01.c v6>,<0x1000000261a/vfork v1598>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000274d:[<0x1000000274a/vfork01.c v14>,<0x1000000261a/vfork v1598>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.494 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.067+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002797 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/getsockname01.c auth v10 ap=1 s=5528 n(v0 rc2024-06-06T08:42:56.320810+0000 b5528 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792760580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002797:[<0x10000002795/getsockname01.c v4>,<0x1000000261a/getsockname v762>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002797:[<0x10000002795/getsockname01.c v10>,<0x1000000261a/getsockname v762>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.494 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.067+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002796 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/Makefile auth v12 ap=1 s=915 n(v0 rc2024-06-06T08:42:56.319810+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792760b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002796:[<0x10000002795/Makefile v2>,<0x1000000261a/getsockname v762>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002796:[<0x10000002795/Makefile v12>,<0x1000000261a/getsockname v762>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.494 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.070+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026c4 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd03.c auth v10 ap=1 s=9620 n(v0 rc2024-06-06T08:42:56.038817+0000 b9620 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562792762100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026c4:[<0x100000026c1/timerfd03.c v10>,<0x1000000261a/timerfd v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026c4:[<0x100000026c1/timerfd03.c v10>,<0x1000000261a/timerfd v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.494 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.071+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026c3 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/Makefile auth v20 ap=1 s=943 n(v0 rc2024-06-06T08:42:56.037817+0000 b943 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792762680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026c3:[<0x100000026c1/Makefile v4>,<0x1000000261a/timerfd v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026c3:[<0x100000026c1/Makefile v20>,<0x1000000261a/timerfd v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.494 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.071+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026c5 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd01.c auth v18 ap=1 s=7227 n(v0 rc2024-06-06T08:42:56.040817+0000 b7227 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792762c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026c5:[<0x100000026c1/timerfd01.c v8>,<0x1000000261a/timerfd v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026c5:[<0x100000026c1/timerfd01.c v18>,<0x1000000261a/timerfd v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.494 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.071+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026c2 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd02.c auth v22 ap=1 s=9871 n(v0 rc2024-06-06T08:42:56.035817+0000 b9871 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792763180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026c2:[<0x100000026c1/timerfd02.c v2>,<0x1000000261a/timerfd v1596>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026c2:[<0x100000026c1/timerfd02.c v22>,<0x1000000261a/timerfd v1596>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.080+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002906 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit01.c auth v14 ap=1 s=3494 n(v0 rc2024-06-06T08:42:57.014792+0000 b3494 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792780100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002906:[<0x10000002903/exit01.c v6>,<0x1000000261a/exit v1466>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002906:[<0x10000002903/exit01.c v14>,<0x1000000261a/exit v1466>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.080+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002905 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/Makefile auth v16 ap=1 s=915 n(v0 rc2024-06-06T08:42:57.013792+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792780680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002905:[<0x10000002903/Makefile v4>,<0x1000000261a/exit v1466>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002905:[<0x10000002903/Makefile v16>,<0x1000000261a/exit v1466>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.080+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002904 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit02.c auth v18 ap=1 s=4843 n(v0 rc2024-06-06T08:42:57.012792+0000 b4843 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792780c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002904:[<0x10000002903/exit02.c v2>,<0x1000000261a/exit v1466>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002904:[<0x10000002903/exit02.c v18>,<0x1000000261a/exit v1466>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.094+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000029c6 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/Makefile auth v20 ap=1 s=915 n(v0 rc2024-06-06T08:42:57.195788+0000 b915 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793060680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000029c6:[<0x100000029c4/Makefile v4>,<0x1000000261a/getsid v1508>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000029c6:[<0x100000029c4/Makefile v20>,<0x1000000261a/getsid v1508>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.094+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000029c7 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid02.c auth v22 ap=1 s=4560 n(v0 rc2024-06-06T08:42:57.196788+0000 b4560 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793060c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000029c7:[<0x100000029c4/getsid02.c v6>,<0x1000000261a/getsid v1508>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000029c7:[<0x100000029c4/getsid02.c v22>,<0x1000000261a/getsid v1508>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.094+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000029c5 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid01.c auth v24 ap=1 s=4293 n(v0 rc2024-06-06T08:42:57.194788+0000 b4293 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793061180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000029c5:[<0x100000029c4/getsid01.c v2>,<0x1000000261a/getsid v1508>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000029c5:[<0x100000029c4/getsid01.c v24>,<0x1000000261a/getsid v1508>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.098+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000284c [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink05.c auth v28 ap=1 s=5868 n(v0 rc2024-06-06T08:42:56.609803+0000 b5868 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792781180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000284c:[<0x10000002849/symlink05.c v6>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000284c:[<0x10000002849/symlink05.c v28>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.098+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000284f [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink04.c auth v16 ap=1 s=6404 n(v0 rc2024-06-06T08:42:56.611802+0000 b6404 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279275a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000284f:[<0x10000002849/symlink04.c v16>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000284f:[<0x10000002849/symlink04.c v16>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.098+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000284e [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink03.c auth v14 ap=1 s=12133 n(v0 rc2024-06-06T08:42:56.610802+0000 b12133 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279275ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000284e:[<0x10000002849/symlink03.c v14>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000284e:[<0x10000002849/symlink03.c v14>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.495 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.098+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000284a [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink02.c auth v32 ap=1 s=6898 n(v0 rc2024-06-06T08:42:56.607803+0000 b6898 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562792781700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000284a:[<0x10000002849/symlink02.c v2>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000284a:[<0x10000002849/symlink02.c v32>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.496 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.098+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000284b [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink01.c auth v30 ap=1 s=64086 n(v0 rc2024-06-06T08:42:56.608803+0000 b64086 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279275b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000284b:[<0x10000002849/symlink01.c v4>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000284b:[<0x10000002849/symlink01.c v30>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.496 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.108+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000027b2 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/exit_group01.c auth v6 ap=1 s=8412 n(v0 rc2024-06-06T08:42:56.408808+0000 b8412 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279277eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000027b2:[<0x100000027b1/exit_group01.c v6>,<0x1000000261a/exit_group v1668>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000027b2:[<0x100000027b1/exit_group01.c v6>,<0x1000000261a/exit_group v1668>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.496 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.153+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000027b3 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/Makefile auth v8 ap=1 s=915 n(v0 rc2024-06-06T08:42:56.409808+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279277e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000027b3:[<0x100000027b1/Makefile v8>,<0x1000000261a/exit_group v1668>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000027b3:[<0x100000027b1/Makefile v8>,<0x1000000261a/exit_group v1946>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.496 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.153+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000284d [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/Makefile auth v26 ap=1 s=915 n(v0 rc2009-10-09T17:56:02.000000+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279275a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000284d:[<0x10000002849/Makefile v8>,<0x1000000261a/symlink v1568>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000284d:[<0x10000002849/Makefile v26>,<0x1000000261a/symlink v1794>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.507 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.010+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002389 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/event_generator.c auth{0=1} v28 ap=1 s=5538 n(v0 rc2024-06-06T08:42:54.900846+0000 b5538 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561caa5a1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002389:[<0x10000002388/event_generator.c v2>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v823>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002389:[<0x10000002388/event_generator.c v28>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v857>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.507 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.010+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000238c [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/run_pec_test auth{0=1} v22 ap=1 s=3319 n(v0 rc2024-06-06T08:42:54.903846+0000 b3319 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561caa5a0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000238c:[<0x10000002388/run_pec_test v8>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v823>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000238c:[<0x10000002388/run_pec_test v22>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v857>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.507 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.010+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000238d [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/pec_listener.c auth v12 ap=1 s=8232 n(v0 rc2024-06-06T08:42:54.904846+0000 b8232 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa5a1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000238d:[<0x10000002388/pec_listener.c v12>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v823>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000238d:[<0x10000002388/pec_listener.c v12>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v857>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.507 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.010+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000238b [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/README auth{0=1} v24 ap=1 s=1098 n(v0 rc2024-06-06T08:42:54.902846+0000 b1098 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561caa458580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000238b:[<0x10000002388/README v6>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v823>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000238b:[<0x10000002388/README v24>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v857>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.507 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.011+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000238a [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/Makefile auth{0=1} v26 ap=1 s=1033 n(v0 rc2024-06-06T08:42:54.901846+0000 b1033 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561caa458000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000238a:[<0x10000002388/Makefile v4>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v823>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000238a:[<0x10000002388/Makefile v26>,<0x10000002386/pec v22>,<0x10000001f2c/connectors v857>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.507 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.042+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026e5 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/asyncio02.c auth v6 ap=1 s=9519 n(v0 rc2009-08-28T11:17:39.000000+0000 b9519 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561ca9f71600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026e5:[<0x100000026e3/asyncio02.c v6>,<0x1000000261a/asyncio v1572>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026e5:[<0x100000026e3/asyncio02.c v6>,<0x1000000261a/asyncio v1608>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.507 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.042+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026e4 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/Makefile auth v10 ap=1 s=915 n(v0 rc2024-06-06T08:42:56.097815+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9f71b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026e4:[<0x100000026e3/Makefile v2>,<0x1000000261a/asyncio v1572>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026e4:[<0x100000026e3/Makefile v10>,<0x1000000261a/asyncio v1608>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.508 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.058+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000297c [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind02.c auth v16 ap=1 s=4080 n(v0 rc2024-06-06T08:42:57.122789+0000 b4080 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000297c:[<0x1000000297b/bind02.c v2>,<0x1000000261a/bind v1618>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000297c:[<0x1000000297b/bind02.c v16>,<0x1000000261a/bind v1618>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.508 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.059+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000297e [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind01.c auth v8 ap=1 s=5648 n(v0 rc2024-06-06T08:42:57.124789+0000 b5648 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa11f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000297e:[<0x1000000297b/bind01.c v8>,<0x1000000261a/bind v1618>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000297e:[<0x1000000297b/bind01.c v8>,<0x1000000261a/bind v1618>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.508 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.059+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000297d [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/Makefile auth v14 ap=1 s=915 n(v0 rc2024-06-06T08:42:57.123789+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000297d:[<0x1000000297b/Makefile v4>,<0x1000000261a/bind v1618>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000297d:[<0x1000000297b/Makefile v14>,<0x1000000261a/bind v1618>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.508 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.068+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002881 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/execv01.c auth v6 ap=1 s=7417 n(v0 rc2024-06-06T08:42:56.655801+0000 b7417 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa0af600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002881:[<0x1000000287f/execv01.c v6>,<0x1000000261a/execv v1634>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002881:[<0x1000000287f/execv01.c v6>,<0x1000000261a/execv v1634>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.509 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.069+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002880 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/Makefile auth v10 ap=1 s=915 n(v0 rc2024-06-06T08:42:56.654801+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0afb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002880:[<0x1000000287f/Makefile v2>,<0x1000000261a/execv v1634>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002880:[<0x1000000287f/Makefile v10>,<0x1000000261a/execv v1634>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.509 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.076+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026ec [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/Makefile auth v20 ap=1 s=915 n(v0 rc2024-06-06T08:42:56.107815+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9f70000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026ec:[<0x100000026ea/Makefile v4>,<0x1000000261a/munmap v1586>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026ec:[<0x100000026ea/Makefile v20>,<0x1000000261a/munmap v1612>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.509 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.076+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026ed [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap02.c auth v12 ap=1 s=9095 n(v0 rc2024-06-06T08:42:56.108815+0000 b9095 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561ca9f70580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026ed:[<0x100000026ea/munmap02.c v12>,<0x1000000261a/munmap v1586>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026ed:[<0x100000026ea/munmap02.c v12>,<0x1000000261a/munmap v1612>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.509 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.076+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026ee [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap01.c auth v18 ap=1 s=8309 n(v0 rc2009-03-23T13:35:58.000000+0000 b8309 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9f70b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026ee:[<0x100000026ea/munmap01.c v8>,<0x1000000261a/munmap v1586>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026ee:[<0x100000026ea/munmap01.c v18>,<0x1000000261a/munmap v1612>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.509 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.077+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000026eb [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap03.c auth v10 ap=1 s=7654 n(v0 rc2024-06-06T08:42:56.105815+0000 b7654 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561ca9f71080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000026eb:[<0x100000026ea/munmap03.c v10>,<0x1000000261a/munmap v1586>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000026eb:[<0x100000026ea/munmap03.c v10>,<0x1000000261a/munmap v1612>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.510 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.079+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000027ea [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/Makefile auth v28 ap=1 s=915 n(v0 rc2024-06-06T08:42:56.499805+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0b0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000027ea:[<0x100000027e8/Makefile v4>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000027ea:[<0x100000027e8/Makefile v28>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.510 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.079+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000027ed [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink06.c auth v22 ap=1 s=6848 n(v0 rc2024-06-06T08:42:56.502805+0000 b6848 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0b0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000027ed:[<0x100000027e8/unlink06.c v10>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000027ed:[<0x100000027e8/unlink06.c v22>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.510 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.080+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000027e9 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink05.c auth v12 ap=1 s=6776 n(v0 rc2024-06-06T08:42:56.498805+0000 b6776 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa0b1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000027e9:[<0x100000027e8/unlink05.c v12>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000027e9:[<0x100000027e8/unlink05.c v12>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.510 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.080+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000027ec [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink08.c auth v26 ap=1 s=11126 n(v0 rc2024-06-06T08:42:56.501805+0000 b11126 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0b1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000027ec:[<0x100000027e8/unlink08.c v8>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000027ec:[<0x100000027e8/unlink08.c v26>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.510 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.080+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000027eb [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink07.c auth v24 ap=1 s=8875 n(v0 rc2024-06-06T08:42:56.500805+0000 b8875 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0b0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000027eb:[<0x100000027e8/unlink07.c v6>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000027eb:[<0x100000027e8/unlink07.c v24>,<0x1000000261a/unlink v1556>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.510 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.104+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000028c7 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam03.c auth v20 ap=1 s=5102 n(v0 rc2009-03-23T13:36:01.000000+0000 b5102 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0aeb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000028c7:[<0x100000028c6/sched_getparam03.c v2>,<0x1000000261a/sched_getparam v1686>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000028c7:[<0x100000028c6/sched_getparam03.c v20>,<0x1000000261a/sched_getparam v1686>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.510 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.104+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000028c9 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam01.c auth v18 ap=1 s=4421 n(v0 rc2024-06-06T08:42:56.722800+0000 b4421 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0ae000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000028c9:[<0x100000028c6/sched_getparam01.c v6>,<0x1000000261a/sched_getparam v1686>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000028c9:[<0x100000028c6/sched_getparam01.c v18>,<0x1000000261a/sched_getparam v1686>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.510 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.104+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000028c8 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/Makefile auth v10 ap=1 s=915 n(v0 rc2024-06-06T08:42:56.721800+0000 b915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa0ae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000028c8:[<0x100000028c6/Makefile v10>,<0x1000000261a/sched_getparam v1686>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000028c8:[<0x100000028c6/Makefile v10>,<0x1000000261a/sched_getparam v1686>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.511 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.104+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000028ca [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam02.c auth v12 ap=1 s=4994 n(v0 rc2024-06-06T08:42:56.723800+0000 b4994 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa0af080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000028ca:[<0x100000028c6/sched_getparam02.c v12>,<0x1000000261a/sched_getparam v1686>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000028ca:[<0x100000028c6/sched_getparam02.c v12>,<0x1000000261a/sched_getparam v1686>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v582>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v61>,<0x10000000006/fsstress v60>,<0x10000000005/tmp v51>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v37>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:10.823 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: pgmap v90: 129 pgs: 129 active+clean; 95 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 2.17k op/s 2024-06-06T08:43:10.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002389 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/event_generator.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.826 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000238c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/run_pec_test) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000238d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/pec_listener.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000238b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/README) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000238a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002819 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000281b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir05.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000281a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir04.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000281d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir03.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002818 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.827 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002817 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir08.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.828 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002816 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir09.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.828 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000281c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.828 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.828 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.828 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000029fe (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.828 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000029ff (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/timer_getoverrun01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.828 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.828 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.828 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000266c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.829 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000266b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.829 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000266a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.829 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.830 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.830 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026e5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/asyncio02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026e4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000274c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000274b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000274d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000297c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000297e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000297d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.831 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002797 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/getsockname01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002796 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002881 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/execv01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002880 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026c4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd03.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026c3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026c5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026c2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.832 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026ec (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026ed (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026ee (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000026eb (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000027ea (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002906 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000027ed (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink06.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000027e9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink05.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002905 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.833 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000027ec (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink08.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002904 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000027eb (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink07.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000029c6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000029c7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000029c5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000284c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink05.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.834 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000284f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink04.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000284e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink03.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000284a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000284b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000028c7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000028c9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000028c8 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000028ca (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000027b2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/exit_group01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000027b3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000284d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.835 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.836 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.836 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:10.836 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: pgmap v90: 129 pgs: 129 active+clean; 95 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 2.17k op/s 2024-06-06T08:43:10.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.838 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002389 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/event_generator.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000238c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/run_pec_test) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000238d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/pec_listener.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000238b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/README) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000238a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002819 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000281b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir05.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000281a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir04.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000281d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir03.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002818 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002817 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir08.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002816 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir09.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000281c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000029fe (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000029ff (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/timer_getoverrun01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.840 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000266c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000266b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000266a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026e5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/asyncio02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026e4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000274c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000274b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000274d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000297c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000297e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000297d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002797 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/getsockname01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002796 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002881 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/execv01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002880 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026c4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd03.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026c3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026c5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026c2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026ec (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026ed (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026ee (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000026eb (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000027ea (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002906 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000027ed (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink06.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000027e9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink05.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002905 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000027ec (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink08.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.844 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002904 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000027eb (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink07.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000029c6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000029c7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000029c5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000284c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink05.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000284f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink04.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000284e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink03.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000284a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000284b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000028c7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000028c9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000028c8 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000028ca (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.846 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000027b2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/exit_group01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.847 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000027b3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.847 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000284d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.847 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.847 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.847 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:10.847 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: pgmap v90: 129 pgs: 129 active+clean; 95 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 2.17k op/s 2024-06-06T08:43:10.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002389 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/event_generator.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000238c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/run_pec_test) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000238d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/pec_listener.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000238b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/README) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000238a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002819 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000281b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir05.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000281a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir04.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000281d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir03.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002818 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002817 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir08.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002816 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir09.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000281c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000029fe (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000029ff (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/timer_getoverrun01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000266c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000266b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000266a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026e5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/asyncio02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026e4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000274c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000274b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000274d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000297c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000297e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000297d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002797 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/getsockname01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002796 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002881 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/execv01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002880 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026c4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd03.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026c3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026c5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026c2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026ec (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026ed (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026ee (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000026eb (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000027ea (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002906 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000027ed (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink06.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000027e9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink05.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002905 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000027ec (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink08.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002904 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000027eb (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink07.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000029c6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000029c7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000029c5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000284c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink05.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000284f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink04.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000284e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink03.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000284a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink02.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000284b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000028c7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000028c9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000028c8 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000028ca (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000027b2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/exit_group01.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000027b3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000284d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:10.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:10.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:10.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:10 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.243 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.829+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000250d [...5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/ auth v160 ap=1 f(v0 m2024-06-06T08:42:55.407833+0000 38=38+0) n(v0 rc2024-06-06T08:42:55.418833+0000 b64374 39=38+1)/n(v0 rc2024-06-06T08:42:55.418833+0000 b52841 39=38+1) old_inodes=1 (inest lock) (ifile excl) caps={24892=pAsLsXsFsx/pAsLsXsFsxcral@1},l=24892 | importingcaps=0 dirfrag=1 caps=1 dirtyrstat=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279302e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000250d:[<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[]","memoryvalue":"(2)0x1000000250d:[<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-06T08:42:55.407833+0000 38=38+0)","ondisk_value.rstat":"n(v0 rc2024-06-06T08:42:55.418833+0000 b64374 39=38+1)","memory_value.dirstat":"f(v0 m2024-06-06T08:42:55.407833+0000 38=38+0)","memory_value.rstat":"n(v0 rc2024-06-06T08:42:55.418833+0000 b64374 39=38+1)","error_str":""},"return_code":0} 2024-06-06T08:43:11.243 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.829+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000252e [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsched.te auth v108 ap=1 s=1031 n(v0 rc2009-05-11T07:22:28.000000+0000 b1031 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279302eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000252e:[<0x1000000250d/test_task_getsched.te v108>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000252e:[<0x1000000250d/test_task_getsched.te v108>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.243 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.830+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002527 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sysctl.te auth v112 ap=1 s=870 n(v0 rc2024-06-06T08:42:55.300836+0000 b870 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279302fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002527:[<0x1000000250d/test_sysctl.te v52>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002527:[<0x1000000250d/test_sysctl.te v112>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.243 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.831+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002518 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_mkdir.te auth v184 ap=1 s=2302 n(v0 rc2024-06-06T08:42:55.285836+0000 b2302 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279302f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002518:[<0x1000000250d/test_mkdir.te v22>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002518:[<0x1000000250d/test_mkdir.te v184>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.243 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.831+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000252f [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_net.te auth v82 ap=1 s=2026 n(v0 rc2024-06-06T08:42:55.323835+0000 b2026 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279302f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000252f:[<0x1000000250d/test_capable_net.te v82>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000252f:[<0x1000000250d/test_capable_net.te v82>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.243 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.831+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002523 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_fdreceive.te auth v170 ap=1 s=2541 n(v0 rc2024-06-06T08:42:55.296836+0000 b2541 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793030100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002523:[<0x1000000250d/test_fdreceive.te v44>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002523:[<0x1000000250d/test_fdreceive.te v170>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.243 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.831+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000252d [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrace.te auth v158 ap=1 s=1965 n(v0 rc2024-06-06T08:42:55.306836+0000 b1965 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793030680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000252d:[<0x1000000250d/test_dyntrace.te v64>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000252d:[<0x1000000250d/test_dyntrace.te v158>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.832+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002513 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ioctl.te auth v96 ap=1 s=1336 n(v0 rc2024-06-06T08:42:55.280836+0000 b1336 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793030c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002513:[<0x1000000250d/test_ioctl.te v96>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002513:[<0x1000000250d/test_ioctl.te v96>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.832+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002514 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setsched.te auth v188 ap=1 s=1307 n(v0 rc2024-06-06T08:42:55.281836+0000 b1307 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793031180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002514:[<0x1000000250d/test_task_setsched.te v14>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002514:[<0x1000000250d/test_task_setsched.te v188>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.832+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002528 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrans.te auth v168 ap=1 s=1178 n(v0 rc2024-06-06T08:42:55.301836+0000 b1178 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793031700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002528:[<0x1000000250d/test_dyntrans.te v54>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002528:[<0x1000000250d/test_dyntrans.te v168>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.833+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000251c [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getpgid.te auth v92 ap=1 s=1001 n(v0 rc2024-06-06T08:42:55.289836+0000 b1001 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793036000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000251c:[<0x1000000250d/test_task_getpgid.te v92>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000251c:[<0x1000000250d/test_task_getpgid.te v92>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.833+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000251f [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_file.te auth v178 ap=1 s=2747 n(v0 rc2024-06-06T08:42:55.292836+0000 b2747 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793036580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000251f:[<0x1000000250d/test_file.te v36>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000251f:[<0x1000000250d/test_file.te v178>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.833+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002516 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_readlink.te auth v186 ap=1 s=1251 n(v0 rc2024-06-06T08:42:55.283836+0000 b1251 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793036b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002516:[<0x1000000250d/test_readlink.te v18>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002516:[<0x1000000250d/test_readlink.te v186>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.833+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000251b [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execute_no_trans.te auth v180 ap=1 s=955 n(v0 rc2024-06-06T08:42:55.288836+0000 b955 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793037080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000251b:[<0x1000000250d/test_execute_no_trans.te v28>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000251b:[<0x1000000250d/test_execute_no_trans.te v180>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.833+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002525 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_open.te auth v94 ap=1 s=1154 n(v0 rc2024-06-06T08:42:55.298836+0000 b1154 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793037600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002525:[<0x1000000250d/test_open.te v94>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002525:[<0x1000000250d/test_open.te v94>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.834+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000251a [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_stat.te auth v78 ap=1 s=864 n(v0 rc2024-06-06T08:42:55.287836+0000 b864 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793037b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000251a:[<0x1000000250d/test_stat.te v78>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000251a:[<0x1000000250d/test_stat.te v78>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.834+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002512 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ipc.te auth v190 ap=1 s=3187 n(v0 rc2024-06-06T08:42:55.279836+0000 b3187 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793038100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002512:[<0x1000000250d/test_ipc.te v10>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002512:[<0x1000000250d/test_ipc.te v190>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.834+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000251d [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_sys.te auth v80 ap=1 s=621 n(v0 rc2024-06-06T08:42:55.290836+0000 b621 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793038680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000251d:[<0x1000000250d/test_capable_sys.te v80>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000251d:[<0x1000000250d/test_capable_sys.te v80>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.244 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.834+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000250f [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setpgid.te auth v90 ap=1 s=1682 n(v0 rc2024-06-06T08:42:55.276836+0000 b1682 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793038c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000250f:[<0x1000000250d/test_task_setpgid.te v90>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000250f:[<0x1000000250d/test_task_setpgid.te v90>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.834+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000252c [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_global.te auth v160 ap=1 s=2555 n(v0 rc2024-06-06T08:42:55.305836+0000 b2555 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793039180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000252c:[<0x1000000250d/test_global.te v62>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000252c:[<0x1000000250d/test_global.te v160>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.835+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002532 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rxdir.te auth v154 ap=1 s=1099 n(v0 rc2024-06-06T08:42:55.407833+0000 b1099 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793039700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002532:[<0x1000000250d/test_rxdir.te v74>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002532:[<0x1000000250d/test_rxdir.te v154>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.835+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002520 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/Makefile auth v176 ap=1 s=1158 n(v0 rc2024-06-06T08:42:55.293836+0000 b1158 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279303c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002520:[<0x1000000250d/Makefile v38>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002520:[<0x1000000250d/Makefile v176>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.835+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002511 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_link.te auth v84 ap=1 s=2781 n(v0 rc2009-05-11T07:22:28.000000+0000 b2781 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279303c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002511:[<0x1000000250d/test_link.te v84>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002511:[<0x1000000250d/test_link.te v84>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.835+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002519 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ptrace.te auth v182 ap=1 s=1333 n(v0 rc2024-06-06T08:42:55.286836+0000 b1333 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279303cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002519:[<0x1000000250d/test_ptrace.te v24>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002519:[<0x1000000250d/test_ptrace.te v182>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.835+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002515 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_file.te auth v98 ap=1 s=2148 n(v0 rc2024-06-06T08:42:55.282836+0000 b2148 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279303d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002515:[<0x1000000250d/test_capable_file.te v98>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002515:[<0x1000000250d/test_capable_file.te v98>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.836+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000252b [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_exectrace.te auth v162 ap=1 s=2339 n(v0 rc2024-06-06T08:42:55.304836+0000 b2339 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279303d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000252b:[<0x1000000250d/test_exectrace.te v60>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000252b:[<0x1000000250d/test_exectrace.te v162>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.836+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002524 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sigkill.te auth v88 ap=1 s=1666 n(v0 rc2024-06-06T08:42:55.297836+0000 b1666 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279303db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002524:[<0x1000000250d/test_sigkill.te v88>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002524:[<0x1000000250d/test_sigkill.te v88>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.836+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000251e [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execshare.te auth v104 ap=1 s=1915 n(v0 rc2024-06-06T08:42:55.291836+0000 b1915 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279303e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000251e:[<0x1000000250d/test_execshare.te v104>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000251e:[<0x1000000250d/test_execshare.te v104>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.836+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002521 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_wait.te auth v174 ap=1 s=1497 n(v0 rc2024-06-06T08:42:55.294836+0000 b1497 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279303e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002521:[<0x1000000250d/test_wait.te v40>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002521:[<0x1000000250d/test_wait.te v174>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.245 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.837+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002531 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_create.te auth v156 ap=1 s=2000 n(v0 rc2024-06-06T08:42:55.400833+0000 b2000 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279303ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002531:[<0x1000000250d/test_task_create.te v72>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002531:[<0x1000000250d/test_task_create.te v156>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.246 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.837+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002510 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setnice.te auth v102 ap=1 s=2022 n(v0 rc2024-06-06T08:42:55.278836+0000 b2022 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x56279303f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002510:[<0x1000000250d/test_setnice.te v102>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002510:[<0x1000000250d/test_setnice.te v102>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.246 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.837+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000252a [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setattr.te auth v164 ap=1 s=1006 n(v0 rc2024-06-06T08:42:55.303836+0000 b1006 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279303f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000252a:[<0x1000000250d/test_setattr.te v58>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000252a:[<0x1000000250d/test_setattr.te v164>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.246 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.837+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002529 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_transition.te auth v166 ap=1 s=1337 n(v0 rc2024-06-06T08:42:55.302836+0000 b1337 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793018000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002529:[<0x1000000250d/test_transition.te v56>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002529:[<0x1000000250d/test_transition.te v166>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.246 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.838+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002526 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_relabel.te auth v106 ap=1 s=1701 n(v0 rc2024-06-06T08:42:55.299836+0000 b1701 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793018580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002526:[<0x1000000250d/test_relabel.te v106>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002526:[<0x1000000250d/test_relabel.te v106>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.246 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.838+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002530 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsid.te auth v86 ap=1 s=971 n(v0 rc2024-06-06T08:42:55.393833+0000 b971 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793018b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002530:[<0x1000000250d/test_task_getsid.te v86>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002530:[<0x1000000250d/test_task_getsid.te v86>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.246 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.838+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000250e [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rename.te auth v100 ap=1 s=4810 n(v0 rc2024-06-06T08:42:55.274836+0000 b4810 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793019080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000250e:[<0x1000000250d/test_rename.te v100>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000250e:[<0x1000000250d/test_rename.te v100>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.246 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.838+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002522 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_entrypoint.te auth v172 ap=1 s=612 n(v0 rc2024-06-06T08:42:55.295836+0000 b612 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562793019600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002522:[<0x1000000250d/test_entrypoint.te v42>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002522:[<0x1000000250d/test_entrypoint.te v172>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.246 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.838+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002517 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_policy.if auth v110 ap=1 s=702 n(v0 rc2024-06-06T08:42:55.284836+0000 b702 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562793019b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002517:[<0x1000000250d/test_policy.if v110>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002517:[<0x1000000250d/test_policy.if v110>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.246 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:10.838+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002533 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_inherit.te auth v114 ap=1 s=2704 n(v0 rc2024-06-06T08:42:55.418833+0000 b2704 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56279301a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002533:[<0x1000000250d/test_inherit.te v76>,<0x1000000250c/5 v114>,<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002533:[<0x1000000250d/test_inherit.te v114>,<0x1000000250c/5 v160>,<0x100000024f5/redhat v252>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.259 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.823+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002500 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_mkdir.te auth{0=1} v245 ap=1 s=2482 n(v0 rc2024-06-06T08:42:55.261837+0000 b2482 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561caa337080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002500:[<0x100000024f5/test_mkdir.te v22>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002500:[<0x100000024f5/test_mkdir.te v245>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.259 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.823+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002541 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsched.te auth v126 ap=1 s=1159 n(v0 rc2024-06-06T08:42:55.460832+0000 b1159 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa336b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002541:[<0x100000024f5/test_task_getsched.te v126>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002541:[<0x100000024f5/test_task_getsched.te v126>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.259 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.823+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000253a [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sysctl.te auth v227 ap=1 s=934 n(v0 rc2024-06-06T08:42:55.450832+0000 b934 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa337b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000253a:[<0x100000024f5/test_sysctl.te v62>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000253a:[<0x100000024f5/test_sysctl.te v227>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.259 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.823+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002538 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/TODO auth v106 ap=1 s=984 n(v0 rc2024-06-06T08:42:55.448832+0000 b984 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa338100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002538:[<0x100000024f5/TODO v106>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002538:[<0x100000024f5/TODO v106>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.259 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.824+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002543 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_net.te auth v127 ap=1 s=2091 n(v0 rc2024-06-06T08:42:55.461832+0000 b2091 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa337600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002543:[<0x100000024f5/test_capable_net.te v127>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002543:[<0x100000024f5/test_capable_net.te v127>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.259 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.824+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002535 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_fdreceive.te auth v179 ap=1 s=2680 n(v0 rc2024-06-06T08:42:55.439832+0000 b2680 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa338680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002535:[<0x100000024f5/test_fdreceive.te v52>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002535:[<0x100000024f5/test_fdreceive.te v179>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.824+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002540 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrace.te auth v92 ap=1 s=2146 n(v0 rc2024-06-06T08:42:55.458832+0000 b2146 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa338c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002540:[<0x100000024f5/test_dyntrace.te v92>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002540:[<0x100000024f5/test_dyntrace.te v92>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.824+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024fb [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ioctl.te auth v96 ap=1 s=1369 n(v0 rc2024-06-06T08:42:55.256837+0000 b1369 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa339180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024fb:[<0x100000024f5/test_ioctl.te v96>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024fb:[<0x100000024f5/test_ioctl.te v96>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.824+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024fc [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setsched.te auth v98 ap=1 s=1435 n(v0 rc2024-06-06T08:42:55.257837+0000 b1435 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa339700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024fc:[<0x100000024f5/test_task_setsched.te v98>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024fc:[<0x100000024f5/test_task_setsched.te v98>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.824+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000253b [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrans.te auth v108 ap=1 s=1375 n(v0 rc2024-06-06T08:42:55.452832+0000 b1375 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561ca9c9e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000253b:[<0x100000024f5/test_dyntrans.te v108>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000253b:[<0x100000024f5/test_dyntrans.te v108>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.825+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002504 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getpgid.te auth v102 ap=1 s=1126 n(v0 rc2009-05-11T07:22:26.000000+0000 b1126 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561ca9c9e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002504:[<0x100000024f5/test_task_getpgid.te v102>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002504:[<0x100000024f5/test_task_getpgid.te v102>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.825+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024fe [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_readlink.te auth v247 ap=1 s=1329 n(v0 rc2024-06-06T08:42:55.259837+0000 b1329 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9c9f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024fe:[<0x100000024f5/test_readlink.te v18>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024fe:[<0x100000024f5/test_readlink.te v247>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.825+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002507 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_file.te auth{0=1} v243 ap=1 s=2732 n(v0 rc2024-06-06T08:42:55.267837+0000 b2732 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561ca9c9eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002507:[<0x100000024f5/test_file.te v36>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002507:[<0x100000024f5/test_file.te v243>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.826+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002503 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execute_no_trans.te auth v123 ap=1 s=1002 n(v0 rc2009-05-11T07:22:26.000000+0000 b1002 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561ca9c9f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002503:[<0x100000024f5/test_execute_no_trans.te v123>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002503:[<0x100000024f5/test_execute_no_trans.te v123>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.826+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002537 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_open.te auth{0=1} v231 ap=1 s=1370 n(v0 rc2024-06-06T08:42:55.446832+0000 b1370 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561ca9c9fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002537:[<0x100000024f5/test_open.te v56>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002537:[<0x100000024f5/test_open.te v231>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.260 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.826+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002502 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_stat.te auth v177 ap=1 s=934 n(v0 rc2024-06-06T08:42:55.263837+0000 b934 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9ca0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002502:[<0x100000024f5/test_stat.te v145>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002502:[<0x100000024f5/test_stat.te v177>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.826+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024fa [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ipc.te auth v178 ap=1 s=3336 n(v0 rc2024-06-06T08:42:55.255837+0000 b3336 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9ca0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024fa:[<0x100000024f5/test_ipc.te v147>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024fa:[<0x100000024f5/test_ipc.te v178>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.827+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002505 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_sys.te auth v174 ap=1 s=691 n(v0 rc2024-06-06T08:42:55.265837+0000 b691 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9ca0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002505:[<0x100000024f5/test_capable_sys.te v139>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002505:[<0x100000024f5/test_capable_sys.te v174>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.827+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024f7 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setpgid.te auth v94 ap=1 s=1803 n(v0 rc2024-06-06T08:42:55.251837+0000 b1803 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561ca9ca1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024f7:[<0x100000024f5/test_task_setpgid.te v94>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024f7:[<0x100000024f5/test_task_setpgid.te v94>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.827+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000253f [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_global.te auth v170 ap=1 s=2583 n(v0 rc2024-06-06T08:42:55.457832+0000 b2583 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9ca1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000253f:[<0x100000024f5/test_global.te v131>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000253f:[<0x100000024f5/test_global.te v170>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.827+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002546 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rxdir.te auth v219 ap=1 s=1170 n(v0 rc2024-06-06T08:42:55.465832+0000 b1170 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5ce580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002546:[<0x100000024f5/test_rxdir.te v84>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002546:[<0x100000024f5/test_rxdir.te v219>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.827+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024f9 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_link.te auth{0=1} v249 ap=1 s=2993 n(v0 rc2024-06-06T08:42:55.254837+0000 b2993 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561caa5cf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024f9:[<0x100000024f5/test_link.te v8>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024f9:[<0x100000024f5/test_link.te v249>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.828+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002508 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/Makefile auth v241 ap=1 s=2341 n(v0 rc2024-06-06T08:42:55.268837+0000 b2341 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5ceb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002508:[<0x100000024f5/Makefile v38>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002508:[<0x100000024f5/Makefile v241>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.828+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002501 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ptrace.te auth v169 ap=1 s=1465 n(v0 rc2024-06-06T08:42:55.262837+0000 b1465 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5cf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002501:[<0x100000024f5/test_ptrace.te v129>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002501:[<0x100000024f5/test_ptrace.te v169>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.828+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024fd [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_file.te auth v100 ap=1 s=2252 n(v0 rc2024-06-06T08:42:55.258837+0000 b2252 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa5cfb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024fd:[<0x100000024f5/test_capable_file.te v100>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024fd:[<0x100000024f5/test_capable_file.te v100>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.261 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.828+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000250b [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_bounds.te auth v124 ap=1 s=1990 n(v0 rc2024-06-06T08:42:55.272836+0000 b1990 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa5d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000250b:[<0x100000024f5/test_bounds.te v124>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000250b:[<0x100000024f5/test_bounds.te v124>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.828+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000253e [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_exectrace.te auth v223 ap=1 s=2500 n(v0 rc2024-06-06T08:42:55.455832+0000 b2500 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5d0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000253e:[<0x100000024f5/test_exectrace.te v70>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000253e:[<0x100000024f5/test_exectrace.te v223>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.829+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002536 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sigkill.te auth v233 ap=1 s=1866 n(v0 rc2024-06-06T08:42:55.443832+0000 b1866 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5d0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002536:[<0x100000024f5/test_sigkill.te v54>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002536:[<0x100000024f5/test_sigkill.te v233>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.829+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002509 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_wait.te auth{0=1} v239 ap=1 s=1675 n(v0 rc2024-06-06T08:42:55.270836+0000 b1675 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561caa5d1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002509:[<0x100000024f5/test_wait.te v40>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002509:[<0x100000024f5/test_wait.te v239>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.829+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002545 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_create.te auth v221 ap=1 s=2118 n(v0 rc2024-06-06T08:42:55.464832+0000 b2118 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5b4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002545:[<0x100000024f5/test_task_create.te v82>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002545:[<0x100000024f5/test_task_create.te v221>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.830+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002534 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.fc auth v122 ap=1 s=8 n(v0 rc2024-06-06T08:42:55.432832+0000 b8 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa5b4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002534:[<0x100000024f5/test_policy.fc v122>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002534:[<0x100000024f5/test_policy.fc v122>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.830+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002506 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execshare.te auth v175 ap=1 s=2054 n(v0 rc2024-06-06T08:42:55.266837+0000 b2054 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5d1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002506:[<0x100000024f5/test_execshare.te v141>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002506:[<0x100000024f5/test_execshare.te v175>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.830+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024f8 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setnice.te auth v90 ap=1 s=2117 n(v0 rc2024-06-06T08:42:55.252837+0000 b2117 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa5b4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024f8:[<0x100000024f5/test_setnice.te v90>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024f8:[<0x100000024f5/test_setnice.te v90>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.831+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000253d [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setattr.te auth v225 ap=1 s=1074 n(v0 rc2024-06-06T08:42:55.454832+0000 b1074 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5b5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000253d:[<0x100000024f5/test_setattr.te v68>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000253d:[<0x100000024f5/test_setattr.te v225>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.831+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000253c [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_transition.te auth v125 ap=1 s=1491 n(v0 rc2024-06-06T08:42:55.453832+0000 b1491 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa5b5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000253c:[<0x100000024f5/test_transition.te v125>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000253c:[<0x100000024f5/test_transition.te v125>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.262 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.832+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002539 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_relabel.te auth v229 ap=1 s=1814 n(v0 rc2024-06-06T08:42:55.449832+0000 b1814 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5b5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002539:[<0x100000024f5/test_relabel.te v60>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002539:[<0x100000024f5/test_relabel.te v229>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.263 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.832+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002544 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsid.te auth v172 ap=1 s=1093 n(v0 rc2024-06-06T08:42:55.462832+0000 b1093 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5b6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002544:[<0x100000024f5/test_task_getsid.te v135>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002544:[<0x100000024f5/test_task_getsid.te v172>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.263 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.832+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024f6 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rename.te auth v171 ap=1 s=5108 n(v0 rc2024-06-06T08:42:55.250837+0000 b5108 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5b6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024f6:[<0x100000024f5/test_rename.te v133>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024f6:[<0x100000024f5/test_rename.te v171>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.263 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.833+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000250a [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_entrypoint.te auth{0=1} v237 ap=1 s=652 n(v0 rc2024-06-06T08:42:55.271836+0000 b652 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561caa5b6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000250a:[<0x100000024f5/test_entrypoint.te v42>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000250a:[<0x100000024f5/test_entrypoint.te v237>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.263 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.833+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000024ff [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.if auth v173 ap=1 s=1765 n(v0 rc2024-06-06T08:42:55.260837+0000 b1765 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5b7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000024ff:[<0x100000024f5/test_policy.if v137>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000024ff:[<0x100000024f5/test_policy.if v173>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.263 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.833+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002547 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_inherit.te auth{0=1} v217 ap=1 s=2882 n(v0 rc2024-06-06T08:42:55.466831+0000 b2882 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x561caa5b7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002547:[<0x100000024f5/test_inherit.te v86>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002547:[<0x100000024f5/test_inherit.te v217>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.263 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.835+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000250c [...5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/ auth{0=1,1=1} v252 ap=2 RANDEPHEMERALPIN f(v0 m2024-06-06T08:42:55.273836+0000 1=0+1) n(v3 rc2024-06-06T08:42:55.418833+0000 b52841 40=38+2) old_inodes=1 (inest sync->mix g=0,1 scatter_wanted) (ifile mix) caps={24892=pAsLsXs/pAsLsXsFsxcral@3} | dirtyscattered=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 randepin 0x561caa5ce000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000250c:[<0x100000024f5/redhat v143>,<0x100000024cb/refpolicy v327>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[]","memoryvalue":"(2)0x1000000250c:[<0x100000024f5/redhat v235>,<0x100000024cb/refpolicy v538>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-06T08:42:55.273836+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-06-06T08:42:55.418833+0000 b52841 40=38+2)","memory_value.dirstat":"f(v0 m2024-06-06T08:42:55.273836+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-06-06T08:42:55.418833+0000 b52841 40=38+2)","error_str":""},"return_code":0} 2024-06-06T08:43:11.263 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.839+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002460 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/hotplug.fns auth v6 ap=1 s=4249 n(v0 rc2024-06-06T08:42:55.104841+0000 b4249 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561caa5a0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002460:[<0x1000000245e/hotplug.fns v6>,<0x10000002444/include v62>,<0x10000002443/cpu_hotplug v59>,<0x10000001f2c/hotplug v819>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002460:[<0x1000000245e/hotplug.fns v6>,<0x10000002444/include v62>,<0x10000002443/cpu_hotplug v121>,<0x10000001f2c/hotplug v1995>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.265 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:10.839+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000245f [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/testsuite.fns auth v10 ap=1 s=3938 n(v0 rc2024-06-06T08:42:55.103841+0000 b3938 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa5a0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000245f:[<0x1000000245e/testsuite.fns v2>,<0x10000002444/include v62>,<0x10000002443/cpu_hotplug v59>,<0x10000001f2c/hotplug v819>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000245f:[<0x1000000245e/testsuite.fns v10>,<0x10000002444/include v62>,<0x10000002443/cpu_hotplug v121>,<0x10000001f2c/hotplug v1995>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.265 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.015+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ac5 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget02.c auth v32 ap=1 s=4950 n(v0 rc2024-06-06T08:42:57.474781+0000 b4950 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9993700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002ac5:[<0x10000002ac0/msgget02.c v10>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002ac5:[<0x10000002ac0/msgget02.c v32>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.265 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.016+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ac3 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget03.c auth v34 ap=1 s=4606 n(v0 rc2024-06-06T08:42:57.472781+0000 b4606 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002ac3:[<0x10000002ac0/msgget03.c v6>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002ac3:[<0x10000002ac0/msgget03.c v34>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.265 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.016+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ac2 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/Makefile auth v36 ap=1 s=967 n(v0 rc2024-06-06T08:42:57.471781+0000 b967 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002ac2:[<0x10000002ac0/Makefile v4>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002ac2:[<0x10000002ac0/Makefile v36>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.265 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.016+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ac4 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget04.c auth v38 ap=1 s=4958 n(v0 rc2024-06-06T08:42:57.473781+0000 b4958 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002ac4:[<0x10000002ac0/msgget04.c v8>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002ac4:[<0x10000002ac0/msgget04.c v38>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.265 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.016+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ac1 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget01.c auth v40 ap=1 s=5149 n(v0 rc2024-06-06T08:42:57.470781+0000 b5149 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002ac1:[<0x10000002ac0/msgget01.c v2>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002ac1:[<0x10000002ac0/msgget01.c v40>,<0x10000002aa2/msgget v70>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.016+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ae4 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat01.c auth v16 ap=1 s=7513 n(v0 rc2024-06-06T08:42:57.613777+0000 b7513 1=1+0) caps={24892=pAsLsXsFsc/-@1},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002ae4:[<0x10000002ae3/shmat01.c v2>,<0x10000002aa2/shmat v138>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002ae4:[<0x10000002ae3/shmat01.c v16>,<0x10000002aa2/shmat v138>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.017+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ae6 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/Makefile auth v20 ap=1 s=967 n(v0 rc2024-06-06T08:42:57.622777+0000 b967 1=1+0) caps={24892=pAsLsXsFsc/-@1},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002ae6:[<0x10000002ae3/Makefile v6>,<0x10000002aa2/shmat v138>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002ae6:[<0x10000002ae3/Makefile v20>,<0x10000002aa2/shmat v138>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.017+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ae5 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat03.c auth v24 ap=1 s=5091 n(v0 rc2024-06-06T08:42:57.618777+0000 b5091 1=1+0) caps={24892=pAsLsXsFsc/-@1},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002ae5:[<0x10000002ae3/shmat03.c v4>,<0x10000002aa2/shmat v138>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002ae5:[<0x10000002ae3/shmat03.c v24>,<0x10000002aa2/shmat v138>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.017+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ae7 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat02.c auth v12 ap=1 s=6627 n(v0 rc2024-06-06T08:42:57.631777+0000 b6627 1=1+0) caps={24892=pAsLsXsFsc/-@1},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa11e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002ae7:[<0x10000002ae3/shmat02.c v8>,<0x10000002aa2/shmat v138>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002ae7:[<0x10000002ae3/shmat02.c v12>,<0x10000002aa2/shmat v138>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.018+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002aa7 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv04.c auth v56 ap=1 s=4958 n(v0 rc2024-06-06T08:42:57.433782+0000 b4958 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0e3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002aa7:[<0x10000002aa3/msgrcv04.c v8>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002aa7:[<0x10000002aa3/msgrcv04.c v56>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.018+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002aaa [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv06.c auth v44 ap=1 s=6537 n(v0 rc2024-06-06T08:42:57.437782+0000 b6537 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9992100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002aaa:[<0x10000002aa3/msgrcv06.c v14>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002aaa:[<0x10000002aa3/msgrcv06.c v44>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.019+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002aa5 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv05.c auth v54 ap=1 s=6678 n(v0 rc2024-06-06T08:42:57.431782+0000 b6678 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0e2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002aa5:[<0x10000002aa3/msgrcv05.c v4>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002aa5:[<0x10000002aa3/msgrcv05.c v54>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.019+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002aa4 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv02.c auth v46 ap=1 s=5852 n(v0 rc2024-06-06T08:42:57.430782+0000 b5852 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9992680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002aa4:[<0x10000002aa3/msgrcv02.c v2>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002aa4:[<0x10000002aa3/msgrcv02.c v46>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.019+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002aa8 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/Makefile auth v52 ap=1 s=967 n(v0 rc2024-06-06T08:42:57.435782+0000 b967 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa0e2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002aa8:[<0x10000002aa3/Makefile v10>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002aa8:[<0x10000002aa3/Makefile v52>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.266 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.019+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002aa9 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv03.c auth v48 ap=1 s=4515 n(v0 rc2024-06-06T08:42:57.436782+0000 b4515 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9992c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002aa9:[<0x10000002aa3/msgrcv03.c v12>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002aa9:[<0x10000002aa3/msgrcv03.c v48>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.267 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.019+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002aa6 [5,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv01.c auth v50 ap=1 s=5460 n(v0 rc2024-06-06T08:42:57.432782+0000 b5460 1=1+0) caps={24892=pAsLsXsFsc/-@2},l=24892(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9993180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002aa6:[<0x10000002aa3/msgrcv01.c v6>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v1624>,<0x10000001f2c/syscalls v794>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002aa6:[<0x10000002aa3/msgrcv01.c v50>,<0x10000002aa2/msgrcv v16>,<0x1000000261a/ipc v2273>,<0x10000001f2c/syscalls v1581>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002500 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_mkdir.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002541 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsched.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000253a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sysctl.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002538 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/TODO) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002543 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_net.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002535 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_fdreceive.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002540 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrace.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024fb (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ioctl.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024fc (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setsched.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000253b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrans.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002504 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getpgid.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024fe (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_readlink.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.960 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002507 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_file.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002503 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execute_no_trans.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002537 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_open.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002502 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_stat.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024fa (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ipc.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002505 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_sys.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024f7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setpgid.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000253f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_global.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002546 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rxdir.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.961 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024f9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_link.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002508 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002501 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ptrace.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024fd (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_file.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000250b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_bounds.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000250d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000253e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_exectrace.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002536 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sigkill.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000252e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsched.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002509 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_wait.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002545 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_create.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002527 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sysctl.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002534 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.fc) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002506 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execshare.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002518 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_mkdir.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024f8 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setnice.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000252f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_net.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000253d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setattr.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002523 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_fdreceive.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000252d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrace.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002513 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ioctl.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000253c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_transition.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.963 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002539 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_relabel.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002514 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setsched.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002544 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsid.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002528 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrans.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000251c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getpgid.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000251f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_file.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024f6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rename.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002516 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_readlink.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000250a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_entrypoint.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000251b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execute_no_trans.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000024ff (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.if) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.964 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002525 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_open.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002547 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_inherit.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000251a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_stat.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002512 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ipc.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000251d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_sys.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000250f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setpgid.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000252c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_global.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002532 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rxdir.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002520 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.965 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002511 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_link.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002519 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ptrace.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002515 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_file.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000250c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000252b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_exectrace.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002524 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sigkill.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000251e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execshare.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002521 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_wait.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002531 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_create.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.966 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002510 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setnice.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000252a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setattr.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002529 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_transition.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002526 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_relabel.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002530 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsid.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000250e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rename.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002522 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_entrypoint.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002517 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_policy.if) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002533 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_inherit.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.967 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002460 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/hotplug.fns) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000245f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/testsuite.fns) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.968 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002ac5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002ac3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002ac2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002ac4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget04.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002ac1 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002ae4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002ae6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002ae5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002ae7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.969 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002aa7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv04.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002aaa (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv06.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002aa5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv05.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002aa4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002aa8 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002aa9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002aa6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:11.971 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:11 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.990 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:11.556+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025af [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_target.c auth v20 ap=1 s=371 n(v0 rc2024-06-06T08:42:55.604828+0000 b371 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5627917c4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000025af:[<0x100000025ac/selinux_task_getpgid_target.c v6>,<0x10000002549/task_getpgid v206>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000025af:[<0x100000025ac/selinux_task_getpgid_target.c v20>,<0x10000002549/task_getpgid v206>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.990 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:11.556+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025ad [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid.sh auth v24 ap=1 s=1650 n(v0 rc2024-06-06T08:42:55.601828+0000 b1650 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5627917c5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000025ad:[<0x100000025ac/selinux_task_getpgid.sh v2>,<0x10000002549/task_getpgid v206>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000025ad:[<0x100000025ac/selinux_task_getpgid.sh v24>,<0x10000002549/task_getpgid v206>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.990 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:11.556+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025ae [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/Makefile auth v22 ap=1 s=1076 n(v0 rc2024-06-06T08:42:55.603828+0000 b1076 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5627917c5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000025ae:[<0x100000025ac/Makefile v4>,<0x10000002549/task_getpgid v206>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000025ae:[<0x100000025ac/Makefile v22>,<0x10000002549/task_getpgid v206>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.990 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:43:11.556+0000 7f04e729b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025b0 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_source.c auth v18 ap=1 s=700 n(v0 rc2024-06-06T08:42:55.605828+0000 b700 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5627917c5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000025b0:[<0x100000025ac/selinux_task_getpgid_source.c v8>,<0x10000002549/task_getpgid v206>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000025b0:[<0x100000025ac/selinux_task_getpgid_source.c v18>,<0x10000002549/task_getpgid v206>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:11.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002500 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_mkdir.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002541 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsched.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000253a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sysctl.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002538 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/TODO) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002543 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_net.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002535 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_fdreceive.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002540 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrace.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024fb (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ioctl.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024fc (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setsched.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000253b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrans.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002504 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getpgid.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024fe (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_readlink.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.994 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002507 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_file.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002503 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execute_no_trans.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002537 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_open.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002502 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_stat.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024fa (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ipc.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002505 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_sys.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024f7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setpgid.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000253f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_global.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002546 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rxdir.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024f9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_link.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002508 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002501 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ptrace.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024fd (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_file.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000250b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_bounds.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000250d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000253e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_exectrace.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002536 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sigkill.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000252e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsched.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002509 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_wait.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002545 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_create.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002527 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sysctl.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002534 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.fc) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002506 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execshare.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002518 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_mkdir.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024f8 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setnice.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000252f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_net.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000253d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setattr.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002523 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_fdreceive.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000252d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrace.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002513 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ioctl.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000253c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_transition.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002539 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_relabel.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002514 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setsched.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002544 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsid.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002528 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrans.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000251c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getpgid.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000251f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_file.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024f6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rename.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002516 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_readlink.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000250a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_entrypoint.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000251b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execute_no_trans.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000024ff (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.if) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002525 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_open.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002547 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_inherit.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000251a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_stat.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002512 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ipc.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000251d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_sys.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.998 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000250f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setpgid.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000252c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_global.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002532 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rxdir.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002520 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002511 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_link.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002519 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ptrace.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002515 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_file.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000250c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat) see mds.h log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000252b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_exectrace.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002524 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sigkill.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:11.999 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000251e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execshare.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002521 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_wait.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002531 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_create.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002510 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setnice.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000252a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setattr.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002529 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_transition.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002526 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_relabel.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.000 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002530 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsid.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000250e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rename.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002522 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_entrypoint.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002517 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_policy.if) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002533 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_inherit.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002460 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/hotplug.fns) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000245f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/testsuite.fns) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.001 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002ac5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002ac3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002ac2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002ac4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget04.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002ac1 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002ae4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002ae6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002ae5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002ae7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.002 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002aa7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv04.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002aaa (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv06.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002aa5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv05.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002aa4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002aa8 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002aa9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002aa6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.003 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:11 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.007 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.591+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025a2 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_bind.c auth v20 ap=1 s=670 n(v0 rc2024-06-06T08:42:55.587828+0000 b670 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9f72680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000025a2:[<0x1000000259f/selinux_bind.c v6>,<0x10000002549/capable_net v436>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000025a2:[<0x1000000259f/selinux_bind.c v20>,<0x10000002549/capable_net v788>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:12.007 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.591+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025a0 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_capable_net.sh auth v10 ap=1 s=2896 n(v0 rc2024-06-06T08:42:55.584828+0000 b2896 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561ca9f72100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000025a0:[<0x1000000259f/selinux_capable_net.sh v10>,<0x10000002549/capable_net v436>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000025a0:[<0x1000000259f/selinux_capable_net.sh v10>,<0x10000002549/capable_net v788>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:12.007 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.591+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025a3 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_raw.c auth v18 ap=1 s=620 n(v0 rc2024-06-06T08:42:55.588828+0000 b620 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9f73180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000025a3:[<0x1000000259f/selinux_raw.c v8>,<0x10000002549/capable_net v436>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000025a3:[<0x1000000259f/selinux_raw.c v18>,<0x10000002549/capable_net v788>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:12.007 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.591+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025a1 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/Makefile auth v12 ap=1 s=1075 n(v0 rc2024-06-06T08:42:55.586828+0000 b1075 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@3} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x561ca9f72c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000025a1:[<0x1000000259f/Makefile v12>,<0x10000002549/capable_net v436>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x100000025a1:[<0x1000000259f/Makefile v12>,<0x10000002549/capable_net v788>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:12.008 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.591+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002571 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create_parent.c auth v14 ap=1 s=535 n(v0 rc2024-06-06T08:42:55.520830+0000 b535 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561ca9f73700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002571:[<0x1000000256e/selinux_task_create_parent.c v6>,<0x10000002549/task_create v80>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002571:[<0x1000000256e/selinux_task_create_parent.c v14>,<0x10000002549/task_create v858>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:12.008 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.591+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000256f [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create.sh auth v18 ap=1 s=1362 n(v0 rc2024-06-06T08:42:55.517830+0000 b1362 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa336000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000256f:[<0x1000000256e/selinux_task_create.sh v2>,<0x10000002549/task_create v80>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x1000000256f:[<0x1000000256e/selinux_task_create.sh v18>,<0x10000002549/task_create v858>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:12.008 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:43:11.592+0000 7f8c972d4700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002570 [4,head] /volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/Makefile auth v16 ap=1 s=1075 n(v0 rc2024-06-06T08:42:55.519830+0000 b1075 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24892=0-4194304@4} caps={24892=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=24892 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x561caa336580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000002570:[<0x1000000256e/Makefile v4>,<0x10000002549/task_create v80>,<0x100000024cb/tests v300>,<0x10000002494/selinux-testsuite v199>,<0x10000001f2c/security v821>,<0x10000000204/kernel v584>,<0x10000000202/testcases v84>,<0x10000000007/ltp-full-20091231 v56>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v64>,<0x10000000001/sv_1 v86>,<0x10000000000/qa v80>,<0x1/volumes v75>]//[3]","memoryvalue":"(4)0x10000002570:[<0x1000000256e/Makefile v16>,<0x10000002549/task_create v858>,<0x100000024cb/tests v532>,<0x10000002494/selinux-testsuite v384>,<0x10000001f2c/security v2001>,<0x10000000204/kernel v1057>,<0x10000000202/testcases v118>,<0x10000000007/ltp-full-20091231 v188>,<0x10000000006/fsstress v99>,<0x10000000005/tmp v63>,<0x100000001fb/client.0 v26>,<0x100000001fa/97e6dc45-7eb6-4039-a0c7-4313ff988f86 v68>,<0x10000000001/sv_1 v78>,<0x10000000000/qa v93>,<0x1/volumes v79>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T08:43:12.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002500 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_mkdir.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002541 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsched.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000253a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sysctl.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002538 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/TODO) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002543 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_net.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002535 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_fdreceive.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002540 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrace.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024fb (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ioctl.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024fc (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setsched.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000253b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrans.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002504 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getpgid.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024fe (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_readlink.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002507 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_file.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002503 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execute_no_trans.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002537 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_open.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002502 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_stat.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024fa (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ipc.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002505 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_sys.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024f7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setpgid.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000253f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_global.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002546 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rxdir.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024f9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_link.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002508 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002501 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ptrace.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024fd (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_file.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000250b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_bounds.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000250d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000253e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_exectrace.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002536 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sigkill.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000252e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsched.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002509 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_wait.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002545 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_create.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002527 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sysctl.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002534 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.fc) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002506 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execshare.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002518 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_mkdir.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024f8 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setnice.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000252f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_net.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000253d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setattr.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002523 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_fdreceive.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000252d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrace.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002513 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ioctl.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000253c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_transition.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002539 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_relabel.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002514 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setsched.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.014 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002544 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsid.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002528 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrans.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000251c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getpgid.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000251f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_file.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024f6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rename.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002516 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_readlink.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000250a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_entrypoint.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000251b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execute_no_trans.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000024ff (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.if) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002525 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_open.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002547 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_inherit.te) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000251a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_stat.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002512 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ipc.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000251d (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_sys.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000250f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setpgid.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000252c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_global.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002532 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rxdir.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002520 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002511 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_link.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002519 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ptrace.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002515 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_file.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000250c (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000252b (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_exectrace.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002524 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sigkill.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000251e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execshare.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002521 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_wait.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002531 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_create.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002510 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setnice.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000252a (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setattr.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002529 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_transition.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002526 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_relabel.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002530 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsid.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000250e (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rename.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002522 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_entrypoint.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002517 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_policy.if) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002533 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_inherit.te) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002460 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/hotplug.fns) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000245f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/testsuite.fns) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002ac5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002ac3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002ac2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002ac4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget04.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002ac1 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002ae4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002ae6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002ae5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002ae7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002aa7 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv04.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002aaa (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv06.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002aa5 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv05.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002aa4 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv02.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002aa8 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002aa9 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv03.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002aa6 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv01.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:12.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:11 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: pgmap v91: 129 pgs: 129 active+clean; 95 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 1.85k op/s 2024-06-06T08:43:12.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:12.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: active paths [/] 2024-06-06T08:43:12.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000025af (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_target.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000025ad (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid.sh) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000025ae (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000025b0 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_source.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000025a2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_bind.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000025a0 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_capable_net.sh) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000025a3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_raw.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x100000025a1 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002571 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create_parent.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x1000000256f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create.sh) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: Scrub error on inode 0x10000002570 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:12 smithi027 ceph-mon[32335]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: pgmap v91: 129 pgs: 129 active+clean; 95 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 1.85k op/s 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.991 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: active paths [/] 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000025af (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_target.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000025ad (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid.sh) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000025ae (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000025b0 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_source.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000025a2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_bind.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000025a0 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_capable_net.sh) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000025a3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_raw.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x100000025a1 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002571 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create_parent.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x1000000256f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create.sh) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: Scrub error on inode 0x10000002570 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:12.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:12.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:12 smithi203 ceph-mon[36579]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: pgmap v91: 129 pgs: 129 active+clean; 95 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 1.85k op/s 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: active paths [/] 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000025af (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_target.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000025ad (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid.sh) see mds.i log and `damage ls` output for details 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000025ae (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/Makefile) see mds.i log and `damage ls` output for details 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000025b0 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_source.c) see mds.i log and `damage ls` output for details 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000025a2 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_bind.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000025a0 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_capable_net.sh) see mds.h log and `damage ls` output for details 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000025a3 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_raw.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:13.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x100000025a1 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:13.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002571 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create_parent.c) see mds.h log and `damage ls` output for details 2024-06-06T08:43:13.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x1000000256f (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create.sh) see mds.h log and `damage ls` output for details 2024-06-06T08:43:13.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: Scrub error on inode 0x10000002570 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/Makefile) see mds.h log and `damage ls` output for details 2024-06-06T08:43:13.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:12 smithi176 ceph-mon[36786]: scrub summary: idle+waiting paths [/] 2024-06-06T08:43:13.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T08:43:13.521 INFO:tasks.workunit.client.0.smithi027.stdout:2/11: sync 2024-06-06T08:43:13.521 INFO:tasks.workunit.client.0.smithi027.stdout:6/6: sync 2024-06-06T08:43:13.522 INFO:tasks.workunit.client.0.smithi027.stdout:7/99: sync 2024-06-06T08:43:13.522 INFO:tasks.workunit.client.0.smithi027.stdout:1/25: sync 2024-06-06T08:43:13.522 INFO:tasks.workunit.client.0.smithi027.stdout:2/12: rmdir - no directory 2024-06-06T08:43:13.522 INFO:tasks.workunit.client.0.smithi027.stdout:9/4: sync 2024-06-06T08:43:13.522 INFO:tasks.workunit.client.0.smithi027.stdout:4/53: sync 2024-06-06T08:43:13.522 INFO:tasks.workunit.client.0.smithi027.stdout:8/0: sync 2024-06-06T08:43:13.522 INFO:tasks.workunit.client.0.smithi027.stdout:0/4: sync 2024-06-06T08:43:13.522 INFO:tasks.workunit.client.0.smithi027.stdout:8/1: write - no filename 2024-06-06T08:43:13.525 INFO:tasks.workunit.client.0.smithi027.stdout:8/2: rename - no filename 2024-06-06T08:43:13.525 INFO:tasks.workunit.client.0.smithi027.stdout:8/3: dwrite - no filename 2024-06-06T08:43:13.525 INFO:tasks.workunit.client.0.smithi027.stdout:8/4: chown . 84 1 2024-06-06T08:43:13.525 INFO:tasks.workunit.client.0.smithi027.stdout:8/5: write - no filename 2024-06-06T08:43:13.525 INFO:tasks.workunit.client.0.smithi027.stdout:8/6: write - no filename 2024-06-06T08:43:13.525 INFO:tasks.workunit.client.0.smithi027.stdout:3/8: sync 2024-06-06T08:43:13.526 INFO:tasks.workunit.client.0.smithi027.stdout:3/9: readlink l0 0 2024-06-06T08:43:13.526 INFO:tasks.workunit.client.0.smithi027.stdout:5/107: sync 2024-06-06T08:43:13.526 INFO:tasks.workunit.client.0.smithi027.stdout:4/54: unlink d0/d1/fa 0 2024-06-06T08:43:13.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/5: mkdir d3 0 2024-06-06T08:43:13.527 INFO:tasks.workunit.client.0.smithi027.stdout:9/6: chown d3 27804969 1 2024-06-06T08:43:13.527 INFO:tasks.workunit.client.0.smithi027.stdout:9/7: write - no filename 2024-06-06T08:43:13.527 INFO:tasks.workunit.client.0.smithi027.stdout:6/7: creat f3 x:0 0 0 2024-06-06T08:43:13.527 INFO:tasks.workunit.client.0.smithi027.stdout:1/26: creat f5 x:0 0 0 2024-06-06T08:43:13.527 INFO:tasks.workunit.client.0.smithi027.stdout:2/13: rename f0 to f3 0 2024-06-06T08:43:13.528 INFO:tasks.workunit.client.0.smithi027.stdout:3/10: symlink d1/l4 0 2024-06-06T08:43:13.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/8: creat d3/f4 x:0 0 0 2024-06-06T08:43:13.529 INFO:tasks.workunit.client.0.smithi027.stdout:6/8: mkdir d4 0 2024-06-06T08:43:13.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/9: dread - d3/f4 zero size 2024-06-06T08:43:13.530 INFO:tasks.workunit.client.0.smithi027.stdout:1/27: mknod c6 0 2024-06-06T08:43:13.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/10: write d3/f4 [823856,33793] 0 2024-06-06T08:43:13.530 INFO:tasks.workunit.client.0.smithi027.stdout:2/14: mknod c4 0 2024-06-06T08:43:13.530 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:43:13.531 INFO:tasks.workunit.client.0.smithi027.stdout:8/7: mknod c0 0 2024-06-06T08:43:13.531 INFO:tasks.workunit.client.0.smithi027.stdout:8/8: readlink - no filename 2024-06-06T08:43:13.531 INFO:tasks.workunit.client.0.smithi027.stdout:8/9: readlink - no filename 2024-06-06T08:43:13.532 INFO:tasks.workunit.client.0.smithi027.stdout:3/11: unlink d1/c3 0 2024-06-06T08:43:13.533 INFO:tasks.workunit.client.0.smithi027.stdout:2/15: creat f5 x:0 0 0 2024-06-06T08:43:13.534 INFO:tasks.workunit.client.0.smithi027.stdout:6/9: dread f0 [0,4194304] 0 2024-06-06T08:43:13.538 INFO:tasks.workunit.client.0.smithi027.stdout:2/16: mkdir d6 0 2024-06-06T08:43:13.539 INFO:tasks.workunit.client.0.smithi027.stdout:2/17: dread - f5 zero size 2024-06-06T08:43:13.539 INFO:tasks.workunit.client.0.smithi027.stdout:8/10: rename c0 to c1 0 2024-06-06T08:43:13.539 INFO:tasks.workunit.client.0.smithi027.stdout:8/11: write - no filename 2024-06-06T08:43:13.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/18: creat d6/f7 x:0 0 0 2024-06-06T08:43:13.540 INFO:tasks.workunit.client.0.smithi027.stdout:6/10: creat d4/f5 x:0 0 0 2024-06-06T08:43:13.540 INFO:tasks.workunit.client.0.smithi027.stdout:6/11: fdatasync d4/f5 0 2024-06-06T08:43:13.542 INFO:tasks.workunit.client.0.smithi027.stdout:0/5: dwrite f0 [0,4194304] 0 2024-06-06T08:43:13.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/19: creat d6/f8 x:0 0 0 2024-06-06T08:43:13.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/20: readlink - no filename 2024-06-06T08:43:13.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/21: chown f5 5 1 2024-06-06T08:43:13.543 INFO:tasks.workunit.client.0.smithi027.stdout:6/12: symlink d4/l6 0 2024-06-06T08:43:13.543 INFO:tasks.workunit.client.0.smithi027.stdout:6/13: truncate f0 311394 0 2024-06-06T08:43:13.547 INFO:tasks.workunit.client.0.smithi027.stdout:2/22: creat d6/f9 x:0 0 0 2024-06-06T08:43:13.548 INFO:tasks.workunit.client.0.smithi027.stdout:5/108: dwrite d0/fe [0,4194304] 0 2024-06-06T08:43:13.549 INFO:tasks.workunit.client.0.smithi027.stdout:5/109: chown d0/da/db/f17 3345042 1 2024-06-06T08:43:13.549 INFO:tasks.workunit.client.0.smithi027.stdout:6/14: mkdir d4/d7 0 2024-06-06T08:43:13.550 INFO:tasks.workunit.client.0.smithi027.stdout:2/23: rename c4 to d6/ca 0 2024-06-06T08:43:13.555 INFO:tasks.workunit.client.0.smithi027.stdout:5/110: unlink d0/l1d 0 2024-06-06T08:43:13.556 INFO:tasks.workunit.client.0.smithi027.stdout:6/15: rename l2 to d4/d7/l8 0 2024-06-06T08:43:13.557 INFO:tasks.workunit.client.0.smithi027.stdout:2/24: mknod d6/cb 0 2024-06-06T08:43:13.557 INFO:tasks.workunit.client.0.smithi027.stdout:1/28: dwrite f5 [0,4194304] 0 2024-06-06T08:43:13.559 INFO:tasks.workunit.client.0.smithi027.stdout:5/111: symlink d0/da/l21 0 2024-06-06T08:43:13.559 INFO:tasks.workunit.client.0.smithi027.stdout:4/55: dwrite d0/d1/fb [0,4194304] 0 2024-06-06T08:43:13.559 INFO:tasks.workunit.client.0.smithi027.stdout:4/56: fdatasync d0/fd 0 2024-06-06T08:43:13.560 INFO:tasks.workunit.client.0.smithi027.stdout:3/12: dwrite d1/f2 [0,4194304] 0 2024-06-06T08:43:13.560 INFO:tasks.workunit.client.0.smithi027.stdout:0/6: write f0 [1743502,76486] 0 2024-06-06T08:43:13.561 INFO:tasks.workunit.client.0.smithi027.stdout:6/16: mkdir d4/d9 0 2024-06-06T08:43:13.561 INFO:tasks.workunit.client.0.smithi027.stdout:6/17: read - d4/f5 zero size 2024-06-06T08:43:13.565 INFO:tasks.workunit.client.0.smithi027.stdout:1/29: symlink l7 0 2024-06-06T08:43:13.565 INFO:tasks.workunit.client.0.smithi027.stdout:1/30: fsync f5 0 2024-06-06T08:43:13.570 INFO:tasks.workunit.client.0.smithi027.stdout:5/112: rmdir d0/da 39 2024-06-06T08:43:13.572 INFO:tasks.workunit.client.0.smithi027.stdout:4/57: creat d0/f11 x:0 0 0 2024-06-06T08:43:13.572 INFO:tasks.workunit.client.0.smithi027.stdout:2/25: write f3 [4041510,35156] 0 2024-06-06T08:43:13.575 INFO:tasks.workunit.client.0.smithi027.stdout:3/13: creat d1/f5 x:0 0 0 2024-06-06T08:43:13.577 INFO:tasks.workunit.client.0.smithi027.stdout:6/18: mknod d4/d7/ca 0 2024-06-06T08:43:13.577 INFO:tasks.workunit.client.0.smithi027.stdout:0/7: getdents . 0 2024-06-06T08:43:13.578 INFO:tasks.workunit.client.0.smithi027.stdout:0/8: chown f0 507 1 2024-06-06T08:43:13.580 INFO:tasks.workunit.client.0.smithi027.stdout:1/31: symlink l8 0 2024-06-06T08:43:13.580 INFO:tasks.workunit.client.0.smithi027.stdout:1/32: chown f5 3 1 2024-06-06T08:43:13.581 INFO:tasks.workunit.client.0.smithi027.stdout:5/113: getdents d0/da/db 0 2024-06-06T08:43:13.581 INFO:tasks.workunit.client.0.smithi027.stdout:5/114: dread - d0/da/db/f1b zero size 2024-06-06T08:43:13.583 INFO:tasks.workunit.client.0.smithi027.stdout:2/26: rmdir d6 39 2024-06-06T08:43:13.583 INFO:tasks.workunit.client.0.smithi027.stdout:2/27: truncate d6/f9 369933 0 2024-06-06T08:43:13.583 INFO:tasks.workunit.client.0.smithi027.stdout:2/28: dread - d6/f8 zero size 2024-06-06T08:43:13.583 INFO:tasks.workunit.client.0.smithi027.stdout:6/19: mknod d4/cb 0 2024-06-06T08:43:13.584 INFO:tasks.workunit.client.0.smithi027.stdout:6/20: readlink d4/l6 0 2024-06-06T08:43:13.584 INFO:tasks.workunit.client.0.smithi027.stdout:0/9: link f0 f1 0 2024-06-06T08:43:13.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/12: rename c1 to c2 0 2024-06-06T08:43:13.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/13: readlink - no filename 2024-06-06T08:43:13.586 INFO:tasks.workunit.client.0.smithi027.stdout:8/14: dwrite - no filename 2024-06-06T08:43:13.587 INFO:tasks.workunit.client.0.smithi027.stdout:2/29: mkdir d6/dc 0 2024-06-06T08:43:13.589 INFO:tasks.workunit.client.0.smithi027.stdout:6/21: mkdir d4/d9/dc 0 2024-06-06T08:43:13.589 INFO:tasks.workunit.client.0.smithi027.stdout:2/30: mkdir d6/dd 0 2024-06-06T08:43:13.592 INFO:tasks.workunit.client.0.smithi027.stdout:1/33: read f5 [1451049,64485] 0 2024-06-06T08:43:13.593 INFO:tasks.workunit.client.0.smithi027.stdout:2/31: creat d6/fe x:0 0 0 2024-06-06T08:43:13.593 INFO:tasks.workunit.client.0.smithi027.stdout:1/34: write f5 [2090947,58713] 0 2024-06-06T08:43:13.593 INFO:tasks.workunit.client.0.smithi027.stdout:1/35: chown l1 45234379 1 2024-06-06T08:43:13.593 INFO:tasks.workunit.client.0.smithi027.stdout:1/36: fdatasync f3 0 2024-06-06T08:43:13.593 INFO:tasks.workunit.client.0.smithi027.stdout:1/37: read - f3 zero size 2024-06-06T08:43:13.593 INFO:tasks.workunit.client.0.smithi027.stdout:2/32: creat d6/ff x:0 0 0 2024-06-06T08:43:13.593 INFO:tasks.workunit.client.0.smithi027.stdout:2/33: write f5 [703695,109152] 0 2024-06-06T08:43:13.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/115: dread d0/fe [0,4194304] 0 2024-06-06T08:43:13.599 INFO:tasks.workunit.client.0.smithi027.stdout:1/38: unlink l4 0 2024-06-06T08:43:13.599 INFO:tasks.workunit.client.0.smithi027.stdout:0/10: fsync f0 0 2024-06-06T08:43:13.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/11: write f1 [5153407,63034] 0 2024-06-06T08:43:13.600 INFO:tasks.workunit.client.0.smithi027.stdout:8/15: chown c2 398 1 2024-06-06T08:43:13.600 INFO:tasks.workunit.client.0.smithi027.stdout:1/39: rename c6 to c9 0 2024-06-06T08:43:13.600 INFO:tasks.workunit.client.0.smithi027.stdout:1/40: fdatasync f3 0 2024-06-06T08:43:13.600 INFO:tasks.workunit.client.0.smithi027.stdout:1/41: read f5 [1297992,32764] 0 2024-06-06T08:43:13.601 INFO:tasks.workunit.client.0.smithi027.stdout:4/58: dwrite d0/d1/ff [0,4194304] 0 2024-06-06T08:43:13.601 INFO:tasks.workunit.client.0.smithi027.stdout:0/12: creat f2 x:0 0 0 2024-06-06T08:43:13.601 INFO:tasks.workunit.client.0.smithi027.stdout:8/16: stat c2 0 2024-06-06T08:43:13.601 INFO:tasks.workunit.client.0.smithi027.stdout:8/17: dwrite - no filename 2024-06-06T08:43:13.601 INFO:tasks.workunit.client.0.smithi027.stdout:8/18: read - no filename 2024-06-06T08:43:13.602 INFO:tasks.workunit.client.0.smithi027.stdout:1/42: mknod ca 0 2024-06-06T08:43:13.602 INFO:tasks.workunit.client.0.smithi027.stdout:1/43: stat ca 0 2024-06-06T08:43:13.602 INFO:tasks.workunit.client.0.smithi027.stdout:1/44: write f3 [55581,124370] 0 2024-06-06T08:43:13.603 INFO:tasks.workunit.client.0.smithi027.stdout:4/59: creat d0/d1/f12 x:0 0 0 2024-06-06T08:43:13.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/13: symlink l3 0 2024-06-06T08:43:13.605 INFO:tasks.workunit.client.0.smithi027.stdout:5/116: dread d0/da/f13 [0,4194304] 0 2024-06-06T08:43:13.605 INFO:tasks.workunit.client.0.smithi027.stdout:4/60: link d0/fd d0/f13 0 2024-06-06T08:43:13.606 INFO:tasks.workunit.client.0.smithi027.stdout:4/61: write d0/f13 [9249521,113934] 0 2024-06-06T08:43:13.608 INFO:tasks.workunit.client.0.smithi027.stdout:0/14: creat f4 x:0 0 0 2024-06-06T08:43:13.608 INFO:tasks.workunit.client.0.smithi027.stdout:8/19: rename c2 to c3 0 2024-06-06T08:43:13.608 INFO:tasks.workunit.client.0.smithi027.stdout:8/20: write - no filename 2024-06-06T08:43:13.608 INFO:tasks.workunit.client.0.smithi027.stdout:5/117: getdents d0 0 2024-06-06T08:43:13.614 INFO:tasks.workunit.client.0.smithi027.stdout:2/34: dwrite f3 [4194304,4194304] 0 2024-06-06T08:43:13.615 INFO:tasks.workunit.client.0.smithi027.stdout:2/35: write d6/f7 [440184,16255] 0 2024-06-06T08:43:13.616 INFO:tasks.workunit.client.0.smithi027.stdout:6/22: dwrite d4/f5 [0,4194304] 0 2024-06-06T08:43:13.616 INFO:tasks.workunit.client.0.smithi027.stdout:2/36: symlink d6/dc/l10 0 2024-06-06T08:43:13.619 INFO:tasks.workunit.client.0.smithi027.stdout:2/37: mknod d6/dd/c11 0 2024-06-06T08:43:13.619 INFO:tasks.workunit.client.0.smithi027.stdout:2/38: write f1 [5099750,38864] 0 2024-06-06T08:43:13.621 INFO:tasks.workunit.client.0.smithi027.stdout:2/39: fsync d6/fe 0 2024-06-06T08:43:13.625 INFO:tasks.workunit.client.0.smithi027.stdout:0/15: dread f0 [0,4194304] 0 2024-06-06T08:43:13.627 INFO:tasks.workunit.client.0.smithi027.stdout:0/16: creat f5 x:0 0 0 2024-06-06T08:43:13.627 INFO:tasks.workunit.client.0.smithi027.stdout:0/17: chown f1 12963988 1 2024-06-06T08:43:13.629 INFO:tasks.workunit.client.0.smithi027.stdout:0/18: rename f0 to f6 0 2024-06-06T08:43:13.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/45: dwrite f5 [0,4194304] 0 2024-06-06T08:43:13.633 INFO:tasks.workunit.client.0.smithi027.stdout:0/19: creat f7 x:0 0 0 2024-06-06T08:43:13.634 INFO:tasks.workunit.client.0.smithi027.stdout:0/20: write f1 [2314849,17388] 0 2024-06-06T08:43:13.635 INFO:tasks.workunit.client.0.smithi027.stdout:1/46: symlink lb 0 2024-06-06T08:43:13.638 INFO:tasks.workunit.client.0.smithi027.stdout:6/23: dwrite d4/f5 [0,4194304] 0 2024-06-06T08:43:13.640 INFO:tasks.workunit.client.0.smithi027.stdout:0/21: getdents . 0 2024-06-06T08:43:13.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/24: symlink d4/d9/ld 0 2024-06-06T08:43:13.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/25: truncate d4/f5 4990260 0 2024-06-06T08:43:13.644 INFO:tasks.workunit.client.0.smithi027.stdout:0/22: link f7 f8 0 2024-06-06T08:43:13.644 INFO:tasks.workunit.client.0.smithi027.stdout:0/23: write f8 [478399,77716] 0 2024-06-06T08:43:13.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/26: mkdir d4/de 0 2024-06-06T08:43:13.646 INFO:tasks.workunit.client.0.smithi027.stdout:6/27: symlink d4/lf 0 2024-06-06T08:43:13.648 INFO:tasks.workunit.client.0.smithi027.stdout:6/28: dread f0 [0,4194304] 0 2024-06-06T08:43:13.648 INFO:tasks.workunit.client.0.smithi027.stdout:6/29: write f0 [1148741,72110] 0 2024-06-06T08:43:13.648 INFO:tasks.workunit.client.0.smithi027.stdout:6/30: fsync d4/f5 0 2024-06-06T08:43:13.652 INFO:tasks.workunit.client.0.smithi027.stdout:1/47: dwrite f3 [0,4194304] 0 2024-06-06T08:43:13.653 INFO:tasks.workunit.client.0.smithi027.stdout:6/31: fsync f0 0 2024-06-06T08:43:13.654 INFO:tasks.workunit.client.0.smithi027.stdout:0/24: getdents . 0 2024-06-06T08:43:13.656 INFO:tasks.workunit.client.0.smithi027.stdout:1/48: mkdir dc 0 2024-06-06T08:43:13.656 INFO:tasks.workunit.client.0.smithi027.stdout:1/49: getdents dc 0 2024-06-06T08:43:13.658 INFO:tasks.workunit.client.0.smithi027.stdout:1/50: creat dc/fd x:0 0 0 2024-06-06T08:43:13.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/32: write f0 [10248,3088] 0 2024-06-06T08:43:13.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/33: chown d4/d9/dc 370 1 2024-06-06T08:43:13.660 INFO:tasks.workunit.client.0.smithi027.stdout:6/34: creat d4/f10 x:0 0 0 2024-06-06T08:43:13.666 INFO:tasks.workunit.client.0.smithi027.stdout:6/35: truncate f0 1050333 0 2024-06-06T08:43:13.666 INFO:tasks.workunit.client.0.smithi027.stdout:6/36: chown d4/de 28568 1 2024-06-06T08:43:13.667 INFO:tasks.workunit.client.0.smithi027.stdout:6/37: creat d4/de/f11 x:0 0 0 2024-06-06T08:43:13.669 INFO:tasks.workunit.client.0.smithi027.stdout:6/38: mknod d4/de/c12 0 2024-06-06T08:43:13.669 INFO:tasks.workunit.client.0.smithi027.stdout:6/39: fdatasync d4/f5 0 2024-06-06T08:43:13.669 INFO:tasks.workunit.client.0.smithi027.stdout:6/40: getdents d4 0 2024-06-06T08:43:13.672 INFO:tasks.workunit.client.0.smithi027.stdout:6/41: mkdir d4/d13 0 2024-06-06T08:43:13.672 INFO:tasks.workunit.client.0.smithi027.stdout:0/25: dwrite f4 [0,4194304] 0 2024-06-06T08:43:13.673 INFO:tasks.workunit.client.0.smithi027.stdout:6/42: truncate d4/f10 77455 0 2024-06-06T08:43:13.673 INFO:tasks.workunit.client.0.smithi027.stdout:6/43: fdatasync d4/f10 0 2024-06-06T08:43:13.675 INFO:tasks.workunit.client.0.smithi027.stdout:0/26: creat f9 x:0 0 0 2024-06-06T08:43:13.675 INFO:tasks.workunit.client.0.smithi027.stdout:0/27: read - f5 zero size 2024-06-06T08:43:13.676 INFO:tasks.workunit.client.0.smithi027.stdout:6/44: readlink d4/lf 0 2024-06-06T08:43:13.677 INFO:tasks.workunit.client.0.smithi027.stdout:0/28: creat fa x:0 0 0 2024-06-06T08:43:13.681 INFO:tasks.workunit.client.0.smithi027.stdout:6/45: mknod d4/d7/c14 0 2024-06-06T08:43:13.681 INFO:tasks.workunit.client.0.smithi027.stdout:6/46: symlink d4/d7/l15 0 2024-06-06T08:43:13.694 INFO:tasks.workunit.client.0.smithi027.stdout:0/29: dwrite f7 [0,4194304] 0 2024-06-06T08:43:13.694 INFO:tasks.workunit.client.0.smithi027.stdout:0/30: dread - f9 zero size 2024-06-06T08:43:13.699 INFO:tasks.workunit.client.0.smithi027.stdout:6/47: dwrite f0 [0,4194304] 0 2024-06-06T08:43:13.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/48: mknod d4/c16 0 2024-06-06T08:43:13.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/49: mkdir d4/d13/d17 0 2024-06-06T08:43:13.705 INFO:tasks.workunit.client.0.smithi027.stdout:6/50: rename d4/d7 to d4/d9/dc/d18 0 2024-06-06T08:43:13.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:13 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[32331]: 2024-06-06T08:43:13.588+0000 7fc898df5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T08:43:13.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:13 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:13.709 INFO:tasks.workunit.client.0.smithi027.stdout:6/51: chown d4/l6 431925 1 2024-06-06T08:43:13.709 INFO:tasks.workunit.client.0.smithi027.stdout:6/52: readlink d4/lf 0 2024-06-06T08:43:13.724 INFO:tasks.workunit.client.0.smithi027.stdout:6/53: dwrite d4/f5 [0,4194304] 0 2024-06-06T08:43:13.725 INFO:tasks.workunit.client.0.smithi027.stdout:6/54: write f0 [5216908,120342] 0 2024-06-06T08:43:13.730 INFO:tasks.workunit.client.0.smithi027.stdout:6/55: getdents d4/d9/dc 0 2024-06-06T08:43:13.730 INFO:tasks.workunit.client.0.smithi027.stdout:6/56: chown d4/d9/ld 11 1 2024-06-06T08:43:13.730 INFO:tasks.workunit.client.0.smithi027.stdout:6/57: truncate d4/f5 5619733 0 2024-06-06T08:43:13.730 INFO:tasks.workunit.client.0.smithi027.stdout:6/58: write f0 [5149949,58600] 0 2024-06-06T08:43:13.732 INFO:tasks.workunit.client.0.smithi027.stdout:6/59: truncate d4/f5 1217803 0 2024-06-06T08:43:13.732 INFO:tasks.workunit.client.0.smithi027.stdout:6/60: getdents d4/d13/d17 0 2024-06-06T08:43:13.747 INFO:tasks.workunit.client.0.smithi027.stdout:6/61: dwrite f3 [0,4194304] 0 2024-06-06T08:43:13.752 INFO:tasks.workunit.client.0.smithi027.stdout:6/62: write f3 [4412582,33070] 0 2024-06-06T08:43:13.763 INFO:tasks.workunit.client.0.smithi027.stdout:6/63: read f0 [1754793,11299] 0 2024-06-06T08:43:13.763 INFO:tasks.workunit.client.0.smithi027.stdout:6/64: truncate f0 6169963 0 2024-06-06T08:43:13.764 INFO:tasks.workunit.client.0.smithi027.stdout:6/65: chown d4/de/f11 226814 1 2024-06-06T08:43:13.780 INFO:tasks.workunit.client.0.smithi027.stdout:6/66: dwrite d4/de/f11 [0,4194304] 0 2024-06-06T08:43:13.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/67: dwrite f3 [0,4194304] 0 2024-06-06T08:43:13.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/68: write f0 [2835890,127075] 0 2024-06-06T08:43:13.800 INFO:tasks.workunit.client.0.smithi027.stdout:6/69: truncate f3 760291 0 2024-06-06T08:43:13.818 INFO:tasks.workunit.client.0.smithi027.stdout:6/70: dwrite f0 [4194304,4194304] 0 2024-06-06T08:43:13.819 INFO:tasks.workunit.client.0.smithi027.stdout:6/71: creat d4/d9/f19 x:0 0 0 2024-06-06T08:43:13.828 INFO:tasks.workunit.client.0.smithi027.stdout:6/72: dread f0 [4194304,4194304] 0 2024-06-06T08:43:13.832 INFO:tasks.workunit.client.0.smithi027.stdout:6/73: dread d4/f5 [0,4194304] 0 2024-06-06T08:43:13.832 INFO:tasks.workunit.client.0.smithi027.stdout:6/74: chown d4/c16 733386273 1 2024-06-06T08:43:13.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/75: fsync d4/f5 0 2024-06-06T08:43:13.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/76: getdents d4/d13 0 2024-06-06T08:43:13.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/77: write f0 [5554518,51101] 0 2024-06-06T08:43:13.841 INFO:tasks.workunit.client.0.smithi027.stdout:6/78: mkdir d4/d1a 0 2024-06-06T08:43:13.854 INFO:tasks.workunit.client.0.smithi027.stdout:7/100: sync 2024-06-06T08:43:13.859 INFO:tasks.workunit.client.0.smithi027.stdout:6/79: dwrite d4/de/f11 [0,4194304] 0 2024-06-06T08:43:13.873 INFO:tasks.workunit.client.0.smithi027.stdout:7/101: dwrite d17/f1b [0,4194304] 0 2024-06-06T08:43:13.874 INFO:tasks.workunit.client.0.smithi027.stdout:7/102: creat d17/f21 x:0 0 0 2024-06-06T08:43:13.876 INFO:tasks.workunit.client.0.smithi027.stdout:7/103: symlink d17/l22 0 2024-06-06T08:43:13.876 INFO:tasks.workunit.client.0.smithi027.stdout:7/104: write d17/f21 [745163,70837] 0 2024-06-06T08:43:13.877 INFO:tasks.workunit.client.0.smithi027.stdout:7/105: write ff [86877,99241] 0 2024-06-06T08:43:13.880 INFO:tasks.workunit.client.0.smithi027.stdout:6/80: dwrite f0 [0,4194304] 0 2024-06-06T08:43:13.896 INFO:tasks.workunit.client.0.smithi027.stdout:7/106: dwrite d17/f21 [0,4194304] 0 2024-06-06T08:43:13.896 INFO:tasks.workunit.client.0.smithi027.stdout:7/107: fsync f15 0 2024-06-06T08:43:13.913 INFO:tasks.workunit.client.0.smithi027.stdout:7/108: dwrite d17/f1b [0,4194304] 0 2024-06-06T08:43:13.915 INFO:tasks.workunit.client.0.smithi027.stdout:7/109: creat d17/f23 x:0 0 0 2024-06-06T08:43:13.915 INFO:tasks.workunit.client.0.smithi027.stdout:7/110: stat ff 0 2024-06-06T08:43:13.915 INFO:tasks.workunit.client.0.smithi027.stdout:7/111: write d17/f1d [743665,43840] 0 2024-06-06T08:43:13.919 INFO:tasks.workunit.client.0.smithi027.stdout:7/112: getdents d17 0 2024-06-06T08:43:13.921 INFO:tasks.workunit.client.0.smithi027.stdout:7/113: symlink d17/l24 0 2024-06-06T08:43:13.921 INFO:tasks.workunit.client.0.smithi027.stdout:7/114: fdatasync f11 0 2024-06-06T08:43:13.921 INFO:tasks.workunit.client.0.smithi027.stdout:7/115: mknod d17/c25 0 2024-06-06T08:43:13.922 INFO:tasks.workunit.client.0.smithi027.stdout:7/116: dread - d17/f23 zero size 2024-06-06T08:43:13.923 INFO:tasks.workunit.client.0.smithi027.stdout:7/117: creat d17/f26 x:0 0 0 2024-06-06T08:43:13.941 INFO:tasks.workunit.client.0.smithi027.stdout:7/118: dwrite d17/f1f [0,4194304] 0 2024-06-06T08:43:13.943 INFO:tasks.workunit.client.0.smithi027.stdout:7/119: creat d17/f27 x:0 0 0 2024-06-06T08:43:13.944 INFO:tasks.workunit.client.0.smithi027.stdout:7/120: rmdir d17 39 2024-06-06T08:43:13.945 INFO:tasks.workunit.client.0.smithi027.stdout:7/121: read - d17/f1e zero size 2024-06-06T08:43:13.945 INFO:tasks.workunit.client.0.smithi027.stdout:7/122: dread - d17/f26 zero size 2024-06-06T08:43:13.946 INFO:tasks.workunit.client.0.smithi027.stdout:7/123: mkdir d17/d28 0 2024-06-06T08:43:13.947 INFO:tasks.workunit.client.0.smithi027.stdout:7/124: creat d17/f29 x:0 0 0 2024-06-06T08:43:13.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/125: dwrite d17/f26 [0,4194304] 0 2024-06-06T08:43:13.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/126: chown d17/f27 2 1 2024-06-06T08:43:13.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:13 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:14.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:13 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:14.849 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:43:14.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:14 smithi027 ceph-mon[32335]: pgmap v92: 129 pgs: 129 active+clean; 111 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 1.99k op/s 2024-06-06T08:43:14.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:14 smithi027 ceph-mon[32335]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T08:43:14.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:14 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] up:active 2024-06-06T08:43:14.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:14 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:14.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:14 smithi203 ceph-mon[36579]: pgmap v92: 129 pgs: 129 active+clean; 111 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 1.99k op/s 2024-06-06T08:43:14.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:14 smithi203 ceph-mon[36579]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T08:43:14.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:14 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] up:active 2024-06-06T08:43:14.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:14 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:15.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:14 smithi176 ceph-mon[36786]: pgmap v92: 129 pgs: 129 active+clean; 111 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 1.99k op/s 2024-06-06T08:43:15.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:14 smithi176 ceph-mon[36786]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T08:43:15.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:14 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] up:active 2024-06-06T08:43:15.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:14 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:15.190 INFO:tasks.workunit.client.0.smithi027.stdout:7/127: sync 2024-06-06T08:43:15.190 INFO:tasks.workunit.client.0.smithi027.stdout:2/40: sync 2024-06-06T08:43:15.190 INFO:tasks.workunit.client.0.smithi027.stdout:8/21: sync 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/51: sync 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:8/22: truncate - no filename 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:8/23: dwrite - no filename 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:8/24: dread - no filename 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:5/118: sync 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:9/11: sync 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/52: stat l1 0 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:0/31: sync 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/53: dread - dc/fd zero size 2024-06-06T08:43:15.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/54: stat l8 0 2024-06-06T08:43:15.192 INFO:tasks.workunit.client.0.smithi027.stdout:4/62: sync 2024-06-06T08:43:15.192 INFO:tasks.workunit.client.0.smithi027.stdout:4/63: dread - d0/d1/f12 zero size 2024-06-06T08:43:15.192 INFO:tasks.workunit.client.0.smithi027.stdout:3/14: sync 2024-06-06T08:43:15.192 INFO:tasks.workunit.client.0.smithi027.stdout:6/81: sync 2024-06-06T08:43:15.192 INFO:tasks.workunit.client.0.smithi027.stdout:3/15: write d1/f5 [823690,82938] 0 2024-06-06T08:43:15.192 INFO:tasks.workunit.client.0.smithi027.stdout:2/41: truncate d6/f9 92080 0 2024-06-06T08:43:15.194 INFO:tasks.workunit.client.0.smithi027.stdout:7/128: mkdir d17/d28/d2a 0 2024-06-06T08:43:15.194 INFO:tasks.workunit.client.0.smithi027.stdout:8/25: chown c3 1260 1 2024-06-06T08:43:15.195 INFO:tasks.workunit.client.0.smithi027.stdout:2/42: creat d6/dc/f12 x:0 0 0 2024-06-06T08:43:15.197 INFO:tasks.workunit.client.0.smithi027.stdout:7/129: symlink d17/d28/l2b 0 2024-06-06T08:43:15.200 INFO:tasks.workunit.client.0.smithi027.stdout:8/26: mknod c4 0 2024-06-06T08:43:15.201 INFO:tasks.workunit.client.0.smithi027.stdout:8/27: dread - no filename 2024-06-06T08:43:15.203 INFO:tasks.workunit.client.0.smithi027.stdout:8/28: creat f5 x:0 0 0 2024-06-06T08:43:15.203 INFO:tasks.workunit.client.0.smithi027.stdout:8/29: dread - f5 zero size 2024-06-06T08:43:15.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/30: fsync f5 0 2024-06-06T08:43:15.209 INFO:tasks.workunit.client.0.smithi027.stdout:5/119: dread d0/f4 [0,4194304] 0 2024-06-06T08:43:15.210 INFO:tasks.workunit.client.0.smithi027.stdout:4/64: dread d0/f13 [4194304,4194304] 0 2024-06-06T08:43:15.211 INFO:tasks.workunit.client.0.smithi027.stdout:8/31: creat f6 x:0 0 0 2024-06-06T08:43:15.211 INFO:tasks.workunit.client.0.smithi027.stdout:8/32: dread - f6 zero size 2024-06-06T08:43:15.211 INFO:tasks.workunit.client.0.smithi027.stdout:8/33: readlink - no filename 2024-06-06T08:43:15.213 INFO:tasks.workunit.client.0.smithi027.stdout:5/120: rename d0/da/f18 to d0/da/db/f22 0 2024-06-06T08:43:15.213 INFO:tasks.workunit.client.0.smithi027.stdout:5/121: fsync d0/f3 0 2024-06-06T08:43:15.213 INFO:tasks.workunit.client.0.smithi027.stdout:3/16: dwrite d1/f2 [4194304,4194304] 0 2024-06-06T08:43:15.214 INFO:tasks.workunit.client.0.smithi027.stdout:4/65: link d0/fd d0/d1/f14 0 2024-06-06T08:43:15.217 INFO:tasks.workunit.client.0.smithi027.stdout:3/17: rename d1/f5 to d1/f6 0 2024-06-06T08:43:15.217 INFO:tasks.workunit.client.0.smithi027.stdout:6/82: dread d4/de/f11 [0,4194304] 0 2024-06-06T08:43:15.218 INFO:tasks.workunit.client.0.smithi027.stdout:9/12: dwrite d3/f4 [0,4194304] 0 2024-06-06T08:43:15.223 INFO:tasks.workunit.client.0.smithi027.stdout:3/18: dread d1/f6 [0,4194304] 0 2024-06-06T08:43:15.223 INFO:tasks.workunit.client.0.smithi027.stdout:4/66: mknod d0/d1/d6/d8/c15 0 2024-06-06T08:43:15.224 INFO:tasks.workunit.client.0.smithi027.stdout:3/19: truncate d1/f6 777532 0 2024-06-06T08:43:15.224 INFO:tasks.workunit.client.0.smithi027.stdout:4/67: getdents d0 0 2024-06-06T08:43:15.226 INFO:tasks.workunit.client.0.smithi027.stdout:4/68: truncate d0/d1/fb 4193446 0 2024-06-06T08:43:15.226 INFO:tasks.workunit.client.0.smithi027.stdout:7/130: write d17/f1b [3237780,107059] 0 2024-06-06T08:43:15.227 INFO:tasks.workunit.client.0.smithi027.stdout:7/131: mkdir d17/d28/d2c 0 2024-06-06T08:43:15.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/132: fsync d17/f21 0 2024-06-06T08:43:15.229 INFO:tasks.workunit.client.0.smithi027.stdout:7/133: mkdir d17/d28/d2c/d2d 0 2024-06-06T08:43:15.231 INFO:tasks.workunit.client.0.smithi027.stdout:9/13: dread d3/f4 [0,4194304] 0 2024-06-06T08:43:15.231 INFO:tasks.workunit.client.0.smithi027.stdout:9/14: stat d3/f4 0 2024-06-06T08:43:15.231 INFO:tasks.workunit.client.0.smithi027.stdout:9/15: fdatasync d3/f4 0 2024-06-06T08:43:15.235 INFO:tasks.workunit.client.0.smithi027.stdout:2/43: dwrite d6/ff [0,4194304] 0 2024-06-06T08:43:15.235 INFO:tasks.workunit.client.0.smithi027.stdout:2/44: fdatasync f3 0 2024-06-06T08:43:15.237 INFO:tasks.workunit.client.0.smithi027.stdout:9/16: dread d3/f4 [0,4194304] 0 2024-06-06T08:43:15.238 INFO:tasks.workunit.client.0.smithi027.stdout:9/17: rename d3 to d3/d5 22 2024-06-06T08:43:15.238 INFO:tasks.workunit.client.0.smithi027.stdout:2/45: getdents d6/dc 0 2024-06-06T08:43:15.238 INFO:tasks.workunit.client.0.smithi027.stdout:2/46: truncate d6/f9 242882 0 2024-06-06T08:43:15.238 INFO:tasks.workunit.client.0.smithi027.stdout:2/47: chown d6/ca 426598 1 2024-06-06T08:43:15.238 INFO:tasks.workunit.client.0.smithi027.stdout:2/48: chown f3 0 1 2024-06-06T08:43:15.239 INFO:tasks.workunit.client.0.smithi027.stdout:9/18: truncate d3/f4 4171359 0 2024-06-06T08:43:15.239 INFO:tasks.workunit.client.0.smithi027.stdout:7/134: dread d17/f1d [0,4194304] 0 2024-06-06T08:43:15.242 INFO:tasks.workunit.client.0.smithi027.stdout:2/49: rename d6/f9 to d6/dd/f13 0 2024-06-06T08:43:15.242 INFO:tasks.workunit.client.0.smithi027.stdout:2/50: write f1 [915399,101212] 0 2024-06-06T08:43:15.242 INFO:tasks.workunit.client.0.smithi027.stdout:2/51: stat d6/fe 0 2024-06-06T08:43:15.242 INFO:tasks.workunit.client.0.smithi027.stdout:2/52: chown f3 0 1 2024-06-06T08:43:15.245 INFO:tasks.workunit.client.0.smithi027.stdout:9/19: symlink d3/l6 0 2024-06-06T08:43:15.246 INFO:tasks.workunit.client.0.smithi027.stdout:3/20: dread d1/f2 [4194304,4194304] 0 2024-06-06T08:43:15.247 INFO:tasks.workunit.client.0.smithi027.stdout:7/135: creat d17/f2e x:0 0 0 2024-06-06T08:43:15.248 INFO:tasks.workunit.client.0.smithi027.stdout:8/34: dwrite f6 [0,4194304] 0 2024-06-06T08:43:15.248 INFO:tasks.workunit.client.0.smithi027.stdout:7/136: write fd [360570,56836] 0 2024-06-06T08:43:15.249 INFO:tasks.workunit.client.0.smithi027.stdout:2/53: link f3 d6/f14 0 2024-06-06T08:43:15.250 INFO:tasks.workunit.client.0.smithi027.stdout:9/20: mknod d3/c7 0 2024-06-06T08:43:15.250 INFO:tasks.workunit.client.0.smithi027.stdout:9/21: fdatasync d3/f4 0 2024-06-06T08:43:15.251 INFO:tasks.workunit.client.0.smithi027.stdout:5/122: write d0/da/f12 [1120692,13117] 0 2024-06-06T08:43:15.251 INFO:tasks.workunit.client.0.smithi027.stdout:0/32: dwrite f1 [0,4194304] 0 2024-06-06T08:43:15.252 INFO:tasks.workunit.client.0.smithi027.stdout:3/21: creat d1/f7 x:0 0 0 2024-06-06T08:43:15.255 INFO:tasks.workunit.client.0.smithi027.stdout:7/137: creat d17/d28/d2c/d2d/f2f x:0 0 0 2024-06-06T08:43:15.256 INFO:tasks.workunit.client.0.smithi027.stdout:2/54: rename c2 to d6/dd/c15 0 2024-06-06T08:43:15.257 INFO:tasks.workunit.client.0.smithi027.stdout:3/22: mkdir d1/d8 0 2024-06-06T08:43:15.258 INFO:tasks.workunit.client.0.smithi027.stdout:8/35: mkdir d7 0 2024-06-06T08:43:15.259 INFO:tasks.workunit.client.0.smithi027.stdout:4/69: read d0/d1/ff [3237529,12028] 0 2024-06-06T08:43:15.259 INFO:tasks.workunit.client.0.smithi027.stdout:6/83: dwrite f0 [0,4194304] 0 2024-06-06T08:43:15.259 INFO:tasks.workunit.client.0.smithi027.stdout:7/138: link c13 d17/d28/d2a/c30 0 2024-06-06T08:43:15.260 INFO:tasks.workunit.client.0.smithi027.stdout:9/22: write d3/f4 [4002352,48397] 0 2024-06-06T08:43:15.261 INFO:tasks.workunit.client.0.smithi027.stdout:2/55: creat d6/f16 x:0 0 0 2024-06-06T08:43:15.262 INFO:tasks.workunit.client.0.smithi027.stdout:3/23: creat d1/d8/f9 x:0 0 0 2024-06-06T08:43:15.264 INFO:tasks.workunit.client.0.smithi027.stdout:0/33: write f7 [621515,100764] 0 2024-06-06T08:43:15.264 INFO:tasks.workunit.client.0.smithi027.stdout:6/84: unlink f3 0 2024-06-06T08:43:15.267 INFO:tasks.workunit.client.0.smithi027.stdout:9/23: symlink d3/l8 0 2024-06-06T08:43:15.267 INFO:tasks.workunit.client.0.smithi027.stdout:2/56: unlink d6/f7 0 2024-06-06T08:43:15.269 INFO:tasks.workunit.client.0.smithi027.stdout:0/34: rename fa to fb 0 2024-06-06T08:43:15.270 INFO:tasks.workunit.client.0.smithi027.stdout:4/70: write d0/d1/f14 [1661246,48881] 0 2024-06-06T08:43:15.272 INFO:tasks.workunit.client.0.smithi027.stdout:2/57: rename d6/ff to d6/dd/f17 0 2024-06-06T08:43:15.272 INFO:tasks.workunit.client.0.smithi027.stdout:8/36: dread f6 [0,4194304] 0 2024-06-06T08:43:15.273 INFO:tasks.workunit.client.0.smithi027.stdout:0/35: rename f8 to fc 0 2024-06-06T08:43:15.275 INFO:tasks.workunit.client.0.smithi027.stdout:2/58: rename d6/f8 to d6/dd/f18 0 2024-06-06T08:43:15.275 INFO:tasks.workunit.client.0.smithi027.stdout:5/123: dwrite d0/fe [4194304,4194304] 0 2024-06-06T08:43:15.278 INFO:tasks.workunit.client.0.smithi027.stdout:3/24: dread d1/f2 [0,4194304] 0 2024-06-06T08:43:15.278 INFO:tasks.workunit.client.0.smithi027.stdout:0/36: symlink ld 0 2024-06-06T08:43:15.278 INFO:tasks.workunit.client.0.smithi027.stdout:0/37: read - fb zero size 2024-06-06T08:43:15.278 INFO:tasks.workunit.client.0.smithi027.stdout:0/38: truncate f2 346107 0 2024-06-06T08:43:15.279 INFO:tasks.workunit.client.0.smithi027.stdout:0/39: stat fb 0 2024-06-06T08:43:15.279 INFO:tasks.workunit.client.0.smithi027.stdout:0/40: fdatasync f2 0 2024-06-06T08:43:15.280 INFO:tasks.workunit.client.0.smithi027.stdout:2/59: creat d6/dd/f19 x:0 0 0 2024-06-06T08:43:15.280 INFO:tasks.workunit.client.0.smithi027.stdout:2/60: stat d6/dd/f18 0 2024-06-06T08:43:15.281 INFO:tasks.workunit.client.0.smithi027.stdout:5/124: symlink d0/l23 0 2024-06-06T08:43:15.283 INFO:tasks.workunit.client.0.smithi027.stdout:3/25: mkdir d1/da 0 2024-06-06T08:43:15.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/24: dread d3/f4 [0,4194304] 0 2024-06-06T08:43:15.285 INFO:tasks.workunit.client.0.smithi027.stdout:6/85: dwrite d4/de/f11 [4194304,4194304] 0 2024-06-06T08:43:15.285 INFO:tasks.workunit.client.0.smithi027.stdout:0/41: unlink f6 0 2024-06-06T08:43:15.286 INFO:tasks.workunit.client.0.smithi027.stdout:2/61: symlink d6/dd/l1a 0 2024-06-06T08:43:15.287 INFO:tasks.workunit.client.0.smithi027.stdout:3/26: link d1/f6 d1/da/fb 0 2024-06-06T08:43:15.289 INFO:tasks.workunit.client.0.smithi027.stdout:0/42: link l3 le 0 2024-06-06T08:43:15.291 INFO:tasks.workunit.client.0.smithi027.stdout:2/62: rename d6/ca to d6/c1b 0 2024-06-06T08:43:15.291 INFO:tasks.workunit.client.0.smithi027.stdout:9/25: dread d3/f4 [0,4194304] 0 2024-06-06T08:43:15.292 INFO:tasks.workunit.client.0.smithi027.stdout:7/139: dread d17/f1b [0,4194304] 0 2024-06-06T08:43:15.293 INFO:tasks.workunit.client.0.smithi027.stdout:3/27: mknod d1/da/cc 0 2024-06-06T08:43:15.294 INFO:tasks.workunit.client.0.smithi027.stdout:0/43: symlink lf 0 2024-06-06T08:43:15.294 INFO:tasks.workunit.client.0.smithi027.stdout:0/44: read fc [2564136,75370] 0 2024-06-06T08:43:15.296 INFO:tasks.workunit.client.0.smithi027.stdout:9/26: creat d3/f9 x:0 0 0 2024-06-06T08:43:15.300 INFO:tasks.workunit.client.0.smithi027.stdout:3/28: mknod d1/d8/cd 0 2024-06-06T08:43:15.303 INFO:tasks.workunit.client.0.smithi027.stdout:7/140: rename d17/d28 to d17/d31 0 2024-06-06T08:43:15.304 INFO:tasks.workunit.client.0.smithi027.stdout:9/27: dread d3/f4 [0,4194304] 0 2024-06-06T08:43:15.304 INFO:tasks.workunit.client.0.smithi027.stdout:8/37: dwrite f5 [0,4194304] 0 2024-06-06T08:43:15.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/28: mkdir d3/da 0 2024-06-06T08:43:15.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/29: write d3/f9 [185183,36689] 0 2024-06-06T08:43:15.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/30: write d3/f4 [2002453,13313] 0 2024-06-06T08:43:15.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/31: write d3/f9 [849240,53349] 0 2024-06-06T08:43:15.307 INFO:tasks.workunit.client.0.smithi027.stdout:5/125: dwrite d0/da/db/f17 [0,4194304] 0 2024-06-06T08:43:15.310 INFO:tasks.workunit.client.0.smithi027.stdout:9/32: mknod d3/cb 0 2024-06-06T08:43:15.310 INFO:tasks.workunit.client.0.smithi027.stdout:9/33: write d3/f4 [2083411,69754] 0 2024-06-06T08:43:15.310 INFO:tasks.workunit.client.0.smithi027.stdout:9/34: readlink d3/l6 0 2024-06-06T08:43:15.311 INFO:tasks.workunit.client.0.smithi027.stdout:6/86: dwrite d4/de/f11 [0,4194304] 0 2024-06-06T08:43:15.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/141: dread ff [0,4194304] 0 2024-06-06T08:43:15.319 INFO:tasks.workunit.client.0.smithi027.stdout:0/45: dwrite f2 [0,4194304] 0 2024-06-06T08:43:15.321 INFO:tasks.workunit.client.0.smithi027.stdout:2/63: dwrite d6/fe [0,4194304] 0 2024-06-06T08:43:15.330 INFO:tasks.workunit.client.0.smithi027.stdout:4/71: link d0/d1/d6/d8/c15 d0/c16 0 2024-06-06T08:43:15.330 INFO:tasks.workunit.client.0.smithi027.stdout:4/72: fdatasync d0/d1/f12 0 2024-06-06T08:43:15.333 INFO:tasks.workunit.client.0.smithi027.stdout:8/38: dwrite f6 [4194304,4194304] 0 2024-06-06T08:43:15.333 INFO:tasks.workunit.client.0.smithi027.stdout:8/39: chown d7 353 1 2024-06-06T08:43:15.340 INFO:tasks.workunit.client.0.smithi027.stdout:9/35: stat l0 0 2024-06-06T08:43:15.340 INFO:tasks.workunit.client.0.smithi027.stdout:2/64: dread f1 [0,4194304] 0 2024-06-06T08:43:15.340 INFO:tasks.workunit.client.0.smithi027.stdout:2/65: write d6/dd/f19 [751520,16881] 0 2024-06-06T08:43:15.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/29: getdents d1/da 0 2024-06-06T08:43:15.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/142: getdents d17/d31/d2c/d2d 0 2024-06-06T08:43:15.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/143: readlink d17/l24 0 2024-06-06T08:43:15.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/144: fsync d17/f26 0 2024-06-06T08:43:15.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/46: dwrite fb [0,4194304] 0 2024-06-06T08:43:15.348 INFO:tasks.workunit.client.0.smithi027.stdout:6/87: creat d4/de/f1b x:0 0 0 2024-06-06T08:43:15.350 INFO:tasks.workunit.client.0.smithi027.stdout:4/73: dwrite d0/f13 [0,4194304] 0 2024-06-06T08:43:15.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/36: fdatasync d3/f9 0 2024-06-06T08:43:15.355 INFO:tasks.workunit.client.0.smithi027.stdout:2/66: dread d6/f14 [0,4194304] 0 2024-06-06T08:43:15.355 INFO:tasks.workunit.client.0.smithi027.stdout:2/67: truncate d6/dc/f12 1035530 0 2024-06-06T08:43:15.356 INFO:tasks.workunit.client.0.smithi027.stdout:7/145: write d17/f21 [3843939,28359] 0 2024-06-06T08:43:15.370 INFO:tasks.workunit.client.0.smithi027.stdout:8/40: creat d7/f8 x:0 0 0 2024-06-06T08:43:15.370 INFO:tasks.workunit.client.0.smithi027.stdout:8/41: chown f5 3 1 2024-06-06T08:43:15.370 INFO:tasks.workunit.client.0.smithi027.stdout:8/42: chown d7 7 1 2024-06-06T08:43:15.371 INFO:tasks.workunit.client.0.smithi027.stdout:4/74: dwrite d0/d1/fb [0,4194304] 0 2024-06-06T08:43:15.372 INFO:tasks.workunit.client.0.smithi027.stdout:4/75: write d0/d1/fb [5199720,122055] 0 2024-06-06T08:43:15.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/30: mknod d1/ce 0 2024-06-06T08:43:15.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/31: write d1/f7 [313332,66923] 0 2024-06-06T08:43:15.381 INFO:tasks.workunit.client.0.smithi027.stdout:0/47: write fc [384890,92226] 0 2024-06-06T08:43:15.384 INFO:tasks.workunit.client.0.smithi027.stdout:8/43: dwrite f6 [4194304,4194304] 0 2024-06-06T08:43:15.384 INFO:tasks.workunit.client.0.smithi027.stdout:9/37: mkdir d3/da/dc 0 2024-06-06T08:43:15.386 INFO:tasks.workunit.client.0.smithi027.stdout:2/68: getdents d6 0 2024-06-06T08:43:15.391 INFO:tasks.workunit.client.0.smithi027.stdout:8/44: write f6 [4121121,62232] 0 2024-06-06T08:43:15.396 INFO:tasks.workunit.client.0.smithi027.stdout:3/32: truncate d1/f6 141656 0 2024-06-06T08:43:15.398 INFO:tasks.workunit.client.0.smithi027.stdout:4/76: symlink d0/d1/l17 0 2024-06-06T08:43:15.401 INFO:tasks.workunit.client.0.smithi027.stdout:3/33: dread d1/f7 [0,4194304] 0 2024-06-06T08:43:15.401 INFO:tasks.workunit.client.0.smithi027.stdout:3/34: chown d1/d8/cd 38 1 2024-06-06T08:43:15.408 INFO:tasks.workunit.client.0.smithi027.stdout:5/126: rmdir d0 39 2024-06-06T08:43:15.408 INFO:tasks.workunit.client.0.smithi027.stdout:5/127: chown d0/da/db/lf 91588 1 2024-06-06T08:43:15.409 INFO:tasks.workunit.client.0.smithi027.stdout:5/128: chown d0/da/db/f22 9525711 1 2024-06-06T08:43:15.412 INFO:tasks.workunit.client.0.smithi027.stdout:4/77: write d0/d1/fb [2856786,89403] 0 2024-06-06T08:43:15.415 INFO:tasks.workunit.client.0.smithi027.stdout:9/38: symlink d3/da/dc/ld 0 2024-06-06T08:43:15.415 INFO:tasks.workunit.client.0.smithi027.stdout:9/39: chown d3/cb 21667429 1 2024-06-06T08:43:15.416 INFO:tasks.workunit.client.0.smithi027.stdout:2/69: truncate d6/f14 4525338 0 2024-06-06T08:43:15.416 INFO:tasks.workunit.client.0.smithi027.stdout:2/70: write d6/dd/f18 [509197,1664] 0 2024-06-06T08:43:15.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/40: read d3/f9 [496057,107022] 0 2024-06-06T08:43:15.418 INFO:tasks.workunit.client.0.smithi027.stdout:8/45: unlink f6 0 2024-06-06T08:43:15.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/129: truncate d0/da/f13 2068059 0 2024-06-06T08:43:15.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/71: creat d6/dc/f1c x:0 0 0 2024-06-06T08:43:15.421 INFO:tasks.workunit.client.0.smithi027.stdout:9/41: dread d3/f9 [0,4194304] 0 2024-06-06T08:43:15.422 INFO:tasks.workunit.client.0.smithi027.stdout:8/46: creat d7/f9 x:0 0 0 2024-06-06T08:43:15.423 INFO:tasks.workunit.client.0.smithi027.stdout:0/48: rename f5 to f10 0 2024-06-06T08:43:15.424 INFO:tasks.workunit.client.0.smithi027.stdout:5/130: creat d0/f24 x:0 0 0 2024-06-06T08:43:15.425 INFO:tasks.workunit.client.0.smithi027.stdout:2/72: mkdir d6/dc/d1d 0 2024-06-06T08:43:15.426 INFO:tasks.workunit.client.0.smithi027.stdout:8/47: rmdir d7 39 2024-06-06T08:43:15.429 INFO:tasks.workunit.client.0.smithi027.stdout:8/48: write d7/f9 [273155,38545] 0 2024-06-06T08:43:15.430 INFO:tasks.workunit.client.0.smithi027.stdout:0/49: mknod c11 0 2024-06-06T08:43:15.431 INFO:tasks.workunit.client.0.smithi027.stdout:0/50: stat f2 0 2024-06-06T08:43:15.433 INFO:tasks.workunit.client.0.smithi027.stdout:0/51: mknod c12 0 2024-06-06T08:43:15.433 INFO:tasks.workunit.client.0.smithi027.stdout:0/52: dread - f9 zero size 2024-06-06T08:43:15.437 INFO:tasks.workunit.client.0.smithi027.stdout:5/131: write d0/da/db/f19 [2241783,14412] 0 2024-06-06T08:43:15.438 INFO:tasks.workunit.client.0.smithi027.stdout:5/132: creat d0/f25 x:0 0 0 2024-06-06T08:43:15.438 INFO:tasks.workunit.client.0.smithi027.stdout:5/133: truncate d0/da/db/f22 101923 0 2024-06-06T08:43:15.439 INFO:tasks.workunit.client.0.smithi027.stdout:5/134: unlink d0/da/f12 0 2024-06-06T08:43:15.441 INFO:tasks.workunit.client.0.smithi027.stdout:5/135: creat d0/da/f26 x:0 0 0 2024-06-06T08:43:15.442 INFO:tasks.workunit.client.0.smithi027.stdout:5/136: mkdir d0/da/d27 0 2024-06-06T08:43:15.443 INFO:tasks.workunit.client.0.smithi027.stdout:5/137: link d0/da/db/f1b d0/da/db/f28 0 2024-06-06T08:43:15.444 INFO:tasks.workunit.client.0.smithi027.stdout:5/138: creat d0/f29 x:0 0 0 2024-06-06T08:43:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/139: getdents d0/da/d20 0 2024-06-06T08:43:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/140: chown d0/da/db/f22 3135 1 2024-06-06T08:43:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:4/78: dwrite d0/d1/f14 [0,4194304] 0 2024-06-06T08:43:15.446 INFO:tasks.workunit.client.0.smithi027.stdout:5/141: mknod d0/da/d27/c2a 0 2024-06-06T08:43:15.448 INFO:tasks.workunit.client.0.smithi027.stdout:4/79: symlink d0/d1/d6/l18 0 2024-06-06T08:43:15.450 INFO:tasks.workunit.client.0.smithi027.stdout:5/142: unlink d0/da/db/f22 0 2024-06-06T08:43:15.450 INFO:tasks.workunit.client.0.smithi027.stdout:5/143: write d0/f29 [509534,47666] 0 2024-06-06T08:43:15.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/80: creat d0/f19 x:0 0 0 2024-06-06T08:43:15.453 INFO:tasks.workunit.client.0.smithi027.stdout:5/144: mkdir d0/da/d27/d2b 0 2024-06-06T08:43:15.454 INFO:tasks.workunit.client.0.smithi027.stdout:4/81: rmdir d0/d1/d6 39 2024-06-06T08:43:15.454 INFO:tasks.workunit.client.0.smithi027.stdout:4/82: stat d0/f13 0 2024-06-06T08:43:15.456 INFO:tasks.workunit.client.0.smithi027.stdout:9/42: truncate d3/f9 695708 0 2024-06-06T08:43:15.461 INFO:tasks.workunit.client.0.smithi027.stdout:2/73: truncate d6/dd/f13 107761 0 2024-06-06T08:43:15.462 INFO:tasks.workunit.client.0.smithi027.stdout:2/74: mkdir d6/d1e 0 2024-06-06T08:43:15.462 INFO:tasks.workunit.client.0.smithi027.stdout:1/55: sync 2024-06-06T08:43:15.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/56: mkdir dc/de 0 2024-06-06T08:43:15.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/57: chown c9 421 1 2024-06-06T08:43:15.466 INFO:tasks.workunit.client.0.smithi027.stdout:1/58: creat dc/de/ff x:0 0 0 2024-06-06T08:43:15.467 INFO:tasks.workunit.client.0.smithi027.stdout:1/59: truncate f3 4524671 0 2024-06-06T08:43:15.469 INFO:tasks.workunit.client.0.smithi027.stdout:2/75: rename f3 to d6/d1e/f1f 0 2024-06-06T08:43:15.471 INFO:tasks.workunit.client.0.smithi027.stdout:1/60: link f3 dc/f10 0 2024-06-06T08:43:15.471 INFO:tasks.workunit.client.0.smithi027.stdout:1/61: write dc/fd [960881,116840] 0 2024-06-06T08:43:15.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/145: dwrite d0/da/db/f11 [0,4194304] 0 2024-06-06T08:43:15.474 INFO:tasks.workunit.client.0.smithi027.stdout:1/62: mknod dc/de/c11 0 2024-06-06T08:43:15.478 INFO:tasks.workunit.client.0.smithi027.stdout:5/146: mknod d0/c2c 0 2024-06-06T08:43:15.479 INFO:tasks.workunit.client.0.smithi027.stdout:1/63: mknod dc/c12 0 2024-06-06T08:43:15.479 INFO:tasks.workunit.client.0.smithi027.stdout:1/64: chown dc/f10 972079918 1 2024-06-06T08:43:15.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/147: mknod d0/da/c2d 0 2024-06-06T08:43:15.489 INFO:tasks.workunit.client.0.smithi027.stdout:2/76: dwrite d6/dd/f19 [0,4194304] 0 2024-06-06T08:43:15.489 INFO:tasks.workunit.client.0.smithi027.stdout:2/77: chown d6/cb 7 1 2024-06-06T08:43:15.489 INFO:tasks.workunit.client.0.smithi027.stdout:2/78: chown d6/cb 313652 1 2024-06-06T08:43:15.499 INFO:tasks.workunit.client.0.smithi027.stdout:1/65: dwrite dc/de/ff [0,4194304] 0 2024-06-06T08:43:15.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/79: dwrite d6/dd/f17 [0,4194304] 0 2024-06-06T08:43:15.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/80: chown d6/dc 0 1 2024-06-06T08:43:15.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/81: symlink d6/dc/d1d/l20 0 2024-06-06T08:43:15.514 INFO:tasks.workunit.client.0.smithi027.stdout:1/66: read dc/f10 [2201365,62990] 0 2024-06-06T08:43:15.514 INFO:tasks.workunit.client.0.smithi027.stdout:9/43: dwrite d3/f9 [0,4194304] 0 2024-06-06T08:43:15.515 INFO:tasks.workunit.client.0.smithi027.stdout:9/44: truncate d3/f4 4972805 0 2024-06-06T08:43:15.515 INFO:tasks.workunit.client.0.smithi027.stdout:1/67: truncate f3 3422757 0 2024-06-06T08:43:15.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/68: dread dc/fd [0,4194304] 0 2024-06-06T08:43:15.525 INFO:tasks.workunit.client.0.smithi027.stdout:2/82: write d6/fe [3210766,13499] 0 2024-06-06T08:43:15.525 INFO:tasks.workunit.client.0.smithi027.stdout:2/83: chown d6/dd/l1a 15972 1 2024-06-06T08:43:15.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/45: dwrite d3/f4 [0,4194304] 0 2024-06-06T08:43:15.531 INFO:tasks.workunit.client.0.smithi027.stdout:6/88: rename d4/d13/d17 to d4/d9/dc/d1c 0 2024-06-06T08:43:15.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/46: mknod d3/da/ce 0 2024-06-06T08:43:15.533 INFO:tasks.workunit.client.0.smithi027.stdout:6/89: mknod d4/d1a/c1d 0 2024-06-06T08:43:15.533 INFO:tasks.workunit.client.0.smithi027.stdout:6/90: read f0 [7362187,40029] 0 2024-06-06T08:43:15.537 INFO:tasks.workunit.client.0.smithi027.stdout:6/91: link d4/d9/ld d4/d9/dc/l1e 0 2024-06-06T08:43:15.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/84: dwrite d6/dd/f17 [0,4194304] 0 2024-06-06T08:43:15.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/85: write d6/dc/f1c [680945,70373] 0 2024-06-06T08:43:15.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/86: link d6/f16 d6/f21 0 2024-06-06T08:43:15.543 INFO:tasks.workunit.client.0.smithi027.stdout:2/87: creat d6/dc/f22 x:0 0 0 2024-06-06T08:43:15.545 INFO:tasks.workunit.client.0.smithi027.stdout:2/88: truncate d6/d1e/f1f 2455651 0 2024-06-06T08:43:15.545 INFO:tasks.workunit.client.0.smithi027.stdout:9/47: dread d3/f9 [0,4194304] 0 2024-06-06T08:43:15.545 INFO:tasks.workunit.client.0.smithi027.stdout:9/48: chown l1 31765 1 2024-06-06T08:43:15.545 INFO:tasks.workunit.client.0.smithi027.stdout:2/89: read d6/dc/f1c [229917,16119] 0 2024-06-06T08:43:15.545 INFO:tasks.workunit.client.0.smithi027.stdout:9/49: mkdir d3/da/dc/df 0 2024-06-06T08:43:15.548 INFO:tasks.workunit.client.0.smithi027.stdout:2/90: unlink d6/dd/f13 0 2024-06-06T08:43:15.548 INFO:tasks.workunit.client.0.smithi027.stdout:6/92: write f0 [1002665,51871] 0 2024-06-06T08:43:15.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/91: creat d6/dd/f23 x:0 0 0 2024-06-06T08:43:15.552 INFO:tasks.workunit.client.0.smithi027.stdout:6/93: mknod d4/d1a/c1f 0 2024-06-06T08:43:15.555 INFO:tasks.workunit.client.0.smithi027.stdout:2/92: creat d6/dd/f24 x:0 0 0 2024-06-06T08:43:15.557 INFO:tasks.workunit.client.0.smithi027.stdout:2/93: creat d6/dc/f25 x:0 0 0 2024-06-06T08:43:15.557 INFO:tasks.workunit.client.0.smithi027.stdout:2/94: fsync d6/fe 0 2024-06-06T08:43:15.558 INFO:tasks.workunit.client.0.smithi027.stdout:2/95: readlink d6/dc/d1d/l20 0 2024-06-06T08:43:15.558 INFO:tasks.workunit.client.0.smithi027.stdout:2/96: chown d6/dc 3 1 2024-06-06T08:43:15.559 INFO:tasks.workunit.client.0.smithi027.stdout:2/97: creat d6/dd/f26 x:0 0 0 2024-06-06T08:43:15.559 INFO:tasks.workunit.client.0.smithi027.stdout:2/98: stat d6/f16 0 2024-06-06T08:43:15.559 INFO:tasks.workunit.client.0.smithi027.stdout:2/99: dread - d6/f16 zero size 2024-06-06T08:43:15.559 INFO:tasks.workunit.client.0.smithi027.stdout:2/100: stat d6/f21 0 2024-06-06T08:43:15.560 INFO:tasks.workunit.client.0.smithi027.stdout:2/101: creat d6/d1e/f27 x:0 0 0 2024-06-06T08:43:15.560 INFO:tasks.workunit.client.0.smithi027.stdout:2/102: fdatasync d6/dd/f26 0 2024-06-06T08:43:15.560 INFO:tasks.workunit.client.0.smithi027.stdout:2/103: read d6/fe [2899183,110527] 0 2024-06-06T08:43:15.561 INFO:tasks.workunit.client.0.smithi027.stdout:7/146: rename d17/d31/d2c/d2d to d17/d32 0 2024-06-06T08:43:15.561 INFO:tasks.workunit.client.0.smithi027.stdout:6/94: rename d4/d9 to d4/d9/dc/d1c/d20 22 2024-06-06T08:43:15.563 INFO:tasks.workunit.client.0.smithi027.stdout:7/147: creat d17/d32/f33 x:0 0 0 2024-06-06T08:43:15.563 INFO:tasks.workunit.client.0.smithi027.stdout:7/148: truncate d17/f1d 1431954 0 2024-06-06T08:43:15.563 INFO:tasks.workunit.client.0.smithi027.stdout:7/149: chown ff 4950 1 2024-06-06T08:43:15.566 INFO:tasks.workunit.client.0.smithi027.stdout:6/95: write d4/f5 [391480,72765] 0 2024-06-06T08:43:15.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/104: getdents d6/d1e 0 2024-06-06T08:43:15.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/105: dread - d6/dd/f26 zero size 2024-06-06T08:43:15.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/106: write d6/f21 [854735,50451] 0 2024-06-06T08:43:15.567 INFO:tasks.workunit.client.0.smithi027.stdout:6/96: getdents d4/de 0 2024-06-06T08:43:15.568 INFO:tasks.workunit.client.0.smithi027.stdout:6/97: write d4/d9/f19 [190132,59522] 0 2024-06-06T08:43:15.569 INFO:tasks.workunit.client.0.smithi027.stdout:2/107: mknod d6/d1e/c28 0 2024-06-06T08:43:15.570 INFO:tasks.workunit.client.0.smithi027.stdout:2/108: write d6/dc/f25 [615752,68933] 0 2024-06-06T08:43:15.570 INFO:tasks.workunit.client.0.smithi027.stdout:2/109: write d6/dd/f26 [202722,72824] 0 2024-06-06T08:43:15.570 INFO:tasks.workunit.client.0.smithi027.stdout:6/98: creat d4/d13/f21 x:0 0 0 2024-06-06T08:43:15.571 INFO:tasks.workunit.client.0.smithi027.stdout:6/99: rmdir d4/d9/dc/d18 39 2024-06-06T08:43:15.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/110: creat d6/dc/f29 x:0 0 0 2024-06-06T08:43:15.572 INFO:tasks.workunit.client.0.smithi027.stdout:6/100: mknod d4/d13/c22 0 2024-06-06T08:43:15.573 INFO:tasks.workunit.client.0.smithi027.stdout:6/101: write f0 [5919739,17371] 0 2024-06-06T08:43:15.573 INFO:tasks.workunit.client.0.smithi027.stdout:6/102: fdatasync d4/d13/f21 0 2024-06-06T08:43:15.573 INFO:tasks.workunit.client.0.smithi027.stdout:6/103: chown d4/de 29 1 2024-06-06T08:43:15.573 INFO:tasks.workunit.client.0.smithi027.stdout:2/111: chown d6/dd/c15 18 1 2024-06-06T08:43:15.576 INFO:tasks.workunit.client.0.smithi027.stdout:2/112: rename d6/dc/f1c to d6/f2a 0 2024-06-06T08:43:15.576 INFO:tasks.workunit.client.0.smithi027.stdout:2/113: stat d6/dc/f12 0 2024-06-06T08:43:15.582 INFO:tasks.workunit.client.0.smithi027.stdout:7/150: dwrite d17/f1d [0,4194304] 0 2024-06-06T08:43:15.584 INFO:tasks.workunit.client.0.smithi027.stdout:7/151: mkdir d17/d32/d34 0 2024-06-06T08:43:15.589 INFO:tasks.workunit.client.0.smithi027.stdout:7/152: creat d17/d31/d2c/f35 x:0 0 0 2024-06-06T08:43:15.591 INFO:tasks.workunit.client.0.smithi027.stdout:7/153: mknod d17/d31/d2a/c36 0 2024-06-06T08:43:15.594 INFO:tasks.workunit.client.0.smithi027.stdout:7/154: mkdir d17/d31/d2c/d37 0 2024-06-06T08:43:15.596 INFO:tasks.workunit.client.0.smithi027.stdout:6/104: dwrite f0 [0,4194304] 0 2024-06-06T08:43:15.596 INFO:tasks.workunit.client.0.smithi027.stdout:6/105: fdatasync d4/de/f11 0 2024-06-06T08:43:15.597 INFO:tasks.workunit.client.0.smithi027.stdout:6/106: dread - d4/d13/f21 zero size 2024-06-06T08:43:15.597 INFO:tasks.workunit.client.0.smithi027.stdout:6/107: write d4/de/f11 [8810132,56071] 0 2024-06-06T08:43:15.597 INFO:tasks.workunit.client.0.smithi027.stdout:6/108: chown d4/d13/c22 14536 1 2024-06-06T08:43:15.600 INFO:tasks.workunit.client.0.smithi027.stdout:2/114: dwrite d6/dc/f25 [0,4194304] 0 2024-06-06T08:43:15.600 INFO:tasks.workunit.client.0.smithi027.stdout:2/115: write d6/dc/f22 [846475,92377] 0 2024-06-06T08:43:15.603 INFO:tasks.workunit.client.0.smithi027.stdout:2/116: mkdir d6/d1e/d2b 0 2024-06-06T08:43:15.603 INFO:tasks.workunit.client.0.smithi027.stdout:2/117: chown d6/dd/l1a 1 1 2024-06-06T08:43:15.605 INFO:tasks.workunit.client.0.smithi027.stdout:2/118: creat d6/f2c x:0 0 0 2024-06-06T08:43:15.606 INFO:tasks.workunit.client.0.smithi027.stdout:2/119: mknod d6/c2d 0 2024-06-06T08:43:15.607 INFO:tasks.workunit.client.0.smithi027.stdout:2/120: creat d6/f2e x:0 0 0 2024-06-06T08:43:15.612 INFO:tasks.workunit.client.0.smithi027.stdout:6/109: link d4/lf d4/d9/dc/l23 0 2024-06-06T08:43:15.613 INFO:tasks.workunit.client.0.smithi027.stdout:6/110: creat d4/d9/dc/d18/f24 x:0 0 0 2024-06-06T08:43:15.613 INFO:tasks.workunit.client.0.smithi027.stdout:6/111: fsync d4/f5 0 2024-06-06T08:43:15.621 INFO:tasks.workunit.client.0.smithi027.stdout:2/121: dread d6/dd/f17 [0,4194304] 0 2024-06-06T08:43:15.636 INFO:tasks.workunit.client.0.smithi027.stdout:2/122: dwrite d6/dc/f29 [0,4194304] 0 2024-06-06T08:43:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:2/123: dwrite d6/dc/f29 [0,4194304] 0 2024-06-06T08:43:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:2/124: chown d6/f14 10750018 1 2024-06-06T08:43:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:2/125: creat d6/d1e/d2b/f2f x:0 0 0 2024-06-06T08:43:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:2/126: truncate d6/f14 3290251 0 2024-06-06T08:43:15.654 INFO:tasks.workunit.client.0.smithi027.stdout:2/127: fsync d6/dc/f22 0 2024-06-06T08:43:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:2/128: creat d6/dc/f30 x:0 0 0 2024-06-06T08:43:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:2/129: stat d6/dc 0 2024-06-06T08:43:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:2/130: symlink d6/l31 0 2024-06-06T08:43:15.661 INFO:tasks.workunit.client.0.smithi027.stdout:2/131: dread d6/d1e/f1f [0,4194304] 0 2024-06-06T08:43:15.662 INFO:tasks.workunit.client.0.smithi027.stdout:2/132: mknod d6/dc/d1d/c32 0 2024-06-06T08:43:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:2/133: write f5 [524193,80901] 0 2024-06-06T08:43:15.664 INFO:tasks.workunit.client.0.smithi027.stdout:2/134: dread - d6/dc/f30 zero size 2024-06-06T08:43:15.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/135: dread d6/f14 [0,4194304] 0 2024-06-06T08:43:15.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/136: write f5 [1173191,7827] 0 2024-06-06T08:43:15.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/137: chown d6/d1e/d2b/f2f 92438557 1 2024-06-06T08:43:15.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/138: fsync f1 0 2024-06-06T08:43:15.671 INFO:tasks.workunit.client.0.smithi027.stdout:2/139: creat d6/dc/d1d/f33 x:0 0 0 2024-06-06T08:43:15.672 INFO:tasks.workunit.client.0.smithi027.stdout:2/140: rename d6/dc/d1d/c32 to d6/dc/c34 0 2024-06-06T08:43:15.673 INFO:tasks.workunit.client.0.smithi027.stdout:2/141: mknod d6/d1e/c35 0 2024-06-06T08:43:15.682 INFO:tasks.workunit.client.0.smithi027.stdout:2/142: dread d6/fe [0,4194304] 0 2024-06-06T08:43:15.684 INFO:tasks.workunit.client.0.smithi027.stdout:2/143: rename d6/dd/f19 to d6/dc/d1d/f36 0 2024-06-06T08:43:15.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/144: truncate d6/f2a 1565579 0 2024-06-06T08:43:15.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/145: creat d6/d1e/d2b/f37 x:0 0 0 2024-06-06T08:43:15.787 INFO:tasks.workunit.client.0.smithi027.stdout:3/35: sync 2024-06-06T08:43:15.788 INFO:tasks.workunit.client.0.smithi027.stdout:3/36: truncate d1/d8/f9 943256 0 2024-06-06T08:43:15.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/37: symlink d1/lf 0 2024-06-06T08:43:15.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/38: write d1/f7 [601717,22580] 0 2024-06-06T08:43:15.790 INFO:tasks.workunit.client.0.smithi027.stdout:3/39: write d1/f6 [58591,19834] 0 2024-06-06T08:43:15.792 INFO:tasks.workunit.client.0.smithi027.stdout:3/40: fdatasync d1/da/fb 0 2024-06-06T08:43:15.793 INFO:tasks.workunit.client.0.smithi027.stdout:3/41: symlink d1/l10 0 2024-06-06T08:43:15.793 INFO:tasks.workunit.client.0.smithi027.stdout:3/42: write d1/f7 [1038904,103001] 0 2024-06-06T08:43:15.793 INFO:tasks.workunit.client.0.smithi027.stdout:3/43: readlink d1/l4 0 2024-06-06T08:43:15.808 INFO:tasks.workunit.client.0.smithi027.stdout:3/44: dwrite d1/d8/f9 [0,4194304] 0 2024-06-06T08:43:15.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/45: stat d1/da/cc 0 2024-06-06T08:43:15.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/46: fdatasync d1/f2 0 2024-06-06T08:43:15.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/47: fsync d1/d8/f9 0 2024-06-06T08:43:15.811 INFO:tasks.workunit.client.0.smithi027.stdout:3/48: symlink d1/da/l11 0 2024-06-06T08:43:15.812 INFO:tasks.workunit.client.0.smithi027.stdout:3/49: getdents d1/d8 0 2024-06-06T08:43:15.814 INFO:tasks.workunit.client.0.smithi027.stdout:3/50: mkdir d1/d8/d12 0 2024-06-06T08:43:15.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/51: dwrite d1/da/fb [0,4194304] 0 2024-06-06T08:43:15.830 INFO:tasks.workunit.client.0.smithi027.stdout:3/52: creat d1/f13 x:0 0 0 2024-06-06T08:43:15.830 INFO:tasks.workunit.client.0.smithi027.stdout:3/53: fdatasync d1/f13 0 2024-06-06T08:43:15.830 INFO:tasks.workunit.client.0.smithi027.stdout:3/54: fdatasync d1/da/fb 0 2024-06-06T08:43:15.837 INFO:tasks.workunit.client.0.smithi027.stdout:3/55: dread d1/f7 [0,4194304] 0 2024-06-06T08:43:15.841 INFO:tasks.workunit.client.0.smithi027.stdout:3/56: truncate d1/f7 854139 0 2024-06-06T08:43:15.845 INFO:tasks.workunit.client.0.smithi027.stdout:3/57: link d1/f7 d1/d8/d12/f14 0 2024-06-06T08:43:15.847 INFO:tasks.workunit.client.0.smithi027.stdout:3/58: creat d1/d8/f15 x:0 0 0 2024-06-06T08:43:15.849 INFO:tasks.workunit.client.0.smithi027.stdout:3/59: creat d1/d8/f16 x:0 0 0 2024-06-06T08:43:15.865 INFO:tasks.workunit.client.0.smithi027.stdout:3/60: dwrite d1/f6 [0,4194304] 0 2024-06-06T08:43:15.867 INFO:tasks.workunit.client.0.smithi027.stdout:3/61: rmdir d1/d8 39 2024-06-06T08:43:15.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/62: creat d1/d8/f17 x:0 0 0 2024-06-06T08:43:15.885 INFO:tasks.workunit.client.0.smithi027.stdout:3/63: dwrite d1/f2 [0,4194304] 0 2024-06-06T08:43:15.885 INFO:tasks.workunit.client.0.smithi027.stdout:3/64: chown d1/da 20993970 1 2024-06-06T08:43:15.885 INFO:tasks.workunit.client.0.smithi027.stdout:3/65: chown d1/d8/f15 0 1 2024-06-06T08:43:15.895 INFO:tasks.workunit.client.0.smithi027.stdout:3/66: dread d1/da/fb [0,4194304] 0 2024-06-06T08:43:15.895 INFO:tasks.workunit.client.0.smithi027.stdout:3/67: stat d1/d8/f16 0 2024-06-06T08:43:15.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/68: dwrite d1/d8/d12/f14 [0,4194304] 0 2024-06-06T08:43:15.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/69: dread - d1/d8/f15 zero size 2024-06-06T08:43:15.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/70: truncate d1/d8/d12/f14 4907516 0 2024-06-06T08:43:15.922 INFO:tasks.workunit.client.0.smithi027.stdout:8/49: sync 2024-06-06T08:43:15.922 INFO:tasks.workunit.client.0.smithi027.stdout:8/50: fdatasync d7/f9 0 2024-06-06T08:43:15.926 INFO:tasks.workunit.client.0.smithi027.stdout:8/51: dread d7/f9 [0,4194304] 0 2024-06-06T08:43:15.926 INFO:tasks.workunit.client.0.smithi027.stdout:8/52: truncate d7/f8 124755 0 2024-06-06T08:43:15.926 INFO:tasks.workunit.client.0.smithi027.stdout:3/71: dwrite d1/d8/d12/f14 [0,4194304] 0 2024-06-06T08:43:15.927 INFO:tasks.workunit.client.0.smithi027.stdout:8/53: creat d7/fa x:0 0 0 2024-06-06T08:43:15.927 INFO:tasks.workunit.client.0.smithi027.stdout:8/54: readlink - no filename 2024-06-06T08:43:15.927 INFO:tasks.workunit.client.0.smithi027.stdout:8/55: fsync d7/f9 0 2024-06-06T08:43:15.930 INFO:tasks.workunit.client.0.smithi027.stdout:3/72: creat d1/da/f18 x:0 0 0 2024-06-06T08:43:15.937 INFO:tasks.workunit.client.0.smithi027.stdout:3/73: dread d1/f2 [4194304,4194304] 0 2024-06-06T08:43:15.937 INFO:tasks.workunit.client.0.smithi027.stdout:3/74: chown d1/da 26374 1 2024-06-06T08:43:15.937 INFO:tasks.workunit.client.0.smithi027.stdout:3/75: readlink d1/lf 0 2024-06-06T08:43:15.937 INFO:tasks.workunit.client.0.smithi027.stdout:3/76: chown d1/lf 1609090628 1 2024-06-06T08:43:15.938 INFO:tasks.workunit.client.0.smithi027.stdout:3/77: symlink d1/da/l19 0 2024-06-06T08:43:15.940 INFO:tasks.workunit.client.0.smithi027.stdout:3/78: rmdir d1/da 39 2024-06-06T08:43:15.941 INFO:tasks.workunit.client.0.smithi027.stdout:3/79: creat d1/d8/f1a x:0 0 0 2024-06-06T08:43:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:1/69: sync 2024-06-06T08:43:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:6/112: sync 2024-06-06T08:43:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:9/50: sync 2024-06-06T08:43:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:5/148: sync 2024-06-06T08:43:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:7/155: sync 2024-06-06T08:43:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:4/83: sync 2024-06-06T08:43:15.943 INFO:tasks.workunit.client.0.smithi027.stdout:2/146: sync 2024-06-06T08:43:15.943 INFO:tasks.workunit.client.0.smithi027.stdout:0/53: sync 2024-06-06T08:43:15.943 INFO:tasks.workunit.client.0.smithi027.stdout:7/156: chown d17/d32/f33 37931846 1 2024-06-06T08:43:15.943 INFO:tasks.workunit.client.0.smithi027.stdout:0/54: fsync f2 0 2024-06-06T08:43:15.943 INFO:tasks.workunit.client.0.smithi027.stdout:7/157: chown l9 753 1 2024-06-06T08:43:15.943 INFO:tasks.workunit.client.0.smithi027.stdout:7/158: chown d17/f1b 0 1 2024-06-06T08:43:15.943 INFO:tasks.workunit.client.0.smithi027.stdout:3/80: link d1/d8/cd d1/d8/d12/c1b 0 2024-06-06T08:43:15.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/70: creat dc/de/f13 x:0 0 0 2024-06-06T08:43:15.944 INFO:tasks.workunit.client.0.smithi027.stdout:9/51: mknod d3/da/c10 0 2024-06-06T08:43:15.945 INFO:tasks.workunit.client.0.smithi027.stdout:8/56: dwrite f5 [0,4194304] 0 2024-06-06T08:43:15.945 INFO:tasks.workunit.client.0.smithi027.stdout:8/57: read - d7/fa zero size 2024-06-06T08:43:15.945 INFO:tasks.workunit.client.0.smithi027.stdout:5/149: mknod d0/da/d20/c2e 0 2024-06-06T08:43:15.947 INFO:tasks.workunit.client.0.smithi027.stdout:4/84: rename d0/f11 to d0/d1/f1a 0 2024-06-06T08:43:15.947 INFO:tasks.workunit.client.0.smithi027.stdout:0/55: readlink ld 0 2024-06-06T08:43:15.948 INFO:tasks.workunit.client.0.smithi027.stdout:2/147: creat d6/dc/d1d/f38 x:0 0 0 2024-06-06T08:43:15.948 INFO:tasks.workunit.client.0.smithi027.stdout:1/71: mkdir dc/de/d14 0 2024-06-06T08:43:15.950 INFO:tasks.workunit.client.0.smithi027.stdout:4/85: mkdir d0/d1/d1b 0 2024-06-06T08:43:15.951 INFO:tasks.workunit.client.0.smithi027.stdout:2/148: symlink d6/dc/l39 0 2024-06-06T08:43:15.951 INFO:tasks.workunit.client.0.smithi027.stdout:2/149: truncate d6/dc/d1d/f33 846323 0 2024-06-06T08:43:15.952 INFO:tasks.workunit.client.0.smithi027.stdout:7/159: rename d17/f1f to d17/d32/f38 0 2024-06-06T08:43:15.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:15 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] up:active 2024-06-06T08:43:15.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:15 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:15.961 INFO:tasks.workunit.client.0.smithi027.stdout:9/52: write d3/f9 [2945193,100746] 0 2024-06-06T08:43:15.963 INFO:tasks.workunit.client.0.smithi027.stdout:6/113: dwrite d4/f5 [0,4194304] 0 2024-06-06T08:43:15.963 INFO:tasks.workunit.client.0.smithi027.stdout:6/114: fsync d4/de/f1b 0 2024-06-06T08:43:15.965 INFO:tasks.workunit.client.0.smithi027.stdout:6/115: creat d4/f25 x:0 0 0 2024-06-06T08:43:15.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/116: dread d4/f10 [0,4194304] 0 2024-06-06T08:43:15.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/117: read - d4/de/f1b zero size 2024-06-06T08:43:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:5/150: dwrite d0/fe [4194304,4194304] 0 2024-06-06T08:43:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:1/72: dread dc/fd [0,4194304] 0 2024-06-06T08:43:15.977 INFO:tasks.workunit.client.0.smithi027.stdout:5/151: link d0/fe d0/da/d27/f2f 0 2024-06-06T08:43:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:1/73: rename dc/f10 to dc/de/d14/f15 0 2024-06-06T08:43:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:1/74: write f3 [1754346,113966] 0 2024-06-06T08:43:15.979 INFO:tasks.workunit.client.0.smithi027.stdout:1/75: chown dc/de/ff 314 1 2024-06-06T08:43:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:1/76: write dc/de/f13 [266476,45572] 0 2024-06-06T08:43:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:5/152: symlink d0/da/l30 0 2024-06-06T08:43:15.982 INFO:tasks.workunit.client.0.smithi027.stdout:1/77: mknod dc/c16 0 2024-06-06T08:43:15.982 INFO:tasks.workunit.client.0.smithi027.stdout:1/78: chown l7 246182 1 2024-06-06T08:43:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:5/153: rename d0/da/c2d to d0/da/db/c31 0 2024-06-06T08:43:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:5/154: fdatasync d0/fe 0 2024-06-06T08:43:15.984 INFO:tasks.workunit.client.0.smithi027.stdout:5/155: chown d0/da/l1c 15 1 2024-06-06T08:43:15.985 INFO:tasks.workunit.client.0.smithi027.stdout:4/86: dwrite d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:15.985 INFO:tasks.workunit.client.0.smithi027.stdout:5/156: mknod d0/da/d27/c32 0 2024-06-06T08:43:15.986 INFO:tasks.workunit.client.0.smithi027.stdout:4/87: getdents d0/d1/d6 0 2024-06-06T08:43:15.987 INFO:tasks.workunit.client.0.smithi027.stdout:7/160: dwrite f15 [0,4194304] 0 2024-06-06T08:43:15.988 INFO:tasks.workunit.client.0.smithi027.stdout:9/53: dwrite d3/f4 [4194304,4194304] 0 2024-06-06T08:43:15.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:15 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] up:active 2024-06-06T08:43:15.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:15 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:7/161: rmdir d17/d32/d34 0 2024-06-06T08:43:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:7/162: dread - d17/f27 zero size 2024-06-06T08:43:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:7/163: dread - d17/d31/d2c/f35 zero size 2024-06-06T08:43:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:6/118: dwrite d4/de/f1b [0,4194304] 0 2024-06-06T08:43:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:9/54: creat d3/f11 x:0 0 0 2024-06-06T08:43:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/81: dwrite d1/f7 [0,4194304] 0 2024-06-06T08:43:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:8/58: dread f5 [0,4194304] 0 2024-06-06T08:43:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/82: chown d1/d8/f16 45504 1 2024-06-06T08:43:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/83: chown d1/l4 865093136 1 2024-06-06T08:43:15.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/84: chown d1/d8/d12/f14 1640280 1 2024-06-06T08:43:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:5/157: dread d0/da/db/f19 [0,4194304] 0 2024-06-06T08:43:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:5/158: write d0/f24 [732895,56826] 0 2024-06-06T08:43:16.000 INFO:tasks.workunit.client.0.smithi027.stdout:5/159: truncate d0/f25 921233 0 2024-06-06T08:43:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:9/55: rmdir d3/da 39 2024-06-06T08:43:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:9/56: write d3/f9 [4085327,65969] 0 2024-06-06T08:43:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:2/150: dwrite d6/dc/f25 [0,4194304] 0 2024-06-06T08:43:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:6/119: creat d4/d9/f26 x:0 0 0 2024-06-06T08:43:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:8/59: mkdir d7/db 0 2024-06-06T08:43:16.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:15 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] up:active 2024-06-06T08:43:16.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:15 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:5/160: symlink d0/da/db/l33 0 2024-06-06T08:43:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:4/88: dwrite d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:7/164: mkdir d17/d39 0 2024-06-06T08:43:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:7/165: chown f15 7 1 2024-06-06T08:43:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:7/166: chown c13 1489662 1 2024-06-06T08:43:16.010 INFO:tasks.workunit.client.0.smithi027.stdout:7/167: readlink d17/l24 0 2024-06-06T08:43:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:2/151: symlink d6/d1e/d2b/l3a 0 2024-06-06T08:43:16.016 INFO:tasks.workunit.client.0.smithi027.stdout:1/79: fsync f3 0 2024-06-06T08:43:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:8/60: dread f5 [0,4194304] 0 2024-06-06T08:43:16.019 INFO:tasks.workunit.client.0.smithi027.stdout:3/85: dwrite d1/f13 [0,4194304] 0 2024-06-06T08:43:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:5/161: mknod d0/da/db/d1a/c34 0 2024-06-06T08:43:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:5/162: chown d0/da/db/c31 71054625 1 2024-06-06T08:43:16.023 INFO:tasks.workunit.client.0.smithi027.stdout:7/168: creat d17/d32/f3a x:0 0 0 2024-06-06T08:43:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:6/120: symlink d4/d9/dc/l27 0 2024-06-06T08:43:16.025 INFO:tasks.workunit.client.0.smithi027.stdout:7/169: mkdir d17/d31/d2a/d3b 0 2024-06-06T08:43:16.026 INFO:tasks.workunit.client.0.smithi027.stdout:7/170: dread - d17/f27 zero size 2024-06-06T08:43:16.026 INFO:tasks.workunit.client.0.smithi027.stdout:7/171: dread - d17/f2e zero size 2024-06-06T08:43:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:7/172: dread ff [0,4194304] 0 2024-06-06T08:43:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:7/173: write fd [637670,12142] 0 2024-06-06T08:43:16.029 INFO:tasks.workunit.client.0.smithi027.stdout:7/174: chown l16 1269 1 2024-06-06T08:43:16.029 INFO:tasks.workunit.client.0.smithi027.stdout:9/57: getdents d3/da 0 2024-06-06T08:43:16.029 INFO:tasks.workunit.client.0.smithi027.stdout:2/152: mkdir d6/d3b 0 2024-06-06T08:43:16.030 INFO:tasks.workunit.client.0.smithi027.stdout:4/89: dwrite d0/fd [8388608,4194304] 0 2024-06-06T08:43:16.030 INFO:tasks.workunit.client.0.smithi027.stdout:1/80: dread dc/de/d14/f15 [0,4194304] 0 2024-06-06T08:43:16.030 INFO:tasks.workunit.client.0.smithi027.stdout:6/121: creat d4/d9/f28 x:0 0 0 2024-06-06T08:43:16.031 INFO:tasks.workunit.client.0.smithi027.stdout:3/86: rmdir d1/da 39 2024-06-06T08:43:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/87: stat d1/d8/d12/f14 0 2024-06-06T08:43:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/88: truncate d1/d8/d12/f14 5848406 0 2024-06-06T08:43:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/89: fdatasync d1/d8/f15 0 2024-06-06T08:43:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/90: fsync d1/da/fb 0 2024-06-06T08:43:16.034 INFO:tasks.workunit.client.0.smithi027.stdout:7/175: dread - d17/d32/f2f zero size 2024-06-06T08:43:16.035 INFO:tasks.workunit.client.0.smithi027.stdout:9/58: rename l1 to d3/da/dc/df/l12 0 2024-06-06T08:43:16.038 INFO:tasks.workunit.client.0.smithi027.stdout:8/61: dwrite f5 [0,4194304] 0 2024-06-06T08:43:16.038 INFO:tasks.workunit.client.0.smithi027.stdout:8/62: truncate d7/f9 612306 0 2024-06-06T08:43:16.038 INFO:tasks.workunit.client.0.smithi027.stdout:8/63: chown c4 403420588 1 2024-06-06T08:43:16.038 INFO:tasks.workunit.client.0.smithi027.stdout:8/64: chown c4 6 1 2024-06-06T08:43:16.041 INFO:tasks.workunit.client.0.smithi027.stdout:5/163: dwrite d0/f1 [0,4194304] 0 2024-06-06T08:43:16.045 INFO:tasks.workunit.client.0.smithi027.stdout:2/153: mkdir d6/dd/d3c 0 2024-06-06T08:43:16.045 INFO:tasks.workunit.client.0.smithi027.stdout:4/90: unlink d0/d1/d6/d8/cc 0 2024-06-06T08:43:16.046 INFO:tasks.workunit.client.0.smithi027.stdout:8/65: mkdir d7/dc 0 2024-06-06T08:43:16.047 INFO:tasks.workunit.client.0.smithi027.stdout:5/164: link d0/da/f13 d0/da/f35 0 2024-06-06T08:43:16.048 INFO:tasks.workunit.client.0.smithi027.stdout:2/154: creat d6/d1e/f3d x:0 0 0 2024-06-06T08:43:16.049 INFO:tasks.workunit.client.0.smithi027.stdout:8/66: mknod d7/db/cd 0 2024-06-06T08:43:16.052 INFO:tasks.workunit.client.0.smithi027.stdout:5/165: link d0/f4 d0/da/d20/f36 0 2024-06-06T08:43:16.052 INFO:tasks.workunit.client.0.smithi027.stdout:5/166: chown d0/da/d20/c2e 1024450110 1 2024-06-06T08:43:16.052 INFO:tasks.workunit.client.0.smithi027.stdout:1/81: dwrite dc/de/d14/f15 [0,4194304] 0 2024-06-06T08:43:16.053 INFO:tasks.workunit.client.0.smithi027.stdout:2/155: chown d6/d1e/f27 343 1 2024-06-06T08:43:16.053 INFO:tasks.workunit.client.0.smithi027.stdout:2/156: fsync d6/dc/f25 0 2024-06-06T08:43:16.053 INFO:tasks.workunit.client.0.smithi027.stdout:2/157: write d6/d1e/d2b/f2f [995389,66193] 0 2024-06-06T08:43:16.054 INFO:tasks.workunit.client.0.smithi027.stdout:1/82: mknod dc/de/d14/c17 0 2024-06-06T08:43:16.055 INFO:tasks.workunit.client.0.smithi027.stdout:2/158: creat d6/d1e/d2b/f3e x:0 0 0 2024-06-06T08:43:16.056 INFO:tasks.workunit.client.0.smithi027.stdout:1/83: mknod dc/c18 0 2024-06-06T08:43:16.057 INFO:tasks.workunit.client.0.smithi027.stdout:2/159: mkdir d6/dc/d3f 0 2024-06-06T08:43:16.057 INFO:tasks.workunit.client.0.smithi027.stdout:2/160: stat d6/d1e/d2b 0 2024-06-06T08:43:16.059 INFO:tasks.workunit.client.0.smithi027.stdout:1/84: creat dc/de/d14/f19 x:0 0 0 2024-06-06T08:43:16.061 INFO:tasks.workunit.client.0.smithi027.stdout:2/161: creat d6/dc/f40 x:0 0 0 2024-06-06T08:43:16.061 INFO:tasks.workunit.client.0.smithi027.stdout:2/162: write d6/f14 [58371,52445] 0 2024-06-06T08:43:16.062 INFO:tasks.workunit.client.0.smithi027.stdout:3/91: read d1/d8/f9 [2643732,64908] 0 2024-06-06T08:43:16.062 INFO:tasks.workunit.client.0.smithi027.stdout:1/85: rmdir dc/de/d14 39 2024-06-06T08:43:16.065 INFO:tasks.workunit.client.0.smithi027.stdout:4/91: dwrite d0/d1/f1a [0,4194304] 0 2024-06-06T08:43:16.065 INFO:tasks.workunit.client.0.smithi027.stdout:4/92: dread - d0/d1/f12 zero size 2024-06-06T08:43:16.065 INFO:tasks.workunit.client.0.smithi027.stdout:4/93: fdatasync d0/d1/f14 0 2024-06-06T08:43:16.066 INFO:tasks.workunit.client.0.smithi027.stdout:3/92: unlink d1/lf 0 2024-06-06T08:43:16.067 INFO:tasks.workunit.client.0.smithi027.stdout:1/86: truncate f5 228326 0 2024-06-06T08:43:16.069 INFO:tasks.workunit.client.0.smithi027.stdout:6/122: rmdir d4/d9 39 2024-06-06T08:43:16.071 INFO:tasks.workunit.client.0.smithi027.stdout:1/87: dread dc/fd [0,4194304] 0 2024-06-06T08:43:16.072 INFO:tasks.workunit.client.0.smithi027.stdout:3/93: unlink d1/f7 0 2024-06-06T08:43:16.072 INFO:tasks.workunit.client.0.smithi027.stdout:5/167: dwrite d0/da/f35 [0,4194304] 0 2024-06-06T08:43:16.072 INFO:tasks.workunit.client.0.smithi027.stdout:5/168: write d0/da/f35 [5069282,39732] 0 2024-06-06T08:43:16.074 INFO:tasks.workunit.client.0.smithi027.stdout:2/163: dread d6/fe [0,4194304] 0 2024-06-06T08:43:16.074 INFO:tasks.workunit.client.0.smithi027.stdout:6/123: creat d4/f29 x:0 0 0 2024-06-06T08:43:16.075 INFO:tasks.workunit.client.0.smithi027.stdout:2/164: write d6/dc/f12 [1771368,95327] 0 2024-06-06T08:43:16.077 INFO:tasks.workunit.client.0.smithi027.stdout:1/88: creat dc/de/d14/f1a x:0 0 0 2024-06-06T08:43:16.077 INFO:tasks.workunit.client.0.smithi027.stdout:4/94: dread d0/d1/ff [0,4194304] 0 2024-06-06T08:43:16.083 INFO:tasks.workunit.client.0.smithi027.stdout:3/94: rename d1/da/fb to d1/da/f1c 0 2024-06-06T08:43:16.083 INFO:tasks.workunit.client.0.smithi027.stdout:3/95: readlink d1/l10 0 2024-06-06T08:43:16.083 INFO:tasks.workunit.client.0.smithi027.stdout:3/96: truncate d1/da/f18 203917 0 2024-06-06T08:43:16.083 INFO:tasks.workunit.client.0.smithi027.stdout:3/97: truncate d1/d8/f1a 297655 0 2024-06-06T08:43:16.083 INFO:tasks.workunit.client.0.smithi027.stdout:3/98: write d1/f13 [4576041,91067] 0 2024-06-06T08:43:16.091 INFO:tasks.workunit.client.0.smithi027.stdout:2/165: rmdir d6/d1e/d2b 39 2024-06-06T08:43:16.092 INFO:tasks.workunit.client.0.smithi027.stdout:1/89: dread f3 [0,4194304] 0 2024-06-06T08:43:16.092 INFO:tasks.workunit.client.0.smithi027.stdout:1/90: write dc/de/d14/f1a [29588,130118] 0 2024-06-06T08:43:16.093 INFO:tasks.workunit.client.0.smithi027.stdout:3/99: write d1/da/f1c [405315,68489] 0 2024-06-06T08:43:16.099 INFO:tasks.workunit.client.0.smithi027.stdout:3/100: creat d1/d8/f1d x:0 0 0 2024-06-06T08:43:16.101 INFO:tasks.workunit.client.0.smithi027.stdout:6/124: creat d4/d9/dc/d1c/f2a x:0 0 0 2024-06-06T08:43:16.101 INFO:tasks.workunit.client.0.smithi027.stdout:6/125: fdatasync d4/de/f11 0 2024-06-06T08:43:16.101 INFO:tasks.workunit.client.0.smithi027.stdout:6/126: dread - d4/d13/f21 zero size 2024-06-06T08:43:16.102 INFO:tasks.workunit.client.0.smithi027.stdout:2/166: mknod d6/dd/c41 0 2024-06-06T08:43:16.107 INFO:tasks.workunit.client.0.smithi027.stdout:3/101: creat d1/f1e x:0 0 0 2024-06-06T08:43:16.108 INFO:tasks.workunit.client.0.smithi027.stdout:6/127: mknod d4/d13/c2b 0 2024-06-06T08:43:16.108 INFO:tasks.workunit.client.0.smithi027.stdout:6/128: fsync d4/f5 0 2024-06-06T08:43:16.109 INFO:tasks.workunit.client.0.smithi027.stdout:2/167: mknod d6/dc/d3f/c42 0 2024-06-06T08:43:16.110 INFO:tasks.workunit.client.0.smithi027.stdout:3/102: mkdir d1/d1f 0 2024-06-06T08:43:16.111 INFO:tasks.workunit.client.0.smithi027.stdout:1/91: dwrite dc/de/d14/f1a [0,4194304] 0 2024-06-06T08:43:16.111 INFO:tasks.workunit.client.0.smithi027.stdout:1/92: chown dc/de 255028 1 2024-06-06T08:43:16.114 INFO:tasks.workunit.client.0.smithi027.stdout:2/168: unlink d6/f21 0 2024-06-06T08:43:16.115 INFO:tasks.workunit.client.0.smithi027.stdout:2/169: chown d6/fe 1661370 1 2024-06-06T08:43:16.115 INFO:tasks.workunit.client.0.smithi027.stdout:2/170: write d6/f14 [2609713,105423] 0 2024-06-06T08:43:16.118 INFO:tasks.workunit.client.0.smithi027.stdout:2/171: creat d6/d3b/f43 x:0 0 0 2024-06-06T08:43:16.118 INFO:tasks.workunit.client.0.smithi027.stdout:2/172: dread - d6/dc/f30 zero size 2024-06-06T08:43:16.124 INFO:tasks.workunit.client.0.smithi027.stdout:0/56: sync 2024-06-06T08:43:16.124 INFO:tasks.workunit.client.0.smithi027.stdout:7/176: rename c4 to d17/d31/d2a/c3c 0 2024-06-06T08:43:16.125 INFO:tasks.workunit.client.0.smithi027.stdout:0/57: truncate f9 505236 0 2024-06-06T08:43:16.125 INFO:tasks.workunit.client.0.smithi027.stdout:7/177: write d17/f23 [518397,69411] 0 2024-06-06T08:43:16.126 INFO:tasks.workunit.client.0.smithi027.stdout:1/93: dwrite dc/de/d14/f1a [0,4194304] 0 2024-06-06T08:43:16.126 INFO:tasks.workunit.client.0.smithi027.stdout:9/59: rename d3/da/dc/ld to d3/l13 0 2024-06-06T08:43:16.130 INFO:tasks.workunit.client.0.smithi027.stdout:8/67: rename c3 to d7/dc/ce 0 2024-06-06T08:43:16.131 INFO:tasks.workunit.client.0.smithi027.stdout:9/60: link d3/l6 d3/l14 0 2024-06-06T08:43:16.133 INFO:tasks.workunit.client.0.smithi027.stdout:0/58: rename f7 to f13 0 2024-06-06T08:43:16.135 INFO:tasks.workunit.client.0.smithi027.stdout:6/129: rename d4/de/f1b to d4/d1a/f2c 0 2024-06-06T08:43:16.138 INFO:tasks.workunit.client.0.smithi027.stdout:1/94: truncate dc/de/d14/f1a 2506225 0 2024-06-06T08:43:16.139 INFO:tasks.workunit.client.0.smithi027.stdout:8/68: dread f5 [0,4194304] 0 2024-06-06T08:43:16.140 INFO:tasks.workunit.client.0.smithi027.stdout:8/69: fsync d7/f8 0 2024-06-06T08:43:16.140 INFO:tasks.workunit.client.0.smithi027.stdout:2/173: rename d6/f16 to d6/dc/f44 0 2024-06-06T08:43:16.140 INFO:tasks.workunit.client.0.smithi027.stdout:7/178: rmdir d17/d31 39 2024-06-06T08:43:16.141 INFO:tasks.workunit.client.0.smithi027.stdout:7/179: write d17/d32/f33 [629366,96076] 0 2024-06-06T08:43:16.142 INFO:tasks.workunit.client.0.smithi027.stdout:7/180: symlink d17/d31/d2c/d37/l3d 0 2024-06-06T08:43:16.144 INFO:tasks.workunit.client.0.smithi027.stdout:1/95: symlink dc/de/d14/l1b 0 2024-06-06T08:43:16.145 INFO:tasks.workunit.client.0.smithi027.stdout:1/96: stat dc/de/d14/c17 0 2024-06-06T08:43:16.145 INFO:tasks.workunit.client.0.smithi027.stdout:8/70: symlink d7/lf 0 2024-06-06T08:43:16.146 INFO:tasks.workunit.client.0.smithi027.stdout:2/174: creat d6/dc/d1d/f45 x:0 0 0 2024-06-06T08:43:16.147 INFO:tasks.workunit.client.0.smithi027.stdout:6/130: dread d4/f5 [0,4194304] 0 2024-06-06T08:43:16.148 INFO:tasks.workunit.client.0.smithi027.stdout:0/59: rename f1 to f14 0 2024-06-06T08:43:16.150 INFO:tasks.workunit.client.0.smithi027.stdout:1/97: mkdir dc/de/d1c 0 2024-06-06T08:43:16.150 INFO:tasks.workunit.client.0.smithi027.stdout:1/98: getdents dc/de/d1c 0 2024-06-06T08:43:16.150 INFO:tasks.workunit.client.0.smithi027.stdout:8/71: getdents d7/dc 0 2024-06-06T08:43:16.151 INFO:tasks.workunit.client.0.smithi027.stdout:6/131: truncate d4/d1a/f2c 133215 0 2024-06-06T08:43:16.155 INFO:tasks.workunit.client.0.smithi027.stdout:6/132: creat d4/d9/f2d x:0 0 0 2024-06-06T08:43:16.156 INFO:tasks.workunit.client.0.smithi027.stdout:6/133: fsync d4/d1a/f2c 0 2024-06-06T08:43:16.156 INFO:tasks.workunit.client.0.smithi027.stdout:6/134: chown d4/de/c12 725 1 2024-06-06T08:43:16.158 INFO:tasks.workunit.client.0.smithi027.stdout:7/181: rename d17/d39 to d17/d3e 0 2024-06-06T08:43:16.158 INFO:tasks.workunit.client.0.smithi027.stdout:1/99: creat dc/f1d x:0 0 0 2024-06-06T08:43:16.159 INFO:tasks.workunit.client.0.smithi027.stdout:8/72: symlink d7/dc/l10 0 2024-06-06T08:43:16.160 INFO:tasks.workunit.client.0.smithi027.stdout:2/175: mkdir d6/dd/d46 0 2024-06-06T08:43:16.161 INFO:tasks.workunit.client.0.smithi027.stdout:0/60: mkdir d15 0 2024-06-06T08:43:16.161 INFO:tasks.workunit.client.0.smithi027.stdout:6/135: mkdir d4/d1a/d2e 0 2024-06-06T08:43:16.162 INFO:tasks.workunit.client.0.smithi027.stdout:6/136: chown d4/l6 2603426 1 2024-06-06T08:43:16.163 INFO:tasks.workunit.client.0.smithi027.stdout:6/137: creat d4/d9/dc/d18/f2f x:0 0 0 2024-06-06T08:43:16.163 INFO:tasks.workunit.client.0.smithi027.stdout:6/138: fsync d4/d9/f19 0 2024-06-06T08:43:16.163 INFO:tasks.workunit.client.0.smithi027.stdout:6/139: chown d4/d9/dc/d18/ca 11 1 2024-06-06T08:43:16.166 INFO:tasks.workunit.client.0.smithi027.stdout:1/100: link dc/c18 dc/de/d14/c1e 0 2024-06-06T08:43:16.167 INFO:tasks.workunit.client.0.smithi027.stdout:8/73: rmdir d7/db 39 2024-06-06T08:43:16.167 INFO:tasks.workunit.client.0.smithi027.stdout:8/74: chown d7/dc 0 1 2024-06-06T08:43:16.167 INFO:tasks.workunit.client.0.smithi027.stdout:0/61: creat d15/f16 x:0 0 0 2024-06-06T08:43:16.167 INFO:tasks.workunit.client.0.smithi027.stdout:0/62: readlink ld 0 2024-06-06T08:43:16.169 INFO:tasks.workunit.client.0.smithi027.stdout:1/101: unlink f3 0 2024-06-06T08:43:16.170 INFO:tasks.workunit.client.0.smithi027.stdout:6/140: creat d4/d9/dc/d1c/f30 x:0 0 0 2024-06-06T08:43:16.170 INFO:tasks.workunit.client.0.smithi027.stdout:6/141: dread - d4/f25 zero size 2024-06-06T08:43:16.170 INFO:tasks.workunit.client.0.smithi027.stdout:6/142: truncate d4/d9/dc/d1c/f30 185109 0 2024-06-06T08:43:16.170 INFO:tasks.workunit.client.0.smithi027.stdout:6/143: chown d4/d13/f21 389092023 1 2024-06-06T08:43:16.171 INFO:tasks.workunit.client.0.smithi027.stdout:8/75: unlink d7/db/cd 0 2024-06-06T08:43:16.173 INFO:tasks.workunit.client.0.smithi027.stdout:0/63: mkdir d15/d17 0 2024-06-06T08:43:16.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/102: rmdir dc/de 39 2024-06-06T08:43:16.178 INFO:tasks.workunit.client.0.smithi027.stdout:7/182: dwrite d17/d32/f2f [0,4194304] 0 2024-06-06T08:43:16.178 INFO:tasks.workunit.client.0.smithi027.stdout:8/76: creat d7/db/f11 x:0 0 0 2024-06-06T08:43:16.178 INFO:tasks.workunit.client.0.smithi027.stdout:7/183: chown d17/f1e 9 1 2024-06-06T08:43:16.178 INFO:tasks.workunit.client.0.smithi027.stdout:8/77: readlink d7/lf 0 2024-06-06T08:43:16.179 INFO:tasks.workunit.client.0.smithi027.stdout:0/64: mkdir d15/d18 0 2024-06-06T08:43:16.182 INFO:tasks.workunit.client.0.smithi027.stdout:8/78: creat d7/dc/f12 x:0 0 0 2024-06-06T08:43:16.182 INFO:tasks.workunit.client.0.smithi027.stdout:8/79: truncate f5 4593282 0 2024-06-06T08:43:16.182 INFO:tasks.workunit.client.0.smithi027.stdout:8/80: truncate d7/dc/f12 782650 0 2024-06-06T08:43:16.183 INFO:tasks.workunit.client.0.smithi027.stdout:0/65: mknod d15/c19 0 2024-06-06T08:43:16.183 INFO:tasks.workunit.client.0.smithi027.stdout:8/81: mkdir d7/d13 0 2024-06-06T08:43:16.185 INFO:tasks.workunit.client.0.smithi027.stdout:8/82: symlink d7/db/l14 0 2024-06-06T08:43:16.186 INFO:tasks.workunit.client.0.smithi027.stdout:8/83: dread d7/dc/f12 [0,4194304] 0 2024-06-06T08:43:16.190 INFO:tasks.workunit.client.0.smithi027.stdout:2/176: dwrite d6/d3b/f43 [0,4194304] 0 2024-06-06T08:43:16.192 INFO:tasks.workunit.client.0.smithi027.stdout:0/66: dread f13 [0,4194304] 0 2024-06-06T08:43:16.196 INFO:tasks.workunit.client.0.smithi027.stdout:2/177: creat d6/dc/d1d/f47 x:0 0 0 2024-06-06T08:43:16.196 INFO:tasks.workunit.client.0.smithi027.stdout:0/67: symlink d15/l1a 0 2024-06-06T08:43:16.196 INFO:tasks.workunit.client.0.smithi027.stdout:2/178: fdatasync f1 0 2024-06-06T08:43:16.198 INFO:tasks.workunit.client.0.smithi027.stdout:0/68: rmdir d15/d17 0 2024-06-06T08:43:16.200 INFO:tasks.workunit.client.0.smithi027.stdout:0/69: symlink d15/d18/l1b 0 2024-06-06T08:43:16.202 INFO:tasks.workunit.client.0.smithi027.stdout:7/184: dwrite d17/f26 [0,4194304] 0 2024-06-06T08:43:16.202 INFO:tasks.workunit.client.0.smithi027.stdout:7/185: write d17/f26 [4841134,65188] 0 2024-06-06T08:43:16.204 INFO:tasks.workunit.client.0.smithi027.stdout:9/61: dwrite d3/f9 [0,4194304] 0 2024-06-06T08:43:16.205 INFO:tasks.workunit.client.0.smithi027.stdout:6/144: dwrite d4/d1a/f2c [0,4194304] 0 2024-06-06T08:43:16.207 INFO:tasks.workunit.client.0.smithi027.stdout:7/186: rename d17/f29 to d17/f3f 0 2024-06-06T08:43:16.207 INFO:tasks.workunit.client.0.smithi027.stdout:7/187: dread - d17/d31/d2c/f35 zero size 2024-06-06T08:43:16.207 INFO:tasks.workunit.client.0.smithi027.stdout:7/188: write fd [1202377,72459] 0 2024-06-06T08:43:16.208 INFO:tasks.workunit.client.0.smithi027.stdout:9/62: creat d3/f15 x:0 0 0 2024-06-06T08:43:16.209 INFO:tasks.workunit.client.0.smithi027.stdout:0/70: getdents d15 0 2024-06-06T08:43:16.216 INFO:tasks.workunit.client.0.smithi027.stdout:9/63: link c2 d3/da/dc/c16 0 2024-06-06T08:43:16.217 INFO:tasks.workunit.client.0.smithi027.stdout:0/71: creat d15/d18/f1c x:0 0 0 2024-06-06T08:43:16.218 INFO:tasks.workunit.client.0.smithi027.stdout:9/64: mkdir d3/d17 0 2024-06-06T08:43:16.220 INFO:tasks.workunit.client.0.smithi027.stdout:2/179: dwrite d6/f2c [0,4194304] 0 2024-06-06T08:43:16.221 INFO:tasks.workunit.client.0.smithi027.stdout:8/84: dwrite f5 [0,4194304] 0 2024-06-06T08:43:16.221 INFO:tasks.workunit.client.0.smithi027.stdout:6/145: dread d4/f5 [0,4194304] 0 2024-06-06T08:43:16.221 INFO:tasks.workunit.client.0.smithi027.stdout:9/65: rmdir d3/d17 0 2024-06-06T08:43:16.224 INFO:tasks.workunit.client.0.smithi027.stdout:6/146: symlink d4/de/l31 0 2024-06-06T08:43:16.225 INFO:tasks.workunit.client.0.smithi027.stdout:9/66: creat d3/da/dc/f18 x:0 0 0 2024-06-06T08:43:16.225 INFO:tasks.workunit.client.0.smithi027.stdout:9/67: readlink l0 0 2024-06-06T08:43:16.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/68: unlink d3/da/dc/f18 0 2024-06-06T08:43:16.227 INFO:tasks.workunit.client.0.smithi027.stdout:7/189: dread f15 [0,4194304] 0 2024-06-06T08:43:16.229 INFO:tasks.workunit.client.0.smithi027.stdout:9/69: unlink d3/da/dc/c16 0 2024-06-06T08:43:16.231 INFO:tasks.workunit.client.0.smithi027.stdout:9/70: rename d3/da/dc/df/l12 to d3/da/dc/l19 0 2024-06-06T08:43:16.232 INFO:tasks.workunit.client.0.smithi027.stdout:7/190: rmdir d17/d32 39 2024-06-06T08:43:16.232 INFO:tasks.workunit.client.0.smithi027.stdout:7/191: getdents d17/d31/d2a/d3b 0 2024-06-06T08:43:16.234 INFO:tasks.workunit.client.0.smithi027.stdout:8/85: dread f5 [0,4194304] 0 2024-06-06T08:43:16.236 INFO:tasks.workunit.client.0.smithi027.stdout:7/192: truncate f15 1556127 0 2024-06-06T08:43:16.239 INFO:tasks.workunit.client.0.smithi027.stdout:0/72: write f14 [1214319,86381] 0 2024-06-06T08:43:16.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/73: symlink d15/d18/l1d 0 2024-06-06T08:43:16.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/74: write fc [372671,54941] 0 2024-06-06T08:43:16.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/75: fsync f10 0 2024-06-06T08:43:16.241 INFO:tasks.workunit.client.0.smithi027.stdout:7/193: dread d17/d32/f33 [0,4194304] 0 2024-06-06T08:43:16.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/147: write f0 [2101321,60013] 0 2024-06-06T08:43:16.243 INFO:tasks.workunit.client.0.smithi027.stdout:6/148: write d4/d9/dc/d1c/f30 [55788,91305] 0 2024-06-06T08:43:16.243 INFO:tasks.workunit.client.0.smithi027.stdout:6/149: dread - d4/d9/dc/d18/f24 zero size 2024-06-06T08:43:16.243 INFO:tasks.workunit.client.0.smithi027.stdout:6/150: chown d4/c16 801819557 1 2024-06-06T08:43:16.243 INFO:tasks.workunit.client.0.smithi027.stdout:6/151: chown d4/d1a/f2c 967850 1 2024-06-06T08:43:16.244 INFO:tasks.workunit.client.0.smithi027.stdout:7/194: symlink d17/d31/d2c/l40 0 2024-06-06T08:43:16.244 INFO:tasks.workunit.client.0.smithi027.stdout:7/195: fsync d17/f2e 0 2024-06-06T08:43:16.244 INFO:tasks.workunit.client.0.smithi027.stdout:2/180: dwrite f1 [0,4194304] 0 2024-06-06T08:43:16.244 INFO:tasks.workunit.client.0.smithi027.stdout:2/181: stat d6/dc 0 2024-06-06T08:43:16.244 INFO:tasks.workunit.client.0.smithi027.stdout:2/182: readlink d6/l31 0 2024-06-06T08:43:16.246 INFO:tasks.workunit.client.0.smithi027.stdout:6/152: link d4/d9/dc/l1e d4/d9/dc/l32 0 2024-06-06T08:43:16.246 INFO:tasks.workunit.client.0.smithi027.stdout:7/196: creat d17/d3e/f41 x:0 0 0 2024-06-06T08:43:16.248 INFO:tasks.workunit.client.0.smithi027.stdout:0/76: truncate f14 3643878 0 2024-06-06T08:43:16.250 INFO:tasks.workunit.client.0.smithi027.stdout:2/183: mkdir d6/dc/d3f/d48 0 2024-06-06T08:43:16.257 INFO:tasks.workunit.client.0.smithi027.stdout:8/86: dwrite d7/dc/f12 [0,4194304] 0 2024-06-06T08:43:16.257 INFO:tasks.workunit.client.0.smithi027.stdout:9/71: dwrite d3/f11 [0,4194304] 0 2024-06-06T08:43:16.257 INFO:tasks.workunit.client.0.smithi027.stdout:2/184: rmdir d6/dc 39 2024-06-06T08:43:16.263 INFO:tasks.workunit.client.0.smithi027.stdout:6/153: link d4/d9/dc/l23 d4/l33 0 2024-06-06T08:43:16.264 INFO:tasks.workunit.client.0.smithi027.stdout:9/72: symlink d3/da/l1a 0 2024-06-06T08:43:16.265 INFO:tasks.workunit.client.0.smithi027.stdout:8/87: write f5 [3260574,65123] 0 2024-06-06T08:43:16.265 INFO:tasks.workunit.client.0.smithi027.stdout:8/88: truncate d7/f9 1094841 0 2024-06-06T08:43:16.265 INFO:tasks.workunit.client.0.smithi027.stdout:2/185: creat d6/dc/d3f/f49 x:0 0 0 2024-06-06T08:43:16.267 INFO:tasks.workunit.client.0.smithi027.stdout:8/89: unlink d7/fa 0 2024-06-06T08:43:16.269 INFO:tasks.workunit.client.0.smithi027.stdout:8/90: mkdir d7/d13/d15 0 2024-06-06T08:43:16.269 INFO:tasks.workunit.client.0.smithi027.stdout:8/91: fdatasync d7/dc/f12 0 2024-06-06T08:43:16.269 INFO:tasks.workunit.client.0.smithi027.stdout:7/197: dwrite d17/f21 [0,4194304] 0 2024-06-06T08:43:16.271 INFO:tasks.workunit.client.0.smithi027.stdout:8/92: symlink d7/d13/l16 0 2024-06-06T08:43:16.271 INFO:tasks.workunit.client.0.smithi027.stdout:7/198: creat d17/d31/d2a/d3b/f42 x:0 0 0 2024-06-06T08:43:16.274 INFO:tasks.workunit.client.0.smithi027.stdout:8/93: creat d7/d13/f17 x:0 0 0 2024-06-06T08:43:16.274 INFO:tasks.workunit.client.0.smithi027.stdout:8/94: chown d7/db 49048240 1 2024-06-06T08:43:16.275 INFO:tasks.workunit.client.0.smithi027.stdout:0/77: dwrite f10 [0,4194304] 0 2024-06-06T08:43:16.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/199: chown c8 9166236 1 2024-06-06T08:43:16.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/73: dread d3/f9 [0,4194304] 0 2024-06-06T08:43:16.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/74: rename d3/c7 to d3/da/dc/df/c1b 0 2024-06-06T08:43:16.285 INFO:tasks.workunit.client.0.smithi027.stdout:2/186: dwrite f5 [0,4194304] 0 2024-06-06T08:43:16.290 INFO:tasks.workunit.client.0.smithi027.stdout:7/200: dread d17/d32/f2f [0,4194304] 0 2024-06-06T08:43:16.292 INFO:tasks.workunit.client.0.smithi027.stdout:0/78: write f10 [1983740,30936] 0 2024-06-06T08:43:16.292 INFO:tasks.workunit.client.0.smithi027.stdout:0/79: write f13 [881086,27948] 0 2024-06-06T08:43:16.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/154: dwrite d4/f5 [4194304,4194304] 0 2024-06-06T08:43:16.293 INFO:tasks.workunit.client.0.smithi027.stdout:6/155: truncate d4/f10 596018 0 2024-06-06T08:43:16.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/201: symlink d17/d31/l43 0 2024-06-06T08:43:16.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/202: chown d17/f1d 3 1 2024-06-06T08:43:16.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/203: write d17/f23 [1089186,97723] 0 2024-06-06T08:43:16.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/204: chown d17/d31/d2a/d3b 1 1 2024-06-06T08:43:16.293 INFO:tasks.workunit.client.0.smithi027.stdout:0/80: mknod d15/c1e 0 2024-06-06T08:43:16.293 INFO:tasks.workunit.client.0.smithi027.stdout:0/81: stat f13 0 2024-06-06T08:43:16.293 INFO:tasks.workunit.client.0.smithi027.stdout:6/156: symlink d4/d9/dc/d18/l34 0 2024-06-06T08:43:16.294 INFO:tasks.workunit.client.0.smithi027.stdout:7/205: readlink l10 0 2024-06-06T08:43:16.295 INFO:tasks.workunit.client.0.smithi027.stdout:7/206: mknod d17/d31/d2c/d37/c44 0 2024-06-06T08:43:16.296 INFO:tasks.workunit.client.0.smithi027.stdout:9/75: dread d3/f4 [4194304,4194304] 0 2024-06-06T08:43:16.297 INFO:tasks.workunit.client.0.smithi027.stdout:0/82: link d15/c19 d15/d18/c1f 0 2024-06-06T08:43:16.298 INFO:tasks.workunit.client.0.smithi027.stdout:0/83: dread f9 [0,4194304] 0 2024-06-06T08:43:16.299 INFO:tasks.workunit.client.0.smithi027.stdout:6/157: creat d4/de/f35 x:0 0 0 2024-06-06T08:43:16.300 INFO:tasks.workunit.client.0.smithi027.stdout:8/95: dwrite d7/f8 [0,4194304] 0 2024-06-06T08:43:16.300 INFO:tasks.workunit.client.0.smithi027.stdout:7/207: creat d17/d31/f45 x:0 0 0 2024-06-06T08:43:16.301 INFO:tasks.workunit.client.0.smithi027.stdout:9/76: mknod d3/da/dc/c1c 0 2024-06-06T08:43:16.301 INFO:tasks.workunit.client.0.smithi027.stdout:9/77: stat d3/da/dc/c1c 0 2024-06-06T08:43:16.301 INFO:tasks.workunit.client.0.smithi027.stdout:9/78: stat d3/da/ce 0 2024-06-06T08:43:16.301 INFO:tasks.workunit.client.0.smithi027.stdout:8/96: write d7/f9 [720707,91003] 0 2024-06-06T08:43:16.301 INFO:tasks.workunit.client.0.smithi027.stdout:8/97: truncate d7/db/f11 986538 0 2024-06-06T08:43:16.301 INFO:tasks.workunit.client.0.smithi027.stdout:8/98: truncate d7/d13/f17 241057 0 2024-06-06T08:43:16.314 INFO:tasks.workunit.client.0.smithi027.stdout:9/79: symlink d3/da/dc/df/l1d 0 2024-06-06T08:43:16.314 INFO:tasks.workunit.client.0.smithi027.stdout:6/158: write d4/f5 [4453937,130849] 0 2024-06-06T08:43:16.314 INFO:tasks.workunit.client.0.smithi027.stdout:2/187: dwrite d6/dd/f18 [0,4194304] 0 2024-06-06T08:43:16.314 INFO:tasks.workunit.client.0.smithi027.stdout:2/188: stat d6/dc/f25 0 2024-06-06T08:43:16.315 INFO:tasks.workunit.client.0.smithi027.stdout:9/80: unlink d3/da/dc/df/l1d 0 2024-06-06T08:43:16.318 INFO:tasks.workunit.client.0.smithi027.stdout:6/159: link d4/d1a/c1f d4/de/c36 0 2024-06-06T08:43:16.321 INFO:tasks.workunit.client.0.smithi027.stdout:2/189: mknod d6/c4a 0 2024-06-06T08:43:16.323 INFO:tasks.workunit.client.0.smithi027.stdout:2/190: creat d6/dc/d1d/f4b x:0 0 0 2024-06-06T08:43:16.326 INFO:tasks.workunit.client.0.smithi027.stdout:7/208: dwrite ff [0,4194304] 0 2024-06-06T08:43:16.326 INFO:tasks.workunit.client.0.smithi027.stdout:7/209: dread - d17/d31/d2a/d3b/f42 zero size 2024-06-06T08:43:16.326 INFO:tasks.workunit.client.0.smithi027.stdout:0/84: dread fb [0,4194304] 0 2024-06-06T08:43:16.326 INFO:tasks.workunit.client.0.smithi027.stdout:0/85: read f13 [3697819,69411] 0 2024-06-06T08:43:16.326 INFO:tasks.workunit.client.0.smithi027.stdout:0/86: readlink ld 0 2024-06-06T08:43:16.328 INFO:tasks.workunit.client.0.smithi027.stdout:2/191: dread d6/dc/f22 [0,4194304] 0 2024-06-06T08:43:16.330 INFO:tasks.workunit.client.0.smithi027.stdout:0/87: mknod d15/d18/c20 0 2024-06-06T08:43:16.332 INFO:tasks.workunit.client.0.smithi027.stdout:7/210: rename d17/d3e to d17/d32/d46 0 2024-06-06T08:43:16.332 INFO:tasks.workunit.client.0.smithi027.stdout:2/192: dread d6/dc/f22 [0,4194304] 0 2024-06-06T08:43:16.332 INFO:tasks.workunit.client.0.smithi027.stdout:2/193: dread - d6/dc/d1d/f38 zero size 2024-06-06T08:43:16.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/81: dwrite d3/f15 [0,4194304] 0 2024-06-06T08:43:16.333 INFO:tasks.workunit.client.0.smithi027.stdout:2/194: creat d6/dd/d3c/f4c x:0 0 0 2024-06-06T08:43:16.333 INFO:tasks.workunit.client.0.smithi027.stdout:6/160: dread d4/d9/dc/d1c/f30 [0,4194304] 0 2024-06-06T08:43:16.335 INFO:tasks.workunit.client.0.smithi027.stdout:0/88: rmdir d15 39 2024-06-06T08:43:16.337 INFO:tasks.workunit.client.0.smithi027.stdout:9/82: symlink d3/da/l1e 0 2024-06-06T08:43:16.339 INFO:tasks.workunit.client.0.smithi027.stdout:2/195: mkdir d6/dc/d1d/d4d 0 2024-06-06T08:43:16.339 INFO:tasks.workunit.client.0.smithi027.stdout:2/196: readlink d6/l31 0 2024-06-06T08:43:16.340 INFO:tasks.workunit.client.0.smithi027.stdout:6/161: symlink d4/d1a/d2e/l37 0 2024-06-06T08:43:16.342 INFO:tasks.workunit.client.0.smithi027.stdout:9/83: creat d3/f1f x:0 0 0 2024-06-06T08:43:16.342 INFO:tasks.workunit.client.0.smithi027.stdout:6/162: creat d4/d9/dc/f38 x:0 0 0 2024-06-06T08:43:16.342 INFO:tasks.workunit.client.0.smithi027.stdout:6/163: chown d4/d1a 1 1 2024-06-06T08:43:16.342 INFO:tasks.workunit.client.0.smithi027.stdout:0/89: dread f14 [0,4194304] 0 2024-06-06T08:43:16.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/197: mknod d6/dc/d1d/d4d/c4e 0 2024-06-06T08:43:16.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/198: write d6/d1e/d2b/f3e [440589,44310] 0 2024-06-06T08:43:16.343 INFO:tasks.workunit.client.0.smithi027.stdout:9/84: rename d3/f1f to d3/f20 0 2024-06-06T08:43:16.346 INFO:tasks.workunit.client.0.smithi027.stdout:2/199: getdents d6/d1e 0 2024-06-06T08:43:16.350 INFO:tasks.workunit.client.0.smithi027.stdout:2/200: unlink d6/dd/c11 0 2024-06-06T08:43:16.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/85: dread d3/f4 [4194304,4194304] 0 2024-06-06T08:43:16.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/86: readlink l0 0 2024-06-06T08:43:16.353 INFO:tasks.workunit.client.0.smithi027.stdout:2/201: creat d6/f4f x:0 0 0 2024-06-06T08:43:16.355 INFO:tasks.workunit.client.0.smithi027.stdout:6/164: write d4/de/f11 [1182894,127445] 0 2024-06-06T08:43:16.355 INFO:tasks.workunit.client.0.smithi027.stdout:6/165: chown d4/d13/c22 49917160 1 2024-06-06T08:43:16.355 INFO:tasks.workunit.client.0.smithi027.stdout:6/166: chown d4/d13 563878534 1 2024-06-06T08:43:16.356 INFO:tasks.workunit.client.0.smithi027.stdout:9/87: dread d3/f4 [4194304,4194304] 0 2024-06-06T08:43:16.356 INFO:tasks.workunit.client.0.smithi027.stdout:6/167: creat d4/d9/dc/d18/f39 x:0 0 0 2024-06-06T08:43:16.356 INFO:tasks.workunit.client.0.smithi027.stdout:6/168: stat d4/de/c12 0 2024-06-06T08:43:16.357 INFO:tasks.workunit.client.0.smithi027.stdout:6/169: chown d4/f25 259870 1 2024-06-06T08:43:16.357 INFO:tasks.workunit.client.0.smithi027.stdout:6/170: write d4/f5 [6077952,92755] 0 2024-06-06T08:43:16.357 INFO:tasks.workunit.client.0.smithi027.stdout:6/171: symlink d4/d9/dc/d18/l3a 0 2024-06-06T08:43:16.357 INFO:tasks.workunit.client.0.smithi027.stdout:6/172: truncate d4/d9/dc/d18/f39 449897 0 2024-06-06T08:43:16.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/88: unlink d3/l6 0 2024-06-06T08:43:16.359 INFO:tasks.workunit.client.0.smithi027.stdout:6/173: mknod d4/d1a/c3b 0 2024-06-06T08:43:16.359 INFO:tasks.workunit.client.0.smithi027.stdout:6/174: chown d4/d1a/d2e/l37 22 1 2024-06-06T08:43:16.359 INFO:tasks.workunit.client.0.smithi027.stdout:6/175: write d4/d13/f21 [717010,73634] 0 2024-06-06T08:43:16.359 INFO:tasks.workunit.client.0.smithi027.stdout:0/90: write f4 [2794503,52702] 0 2024-06-06T08:43:16.359 INFO:tasks.workunit.client.0.smithi027.stdout:9/89: creat d3/da/dc/f21 x:0 0 0 2024-06-06T08:43:16.360 INFO:tasks.workunit.client.0.smithi027.stdout:9/90: symlink d3/da/dc/l22 0 2024-06-06T08:43:16.360 INFO:tasks.workunit.client.0.smithi027.stdout:9/91: readlink d3/da/dc/l19 0 2024-06-06T08:43:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:0/91: dread f14 [0,4194304] 0 2024-06-06T08:43:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:0/92: chown d15/f16 385804 1 2024-06-06T08:43:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:0/93: write f4 [1307646,29029] 0 2024-06-06T08:43:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:0/94: dread f9 [0,4194304] 0 2024-06-06T08:43:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:0/95: creat d15/f21 x:0 0 0 2024-06-06T08:43:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:2/202: dwrite d6/d1e/d2b/f37 [0,4194304] 0 2024-06-06T08:43:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:6/176: dwrite d4/d9/dc/d18/f2f [0,4194304] 0 2024-06-06T08:43:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:0/96: creat d15/f22 x:0 0 0 2024-06-06T08:43:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:0/97: write f14 [4566240,113971] 0 2024-06-06T08:43:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:6/177: read d4/f10 [482229,5276] 0 2024-06-06T08:43:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:9/92: dwrite d3/da/dc/f21 [0,4194304] 0 2024-06-06T08:43:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:9/93: dread - d3/f20 zero size 2024-06-06T08:43:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:2/203: creat d6/dc/d3f/f50 x:0 0 0 2024-06-06T08:43:16.384 INFO:tasks.workunit.client.0.smithi027.stdout:2/204: symlink d6/l51 0 2024-06-06T08:43:16.384 INFO:tasks.workunit.client.0.smithi027.stdout:2/205: stat d6/f2e 0 2024-06-06T08:43:16.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/206: creat d6/d3b/f52 x:0 0 0 2024-06-06T08:43:16.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/207: write d6/dd/f26 [557511,104030] 0 2024-06-06T08:43:16.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/208: truncate d6/dd/f24 850732 0 2024-06-06T08:43:16.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/209: dread - d6/dc/d1d/f38 zero size 2024-06-06T08:43:16.388 INFO:tasks.workunit.client.0.smithi027.stdout:2/210: stat d6/d1e/d2b/f3e 0 2024-06-06T08:43:16.392 INFO:tasks.workunit.client.0.smithi027.stdout:9/94: write d3/da/dc/f21 [1868853,106371] 0 2024-06-06T08:43:16.395 INFO:tasks.workunit.client.0.smithi027.stdout:9/95: mkdir d3/da/dc/df/d23 0 2024-06-06T08:43:16.398 INFO:tasks.workunit.client.0.smithi027.stdout:9/96: rename d3/da/ce to d3/da/dc/df/c24 0 2024-06-06T08:43:16.401 INFO:tasks.workunit.client.0.smithi027.stdout:0/98: dwrite f4 [0,4194304] 0 2024-06-06T08:43:16.401 INFO:tasks.workunit.client.0.smithi027.stdout:9/97: mknod d3/da/dc/c25 0 2024-06-06T08:43:16.403 INFO:tasks.workunit.client.0.smithi027.stdout:0/99: mkdir d15/d18/d23 0 2024-06-06T08:43:16.405 INFO:tasks.workunit.client.0.smithi027.stdout:0/100: creat d15/f24 x:0 0 0 2024-06-06T08:43:16.405 INFO:tasks.workunit.client.0.smithi027.stdout:6/178: dwrite d4/d13/f21 [0,4194304] 0 2024-06-06T08:43:16.407 INFO:tasks.workunit.client.0.smithi027.stdout:2/211: dwrite d6/dc/d1d/f45 [0,4194304] 0 2024-06-06T08:43:16.408 INFO:tasks.workunit.client.0.smithi027.stdout:0/101: creat d15/d18/d23/f25 x:0 0 0 2024-06-06T08:43:16.408 INFO:tasks.workunit.client.0.smithi027.stdout:0/102: truncate d15/d18/f1c 17127 0 2024-06-06T08:43:16.409 INFO:tasks.workunit.client.0.smithi027.stdout:0/103: stat f2 0 2024-06-06T08:43:16.410 INFO:tasks.workunit.client.0.smithi027.stdout:9/98: rename d3/da/dc/df/c24 to d3/c26 0 2024-06-06T08:43:16.414 INFO:tasks.workunit.client.0.smithi027.stdout:2/212: rmdir d6/dd/d3c 39 2024-06-06T08:43:16.415 INFO:tasks.workunit.client.0.smithi027.stdout:0/104: mknod d15/d18/c26 0 2024-06-06T08:43:16.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/99: mknod d3/da/dc/df/d23/c27 0 2024-06-06T08:43:16.419 INFO:tasks.workunit.client.0.smithi027.stdout:2/213: getdents d6/dc/d1d 0 2024-06-06T08:43:16.421 INFO:tasks.workunit.client.0.smithi027.stdout:9/100: creat d3/da/dc/df/f28 x:0 0 0 2024-06-06T08:43:16.437 INFO:tasks.workunit.client.0.smithi027.stdout:2/214: dwrite f5 [0,4194304] 0 2024-06-06T08:43:16.438 INFO:tasks.workunit.client.0.smithi027.stdout:2/215: fsync d6/dd/f18 0 2024-06-06T08:43:16.441 INFO:tasks.workunit.client.0.smithi027.stdout:2/216: getdents d6/dc/d3f 0 2024-06-06T08:43:16.443 INFO:tasks.workunit.client.0.smithi027.stdout:9/101: dwrite d3/da/dc/f21 [0,4194304] 0 2024-06-06T08:43:16.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/102: mknod d3/da/dc/c29 0 2024-06-06T08:43:16.446 INFO:tasks.workunit.client.0.smithi027.stdout:9/103: creat d3/da/dc/f2a x:0 0 0 2024-06-06T08:43:16.446 INFO:tasks.workunit.client.0.smithi027.stdout:9/104: creat d3/da/dc/df/f2b x:0 0 0 2024-06-06T08:43:16.447 INFO:tasks.workunit.client.0.smithi027.stdout:9/105: stat d3 0 2024-06-06T08:43:16.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/106: truncate d3/f9 2511270 0 2024-06-06T08:43:16.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/107: readlink d3/da/l1a 0 2024-06-06T08:43:16.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/108: dread - d3/da/dc/df/f28 zero size 2024-06-06T08:43:16.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/109: creat d3/da/f2c x:0 0 0 2024-06-06T08:43:16.449 INFO:tasks.workunit.client.0.smithi027.stdout:9/110: readlink d3/l8 0 2024-06-06T08:43:16.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/111: creat d3/da/dc/f2d x:0 0 0 2024-06-06T08:43:16.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/112: chown d3/da/dc/c1c 8 1 2024-06-06T08:43:16.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/113: write d3/da/dc/df/f28 [831212,4128] 0 2024-06-06T08:43:16.460 INFO:tasks.workunit.client.0.smithi027.stdout:2/217: dwrite d6/d1e/d2b/f3e [0,4194304] 0 2024-06-06T08:43:16.462 INFO:tasks.workunit.client.0.smithi027.stdout:2/218: truncate d6/dc/f25 199809 0 2024-06-06T08:43:16.464 INFO:tasks.workunit.client.0.smithi027.stdout:2/219: link d6/dc/l39 d6/d3b/l53 0 2024-06-06T08:43:16.465 INFO:tasks.workunit.client.0.smithi027.stdout:2/220: creat d6/dc/f54 x:0 0 0 2024-06-06T08:43:16.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:16 smithi027 ceph-mon[32335]: pgmap v93: 129 pgs: 129 active+clean; 165 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 2.42k op/s 2024-06-06T08:43:16.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:16 smithi203 ceph-mon[36579]: pgmap v93: 129 pgs: 129 active+clean; 165 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 2.42k op/s 2024-06-06T08:43:17.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:16 smithi176 ceph-mon[36786]: pgmap v93: 129 pgs: 129 active+clean; 165 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 2.42k op/s 2024-06-06T08:43:18.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.496+0000 7f5286f06700 1 -- 172.21.15.27:0/3241881927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101d50 msgr2=0x7f528010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.496+0000 7f5286f06700 1 --2- 172.21.15.27:0/3241881927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101d50 0x7f528010e240 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f527800b0a0 tx=0x7f527800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.496+0000 7f5286f06700 1 -- 172.21.15.27:0/3241881927 shutdown_connections 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.496+0000 7f5286f06700 1 --2- 172.21.15.27:0/3241881927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101d50 0x7f528010e240 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.496+0000 7f5286f06700 1 --2- 172.21.15.27:0/3241881927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5280107400 0x7f52801077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.496+0000 7f5286f06700 1 --2- 172.21.15.27:0/3241881927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5280101390 0x7f5280101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.496+0000 7f5286f06700 1 -- 172.21.15.27:0/3241881927 >> 172.21.15.27:0/3241881927 conn(0x7f5280078340 msgr2=0x7f5280078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.496+0000 7f5286f06700 1 -- 172.21.15.27:0/3241881927 shutdown_connections 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.496+0000 7f5286f06700 1 -- 172.21.15.27:0/3241881927 wait complete. 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5286f06700 1 Processor -- start 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5286f06700 1 -- start start 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5286f06700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101390 0x7f528019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5286f06700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5280101d50 0x7f528019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5286f06700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5280107400 0x7f52801989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5286f06700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52801116f0 con 0x7f5280101390 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5286f06700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5280111570 con 0x7f5280101d50 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5286f06700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5280111870 con 0x7f5280107400 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5284ca2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101390 0x7f528019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5284ca2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101390 0x7f528019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54170/0 (socket says 172.21.15.27:54170) 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.497+0000 7f5284ca2700 1 -- 172.21.15.27:0/3607456710 learned_addr learned my addr 172.21.15.27:0/3607456710 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f527ffff700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5280101d50 0x7f528019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f52854a3700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5280107400 0x7f52801989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f5284ca2700 1 -- 172.21.15.27:0/3607456710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5280107400 msgr2=0x7f52801989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f5284ca2700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5280107400 0x7f52801989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f5284ca2700 1 -- 172.21.15.27:0/3607456710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5280101d50 msgr2=0x7f528019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f5284ca2700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5280101d50 0x7f528019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f5284ca2700 1 -- 172.21.15.27:0/3607456710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5280199120 con 0x7f5280101390 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f527ffff700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5280101d50 0x7f528019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:43:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f5284ca2700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101390 0x7f528019e800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f527400b530 tx=0x7f527400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f527dffb700 1 -- 172.21.15.27:0/3607456710 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274015020 con 0x7f5280101390 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f527dffb700 1 -- 172.21.15.27:0/3607456710 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5274011d10 con 0x7f5280101390 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f527dffb700 1 -- 172.21.15.27:0/3607456710 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274010e50 con 0x7f5280101390 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5280199410 con 0x7f5280101390 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.498+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a56d0 con 0x7f5280101390 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.499+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5280069180 con 0x7f5280101390 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.500+0000 7f527dffb700 1 -- 172.21.15.27:0/3607456710 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5274018020 con 0x7f5280101390 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.501+0000 7f527dffb700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f52700a2f30 0x7f52700a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.501+0000 7f527ffff700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f52700a2f30 0x7f52700a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.501+0000 7f527dffb700 1 -- 172.21.15.27:0/3607456710 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f52740cef90 con 0x7f5280101390 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.502+0000 7f527ffff700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f52700a2f30 0x7f52700a53f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f526c005fd0 tx=0x7f526c005e20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.506+0000 7f527dffb700 1 -- 172.21.15.27:0/3607456710 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f527401f020 con 0x7f5280101390 2024-06-06T08:43:18.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:18 smithi027 ceph-mon[32335]: pgmap v94: 129 pgs: 129 active+clean; 165 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 1.69k op/s 2024-06-06T08:43:18.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.741+0000 7f6adc042700 1 -- 172.21.15.27:0/879903008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 msgr2=0x7f6ad4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.741+0000 7f6adc042700 1 --2- 172.21.15.27:0/879903008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 0x7f6ad4108920 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00099c0 tx=0x7f6ad0009cd0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.741+0000 7f6adc042700 1 -- 172.21.15.27:0/879903008 shutdown_connections 2024-06-06T08:43:18.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.741+0000 7f6adc042700 1 --2- 172.21.15.27:0/879903008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ad4102e50 0x7f6ad410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.741+0000 7f6adc042700 1 --2- 172.21.15.27:0/879903008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 0x7f6ad4108920 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.741+0000 7f6adc042700 1 --2- 172.21.15.27:0/879903008 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ad4102490 0x7f6ad4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.741+0000 7f6adc042700 1 -- 172.21.15.27:0/879903008 >> 172.21.15.27:0/879903008 conn(0x7f6ad40fdf40 msgr2=0x7f6ad4100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.741+0000 7f6adc042700 1 -- 172.21.15.27:0/879903008 shutdown_connections 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.741+0000 7f6adc042700 1 -- 172.21.15.27:0/879903008 wait complete. 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6adc042700 1 Processor -- start 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6adc042700 1 -- start start 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6adc042700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ad4102490 0x7f6ad419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6adc042700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ad4102e50 0x7f6ad419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6adc042700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 0x7f6ad419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6adc042700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4075370 con 0x7f6ad4108540 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6adc042700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40751f0 con 0x7f6ad4102e50 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6adc042700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40754f0 con 0x7f6ad4102490 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ada5df700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 0x7f6ad419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ada5df700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 0x7f6ad419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54192/0 (socket says 172.21.15.27:54192) 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ada5df700 1 -- 172.21.15.27:0/2848823827 learned_addr learned my addr 172.21.15.27:0/2848823827 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ad95dd700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ad4102e50 0x7f6ad419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ad4102490 0x7f6ad419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ada5df700 1 -- 172.21.15.27:0/2848823827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ad4102490 msgr2=0x7f6ad419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ada5df700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ad4102490 0x7f6ad419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ada5df700 1 -- 172.21.15.27:0/2848823827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ad4102e50 msgr2=0x7f6ad419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ada5df700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ad4102e50 0x7f6ad419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.742+0000 7f6ada5df700 1 -- 172.21.15.27:0/2848823827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad419fd30 con 0x7f6ad4108540 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.743+0000 7f6ada5df700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 0x7f6ad419f5f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6acc007d70 tx=0x7f6acc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.743+0000 7f6acaffd700 1 -- 172.21.15.27:0/2848823827 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc016020 con 0x7f6ad4108540 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.743+0000 7f6adc042700 1 -- 172.21.15.27:0/2848823827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad41a0020 con 0x7f6ad4108540 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.743+0000 7f6acaffd700 1 -- 172.21.15.27:0/2848823827 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6acc00ad40 con 0x7f6ad4108540 2024-06-06T08:43:18.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.743+0000 7f6acaffd700 1 -- 172.21.15.27:0/2848823827 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc011600 con 0x7f6ad4108540 2024-06-06T08:43:18.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.743+0000 7f6adc042700 1 -- 172.21.15.27:0/2848823827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad41a78a0 con 0x7f6ad4108540 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.746+0000 7f6acaffd700 1 -- 172.21.15.27:0/2848823827 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6acc019020 con 0x7f6ad4108540 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ae68c5b80 1 Processor -- start 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ae68c5b80 1 -- start start 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6acaffd700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ac00a2d90 0x7f6ac00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029ef60 0x563a0029f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a00299260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 0x563a00299bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563a0018bf60 con 0x563a0029f880 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563a001ea850 con 0x563a0029ef60 2024-06-06T08:43:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563a00152460 con 0x563a002997a0 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6acaffd700 1 -- 172.21.15.27:0/2848823827 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6acc0cf6d0 con 0x7f6ad4108540 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ad95dd700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ac00a2d90 0x7f6ac00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.747+0000 7f6ad9dde700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029ef60 0x563a0029f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad9dde700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029ef60 0x563a0029f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:57042/0 (socket says 172.21.15.27:57042) 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad9dde700 1 -- 172.21.15.27:0/1334006427 learned_addr learned my addr 172.21.15.27:0/1334006427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ada5df700 1 --2- 172.21.15.27:0/1334006427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 0x563a00299bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad95dd700 1 --2- 172.21.15.27:0/1334006427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a00299260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad9dde700 1 -- 172.21.15.27:0/1334006427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 msgr2=0x563a00299bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/1334006427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 0x563a00299bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad9dde700 1 -- 172.21.15.27:0/1334006427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 msgr2=0x563a00299260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/1334006427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a00299260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad9dde700 1 -- 172.21.15.27:0/1334006427 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a001398d0 con 0x563a0029ef60 2024-06-06T08:43:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad95dd700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ac00a2d90 0x7f6ac00a5250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4108b40 tx=0x7f6ac4013900 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/1334006427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029ef60 0x563a0029f340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00099c0 tx=0x7f6ad0009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b348e36f1f76502d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6abffff700 1 -- 172.21.15.27:0/1334006427 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0027070 con 0x563a0029ef60 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6abffff700 1 -- 172.21.15.27:0/1334006427 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad00050c0 con 0x563a0029ef60 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.748+0000 7f6abffff700 1 -- 172.21.15.27:0/1334006427 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad000bac0 con 0x563a0029ef60 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.749+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1334006427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029ef60 msgr2=0x563a0029f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.749+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/1334006427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029ef60 0x563a0029f340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00099c0 tx=0x7f6ad0009ec0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.749+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1334006427 shutdown_connections 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.749+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/1334006427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 0x563a00299bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.749+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/1334006427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029ef60 0x563a0029f340 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.749+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/1334006427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a00299260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.749+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1334006427 >> 172.21.15.27:0/1334006427 conn(0x563a001c6080 msgr2=0x563a00294220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.750+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1334006427 shutdown_connections 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.750+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1334006427 wait complete. 2024-06-06T08:43:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.750+0000 7f6ae68c5b80 1 -- start start 2024-06-06T08:43:18.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.750+0000 7f6ae68c5b80 1 Processor -- start 2024-06-06T08:43:18.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a002997a0 0x563a002a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0029ef60 0x563a002a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 0x563a002a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563a0018afc0 con 0x563a002997a0 2024-06-06T08:43:18.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ad9dde700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a002997a0 0x563a002a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ad9dde700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a002997a0 0x563a002a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54220/0 (socket says 172.21.15.27:54220) 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ad9dde700 1 -- 172.21.15.27:0/157835597 learned_addr learned my addr 172.21.15.27:0/157835597 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x563a001e6810 con 0x563a0029f880 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563a002a6200 con 0x563a0029ef60 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ada5df700 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 0x563a002a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ae68c5b80 10 client.0 _ll_register_callbacks cb 0x563a001805b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ad95dd700 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0029ef60 0x563a002a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ada5df700 1 -- 172.21.15.27:0/157835597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0029ef60 msgr2=0x563a002a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ada5df700 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0029ef60 0x563a002a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ada5df700 1 -- 172.21.15.27:0/157835597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a002997a0 msgr2=0x563a002a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.751+0000 7f6ada5df700 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a002997a0 0x563a002a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6ada5df700 1 -- 172.21.15.27:0/157835597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a001b0750 con 0x563a0029f880 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a002997a0 0x563a002a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:18.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad00050c0 con 0x563a0029f880 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6ada5df700 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 0x563a002a2e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0ccb70 tx=0x7f6acc09b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15685dc918294eb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6abd7fa700 10 client.25682 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6abd7fa700 1 -- 172.21.15.27:0/157835597 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0b3020 con 0x563a0029f880 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6abd7fa700 1 -- 172.21.15.27:0/157835597 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6acc0ba120 con 0x563a0029f880 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6abd7fa700 1 -- 172.21.15.27:0/157835597 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6acc0afe60 con 0x563a0029f880 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6ae68c5b80 10 client.25682 fetch_fsmap learned FSMap version 33 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.752+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({fsmap=33}) -- 0x563a002b0880 con 0x563a0029f880 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.753+0000 7f6abd7fa700 1 -- 172.21.15.27:0/157835597 <== mon.2 v2:172.21.15.176:3300/0 4 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f6acc0860b0 con 0x563a0029f880 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.753+0000 7f6ae68c5b80 10 client.25682 fetch_fsmap finished waiting for FSMap version 33 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.753+0000 7f6ae68c5b80 10 client.25682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.753+0000 7f6ae68c5b80 20 client.25682 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.753+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563a002b2df0 0x563a002b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.753+0000 7f6ae68c5b80 4 client.25682 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:43:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.753+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a001f7d50 con 0x563a002b2df0 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.753+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563a002b2df0 0x563a002b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.754+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563a002b2df0 0x563a002b5260 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e57e4d8ffa7dbd server_cookie=26b0431061453d9b in_seq=0 out_seq=0 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.754+0000 7f6abd7fa700 10 client.25682 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.754+0000 7f6abd7fa700 1 -- 172.21.15.27:0/157835597 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563a001f7d50 con 0x563a002b2df0 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.754+0000 7f6abd7fa700 10 client.25682 handle_command_reply: tid=0 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.754+0000 7f6ae68c5b80 1 client.25682 shutdown 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.754+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 msgr2=0x563a002a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 0x563a002a2e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0ccb70 tx=0x7f6acc09b390 comp rx=0 tx=0).stop 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 shutdown_connections 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 0x563a002a2e10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563a002b2df0 0x563a002b5260 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6abd7fa700 0 client.25682 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a002997a0 0x563a002a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/157835597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0029ef60 0x563a002a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 >> 172.21.15.27:0/157835597 conn(0x563a001c6080 msgr2=0x563a00293700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 shutdown_connections 2024-06-06T08:43:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/157835597 wait complete. 2024-06-06T08:43:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.755+0000 7f6ae68c5b80 20 client.25682 trim_cache size 0 max 16384 2024-06-06T08:43:18.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.767+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52801a5960 con 0x7f5280101390 2024-06-06T08:43:18.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.768+0000 7f527dffb700 1 -- 172.21.15.27:0/3607456710 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11894 (secure 0 0 0) 0x7f527409b900 con 0x7f5280101390 2024-06-06T08:43:18.766 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:43:18.767 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:43:13.942784+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24458,24460]},"id":1}]} 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f52700a2f30 msgr2=0x7f52700a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f52700a2f30 0x7f52700a53f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f526c005fd0 tx=0x7f526c005e20 comp rx=0 tx=0).stop 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101390 msgr2=0x7f528019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101390 0x7f528019e800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f527400b530 tx=0x7f527400b8f0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 shutdown_connections 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5280107400 0x7f52801989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5280101390 0x7f528019e800 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f52700a2f30 0x7f52700a53f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 --2- 172.21.15.27:0/3607456710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5280101d50 0x7f528019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 >> 172.21.15.27:0/3607456710 conn(0x7f5280078340 msgr2=0x7f5280105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 shutdown_connections 2024-06-06T08:43:18.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.770+0000 7f5286f06700 1 -- 172.21.15.27:0/3607456710 wait complete. 2024-06-06T08:43:18.770 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 33 2024-06-06T08:43:18.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.776+0000 7f6ae68c5b80 1 Processor -- start 2024-06-06T08:43:18.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.776+0000 7f6ae68c5b80 1 -- start start 2024-06-06T08:43:18.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.776+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a002997a0 0x563a002a0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.776+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0029ef60 0x563a00293e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.776+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a002943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.776+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41b2ed0 con 0x563a0029f880 2024-06-06T08:43:18.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.776+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41b06d0 con 0x563a002997a0 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.776+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41ae520 con 0x563a0029ef60 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6ada5df700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a002943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6ada5df700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a002943c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54232/0 (socket says 172.21.15.27:54232) 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6ada5df700 1 -- 172.21.15.27:0/1318167939 learned_addr learned my addr 172.21.15.27:0/1318167939 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6ada5df700 1 -- 172.21.15.27:0/1318167939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0029ef60 msgr2=0x563a00293e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6ada5df700 1 --2- 172.21.15.27:0/1318167939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0029ef60 0x563a00293e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6ada5df700 1 -- 172.21.15.27:0/1318167939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a002997a0 msgr2=0x563a002a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6ada5df700 1 --2- 172.21.15.27:0/1318167939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a002997a0 0x563a002a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6ada5df700 1 -- 172.21.15.27:0/1318167939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a002a0e40 con 0x563a0029f880 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6ada5df700 1 --2- 172.21.15.27:0/1318167939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a002943c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6acc09b8d0 tx=0x7f6acc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da110d35c2b6745 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6abf7fe700 1 -- 172.21.15.27:0/1318167939 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc07f070 con 0x563a0029f880 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6abf7fe700 1 -- 172.21.15.27:0/1318167939 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad00050c0 con 0x563a0029f880 2024-06-06T08:43:18.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.777+0000 7f6abf7fe700 1 -- 172.21.15.27:0/1318167939 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0b3030 con 0x563a0029f880 2024-06-06T08:43:18.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.778+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1318167939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 msgr2=0x563a002943c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.778+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/1318167939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a002943c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6acc09b8d0 tx=0x7f6acc01d000 comp rx=0 tx=0).stop 2024-06-06T08:43:18.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.778+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1318167939 shutdown_connections 2024-06-06T08:43:18.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.778+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/1318167939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029f880 0x563a002943c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.778+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/1318167939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a002997a0 0x563a002a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.778+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/1318167939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0029ef60 0x563a00293e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.778+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1318167939 >> 172.21.15.27:0/1318167939 conn(0x563a001c6080 msgr2=0x563a002a4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.778+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1318167939 shutdown_connections 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.778+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/1318167939 wait complete. 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ae68c5b80 1 -- start start 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ae68c5b80 1 Processor -- start 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 0x563a002ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029ef60 0x563a002aba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ae68c5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 0x563a002a8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41aeb90 con 0x563a0029ef60 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41b0000 con 0x563a0029f880 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ae68c5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad41af260 con 0x563a002997a0 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ae68c5b80 10 client.0 _ll_register_callbacks cb 0x563a001805b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:18.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ad95dd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029ef60 0x563a002aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.779+0000 7f6ad95dd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029ef60 0x563a002aba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54248/0 (socket says 172.21.15.27:54248) 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ad95dd700 1 -- 172.21.15.27:0/3124180698 learned_addr learned my addr 172.21.15.27:0/3124180698 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 0x563a002ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ada5df700 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 0x563a002a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ad95dd700 1 -- 172.21.15.27:0/3124180698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 msgr2=0x563a002ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ad95dd700 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 0x563a002ab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ad95dd700 1 -- 172.21.15.27:0/3124180698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 msgr2=0x563a002a8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ad95dd700 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 0x563a002a8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ad95dd700 1 -- 172.21.15.27:0/3124180698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a002ac100 con 0x563a0029ef60 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ad95dd700 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029ef60 0x563a002aba00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4015fd0 tx=0x7f6ac40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1818a5df94390f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6abd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6abd7fa700 1 -- 172.21.15.27:0/3124180698 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4011e60 con 0x563a0029ef60 2024-06-06T08:43:18.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6abd7fa700 1 -- 172.21.15.27:0/3124180698 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6ac4021c20 con 0x563a0029ef60 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.780+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/3124180698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad00050c0 con 0x563a0029ef60 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6abd7fa700 1 -- 172.21.15.27:0/3124180698 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ac4021e20 con 0x563a0029ef60 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6ae68c5b80 10 client.16179 fetch_fsmap learned FSMap version 33 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/3124180698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563a0031e120 con 0x563a0029ef60 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6abd7fa700 1 -- 172.21.15.27:0/3124180698 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad00050c0 con 0x563a0029ef60 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6abd7fa700 1 -- 172.21.15.27:0/3124180698 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f6ac4028070 con 0x563a0029ef60 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6ae68c5b80 10 client.16179 fetch_fsmap finished waiting for FSMap version 33 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6ae68c5b80 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6ae68c5b80 20 client.16179 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563a002b2df0 0x563a003208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6ae68c5b80 4 client.16179 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/3124180698 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a001f22d0 con 0x563a002b2df0 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.781+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563a002b2df0 0x563a003208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:18.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6abd7fa700 10 client.16179 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:18.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ad9dde700 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563a002b2df0 0x563a003208d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902ca04102e9f538 server_cookie=26dafbf34a3ad865 in_seq=0 out_seq=0 2024-06-06T08:43:18.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6abd7fa700 1 -- 172.21.15.27:0/3124180698 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563a001f22d0 con 0x563a002b2df0 2024-06-06T08:43:18.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6abd7fa700 10 client.16179 handle_command_reply: tid=0 2024-06-06T08:43:18.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 client.16179 shutdown 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/3124180698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029ef60 msgr2=0x563a002aba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029ef60 0x563a002aba00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4015fd0 tx=0x7f6ac40047c0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/3124180698 shutdown_connections 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563a0029f880 0x563a002a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a002997a0 0x563a002ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x563a002b2df0 0x563a003208d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 --2- 172.21.15.27:0/3124180698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0029ef60 0x563a002aba00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/3124180698 >> 172.21.15.27:0/3124180698 conn(0x563a001c6080 msgr2=0x563a002a4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6abd7fa700 0 client.16179 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/3124180698 shutdown_connections 2024-06-06T08:43:18.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.782+0000 7f6ae68c5b80 1 -- 172.21.15.27:0/3124180698 wait complete. 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.783+0000 7f6ae68c5b80 20 client.16179 trim_cache size 0 max 16384 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": { 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stdout: "5ccc9a12-fe19-4f51-aecc-bda187b537e4": { 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/", 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tag": "5ccc9a12-fe19-4f51-aecc-bda187b537e4", 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stdout: "options": "recursive,force" 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:18.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:18.787 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:43:18.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.783+0000 7f6adc042700 1 -- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ac00a2d90 msgr2=0x7f6ac00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.783+0000 7f6adc042700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ac00a2d90 0x7f6ac00a5250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4108b40 tx=0x7f6ac4013900 comp rx=0 tx=0).stop 2024-06-06T08:43:18.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.783+0000 7f6adc042700 1 -- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 msgr2=0x7f6ad419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:18.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.783+0000 7f6adc042700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 0x7f6ad419f5f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6acc007d70 tx=0x7f6acc00a340 comp rx=0 tx=0).stop 2024-06-06T08:43:18.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.784+0000 7f6adc042700 1 -- 172.21.15.27:0/2848823827 shutdown_connections 2024-06-06T08:43:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.784+0000 7f6adc042700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad4108540 0x7f6ad419f5f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.784+0000 7f6adc042700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ad4102490 0x7f6ad419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.784+0000 7f6adc042700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ac00a2d90 0x7f6ac00a5250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.784+0000 7f6adc042700 1 --2- 172.21.15.27:0/2848823827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ad4102e50 0x7f6ad419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.784+0000 7f6adc042700 1 -- 172.21.15.27:0/2848823827 >> 172.21.15.27:0/2848823827 conn(0x7f6ad40fdf40 msgr2=0x7f6ad40fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.784+0000 7f6adc042700 1 -- 172.21.15.27:0/2848823827 shutdown_connections 2024-06-06T08:43:18.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:18.784+0000 7f6adc042700 1 -- 172.21.15.27:0/2848823827 wait complete. 2024-06-06T08:43:18.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:18 smithi203 ceph-mon[36579]: pgmap v94: 129 pgs: 129 active+clean; 165 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 1.69k op/s 2024-06-06T08:43:19.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:18 smithi176 ceph-mon[36786]: pgmap v94: 129 pgs: 129 active+clean; 165 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 1.69k op/s 2024-06-06T08:43:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.397+0000 7f2e44809640 1 -- 172.21.15.27:0/1625788438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 msgr2=0x7f2e4010e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.397+0000 7f2e44809640 1 --2- 172.21.15.27:0/1625788438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 0x7f2e4010e400 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009ca0 tx=0x7f2e3802a950 comp rx=0 tx=0).stop 2024-06-06T08:43:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.397+0000 7f2e44809640 1 -- 172.21.15.27:0/1625788438 shutdown_connections 2024-06-06T08:43:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.397+0000 7f2e44809640 1 --2- 172.21.15.27:0/1625788438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 0x7f2e4010e400 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.397+0000 7f2e44809640 1 --2- 172.21.15.27:0/1625788438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e40103590 0x7f2e4010ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.397+0000 7f2e44809640 1 --2- 172.21.15.27:0/1625788438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e40102c70 0x7f2e40103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.397+0000 7f2e44809640 1 -- 172.21.15.27:0/1625788438 >> 172.21.15.27:0/1625788438 conn(0x7f2e40077a20 msgr2=0x7f2e40077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.397+0000 7f2e44809640 1 -- 172.21.15.27:0/1625788438 shutdown_connections 2024-06-06T08:43:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.397+0000 7f2e44809640 1 -- 172.21.15.27:0/1625788438 wait complete. 2024-06-06T08:43:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e44809640 1 Processor -- start 2024-06-06T08:43:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e44809640 1 -- start start 2024-06-06T08:43:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e44809640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e40102c70 0x7f2e4019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e44809640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e40103590 0x7f2e4019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e44809640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 0x7f2e4019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e44809640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40111db0 con 0x7f2e4010bfb0 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e44809640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40111c30 con 0x7f2e40102c70 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e44809640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40111f30 con 0x7f2e40103590 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e3ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 0x7f2e4019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e3ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 0x7f2e4019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54260/0 (socket says 172.21.15.27:54260) 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e3ed9d640 1 -- 172.21.15.27:0/2157660924 learned_addr learned my addr 172.21.15.27:0/2157660924 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e3e59c640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e40102c70 0x7f2e4019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.398+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e40103590 0x7f2e4019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e3ed9d640 1 -- 172.21.15.27:0/2157660924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e40103590 msgr2=0x7f2e4019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e3ed9d640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e40103590 0x7f2e4019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e3ed9d640 1 -- 172.21.15.27:0/2157660924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e40102c70 msgr2=0x7f2e4019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e3ed9d640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e40102c70 0x7f2e4019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e3ed9d640 1 -- 172.21.15.27:0/2157660924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4019f380 con 0x7f2e4010bfb0 2024-06-06T08:43:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e40103590 0x7f2e4019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:43:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e3ed9d640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 0x7f2e4019ec50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009dc0 tx=0x7f2e38003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e2f7fe640 1 -- 172.21.15.27:0/2157660924 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38034040 con 0x7f2e4010bfb0 2024-06-06T08:43:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e2f7fe640 1 -- 172.21.15.27:0/2157660924 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e3802fe20 con 0x7f2e4010bfb0 2024-06-06T08:43:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e44809640 1 -- 172.21.15.27:0/2157660924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4019f610 con 0x7f2e4010bfb0 2024-06-06T08:43:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e2f7fe640 1 -- 172.21.15.27:0/2157660924 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380047f0 con 0x7f2e4010bfb0 2024-06-06T08:43:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.399+0000 7f2e44809640 1 -- 172.21.15.27:0/2157660924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e401a6f00 con 0x7f2e4010bfb0 2024-06-06T08:43:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.401+0000 7f2e2f7fe640 1 -- 172.21.15.27:0/2157660924 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2e38037020 con 0x7f2e4010bfb0 2024-06-06T08:43:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e475f1740 1 Processor -- start 2024-06-06T08:43:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e475f1740 1 -- start start 2024-06-06T08:43:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e2f7fe640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2e280a2da0 0x7f2e280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 0x55d2c9aba550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e475f1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9a1e4f0 con 0x55d2c9ac5350 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e475f1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d2c99bc870 con 0x55d2c9aba170 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e475f1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9a1a4b0 con 0x55d2c9abaa90 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2e280a2da0 0x7f2e280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e2f7fe640 1 -- 172.21.15.27:0/2157660924 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f2e380f0670 con 0x7f2e4010bfb0 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e3ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e3ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54268/0 (socket says 172.21.15.27:54268) 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.402+0000 7f2e3ed9d640 1 -- 172.21.15.27:0/2858783475 learned_addr learned my addr 172.21.15.27:0/2858783475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e3e59c640 1 --2- 172.21.15.27:0/2858783475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 0x55d2c9aba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/2858783475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e3ed9d640 1 -- 172.21.15.27:0/2858783475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 msgr2=0x55d2c9ac4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e3ed9d640 1 --2- 172.21.15.27:0/2858783475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e3ed9d640 1 -- 172.21.15.27:0/2858783475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 msgr2=0x55d2c9aba550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e3ed9d640 1 --2- 172.21.15.27:0/2858783475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 0x55d2c9aba550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e3ed9d640 1 -- 172.21.15.27:0/2858783475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c99d89b0 con 0x55d2c9ac5350 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e3ed9d640 1 --2- 172.21.15.27:0/2858783475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac7740 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2e380bc810 tx=0x7f2e38004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85afafba8bb79015 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2e280a2da0 0x7f2e280a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e34005d80 tx=0x7f2e34012760 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e2cff9640 1 -- 172.21.15.27:0/2858783475 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380a0560 con 0x55d2c9ac5350 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e2cff9640 1 -- 172.21.15.27:0/2858783475 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e380098d0 con 0x55d2c9ac5350 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.403+0000 7f2e2cff9640 1 -- 172.21.15.27:0/2858783475 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3800a2f0 con 0x55d2c9ac5350 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.404+0000 7f2e475f1740 1 -- 172.21.15.27:0/2858783475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9ac5350 msgr2=0x55d2c9ac7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.404+0000 7f2e475f1740 1 --2- 172.21.15.27:0/2858783475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac7740 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2e380bc810 tx=0x7f2e38004290 comp rx=0 tx=0).stop 2024-06-06T08:43:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.404+0000 7f2e475f1740 1 -- 172.21.15.27:0/2858783475 shutdown_connections 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.404+0000 7f2e475f1740 1 --2- 172.21.15.27:0/2858783475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac7740 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.404+0000 7f2e475f1740 1 --2- 172.21.15.27:0/2858783475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.404+0000 7f2e475f1740 1 --2- 172.21.15.27:0/2858783475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 0x55d2c9aba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.404+0000 7f2e475f1740 1 -- 172.21.15.27:0/2858783475 >> 172.21.15.27:0/2858783475 conn(0x55d2c9ab6680 msgr2=0x55d2c9ab6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.404+0000 7f2e475f1740 1 -- 172.21.15.27:0/2858783475 shutdown_connections 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.404+0000 7f2e475f1740 1 -- 172.21.15.27:0/2858783475 wait complete. 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e475f1740 1 -- start start 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e475f1740 1 Processor -- start 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9ac22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e475f1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9a1e4f0 con 0x55d2c9aba170 2024-06-06T08:43:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e475f1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d2c99a7ae0 con 0x55d2c9abaa90 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e475f1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d2c9ac7d40 con 0x55d2c9ac5350 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e475f1740 10 client.0 _ll_register_callbacks cb 0x55d2c99e09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9ac22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9ac22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54272/0 (socket says 172.21.15.27:54272) 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.405+0000 7f2e3e59c640 1 -- 172.21.15.27:0/862328878 learned_addr learned my addr 172.21.15.27:0/862328878 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e3ed9d640 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e3e59c640 1 -- 172.21.15.27:0/862328878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 msgr2=0x55d2c9ac2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e3e59c640 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e3e59c640 1 -- 172.21.15.27:0/862328878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9abaa90 msgr2=0x55d2c9ac2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e3e59c640 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e3e59c640 1 -- 172.21.15.27:0/862328878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c9732210 con 0x55d2c9aba170 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e3e59c640 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9ac22c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000ba80 tx=0x7f2e3000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc2cbdbf4a34f61 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e25ffb640 1 -- 172.21.15.27:0/862328878 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30011440 con 0x55d2c9aba170 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e25ffb640 1 -- 172.21.15.27:0/862328878 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f2e300115e0 con 0x55d2c9aba170 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e475f1740 1 -- 172.21.15.27:0/862328878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d2c9a31f10 con 0x55d2c9aba170 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e25ffb640 1 -- 172.21.15.27:0/862328878 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2e300117e0 con 0x55d2c9aba170 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e475f1740 10 client.16191 fetch_fsmap learned FSMap version 33 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e475f1740 1 -- 172.21.15.27:0/862328878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d2c97fe620 con 0x55d2c9aba170 2024-06-06T08:43:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.406+0000 7f2e25ffb640 1 -- 172.21.15.27:0/862328878 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d2c9a31f10 con 0x55d2c9aba170 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e25ffb640 1 -- 172.21.15.27:0/862328878 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f2e3001ba00 con 0x55d2c9aba170 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e475f1740 10 client.16191 fetch_fsmap finished waiting for FSMap version 33 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e475f1740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e475f1740 20 client.16191 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e475f1740 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d2c9ad5210 0x55d2c9ad76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e475f1740 4 client.16191 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e475f1740 1 -- 172.21.15.27:0/862328878 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2c9a2bc20 con 0x55d2c9ad5210 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d2c9ad5210 0x55d2c9ad76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e25ffb640 10 client.16191 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.407+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d2c9ad5210 0x55d2c9ad76b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb1cfcb46bb436cc server_cookie=49af80551b67e9c0 in_seq=0 out_seq=0 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e25ffb640 1 -- 172.21.15.27:0/862328878 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d2c9a2bc20 con 0x55d2c9ad5210 2024-06-06T08:43:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e25ffb640 10 client.16191 handle_command_reply: tid=0 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 client.16191 shutdown 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 -- 172.21.15.27:0/862328878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 msgr2=0x55d2c9ac22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9ac22c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000ba80 tx=0x7f2e3000bf40 comp rx=0 tx=0).stop 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 -- 172.21.15.27:0/862328878 shutdown_connections 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d2c9ad5210 0x55d2c9ad76b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e25ffb640 0 client.16191 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 --2- 172.21.15.27:0/862328878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9ac22c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 -- 172.21.15.27:0/862328878 >> 172.21.15.27:0/862328878 conn(0x55d2c9ab6680 msgr2=0x55d2c9abc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 -- 172.21.15.27:0/862328878 shutdown_connections 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 1 -- 172.21.15.27:0/862328878 wait complete. 2024-06-06T08:43:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.408+0000 7f2e475f1740 20 client.16191 trim_cache size 0 max 16384 2024-06-06T08:43:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e475f1740 1 Processor -- start 2024-06-06T08:43:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e475f1740 1 -- start start 2024-06-06T08:43:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 0x55d2c9abcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 0x55d2c9adcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e475f1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401b3970 con 0x55d2c9abaa90 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e475f1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401b11b0 con 0x55d2c9aba170 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e475f1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401af000 con 0x55d2c9ac5350 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54276/0 (socket says 172.21.15.27:54276) 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3dd9b640 1 -- 172.21.15.27:0/355708263 learned_addr learned my addr 172.21.15.27:0/355708263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3e59c640 1 --2- 172.21.15.27:0/355708263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 0x55d2c9abcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3ed9d640 1 --2- 172.21.15.27:0/355708263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 0x55d2c9adcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3dd9b640 1 -- 172.21.15.27:0/355708263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 msgr2=0x55d2c9adcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/355708263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 0x55d2c9adcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3dd9b640 1 -- 172.21.15.27:0/355708263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 msgr2=0x55d2c9abcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/355708263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 0x55d2c9abcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.424+0000 7f2e3dd9b640 1 -- 172.21.15.27:0/355708263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c9aba8c0 con 0x55d2c9abaa90 2024-06-06T08:43:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.425+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/355708263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4200 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55d2c9ac1b20 tx=0x7f2e34015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2125e998fb7f7687 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.425+0000 7f2e27fff640 1 -- 172.21.15.27:0/355708263 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34020450 con 0x55d2c9abaa90 2024-06-06T08:43:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.425+0000 7f2e27fff640 1 -- 172.21.15.27:0/355708263 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e34017070 con 0x55d2c9abaa90 2024-06-06T08:43:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.425+0000 7f2e27fff640 1 -- 172.21.15.27:0/355708263 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34027830 con 0x55d2c9abaa90 2024-06-06T08:43:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 -- 172.21.15.27:0/355708263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9abaa90 msgr2=0x55d2c9ac4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 --2- 172.21.15.27:0/355708263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4200 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55d2c9ac1b20 tx=0x7f2e34015000 comp rx=0 tx=0).stop 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 -- 172.21.15.27:0/355708263 shutdown_connections 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 --2- 172.21.15.27:0/355708263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9ac5350 0x55d2c9adcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 --2- 172.21.15.27:0/355708263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac4200 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 --2- 172.21.15.27:0/355708263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9aba170 0x55d2c9abcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 -- 172.21.15.27:0/355708263 >> 172.21.15.27:0/355708263 conn(0x55d2c9ab6680 msgr2=0x55d2c9ac3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 -- 172.21.15.27:0/355708263 shutdown_connections 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 -- 172.21.15.27:0/355708263 wait complete. 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 -- start start 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 Processor -- start 2024-06-06T08:43:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9acd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401af670 con 0x55d2c9aba170 2024-06-06T08:43:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401b0ae0 con 0x55d2c9ac5350 2024-06-06T08:43:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401afd40 con 0x55d2c9abaa90 2024-06-06T08:43:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e475f1740 10 client.0 _ll_register_callbacks cb 0x7f2e30004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9acd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e3e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9acd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54282/0 (socket says 172.21.15.27:54282) 2024-06-06T08:43:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e3e59c640 1 -- 172.21.15.27:0/344327473 learned_addr learned my addr 172.21.15.27:0/344327473 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.426+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e3ed9d640 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e3e59c640 1 -- 172.21.15.27:0/344327473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 msgr2=0x55d2c9ac8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e3e59c640 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e3e59c640 1 -- 172.21.15.27:0/344327473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9ac5350 msgr2=0x55d2c9ac90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e3e59c640 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e3e59c640 1 -- 172.21.15.27:0/344327473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c9ac98a0 con 0x55d2c9aba170 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e3e59c640 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9acd720 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2e3001cff0 tx=0x7f2e30005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9515bd2d38b0a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e25ffb640 1 -- 172.21.15.27:0/344327473 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3002d070 con 0x55d2c9aba170 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e25ffb640 1 -- 172.21.15.27:0/344327473 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f2e30011cd0 con 0x55d2c9aba170 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e475f1740 1 -- 172.21.15.27:0/344327473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e34017070 con 0x55d2c9aba170 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e25ffb640 1 -- 172.21.15.27:0/344327473 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2e3001b420 con 0x55d2c9aba170 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e475f1740 10 client.16197 fetch_fsmap learned FSMap version 33 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e475f1740 1 -- 172.21.15.27:0/344327473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d2c9b400c0 con 0x55d2c9aba170 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.427+0000 7f2e25ffb640 1 -- 172.21.15.27:0/344327473 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e34017070 con 0x55d2c9aba170 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.428+0000 7f2e25ffb640 1 -- 172.21.15.27:0/344327473 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f2e300108b0 con 0x55d2c9aba170 2024-06-06T08:43:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.428+0000 7f2e475f1740 10 client.16197 fetch_fsmap finished waiting for FSMap version 33 2024-06-06T08:43:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.428+0000 7f2e475f1740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.428+0000 7f2e475f1740 20 client.16197 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.428+0000 7f2e475f1740 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d2c9b408a0 0x55d2c9b42cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.428+0000 7f2e475f1740 4 client.16197 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:43:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.428+0000 7f2e475f1740 1 -- 172.21.15.27:0/344327473 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2c9a25f70 con 0x55d2c9b408a0 2024-06-06T08:43:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.428+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d2c9b408a0 0x55d2c9b42cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.429+0000 7f2e3dd9b640 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d2c9b408a0 0x55d2c9b42cc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f60ba541150825c server_cookie=b5dc9a6574c95993 in_seq=0 out_seq=0 2024-06-06T08:43:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:19.429+0000 7f2e25ffb640 10 client.16197 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:19.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:19 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3607456710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:43:19.866 INFO:tasks.cephfs.filesystem:scrub status for tag:5ccc9a12-fe19-4f51-aecc-bda187b537e4 - {'path': '/', 'tag': '5ccc9a12-fe19-4f51-aecc-bda187b537e4', 'options': 'recursive,force'} 2024-06-06T08:43:19.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:19 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3607456710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:43:20.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:19 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3607456710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:43:20.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:19 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[32331]: 2024-06-06T08:43:19.910+0000 7fc898df5700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T08:43:20.278 INFO:tasks.workunit.client.0.smithi027.stdout:6/179: rename d4/d9/dc/d18/f39 to d4/d1a/d2e/f3c 0 2024-06-06T08:43:20.283 INFO:tasks.workunit.client.0.smithi027.stdout:6/180: rmdir d4/d9/dc/d1c 39 2024-06-06T08:43:20.283 INFO:tasks.workunit.client.0.smithi027.stdout:6/181: chown d4/lf 570356 1 2024-06-06T08:43:20.284 INFO:tasks.workunit.client.0.smithi027.stdout:6/182: write d4/d9/dc/f38 [818115,52408] 0 2024-06-06T08:43:20.284 INFO:tasks.workunit.client.0.smithi027.stdout:6/183: fdatasync d4/de/f35 0 2024-06-06T08:43:20.284 INFO:tasks.workunit.client.0.smithi027.stdout:6/184: mknod d4/d9/dc/d18/c3d 0 2024-06-06T08:43:20.342 INFO:tasks.workunit.client.0.smithi027.stdout:4/95: sync 2024-06-06T08:43:20.343 INFO:tasks.workunit.client.0.smithi027.stdout:4/96: readlink d0/l9 0 2024-06-06T08:43:20.343 INFO:tasks.workunit.client.0.smithi027.stdout:1/103: sync 2024-06-06T08:43:20.344 INFO:tasks.workunit.client.0.smithi027.stdout:5/169: sync 2024-06-06T08:43:20.344 INFO:tasks.workunit.client.0.smithi027.stdout:8/99: sync 2024-06-06T08:43:20.344 INFO:tasks.workunit.client.0.smithi027.stdout:7/211: sync 2024-06-06T08:43:20.344 INFO:tasks.workunit.client.0.smithi027.stdout:3/103: sync 2024-06-06T08:43:20.344 INFO:tasks.workunit.client.0.smithi027.stdout:9/114: sync 2024-06-06T08:43:20.345 INFO:tasks.workunit.client.0.smithi027.stdout:4/97: rmdir d0/d1/d1b 0 2024-06-06T08:43:20.345 INFO:tasks.workunit.client.0.smithi027.stdout:4/98: write d0/f19 [369918,35150] 0 2024-06-06T08:43:20.345 INFO:tasks.workunit.client.0.smithi027.stdout:4/99: write d0/d1/ff [1139576,62367] 0 2024-06-06T08:43:20.346 INFO:tasks.workunit.client.0.smithi027.stdout:5/170: getdents d0/da/d27 0 2024-06-06T08:43:20.348 INFO:tasks.workunit.client.0.smithi027.stdout:1/104: symlink dc/de/d1c/l1f 0 2024-06-06T08:43:20.361 INFO:tasks.workunit.client.0.smithi027.stdout:8/100: dwrite d7/f9 [0,4194304] 0 2024-06-06T08:43:20.362 INFO:tasks.workunit.client.0.smithi027.stdout:9/115: write d3/f11 [156596,26715] 0 2024-06-06T08:43:20.364 INFO:tasks.workunit.client.0.smithi027.stdout:4/100: creat d0/f1c x:0 0 0 2024-06-06T08:43:20.366 INFO:tasks.workunit.client.0.smithi027.stdout:1/105: mknod dc/de/d1c/c20 0 2024-06-06T08:43:20.368 INFO:tasks.workunit.client.0.smithi027.stdout:7/212: mkdir d17/d47 0 2024-06-06T08:43:20.369 INFO:tasks.workunit.client.0.smithi027.stdout:7/213: symlink d17/d31/d2a/d3b/l48 0 2024-06-06T08:43:20.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/101: mknod d7/c18 0 2024-06-06T08:43:20.375 INFO:tasks.workunit.client.0.smithi027.stdout:4/101: mkdir d0/d1d 0 2024-06-06T08:43:20.375 INFO:tasks.workunit.client.0.smithi027.stdout:4/102: write d0/d1/f14 [5341061,128781] 0 2024-06-06T08:43:20.377 INFO:tasks.workunit.client.0.smithi027.stdout:0/105: rename d15/f24 to d15/d18/d23/f27 0 2024-06-06T08:43:20.378 INFO:tasks.workunit.client.0.smithi027.stdout:7/214: creat d17/d47/f49 x:0 0 0 2024-06-06T08:43:20.380 INFO:tasks.workunit.client.0.smithi027.stdout:9/116: getdents d3/da/dc/df/d23 0 2024-06-06T08:43:20.380 INFO:tasks.workunit.client.0.smithi027.stdout:9/117: chown d3/da/dc/c29 895 1 2024-06-06T08:43:20.388 INFO:tasks.workunit.client.0.smithi027.stdout:1/106: mkdir dc/de/d14/d21 0 2024-06-06T08:43:20.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/221: rename d6/dd/d46 to d6/dd/d3c/d55 0 2024-06-06T08:43:20.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/222: getdents d6/dc/d3f/d48 0 2024-06-06T08:43:20.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/223: chown d6/d1e/d2b 1366005 1 2024-06-06T08:43:20.394 INFO:tasks.workunit.client.0.smithi027.stdout:0/106: truncate f2 2367172 0 2024-06-06T08:43:20.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/224: write d6/dc/f44 [773524,16374] 0 2024-06-06T08:43:20.396 INFO:tasks.workunit.client.0.smithi027.stdout:2/225: fsync d6/dc/f25 0 2024-06-06T08:43:20.396 INFO:tasks.workunit.client.0.smithi027.stdout:6/185: link d4/d9/dc/d18/ca d4/de/c3e 0 2024-06-06T08:43:20.397 INFO:tasks.workunit.client.0.smithi027.stdout:9/118: fsync d3/f20 0 2024-06-06T08:43:20.398 INFO:tasks.workunit.client.0.smithi027.stdout:6/186: symlink d4/d9/dc/l3f 0 2024-06-06T08:43:20.398 INFO:tasks.workunit.client.0.smithi027.stdout:6/187: write d4/d9/f26 [411759,1537] 0 2024-06-06T08:43:20.398 INFO:tasks.workunit.client.0.smithi027.stdout:6/188: write d4/f10 [1173407,45298] 0 2024-06-06T08:43:20.400 INFO:tasks.workunit.client.0.smithi027.stdout:5/171: rename d0/da/d27 to d0/da/d37 0 2024-06-06T08:43:20.402 INFO:tasks.workunit.client.0.smithi027.stdout:1/107: getdents dc 0 2024-06-06T08:43:20.403 INFO:tasks.workunit.client.0.smithi027.stdout:1/108: readlink dc/de/d1c/l1f 0 2024-06-06T08:43:20.403 INFO:tasks.workunit.client.0.smithi027.stdout:6/189: dread d4/d9/f19 [0,4194304] 0 2024-06-06T08:43:20.403 INFO:tasks.workunit.client.0.smithi027.stdout:8/102: write d7/f9 [4782869,106961] 0 2024-06-06T08:43:20.408 INFO:tasks.workunit.client.0.smithi027.stdout:1/109: dread dc/de/f13 [0,4194304] 0 2024-06-06T08:43:20.412 INFO:tasks.workunit.client.0.smithi027.stdout:7/215: mknod d17/d32/c4a 0 2024-06-06T08:43:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.415+0000 7f2e25ffb640 1 -- 172.21.15.27:0/344327473 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d2c9a25f70 con 0x55d2c9b408a0 2024-06-06T08:43:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.415+0000 7f2e25ffb640 10 client.16197 handle_command_reply: tid=0 2024-06-06T08:43:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.415+0000 7f2e475f1740 1 client.16197 shutdown 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 -- 172.21.15.27:0/344327473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 msgr2=0x55d2c9acd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9acd720 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2e3001cff0 tx=0x7f2e30005700 comp rx=0 tx=0).stop 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 -- 172.21.15.27:0/344327473 shutdown_connections 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d2c9b408a0 0x55d2c9b42cc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d2c9ac5350 0x55d2c9ac90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d2c9abaa90 0x55d2c9ac8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 --2- 172.21.15.27:0/344327473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d2c9aba170 0x55d2c9acd720 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 -- 172.21.15.27:0/344327473 >> 172.21.15.27:0/344327473 conn(0x55d2c9ab6680 msgr2=0x55d2c9ac4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e25ffb640 0 client.16197 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 -- 172.21.15.27:0/344327473 shutdown_connections 2024-06-06T08:43:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 1 -- 172.21.15.27:0/344327473 wait complete. 2024-06-06T08:43:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.416+0000 7f2e475f1740 20 client.16197 trim_cache size 0 max 16384 2024-06-06T08:43:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:43:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:43:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.417+0000 7f2e44809640 1 -- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2e280a2da0 msgr2=0x7f2e280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.417+0000 7f2e44809640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2e280a2da0 0x7f2e280a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e34005d80 tx=0x7f2e34012760 comp rx=0 tx=0).stop 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.417+0000 7f2e44809640 1 -- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 msgr2=0x7f2e4019ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.417+0000 7f2e44809640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 0x7f2e4019ec50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009dc0 tx=0x7f2e38003f10 comp rx=0 tx=0).stop 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.418+0000 7f2e44809640 1 -- 172.21.15.27:0/2157660924 shutdown_connections 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.418+0000 7f2e44809640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2e280a2da0 0x7f2e280a5260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.418+0000 7f2e44809640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e4010bfb0 0x7f2e4019ec50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.418+0000 7f2e44809640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e40103590 0x7f2e4019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.418+0000 7f2e44809640 1 --2- 172.21.15.27:0/2157660924 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e40102c70 0x7f2e4019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.418+0000 7f2e44809640 1 -- 172.21.15.27:0/2157660924 >> 172.21.15.27:0/2157660924 conn(0x7f2e40077a20 msgr2=0x7f2e400fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.418+0000 7f2e44809640 1 -- 172.21.15.27:0/2157660924 shutdown_connections 2024-06-06T08:43:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:20.418+0000 7f2e44809640 1 -- 172.21.15.27:0/2157660924 wait complete. 2024-06-06T08:43:20.419 INFO:tasks.workunit.client.0.smithi027.stdout:2/226: dwrite d6/dd/f23 [0,4194304] 0 2024-06-06T08:43:20.419 INFO:tasks.workunit.client.0.smithi027.stdout:2/227: truncate d6/dc/f40 810759 0 2024-06-06T08:43:20.422 INFO:tasks.workunit.client.0.smithi027.stdout:5/172: dwrite d0/f24 [0,4194304] 0 2024-06-06T08:43:20.425 INFO:tasks.workunit.client.0.smithi027.stdout:1/110: rename dc/de/d1c to dc/de/d22 0 2024-06-06T08:43:20.426 INFO:tasks.workunit.client.0.smithi027.stdout:4/103: rmdir d0/d1/d6/d8 39 2024-06-06T08:43:20.428 INFO:tasks.workunit.client.0.smithi027.stdout:5/173: write d0/da/d37/f2f [3596362,6660] 0 2024-06-06T08:43:20.429 INFO:tasks.workunit.client.0.smithi027.stdout:4/104: mknod d0/d1/d6/d8/c1e 0 2024-06-06T08:43:20.429 INFO:tasks.workunit.client.0.smithi027.stdout:4/105: chown d0/d1/d6/l10 260 1 2024-06-06T08:43:20.430 INFO:tasks.workunit.client.0.smithi027.stdout:7/216: mknod d17/d47/c4b 0 2024-06-06T08:43:20.430 INFO:tasks.workunit.client.0.smithi027.stdout:7/217: rename d17/d31/d2c to d17/d31/d2c/d37/d4c 22 2024-06-06T08:43:20.442 INFO:tasks.workunit.client.0.smithi027.stdout:1/111: symlink dc/l23 0 2024-06-06T08:43:20.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/174: mknod d0/da/db/c38 0 2024-06-06T08:43:20.446 INFO:tasks.workunit.client.0.smithi027.stdout:9/119: creat d3/da/dc/f2e x:0 0 0 2024-06-06T08:43:20.446 INFO:tasks.workunit.client.0.smithi027.stdout:9/120: getdents d3/da/dc/df/d23 0 2024-06-06T08:43:20.447 INFO:tasks.workunit.client.0.smithi027.stdout:4/106: unlink d0/f19 0 2024-06-06T08:43:20.447 INFO:tasks.workunit.client.0.smithi027.stdout:7/218: dwrite d17/d32/f33 [0,4194304] 0 2024-06-06T08:43:20.448 INFO:tasks.workunit.client.0.smithi027.stdout:5/175: truncate d0/f4 701286 0 2024-06-06T08:43:20.449 INFO:tasks.workunit.client.0.smithi027.stdout:0/107: mkdir d15/d18/d23/d28 0 2024-06-06T08:43:20.450 INFO:tasks.workunit.client.0.smithi027.stdout:0/108: read f9 [471032,20826] 0 2024-06-06T08:43:20.450 INFO:tasks.workunit.client.0.smithi027.stdout:5/176: write d0/fe [4177855,52599] 0 2024-06-06T08:43:20.450 INFO:tasks.workunit.client.0.smithi027.stdout:5/177: chown d0/da 18442410 1 2024-06-06T08:43:20.450 INFO:tasks.workunit.client.0.smithi027.stdout:5/178: fdatasync d0/f4 0 2024-06-06T08:43:20.451 INFO:tasks.workunit.client.0.smithi027.stdout:7/219: creat d17/d47/f4d x:0 0 0 2024-06-06T08:43:20.451 INFO:tasks.workunit.client.0.smithi027.stdout:7/220: fdatasync d17/d31/f45 0 2024-06-06T08:43:20.451 INFO:tasks.workunit.client.0.smithi027.stdout:7/221: fsync d17/f1e 0 2024-06-06T08:43:20.456 INFO:tasks.workunit.client.0.smithi027.stdout:9/121: write d3/f4 [7982233,103758] 0 2024-06-06T08:43:20.458 INFO:tasks.workunit.client.0.smithi027.stdout:9/122: creat d3/da/dc/df/f2f x:0 0 0 2024-06-06T08:43:20.458 INFO:tasks.workunit.client.0.smithi027.stdout:9/123: fsync d3/da/dc/df/f2b 0 2024-06-06T08:43:20.459 INFO:tasks.workunit.client.0.smithi027.stdout:1/112: dread dc/de/d14/f15 [0,4194304] 0 2024-06-06T08:43:20.459 INFO:tasks.workunit.client.0.smithi027.stdout:9/124: creat d3/da/dc/f30 x:0 0 0 2024-06-06T08:43:20.459 INFO:tasks.workunit.client.0.smithi027.stdout:9/125: chown d3/da/dc/c29 11570 1 2024-06-06T08:43:20.462 INFO:tasks.workunit.client.0.smithi027.stdout:9/126: symlink d3/da/dc/df/l31 0 2024-06-06T08:43:20.463 INFO:tasks.workunit.client.0.smithi027.stdout:0/109: dread f13 [0,4194304] 0 2024-06-06T08:43:20.463 INFO:tasks.workunit.client.0.smithi027.stdout:0/110: write f10 [3625593,76734] 0 2024-06-06T08:43:20.465 INFO:tasks.workunit.client.0.smithi027.stdout:1/113: mknod dc/de/d14/c24 0 2024-06-06T08:43:20.466 INFO:tasks.workunit.client.0.smithi027.stdout:6/190: link d4/d9/dc/d18/c3d d4/c40 0 2024-06-06T08:43:20.468 INFO:tasks.workunit.client.0.smithi027.stdout:6/191: mknod d4/d9/dc/d18/c41 0 2024-06-06T08:43:20.469 INFO:tasks.workunit.client.0.smithi027.stdout:6/192: symlink d4/l42 0 2024-06-06T08:43:20.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/179: dwrite d0/da/db/f19 [0,4194304] 0 2024-06-06T08:43:20.469 INFO:tasks.workunit.client.0.smithi027.stdout:6/193: chown d4/d9/f26 2032625 1 2024-06-06T08:43:20.469 INFO:tasks.workunit.client.0.smithi027.stdout:1/114: dread dc/fd [0,4194304] 0 2024-06-06T08:43:20.470 INFO:tasks.workunit.client.0.smithi027.stdout:7/222: dwrite d17/f21 [0,4194304] 0 2024-06-06T08:43:20.471 INFO:tasks.workunit.client.0.smithi027.stdout:1/115: symlink dc/de/d14/d21/l25 0 2024-06-06T08:43:20.471 INFO:tasks.workunit.client.0.smithi027.stdout:1/116: stat dc/de/c11 0 2024-06-06T08:43:20.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/194: chown d4/d1a/d2e/f3c 272 1 2024-06-06T08:43:20.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/195: read d4/f5 [4481493,112163] 0 2024-06-06T08:43:20.473 INFO:tasks.workunit.client.0.smithi027.stdout:7/223: symlink d17/d31/d2c/l4e 0 2024-06-06T08:43:20.474 INFO:tasks.workunit.client.0.smithi027.stdout:1/117: rmdir dc/de/d14 39 2024-06-06T08:43:20.478 INFO:tasks.workunit.client.0.smithi027.stdout:7/224: mkdir d17/d32/d46/d4f 0 2024-06-06T08:43:20.479 INFO:tasks.workunit.client.0.smithi027.stdout:0/111: dwrite d15/d18/d23/f27 [0,4194304] 0 2024-06-06T08:43:20.479 INFO:tasks.workunit.client.0.smithi027.stdout:0/112: fsync fc 0 2024-06-06T08:43:20.480 INFO:tasks.workunit.client.0.smithi027.stdout:7/225: rename d17/d32/f33 to d17/d32/d46/d4f/f50 0 2024-06-06T08:43:20.480 INFO:tasks.workunit.client.0.smithi027.stdout:7/226: write d17/d47/f4d [289892,62424] 0 2024-06-06T08:43:20.484 INFO:tasks.workunit.client.0.smithi027.stdout:9/127: dwrite d3/f20 [0,4194304] 0 2024-06-06T08:43:20.484 INFO:tasks.workunit.client.0.smithi027.stdout:0/113: unlink c12 0 2024-06-06T08:43:20.485 INFO:tasks.workunit.client.0.smithi027.stdout:7/227: truncate d17/d32/f2f 40761 0 2024-06-06T08:43:20.486 INFO:tasks.workunit.client.0.smithi027.stdout:0/114: dread d15/d18/f1c [0,4194304] 0 2024-06-06T08:43:20.486 INFO:tasks.workunit.client.0.smithi027.stdout:0/115: chown d15/d18/d23/f25 483337 1 2024-06-06T08:43:20.487 INFO:tasks.workunit.client.0.smithi027.stdout:1/118: dread dc/de/d14/f1a [0,4194304] 0 2024-06-06T08:43:20.487 INFO:tasks.workunit.client.0.smithi027.stdout:1/119: chown dc/de/ff 12 1 2024-06-06T08:43:20.487 INFO:tasks.workunit.client.0.smithi027.stdout:1/120: chown dc/fd 539 1 2024-06-06T08:43:20.488 INFO:tasks.workunit.client.0.smithi027.stdout:0/116: creat d15/d18/d23/f29 x:0 0 0 2024-06-06T08:43:20.488 INFO:tasks.workunit.client.0.smithi027.stdout:5/180: write d0/da/db/f11 [2967655,23692] 0 2024-06-06T08:43:20.489 INFO:tasks.workunit.client.0.smithi027.stdout:6/196: getdents d4/d9 0 2024-06-06T08:43:20.490 INFO:tasks.workunit.client.0.smithi027.stdout:0/117: mkdir d15/d18/d23/d28/d2a 0 2024-06-06T08:43:20.494 INFO:tasks.workunit.client.0.smithi027.stdout:1/121: write dc/de/d14/f1a [2248558,19983] 0 2024-06-06T08:43:20.494 INFO:tasks.workunit.client.0.smithi027.stdout:0/118: truncate f14 4595003 0 2024-06-06T08:43:20.494 INFO:tasks.workunit.client.0.smithi027.stdout:0/119: readlink ld 0 2024-06-06T08:43:20.494 INFO:tasks.workunit.client.0.smithi027.stdout:1/122: write dc/fd [1325302,56380] 0 2024-06-06T08:43:20.495 INFO:tasks.workunit.client.0.smithi027.stdout:5/181: rmdir d0/da/db 39 2024-06-06T08:43:20.496 INFO:tasks.workunit.client.0.smithi027.stdout:0/120: readlink le 0 2024-06-06T08:43:20.498 INFO:tasks.workunit.client.0.smithi027.stdout:6/197: write d4/d9/dc/d18/f2f [2239850,126092] 0 2024-06-06T08:43:20.498 INFO:tasks.workunit.client.0.smithi027.stdout:6/198: dread - d4/d9/f28 zero size 2024-06-06T08:43:20.498 INFO:tasks.workunit.client.0.smithi027.stdout:6/199: write d4/d9/f19 [236372,52576] 0 2024-06-06T08:43:20.498 INFO:tasks.workunit.client.0.smithi027.stdout:0/121: mknod d15/d18/d23/d28/d2a/c2b 0 2024-06-06T08:43:20.498 INFO:tasks.workunit.client.0.smithi027.stdout:9/128: dread d3/da/dc/f21 [0,4194304] 0 2024-06-06T08:43:20.499 INFO:tasks.workunit.client.0.smithi027.stdout:5/182: mknod d0/da/d20/c39 0 2024-06-06T08:43:20.499 INFO:tasks.workunit.client.0.smithi027.stdout:5/183: write d0/f25 [1456197,13026] 0 2024-06-06T08:43:20.499 INFO:tasks.workunit.client.0.smithi027.stdout:1/123: rmdir dc 39 2024-06-06T08:43:20.500 INFO:tasks.workunit.client.0.smithi027.stdout:6/200: creat d4/d9/dc/f43 x:0 0 0 2024-06-06T08:43:20.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/184: unlink d0/f3 0 2024-06-06T08:43:20.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/185: dread - d0/da/db/f28 zero size 2024-06-06T08:43:20.503 INFO:tasks.workunit.client.0.smithi027.stdout:1/124: symlink dc/de/d22/l26 0 2024-06-06T08:43:20.505 INFO:tasks.workunit.client.0.smithi027.stdout:9/129: symlink d3/da/l32 0 2024-06-06T08:43:20.507 INFO:tasks.workunit.client.0.smithi027.stdout:9/130: link d3/da/dc/df/d23/c27 d3/da/dc/df/c33 0 2024-06-06T08:43:20.510 INFO:tasks.workunit.client.0.smithi027.stdout:9/131: mkdir d3/da/dc/df/d34 0 2024-06-06T08:43:20.512 INFO:tasks.workunit.client.0.smithi027.stdout:9/132: mknod d3/da/dc/df/d23/c35 0 2024-06-06T08:43:20.514 INFO:tasks.workunit.client.0.smithi027.stdout:9/133: mkdir d3/da/dc/df/d23/d36 0 2024-06-06T08:43:20.516 INFO:tasks.workunit.client.0.smithi027.stdout:9/134: creat d3/da/dc/df/d23/d36/f37 x:0 0 0 2024-06-06T08:43:20.518 INFO:tasks.workunit.client.0.smithi027.stdout:1/125: dread dc/de/ff [0,4194304] 0 2024-06-06T08:43:20.518 INFO:tasks.workunit.client.0.smithi027.stdout:1/126: write dc/f1d [193435,56373] 0 2024-06-06T08:43:20.518 INFO:tasks.workunit.client.0.smithi027.stdout:0/122: dwrite f13 [0,4194304] 0 2024-06-06T08:43:20.518 INFO:tasks.workunit.client.0.smithi027.stdout:9/135: mknod d3/da/dc/df/d34/c38 0 2024-06-06T08:43:20.519 INFO:tasks.workunit.client.0.smithi027.stdout:9/136: fsync d3/da/f2c 0 2024-06-06T08:43:20.521 INFO:tasks.workunit.client.0.smithi027.stdout:0/123: getdents d15 0 2024-06-06T08:43:20.521 INFO:tasks.workunit.client.0.smithi027.stdout:9/137: creat d3/da/dc/df/f39 x:0 0 0 2024-06-06T08:43:20.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/127: rmdir dc/de/d14 39 2024-06-06T08:43:20.522 INFO:tasks.workunit.client.0.smithi027.stdout:9/138: unlink d3/da/dc/c29 0 2024-06-06T08:43:20.522 INFO:tasks.workunit.client.0.smithi027.stdout:5/186: dread d0/da/db/f19 [0,4194304] 0 2024-06-06T08:43:20.523 INFO:tasks.workunit.client.0.smithi027.stdout:6/201: dwrite d4/d9/f19 [0,4194304] 0 2024-06-06T08:43:20.526 INFO:tasks.workunit.client.0.smithi027.stdout:5/187: chown d0/da/db/c31 45592 1 2024-06-06T08:43:20.526 INFO:tasks.workunit.client.0.smithi027.stdout:1/128: unlink dc/c16 0 2024-06-06T08:43:20.526 INFO:tasks.workunit.client.0.smithi027.stdout:1/129: chown dc/l23 100735 1 2024-06-06T08:43:20.529 INFO:tasks.workunit.client.0.smithi027.stdout:5/188: mknod d0/da/d20/c3a 0 2024-06-06T08:43:20.530 INFO:tasks.workunit.client.0.smithi027.stdout:1/130: creat dc/de/d14/d21/f27 x:0 0 0 2024-06-06T08:43:20.534 INFO:tasks.workunit.client.0.smithi027.stdout:6/202: mkdir d4/d1a/d2e/d44 0 2024-06-06T08:43:20.535 INFO:tasks.workunit.client.0.smithi027.stdout:5/189: creat d0/da/d37/d2b/f3b x:0 0 0 2024-06-06T08:43:20.538 INFO:tasks.workunit.client.0.smithi027.stdout:1/131: mknod dc/c28 0 2024-06-06T08:43:20.540 INFO:tasks.workunit.client.0.smithi027.stdout:6/203: mkdir d4/de/d45 0 2024-06-06T08:43:20.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/139: dwrite d3/da/dc/df/d23/d36/f37 [0,4194304] 0 2024-06-06T08:43:20.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/140: chown d3/da/dc/df/f39 29236 1 2024-06-06T08:43:20.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/141: write d3/da/dc/f30 [699424,5832] 0 2024-06-06T08:43:20.540 INFO:tasks.workunit.client.0.smithi027.stdout:7/228: rename d17/d31/d2c to d17/d32/d46/d51 0 2024-06-06T08:43:20.542 INFO:tasks.workunit.client.0.smithi027.stdout:1/132: mknod dc/de/d14/d21/c29 0 2024-06-06T08:43:20.543 INFO:tasks.workunit.client.0.smithi027.stdout:9/142: mkdir d3/da/dc/df/d23/d36/d3a 0 2024-06-06T08:43:20.544 INFO:tasks.workunit.client.0.smithi027.stdout:6/204: mkdir d4/d1a/d46 0 2024-06-06T08:43:20.544 INFO:tasks.workunit.client.0.smithi027.stdout:6/205: chown d4/d9/dc/d18/f24 500 1 2024-06-06T08:43:20.544 INFO:tasks.workunit.client.0.smithi027.stdout:1/133: mknod dc/de/d14/c2a 0 2024-06-06T08:43:20.544 INFO:tasks.workunit.client.0.smithi027.stdout:1/134: rename dc to dc/de/d14/d21/d2b 22 2024-06-06T08:43:20.545 INFO:tasks.workunit.client.0.smithi027.stdout:1/135: read - dc/de/d14/f19 zero size 2024-06-06T08:43:20.546 INFO:tasks.workunit.client.0.smithi027.stdout:1/136: creat dc/de/d22/f2c x:0 0 0 2024-06-06T08:43:20.546 INFO:tasks.workunit.client.0.smithi027.stdout:6/206: dread d4/d9/dc/d1c/f30 [0,4194304] 0 2024-06-06T08:43:20.546 INFO:tasks.workunit.client.0.smithi027.stdout:6/207: readlink d4/d9/dc/d18/l15 0 2024-06-06T08:43:20.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/137: symlink dc/l2d 0 2024-06-06T08:43:20.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/138: stat dc/de/d14/d21 0 2024-06-06T08:43:20.547 INFO:tasks.workunit.client.0.smithi027.stdout:7/229: mkdir d17/d32/d46/d51/d52 0 2024-06-06T08:43:20.548 INFO:tasks.workunit.client.0.smithi027.stdout:7/230: dread - d17/f3f zero size 2024-06-06T08:43:20.548 INFO:tasks.workunit.client.0.smithi027.stdout:6/208: symlink d4/l47 0 2024-06-06T08:43:20.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/231: mkdir d17/d32/d46/d4f/d53 0 2024-06-06T08:43:20.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/232: dread - d17/f1e zero size 2024-06-06T08:43:20.552 INFO:tasks.workunit.client.0.smithi027.stdout:7/233: creat d17/d32/d46/d51/f54 x:0 0 0 2024-06-06T08:43:20.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/143: dread d3/da/dc/f21 [0,4194304] 0 2024-06-06T08:43:20.553 INFO:tasks.workunit.client.0.smithi027.stdout:7/234: creat d17/d32/d46/d4f/f55 x:0 0 0 2024-06-06T08:43:20.554 INFO:tasks.workunit.client.0.smithi027.stdout:9/144: getdents d3/da/dc/df/d34 0 2024-06-06T08:43:20.556 INFO:tasks.workunit.client.0.smithi027.stdout:1/139: getdents dc 0 2024-06-06T08:43:20.557 INFO:tasks.workunit.client.0.smithi027.stdout:1/140: write dc/f1d [801671,50399] 0 2024-06-06T08:43:20.557 INFO:tasks.workunit.client.0.smithi027.stdout:1/141: write dc/de/d14/f1a [2754870,74962] 0 2024-06-06T08:43:20.558 INFO:tasks.workunit.client.0.smithi027.stdout:9/145: mknod d3/da/c3b 0 2024-06-06T08:43:20.558 INFO:tasks.workunit.client.0.smithi027.stdout:1/142: mknod dc/de/d14/d21/c2e 0 2024-06-06T08:43:20.558 INFO:tasks.workunit.client.0.smithi027.stdout:5/190: dwrite d0/f29 [0,4194304] 0 2024-06-06T08:43:20.559 INFO:tasks.workunit.client.0.smithi027.stdout:5/191: stat d0/da/db/d1a/c34 0 2024-06-06T08:43:20.560 INFO:tasks.workunit.client.0.smithi027.stdout:1/143: dread f5 [0,4194304] 0 2024-06-06T08:43:20.562 INFO:tasks.workunit.client.0.smithi027.stdout:5/192: fdatasync d0/f1 0 2024-06-06T08:43:20.563 INFO:tasks.workunit.client.0.smithi027.stdout:5/193: fdatasync d0/da/db/f19 0 2024-06-06T08:43:20.566 INFO:tasks.workunit.client.0.smithi027.stdout:1/144: symlink dc/l2f 0 2024-06-06T08:43:20.570 INFO:tasks.workunit.client.0.smithi027.stdout:6/209: unlink d4/d9/dc/l23 0 2024-06-06T08:43:20.570 INFO:tasks.workunit.client.0.smithi027.stdout:6/210: write d4/d9/dc/d1c/f2a [147449,113904] 0 2024-06-06T08:43:20.574 INFO:tasks.workunit.client.0.smithi027.stdout:7/235: dwrite ff [0,4194304] 0 2024-06-06T08:43:20.587 INFO:tasks.workunit.client.0.smithi027.stdout:6/211: write d4/d13/f21 [4008454,109261] 0 2024-06-06T08:43:20.587 INFO:tasks.workunit.client.0.smithi027.stdout:5/194: dwrite d0/da/db/f17 [0,4194304] 0 2024-06-06T08:43:20.587 INFO:tasks.workunit.client.0.smithi027.stdout:6/212: stat d4/f25 0 2024-06-06T08:43:20.588 INFO:tasks.workunit.client.0.smithi027.stdout:1/145: dwrite dc/de/d22/f2c [0,4194304] 0 2024-06-06T08:43:20.589 INFO:tasks.workunit.client.0.smithi027.stdout:6/213: creat d4/de/d45/f48 x:0 0 0 2024-06-06T08:43:20.592 INFO:tasks.workunit.client.0.smithi027.stdout:5/195: link d0/l14 d0/l3c 0 2024-06-06T08:43:20.592 INFO:tasks.workunit.client.0.smithi027.stdout:5/196: dread - d0/da/d37/d2b/f3b zero size 2024-06-06T08:43:20.592 INFO:tasks.workunit.client.0.smithi027.stdout:6/214: fdatasync d4/d13/f21 0 2024-06-06T08:43:20.592 INFO:tasks.workunit.client.0.smithi027.stdout:5/197: write d0/da/db/f11 [4623897,34761] 0 2024-06-06T08:43:20.592 INFO:tasks.workunit.client.0.smithi027.stdout:6/215: chown d4/d9/dc/d1c/f2a 28294 1 2024-06-06T08:43:20.592 INFO:tasks.workunit.client.0.smithi027.stdout:6/216: write d4/d13/f21 [1861367,65413] 0 2024-06-06T08:43:20.593 INFO:tasks.workunit.client.0.smithi027.stdout:5/198: creat d0/da/d37/d2b/f3d x:0 0 0 2024-06-06T08:43:20.595 INFO:tasks.workunit.client.0.smithi027.stdout:6/217: mkdir d4/d9/d49 0 2024-06-06T08:43:20.595 INFO:tasks.workunit.client.0.smithi027.stdout:6/218: truncate d4/d9/dc/d18/f2f 4880515 0 2024-06-06T08:43:20.597 INFO:tasks.workunit.client.0.smithi027.stdout:6/219: mknod d4/d9/c4a 0 2024-06-06T08:43:20.597 INFO:tasks.workunit.client.0.smithi027.stdout:6/220: chown d4/d9/dc/l1e 25 1 2024-06-06T08:43:20.597 INFO:tasks.workunit.client.0.smithi027.stdout:6/221: read f0 [7074581,126037] 0 2024-06-06T08:43:20.598 INFO:tasks.workunit.client.0.smithi027.stdout:6/222: unlink f0 0 2024-06-06T08:43:20.606 INFO:tasks.workunit.client.0.smithi027.stdout:1/146: dwrite dc/de/d14/f19 [0,4194304] 0 2024-06-06T08:43:20.606 INFO:tasks.workunit.client.0.smithi027.stdout:5/199: dread d0/da/d37/f2f [4194304,4194304] 0 2024-06-06T08:43:20.606 INFO:tasks.workunit.client.0.smithi027.stdout:5/200: stat d0/da/d20/c3a 0 2024-06-06T08:43:20.607 INFO:tasks.workunit.client.0.smithi027.stdout:1/147: rmdir dc/de/d22 39 2024-06-06T08:43:20.608 INFO:tasks.workunit.client.0.smithi027.stdout:1/148: getdents dc/de 0 2024-06-06T08:43:20.609 INFO:tasks.workunit.client.0.smithi027.stdout:1/149: creat dc/f30 x:0 0 0 2024-06-06T08:43:20.613 INFO:tasks.workunit.client.0.smithi027.stdout:1/150: mknod dc/de/d14/c31 0 2024-06-06T08:43:20.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/151: mkdir dc/de/d14/d21/d32 0 2024-06-06T08:43:20.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/152: chown dc/fd 4523865 1 2024-06-06T08:43:20.616 INFO:tasks.workunit.client.0.smithi027.stdout:1/153: unlink dc/de/ff 0 2024-06-06T08:43:20.616 INFO:tasks.workunit.client.0.smithi027.stdout:6/223: dwrite d4/f5 [8388608,4194304] 0 2024-06-06T08:43:20.617 INFO:tasks.workunit.client.0.smithi027.stdout:6/224: getdents d4/d1a/d2e/d44 0 2024-06-06T08:43:20.617 INFO:tasks.workunit.client.0.smithi027.stdout:1/154: mknod dc/de/c33 0 2024-06-06T08:43:20.618 INFO:tasks.workunit.client.0.smithi027.stdout:1/155: symlink dc/l34 0 2024-06-06T08:43:20.619 INFO:tasks.workunit.client.0.smithi027.stdout:1/156: creat dc/de/d22/f35 x:0 0 0 2024-06-06T08:43:20.619 INFO:tasks.workunit.client.0.smithi027.stdout:1/157: chown dc/de/d14/d21 3751 1 2024-06-06T08:43:20.621 INFO:tasks.workunit.client.0.smithi027.stdout:1/158: unlink dc/de/c33 0 2024-06-06T08:43:20.622 INFO:tasks.workunit.client.0.smithi027.stdout:1/159: creat dc/f36 x:0 0 0 2024-06-06T08:43:20.622 INFO:tasks.workunit.client.0.smithi027.stdout:1/160: write f5 [1007063,41143] 0 2024-06-06T08:43:20.622 INFO:tasks.workunit.client.0.smithi027.stdout:6/225: truncate d4/d9/dc/d18/f2f 779573 0 2024-06-06T08:43:20.623 INFO:tasks.workunit.client.0.smithi027.stdout:6/226: creat d4/d1a/d46/f4b x:0 0 0 2024-06-06T08:43:20.624 INFO:tasks.workunit.client.0.smithi027.stdout:6/227: creat d4/d9/d49/f4c x:0 0 0 2024-06-06T08:43:20.626 INFO:tasks.workunit.client.0.smithi027.stdout:5/201: dwrite d0/da/db/f1b [0,4194304] 0 2024-06-06T08:43:20.626 INFO:tasks.workunit.client.0.smithi027.stdout:5/202: readlink d0/da/db/l33 0 2024-06-06T08:43:20.627 INFO:tasks.workunit.client.0.smithi027.stdout:5/203: readlink d0/l14 0 2024-06-06T08:43:20.629 INFO:tasks.workunit.client.0.smithi027.stdout:6/228: dread d4/d9/f26 [0,4194304] 0 2024-06-06T08:43:20.644 INFO:tasks.workunit.client.0.smithi027.stdout:1/161: dwrite f5 [0,4194304] 0 2024-06-06T08:43:20.646 INFO:tasks.workunit.client.0.smithi027.stdout:1/162: mkdir dc/de/d14/d21/d32/d37 0 2024-06-06T08:43:20.649 INFO:tasks.workunit.client.0.smithi027.stdout:6/229: dwrite d4/d9/dc/d18/f2f [0,4194304] 0 2024-06-06T08:43:20.650 INFO:tasks.workunit.client.0.smithi027.stdout:6/230: write d4/de/f11 [3126637,45747] 0 2024-06-06T08:43:20.650 INFO:tasks.workunit.client.0.smithi027.stdout:5/204: dwrite d0/fe [4194304,4194304] 0 2024-06-06T08:43:20.652 INFO:tasks.workunit.client.0.smithi027.stdout:6/231: symlink d4/d1a/l4d 0 2024-06-06T08:43:20.653 INFO:tasks.workunit.client.0.smithi027.stdout:6/232: write d4/de/d45/f48 [178077,19266] 0 2024-06-06T08:43:20.655 INFO:tasks.workunit.client.0.smithi027.stdout:6/233: symlink d4/d9/dc/d1c/l4e 0 2024-06-06T08:43:20.655 INFO:tasks.workunit.client.0.smithi027.stdout:6/234: stat d4/c40 0 2024-06-06T08:43:20.660 INFO:tasks.workunit.client.0.smithi027.stdout:5/205: dread d0/f1 [0,4194304] 0 2024-06-06T08:43:20.664 INFO:tasks.workunit.client.0.smithi027.stdout:5/206: dread d0/da/d20/f36 [0,4194304] 0 2024-06-06T08:43:20.664 INFO:tasks.workunit.client.0.smithi027.stdout:5/207: fsync d0/da/d37/f2f 0 2024-06-06T08:43:20.666 INFO:tasks.workunit.client.0.smithi027.stdout:5/208: symlink d0/da/d37/d2b/l3e 0 2024-06-06T08:43:20.667 INFO:tasks.workunit.client.0.smithi027.stdout:5/209: mkdir d0/da/d3f 0 2024-06-06T08:43:20.672 INFO:tasks.workunit.client.0.smithi027.stdout:6/235: dwrite d4/d9/d49/f4c [0,4194304] 0 2024-06-06T08:43:20.674 INFO:tasks.workunit.client.0.smithi027.stdout:6/236: mknod d4/d9/dc/d18/c4f 0 2024-06-06T08:43:20.678 INFO:tasks.workunit.client.0.smithi027.stdout:5/210: write d0/da/d37/f2f [7786852,2577] 0 2024-06-06T08:43:20.679 INFO:tasks.workunit.client.0.smithi027.stdout:5/211: symlink d0/da/db/d1a/l40 0 2024-06-06T08:43:20.679 INFO:tasks.workunit.client.0.smithi027.stdout:5/212: chown d0/da/db/f28 11855456 1 2024-06-06T08:43:20.691 INFO:tasks.workunit.client.0.smithi027.stdout:6/237: rename d4/d9/d49/f4c to d4/d9/dc/d1c/f50 0 2024-06-06T08:43:20.691 INFO:tasks.workunit.client.0.smithi027.stdout:6/238: stat d4/d9/dc/d18 0 2024-06-06T08:43:20.693 INFO:tasks.workunit.client.0.smithi027.stdout:6/239: mknod d4/d9/dc/c51 0 2024-06-06T08:43:20.693 INFO:tasks.workunit.client.0.smithi027.stdout:6/240: chown d4/f10 8097376 1 2024-06-06T08:43:20.694 INFO:tasks.workunit.client.0.smithi027.stdout:5/213: dwrite d0/da/db/f11 [0,4194304] 0 2024-06-06T08:43:20.696 INFO:tasks.workunit.client.0.smithi027.stdout:5/214: rename d0/da/d37/d2b/f3b to d0/da/d37/f41 0 2024-06-06T08:43:20.696 INFO:tasks.workunit.client.0.smithi027.stdout:6/241: mkdir d4/d9/dc/d1c/d52 0 2024-06-06T08:43:20.697 INFO:tasks.workunit.client.0.smithi027.stdout:5/215: rmdir d0/da/d3f 0 2024-06-06T08:43:20.698 INFO:tasks.workunit.client.0.smithi027.stdout:5/216: stat d0/da/db/c31 0 2024-06-06T08:43:20.704 INFO:tasks.workunit.client.0.smithi027.stdout:6/242: dread d4/d9/dc/d18/f2f [0,4194304] 0 2024-06-06T08:43:20.742 INFO:tasks.workunit.client.0.smithi027.stdout:5/217: dwrite d0/fe [0,4194304] 0 2024-06-06T08:43:20.742 INFO:tasks.workunit.client.0.smithi027.stdout:5/218: chown d0/f1 939 1 2024-06-06T08:43:20.752 INFO:tasks.workunit.client.0.smithi027.stdout:5/219: dread d0/da/db/f19 [0,4194304] 0 2024-06-06T08:43:20.753 INFO:tasks.workunit.client.0.smithi027.stdout:5/220: symlink d0/da/db/l42 0 2024-06-06T08:43:20.767 INFO:tasks.workunit.client.0.smithi027.stdout:5/221: dwrite d0/f25 [0,4194304] 0 2024-06-06T08:43:20.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/222: write d0/f25 [2150939,97113] 0 2024-06-06T08:43:20.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/223: read - d0/da/d37/d2b/f3d zero size 2024-06-06T08:43:20.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/224: read d0/da/db/f10 [681873,28547] 0 2024-06-06T08:43:20.781 INFO:tasks.workunit.client.0.smithi027.stdout:5/225: rmdir d0/da/d37/d2b 39 2024-06-06T08:43:20.781 INFO:tasks.workunit.client.0.smithi027.stdout:5/226: chown d0/l3c 0 1 2024-06-06T08:43:20.795 INFO:tasks.workunit.client.0.smithi027.stdout:5/227: dwrite d0/da/db/f19 [4194304,4194304] 0 2024-06-06T08:43:20.796 INFO:tasks.workunit.client.0.smithi027.stdout:5/228: stat d0/c2c 0 2024-06-06T08:43:20.798 INFO:tasks.workunit.client.0.smithi027.stdout:5/229: unlink d0/da/db/d1a/l40 0 2024-06-06T08:43:20.799 INFO:tasks.workunit.client.0.smithi027.stdout:5/230: creat d0/da/f43 x:0 0 0 2024-06-06T08:43:20.800 INFO:tasks.workunit.client.0.smithi027.stdout:5/231: creat d0/f44 x:0 0 0 2024-06-06T08:43:20.801 INFO:tasks.workunit.client.0.smithi027.stdout:5/232: mknod d0/da/db/d1a/c45 0 2024-06-06T08:43:20.801 INFO:tasks.workunit.client.0.smithi027.stdout:5/233: mkdir d0/da/db/d46 0 2024-06-06T08:43:20.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/234: mkdir d0/da/d47 0 2024-06-06T08:43:20.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/235: chown d0/da/f26 486 1 2024-06-06T08:43:20.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/236: write d0/f4 [677865,109761] 0 2024-06-06T08:43:20.805 INFO:tasks.workunit.client.0.smithi027.stdout:5/237: link d0/f24 d0/da/d37/f48 0 2024-06-06T08:43:20.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:20 smithi027 ceph-mon[32335]: pgmap v95: 129 pgs: 129 active+clean; 398 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 49 MiB/s wr, 1.74k op/s 2024-06-06T08:43:20.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:20 smithi027 ceph-mon[32335]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T08:43:20.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:20 smithi203 ceph-mon[36579]: pgmap v95: 129 pgs: 129 active+clean; 398 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 49 MiB/s wr, 1.74k op/s 2024-06-06T08:43:20.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:20 smithi203 ceph-mon[36579]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T08:43:21.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:20 smithi176 ceph-mon[36786]: pgmap v95: 129 pgs: 129 active+clean; 398 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 34 MiB/s rd, 49 MiB/s wr, 1.74k op/s 2024-06-06T08:43:21.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:20 smithi176 ceph-mon[36786]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T08:43:21.477 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-06-06T08:43:21.477 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-06-06T08:43:21.536 INFO:tasks.workunit.client.0.smithi027.stdout:6/243: rename d4/d1a/l4d to d4/d9/dc/d18/l53 0 2024-06-06T08:43:21.539 INFO:tasks.workunit.client.0.smithi027.stdout:2/228: sync 2024-06-06T08:43:21.540 INFO:tasks.workunit.client.0.smithi027.stdout:7/236: sync 2024-06-06T08:43:21.540 INFO:tasks.workunit.client.0.smithi027.stdout:8/103: sync 2024-06-06T08:43:21.540 INFO:tasks.workunit.client.0.smithi027.stdout:1/163: sync 2024-06-06T08:43:21.540 INFO:tasks.workunit.client.0.smithi027.stdout:4/107: sync 2024-06-06T08:43:21.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/146: sync 2024-06-06T08:43:21.540 INFO:tasks.workunit.client.0.smithi027.stdout:3/104: sync 2024-06-06T08:43:21.540 INFO:tasks.workunit.client.0.smithi027.stdout:0/124: sync 2024-06-06T08:43:21.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/147: chown d3/da/l1a 58006 1 2024-06-06T08:43:21.541 INFO:tasks.workunit.client.0.smithi027.stdout:5/238: sync 2024-06-06T08:43:21.541 INFO:tasks.workunit.client.0.smithi027.stdout:5/239: write d0/da/d37/d2b/f3d [531082,82251] 0 2024-06-06T08:43:21.541 INFO:tasks.workunit.client.0.smithi027.stdout:2/229: symlink d6/d1e/l56 0 2024-06-06T08:43:21.541 INFO:tasks.workunit.client.0.smithi027.stdout:2/230: read - d6/d3b/f52 zero size 2024-06-06T08:43:21.542 INFO:tasks.workunit.client.0.smithi027.stdout:7/237: write d17/f23 [109830,30822] 0 2024-06-06T08:43:21.542 INFO:tasks.workunit.client.0.smithi027.stdout:8/104: dread d7/db/f11 [0,4194304] 0 2024-06-06T08:43:21.543 INFO:tasks.workunit.client.0.smithi027.stdout:5/240: unlink d0/da/db/c38 0 2024-06-06T08:43:21.543 INFO:tasks.workunit.client.0.smithi027.stdout:5/241: stat d0/f29 0 2024-06-06T08:43:21.543 INFO:tasks.workunit.client.0.smithi027.stdout:5/242: chown d0/da/d20 394705107 1 2024-06-06T08:43:21.545 INFO:tasks.workunit.client.0.smithi027.stdout:5/243: truncate d0/da/db/f19 6719239 0 2024-06-06T08:43:21.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/238: creat d17/d31/d2a/d3b/f56 x:0 0 0 2024-06-06T08:43:21.549 INFO:tasks.workunit.client.0.smithi027.stdout:6/244: getdents d4/d1a 0 2024-06-06T08:43:21.550 INFO:tasks.workunit.client.0.smithi027.stdout:7/239: link d17/d31/d2a/c36 d17/d32/d46/d51/d37/c57 0 2024-06-06T08:43:21.552 INFO:tasks.workunit.client.0.smithi027.stdout:1/164: dread dc/de/d14/f1a [0,4194304] 0 2024-06-06T08:43:21.552 INFO:tasks.workunit.client.0.smithi027.stdout:1/165: stat dc/c12 0 2024-06-06T08:43:21.558 INFO:tasks.workunit.client.0.smithi027.stdout:3/105: dread d1/f13 [0,4194304] 0 2024-06-06T08:43:21.558 INFO:tasks.workunit.client.0.smithi027.stdout:7/240: creat d17/f58 x:0 0 0 2024-06-06T08:43:21.559 INFO:tasks.workunit.client.0.smithi027.stdout:7/241: dread - d17/d32/f3a zero size 2024-06-06T08:43:21.559 INFO:tasks.workunit.client.0.smithi027.stdout:1/166: getdents dc/de/d22 0 2024-06-06T08:43:21.561 INFO:tasks.workunit.client.0.smithi027.stdout:1/167: rename dc/de/d14/c24 to dc/de/d14/d21/c38 0 2024-06-06T08:43:21.564 INFO:tasks.workunit.client.0.smithi027.stdout:3/106: rmdir d1 39 2024-06-06T08:43:21.564 INFO:tasks.workunit.client.0.smithi027.stdout:5/244: write d0/fe [1629421,32705] 0 2024-06-06T08:43:21.564 INFO:tasks.workunit.client.0.smithi027.stdout:4/108: dread d0/f13 [4194304,4194304] 0 2024-06-06T08:43:21.564 INFO:tasks.workunit.client.0.smithi027.stdout:1/168: rename l1 to dc/de/l39 0 2024-06-06T08:43:21.565 INFO:tasks.workunit.client.0.smithi027.stdout:0/125: dwrite d15/f21 [0,4194304] 0 2024-06-06T08:43:21.565 INFO:tasks.workunit.client.0.smithi027.stdout:0/126: write d15/f16 [202439,99745] 0 2024-06-06T08:43:21.565 INFO:tasks.workunit.client.0.smithi027.stdout:3/107: truncate d1/f6 915180 0 2024-06-06T08:43:21.565 INFO:tasks.workunit.client.0.smithi027.stdout:3/108: write d1/d8/f9 [647967,22819] 0 2024-06-06T08:43:21.567 INFO:tasks.workunit.client.0.smithi027.stdout:4/109: mknod d0/c1f 0 2024-06-06T08:43:21.567 INFO:tasks.workunit.client.0.smithi027.stdout:8/105: write d7/dc/f12 [1627423,117079] 0 2024-06-06T08:43:21.567 INFO:tasks.workunit.client.0.smithi027.stdout:4/110: write d0/fd [13183911,61223] 0 2024-06-06T08:43:21.568 INFO:tasks.workunit.client.0.smithi027.stdout:8/106: write d7/d13/f17 [268325,101723] 0 2024-06-06T08:43:21.569 INFO:tasks.workunit.client.0.smithi027.stdout:1/169: symlink dc/de/l3a 0 2024-06-06T08:43:21.569 INFO:tasks.workunit.client.0.smithi027.stdout:1/170: write dc/de/f13 [277277,47629] 0 2024-06-06T08:43:21.569 INFO:tasks.workunit.client.0.smithi027.stdout:5/245: rmdir d0/da/d37 39 2024-06-06T08:43:21.573 INFO:tasks.workunit.client.0.smithi027.stdout:0/127: mknod d15/c2c 0 2024-06-06T08:43:21.575 INFO:tasks.workunit.client.0.smithi027.stdout:4/111: creat d0/f20 x:0 0 0 2024-06-06T08:43:21.576 INFO:tasks.workunit.client.0.smithi027.stdout:1/171: creat dc/de/d22/f3b x:0 0 0 2024-06-06T08:43:21.576 INFO:tasks.workunit.client.0.smithi027.stdout:1/172: truncate dc/de/f13 454389 0 2024-06-06T08:43:21.577 INFO:tasks.workunit.client.0.smithi027.stdout:0/128: chown l3 3 1 2024-06-06T08:43:21.578 INFO:tasks.workunit.client.0.smithi027.stdout:4/112: mkdir d0/d1/d6/d8/d21 0 2024-06-06T08:43:21.579 INFO:tasks.workunit.client.0.smithi027.stdout:0/129: mkdir d15/d18/d23/d28/d2a/d2d 0 2024-06-06T08:43:21.579 INFO:tasks.workunit.client.0.smithi027.stdout:6/245: dwrite d4/f10 [0,4194304] 0 2024-06-06T08:43:21.579 INFO:tasks.workunit.client.0.smithi027.stdout:6/246: read - d4/f29 zero size 2024-06-06T08:43:21.580 INFO:tasks.workunit.client.0.smithi027.stdout:7/242: dwrite d17/d32/f3a [0,4194304] 0 2024-06-06T08:43:21.580 INFO:tasks.workunit.client.0.smithi027.stdout:7/243: write d17/f3f [377694,83538] 0 2024-06-06T08:43:21.580 INFO:tasks.workunit.client.0.smithi027.stdout:7/244: write d17/f2e [187243,20777] 0 2024-06-06T08:43:21.580 INFO:tasks.workunit.client.0.smithi027.stdout:7/245: truncate d17/d32/f2f 643325 0 2024-06-06T08:43:21.581 INFO:tasks.workunit.client.0.smithi027.stdout:5/246: symlink d0/l49 0 2024-06-06T08:43:21.584 INFO:tasks.workunit.client.0.smithi027.stdout:4/113: getdents d0/d1 0 2024-06-06T08:43:21.584 INFO:tasks.workunit.client.0.smithi027.stdout:4/114: chown d0/d1/d6/d8/c15 0 1 2024-06-06T08:43:21.584 INFO:tasks.workunit.client.0.smithi027.stdout:7/246: fdatasync d17/f2e 0 2024-06-06T08:43:21.585 INFO:tasks.workunit.client.0.smithi027.stdout:4/115: mknod d0/d1/d6/d8/d21/c22 0 2024-06-06T08:43:21.585 INFO:tasks.workunit.client.0.smithi027.stdout:4/116: write d0/d1/f14 [14005605,71149] 0 2024-06-06T08:43:21.585 INFO:tasks.workunit.client.0.smithi027.stdout:3/109: dwrite d1/d8/f1a [0,4194304] 0 2024-06-06T08:43:21.586 INFO:tasks.workunit.client.0.smithi027.stdout:0/130: truncate f10 80884 0 2024-06-06T08:43:21.587 INFO:tasks.workunit.client.0.smithi027.stdout:7/247: dread f14 [0,4194304] 0 2024-06-06T08:43:21.587 INFO:tasks.workunit.client.0.smithi027.stdout:0/131: rmdir d15/d18/d23/d28 39 2024-06-06T08:43:21.592 INFO:tasks.workunit.client.0.smithi027.stdout:6/247: chown d4/d1a/d2e/l37 77 1 2024-06-06T08:43:21.592 INFO:tasks.workunit.client.0.smithi027.stdout:6/248: getdents d4/d9/dc/d1c/d52 0 2024-06-06T08:43:21.593 INFO:tasks.workunit.client.0.smithi027.stdout:6/249: creat d4/d9/d49/f54 x:0 0 0 2024-06-06T08:43:21.593 INFO:tasks.workunit.client.0.smithi027.stdout:6/250: fdatasync d4/d1a/d46/f4b 0 2024-06-06T08:43:21.595 INFO:tasks.workunit.client.0.smithi027.stdout:5/247: rename d0/da/db/f11 to d0/da/d37/d2b/f4a 0 2024-06-06T08:43:21.597 INFO:tasks.workunit.client.0.smithi027.stdout:4/117: mknod d0/d1d/c23 0 2024-06-06T08:43:21.598 INFO:tasks.workunit.client.0.smithi027.stdout:8/107: truncate d7/dc/f12 131244 0 2024-06-06T08:43:21.599 INFO:tasks.workunit.client.0.smithi027.stdout:0/132: dread fb [0,4194304] 0 2024-06-06T08:43:21.600 INFO:tasks.workunit.client.0.smithi027.stdout:1/173: dwrite dc/de/d14/d21/f27 [0,4194304] 0 2024-06-06T08:43:21.601 INFO:tasks.workunit.client.0.smithi027.stdout:0/133: truncate d15/d18/d23/f27 1451900 0 2024-06-06T08:43:21.601 INFO:tasks.workunit.client.0.smithi027.stdout:3/110: creat d1/f20 x:0 0 0 2024-06-06T08:43:21.603 INFO:tasks.workunit.client.0.smithi027.stdout:0/134: creat d15/d18/d23/f2e x:0 0 0 2024-06-06T08:43:21.603 INFO:tasks.workunit.client.0.smithi027.stdout:0/135: chown fb 3998 1 2024-06-06T08:43:21.606 INFO:tasks.workunit.client.0.smithi027.stdout:7/248: getdents d17 0 2024-06-06T08:43:21.609 INFO:tasks.workunit.client.0.smithi027.stdout:5/248: write d0/da/db/f28 [3063146,121701] 0 2024-06-06T08:43:21.609 INFO:tasks.workunit.client.0.smithi027.stdout:4/118: mkdir d0/d1/d24 0 2024-06-06T08:43:21.609 INFO:tasks.workunit.client.0.smithi027.stdout:5/249: stat d0/da/f35 0 2024-06-06T08:43:21.615 INFO:tasks.workunit.client.0.smithi027.stdout:1/174: read dc/de/f13 [421746,73602] 0 2024-06-06T08:43:21.615 INFO:tasks.workunit.client.0.smithi027.stdout:1/175: fsync dc/de/d14/f19 0 2024-06-06T08:43:21.621 INFO:tasks.workunit.client.0.smithi027.stdout:6/251: dwrite d4/d1a/f2c [0,4194304] 0 2024-06-06T08:43:21.623 INFO:tasks.workunit.client.0.smithi027.stdout:3/111: rmdir d1 39 2024-06-06T08:43:21.631 INFO:tasks.workunit.client.0.smithi027.stdout:5/250: dwrite d0/da/d37/d2b/f4a [0,4194304] 0 2024-06-06T08:43:21.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/249: dread d17/d32/d46/d4f/f50 [0,4194304] 0 2024-06-06T08:43:21.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/250: chown d17/d31/d2a 281289 1 2024-06-06T08:43:21.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/176: stat dc/de/l39 0 2024-06-06T08:43:21.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/252: getdents d4/de/d45 0 2024-06-06T08:43:21.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/253: truncate d4/d9/f2d 105753 0 2024-06-06T08:43:21.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/254: chown d4/d1a/c1f 180 1 2024-06-06T08:43:21.633 INFO:tasks.workunit.client.0.smithi027.stdout:7/251: mkdir d17/d31/d2a/d59 0 2024-06-06T08:43:21.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/177: mknod dc/de/c3c 0 2024-06-06T08:43:21.634 INFO:tasks.workunit.client.0.smithi027.stdout:4/119: write d0/d1/f2 [2885452,88089] 0 2024-06-06T08:43:21.635 INFO:tasks.workunit.client.0.smithi027.stdout:7/252: symlink d17/d32/d46/l5a 0 2024-06-06T08:43:21.635 INFO:tasks.workunit.client.0.smithi027.stdout:7/253: write d17/d31/d2a/d3b/f56 [913201,113745] 0 2024-06-06T08:43:21.636 INFO:tasks.workunit.client.0.smithi027.stdout:1/178: mkdir dc/de/d3d 0 2024-06-06T08:43:21.638 INFO:tasks.workunit.client.0.smithi027.stdout:5/251: mknod d0/c4b 0 2024-06-06T08:43:21.640 INFO:tasks.workunit.client.0.smithi027.stdout:7/254: rmdir d17/d32/d46/d4f/d53 0 2024-06-06T08:43:21.641 INFO:tasks.workunit.client.0.smithi027.stdout:1/179: creat dc/de/d3d/f3e x:0 0 0 2024-06-06T08:43:21.641 INFO:tasks.workunit.client.0.smithi027.stdout:1/180: write dc/de/d22/f3b [661102,108269] 0 2024-06-06T08:43:21.642 INFO:tasks.workunit.client.0.smithi027.stdout:7/255: write d17/d32/d46/d51/f35 [164084,122073] 0 2024-06-06T08:43:21.642 INFO:tasks.workunit.client.0.smithi027.stdout:7/256: dread - d17/d32/d46/f41 zero size 2024-06-06T08:43:21.642 INFO:tasks.workunit.client.0.smithi027.stdout:7/257: write d17/d32/f2f [970789,75743] 0 2024-06-06T08:43:21.642 INFO:tasks.workunit.client.0.smithi027.stdout:1/181: creat dc/de/d3d/f3f x:0 0 0 2024-06-06T08:43:21.643 INFO:tasks.workunit.client.0.smithi027.stdout:1/182: write dc/de/f13 [1425672,53319] 0 2024-06-06T08:43:21.643 INFO:tasks.workunit.client.0.smithi027.stdout:7/258: creat d17/d31/d2a/d3b/f5b x:0 0 0 2024-06-06T08:43:21.643 INFO:tasks.workunit.client.0.smithi027.stdout:1/183: mknod dc/de/d14/d21/d32/d37/c40 0 2024-06-06T08:43:21.644 INFO:tasks.workunit.client.0.smithi027.stdout:3/112: dwrite d1/d8/f1a [0,4194304] 0 2024-06-06T08:43:21.644 INFO:tasks.workunit.client.0.smithi027.stdout:1/184: creat dc/de/d14/d21/d32/d37/f41 x:0 0 0 2024-06-06T08:43:21.645 INFO:tasks.workunit.client.0.smithi027.stdout:4/120: dread d0/fd [8388608,4194304] 0 2024-06-06T08:43:21.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/113: mknod d1/da/c21 0 2024-06-06T08:43:21.646 INFO:tasks.workunit.client.0.smithi027.stdout:1/185: creat dc/de/d3d/f42 x:0 0 0 2024-06-06T08:43:21.648 INFO:tasks.workunit.client.0.smithi027.stdout:4/121: creat d0/d1/d6/f25 x:0 0 0 2024-06-06T08:43:21.650 INFO:tasks.workunit.client.0.smithi027.stdout:3/114: mkdir d1/d8/d22 0 2024-06-06T08:43:21.650 INFO:tasks.workunit.client.0.smithi027.stdout:3/115: truncate d1/f20 487495 0 2024-06-06T08:43:21.650 INFO:tasks.workunit.client.0.smithi027.stdout:3/116: stat d1/d8/d22 0 2024-06-06T08:43:21.654 INFO:tasks.workunit.client.0.smithi027.stdout:3/117: mknod d1/d8/c23 0 2024-06-06T08:43:21.656 INFO:tasks.workunit.client.0.smithi027.stdout:7/259: dread d17/f26 [0,4194304] 0 2024-06-06T08:43:21.656 INFO:tasks.workunit.client.0.smithi027.stdout:7/260: getdents d17/d32/d46/d51/d52 0 2024-06-06T08:43:21.656 INFO:tasks.workunit.client.0.smithi027.stdout:3/118: symlink d1/da/l24 0 2024-06-06T08:43:21.656 INFO:tasks.workunit.client.0.smithi027.stdout:3/119: truncate d1/d8/f15 715019 0 2024-06-06T08:43:21.661 INFO:tasks.workunit.client.0.smithi027.stdout:5/252: dwrite d0/da/f43 [0,4194304] 0 2024-06-06T08:43:21.661 INFO:tasks.workunit.client.0.smithi027.stdout:5/253: chown d0/da/d37/d2b/f3d 13 1 2024-06-06T08:43:21.665 INFO:tasks.workunit.client.0.smithi027.stdout:7/261: dread d17/f3f [0,4194304] 0 2024-06-06T08:43:21.666 INFO:tasks.workunit.client.0.smithi027.stdout:1/186: dread dc/de/d22/f2c [0,4194304] 0 2024-06-06T08:43:21.666 INFO:tasks.workunit.client.0.smithi027.stdout:1/187: write dc/f1d [1238163,126037] 0 2024-06-06T08:43:21.670 INFO:tasks.workunit.client.0.smithi027.stdout:1/188: mknod dc/de/d3d/c43 0 2024-06-06T08:43:21.672 INFO:tasks.workunit.client.0.smithi027.stdout:5/254: write d0/da/f43 [1366361,53378] 0 2024-06-06T08:43:21.673 INFO:tasks.workunit.client.0.smithi027.stdout:1/189: creat dc/de/d3d/f44 x:0 0 0 2024-06-06T08:43:21.673 INFO:tasks.workunit.client.0.smithi027.stdout:1/190: dread - dc/de/d3d/f44 zero size 2024-06-06T08:43:21.677 INFO:tasks.workunit.client.0.smithi027.stdout:0/136: rename d15/c2c to d15/d18/d23/d28/d2a/d2d/c2f 0 2024-06-06T08:43:21.679 INFO:tasks.workunit.client.0.smithi027.stdout:0/137: mkdir d15/d18/d23/d28/d2a/d30 0 2024-06-06T08:43:21.681 INFO:tasks.workunit.client.0.smithi027.stdout:0/138: truncate fb 641613 0 2024-06-06T08:43:21.682 INFO:tasks.workunit.client.0.smithi027.stdout:0/139: mkdir d15/d18/d23/d28/d31 0 2024-06-06T08:43:21.683 INFO:tasks.workunit.client.0.smithi027.stdout:7/262: dwrite d17/d32/d46/f41 [0,4194304] 0 2024-06-06T08:43:21.684 INFO:tasks.workunit.client.0.smithi027.stdout:5/255: dread d0/da/f13 [0,4194304] 0 2024-06-06T08:43:21.684 INFO:tasks.workunit.client.0.smithi027.stdout:5/256: readlink d0/da/l1c 0 2024-06-06T08:43:21.684 INFO:tasks.workunit.client.0.smithi027.stdout:5/257: truncate d0/da/f26 230772 0 2024-06-06T08:43:21.688 INFO:tasks.workunit.client.0.smithi027.stdout:7/263: truncate d17/f1d 3101743 0 2024-06-06T08:43:21.689 INFO:tasks.workunit.client.0.smithi027.stdout:5/258: mkdir d0/da/db/d46/d4c 0 2024-06-06T08:43:21.693 INFO:tasks.workunit.client.0.smithi027.stdout:1/191: write dc/de/d14/f15 [1998668,120565] 0 2024-06-06T08:43:21.693 INFO:tasks.workunit.client.0.smithi027.stdout:7/264: dread d17/f23 [0,4194304] 0 2024-06-06T08:43:21.696 INFO:tasks.workunit.client.0.smithi027.stdout:1/192: getdents dc/de/d22 0 2024-06-06T08:43:21.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/265: creat d17/d32/d46/d51/f5c x:0 0 0 2024-06-06T08:43:21.699 INFO:tasks.workunit.client.0.smithi027.stdout:1/193: symlink dc/de/d14/d21/l45 0 2024-06-06T08:43:21.699 INFO:tasks.workunit.client.0.smithi027.stdout:1/194: write dc/de/d3d/f42 [196215,26174] 0 2024-06-06T08:43:21.699 INFO:tasks.workunit.client.0.smithi027.stdout:1/195: chown dc/l2f 79260486 1 2024-06-06T08:43:21.699 INFO:tasks.workunit.client.0.smithi027.stdout:1/196: dread - dc/de/d3d/f44 zero size 2024-06-06T08:43:21.706 INFO:tasks.workunit.client.0.smithi027.stdout:4/122: link d0/d1/ce d0/d1/d6/d8/c26 0 2024-06-06T08:43:21.707 INFO:tasks.workunit.client.0.smithi027.stdout:7/266: write d17/d32/d46/f41 [1060599,69715] 0 2024-06-06T08:43:21.707 INFO:tasks.workunit.client.0.smithi027.stdout:4/123: mknod d0/c27 0 2024-06-06T08:43:21.708 INFO:tasks.workunit.client.0.smithi027.stdout:4/124: mknod d0/d1/c28 0 2024-06-06T08:43:21.708 INFO:tasks.workunit.client.0.smithi027.stdout:4/125: readlink d0/d1/d6/l18 0 2024-06-06T08:43:21.709 INFO:tasks.workunit.client.0.smithi027.stdout:7/267: rmdir d17/d31/d2a/d59 0 2024-06-06T08:43:21.710 INFO:tasks.workunit.client.0.smithi027.stdout:4/126: unlink d0/d1d/c23 0 2024-06-06T08:43:21.712 INFO:tasks.workunit.client.0.smithi027.stdout:0/140: rename d15/c19 to d15/d18/d23/d28/d2a/c32 0 2024-06-06T08:43:21.714 INFO:tasks.workunit.client.0.smithi027.stdout:0/141: creat d15/d18/d23/d28/d2a/f33 x:0 0 0 2024-06-06T08:43:21.715 INFO:tasks.workunit.client.0.smithi027.stdout:0/142: truncate f4 4033229 0 2024-06-06T08:43:21.718 INFO:tasks.workunit.client.0.smithi027.stdout:5/259: rename d0/da/d37/c32 to d0/c4d 0 2024-06-06T08:43:21.719 INFO:tasks.workunit.client.0.smithi027.stdout:7/268: dread d17/f21 [0,4194304] 0 2024-06-06T08:43:21.719 INFO:tasks.workunit.client.0.smithi027.stdout:1/197: rename dc/de/c3c to dc/de/d14/d21/d32/d37/c46 0 2024-06-06T08:43:21.721 INFO:tasks.workunit.client.0.smithi027.stdout:5/260: creat d0/f4e x:0 0 0 2024-06-06T08:43:21.721 INFO:tasks.workunit.client.0.smithi027.stdout:4/127: dread d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:21.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/269: creat d17/d47/f5d x:0 0 0 2024-06-06T08:43:21.722 INFO:tasks.workunit.client.0.smithi027.stdout:1/198: creat dc/de/f47 x:0 0 0 2024-06-06T08:43:21.722 INFO:tasks.workunit.client.0.smithi027.stdout:1/199: truncate dc/de/f47 590636 0 2024-06-06T08:43:21.725 INFO:tasks.workunit.client.0.smithi027.stdout:0/143: write d15/f21 [2222709,97155] 0 2024-06-06T08:43:21.725 INFO:tasks.workunit.client.0.smithi027.stdout:5/261: creat d0/da/d20/f4f x:0 0 0 2024-06-06T08:43:21.725 INFO:tasks.workunit.client.0.smithi027.stdout:0/144: write f10 [705203,23245] 0 2024-06-06T08:43:21.726 INFO:tasks.workunit.client.0.smithi027.stdout:7/270: mkdir d17/d32/d46/d4f/d5e 0 2024-06-06T08:43:21.726 INFO:tasks.workunit.client.0.smithi027.stdout:7/271: truncate d17/d31/d2a/d3b/f42 809752 0 2024-06-06T08:43:21.728 INFO:tasks.workunit.client.0.smithi027.stdout:5/262: rmdir d0/da/d20 39 2024-06-06T08:43:21.739 INFO:tasks.workunit.client.0.smithi027.stdout:5/263: rmdir d0/da 39 2024-06-06T08:43:21.740 INFO:tasks.workunit.client.0.smithi027.stdout:5/264: truncate d0/f4 1593836 0 2024-06-06T08:43:21.743 INFO:tasks.workunit.client.0.smithi027.stdout:0/145: read f13 [2611155,48640] 0 2024-06-06T08:43:21.745 INFO:tasks.workunit.client.0.smithi027.stdout:1/200: dwrite dc/de/d22/f2c [0,4194304] 0 2024-06-06T08:43:21.745 INFO:tasks.workunit.client.0.smithi027.stdout:1/201: readlink l2 0 2024-06-06T08:43:21.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/265: rename d0/da/l1c to d0/da/db/d1a/l50 0 2024-06-06T08:43:21.748 INFO:tasks.workunit.client.0.smithi027.stdout:1/202: rename dc/de/d22/c20 to dc/de/d14/d21/d32/c48 0 2024-06-06T08:43:21.748 INFO:tasks.workunit.client.0.smithi027.stdout:4/128: dwrite d0/d1/ff [0,4194304] 0 2024-06-06T08:43:21.749 INFO:tasks.workunit.client.0.smithi027.stdout:4/129: readlink d0/d1/d6/l10 0 2024-06-06T08:43:21.749 INFO:tasks.workunit.client.0.smithi027.stdout:4/130: write d0/d1/f2 [1955711,35649] 0 2024-06-06T08:43:21.750 INFO:tasks.workunit.client.0.smithi027.stdout:5/266: truncate d0/da/db/f17 2123388 0 2024-06-06T08:43:21.750 INFO:tasks.workunit.client.0.smithi027.stdout:5/267: write d0/f4 [563304,59539] 0 2024-06-06T08:43:21.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/268: chown d0/da/d37/d2b/l3e 25 1 2024-06-06T08:43:21.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/269: chown d0/da/db/d1a/c34 9 1 2024-06-06T08:43:21.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/270: fsync d0/da/db/f28 0 2024-06-06T08:43:21.751 INFO:tasks.workunit.client.0.smithi027.stdout:1/203: creat dc/de/d14/f49 x:0 0 0 2024-06-06T08:43:21.754 INFO:tasks.workunit.client.0.smithi027.stdout:1/204: write dc/de/d22/f3b [196997,120904] 0 2024-06-06T08:43:21.754 INFO:tasks.workunit.client.0.smithi027.stdout:1/205: truncate dc/de/d14/f49 436867 0 2024-06-06T08:43:21.754 INFO:tasks.workunit.client.0.smithi027.stdout:1/206: stat dc/de/d14/d21/c38 0 2024-06-06T08:43:21.754 INFO:tasks.workunit.client.0.smithi027.stdout:1/207: fsync dc/de/d3d/f42 0 2024-06-06T08:43:21.755 INFO:tasks.workunit.client.0.smithi027.stdout:1/208: mkdir dc/de/d3d/d4a 0 2024-06-06T08:43:21.757 INFO:tasks.workunit.client.0.smithi027.stdout:1/209: symlink dc/de/d3d/l4b 0 2024-06-06T08:43:21.759 INFO:tasks.workunit.client.0.smithi027.stdout:1/210: link l7 dc/de/d14/d21/l4c 0 2024-06-06T08:43:21.764 INFO:tasks.workunit.client.0.smithi027.stdout:4/131: read d0/f13 [11660732,117114] 0 2024-06-06T08:43:21.766 INFO:tasks.workunit.client.0.smithi027.stdout:4/132: mkdir d0/d1/d29 0 2024-06-06T08:43:21.767 INFO:tasks.workunit.client.0.smithi027.stdout:7/272: link d17/f58 d17/d32/d46/d51/f5f 0 2024-06-06T08:43:21.770 INFO:tasks.workunit.client.0.smithi027.stdout:4/133: mkdir d0/d1d/d2a 0 2024-06-06T08:43:21.777 INFO:tasks.workunit.client.0.smithi027.stdout:0/146: dwrite f13 [4194304,4194304] 0 2024-06-06T08:43:21.777 INFO:tasks.workunit.client.0.smithi027.stdout:5/271: dwrite d0/da/f26 [0,4194304] 0 2024-06-06T08:43:21.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/272: link d0/f4 d0/da/db/f51 0 2024-06-06T08:43:21.781 INFO:tasks.workunit.client.0.smithi027.stdout:5/273: unlink d0/da/d20/c3a 0 2024-06-06T08:43:21.783 INFO:tasks.workunit.client.0.smithi027.stdout:7/273: dread d17/d32/f38 [0,4194304] 0 2024-06-06T08:43:21.787 INFO:tasks.workunit.client.0.smithi027.stdout:7/274: dread d17/f2e [0,4194304] 0 2024-06-06T08:43:21.788 INFO:tasks.workunit.client.0.smithi027.stdout:7/275: creat d17/f60 x:0 0 0 2024-06-06T08:43:21.793 INFO:tasks.workunit.client.0.smithi027.stdout:0/147: dwrite f13 [4194304,4194304] 0 2024-06-06T08:43:21.794 INFO:tasks.workunit.client.0.smithi027.stdout:7/276: getdents d17/d32/d46/d51 0 2024-06-06T08:43:21.795 INFO:tasks.workunit.client.0.smithi027.stdout:7/277: read - d17/f1e zero size 2024-06-06T08:43:21.795 INFO:tasks.workunit.client.0.smithi027.stdout:7/278: rmdir d17 39 2024-06-06T08:43:21.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/134: dwrite d0/d1/f12 [0,4194304] 0 2024-06-06T08:43:21.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/135: write d0/fd [8805695,60493] 0 2024-06-06T08:43:21.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/136: fsync d0/f1c 0 2024-06-06T08:43:21.799 INFO:tasks.workunit.client.0.smithi027.stdout:4/137: mknod d0/d1/d6/d8/d21/c2b 0 2024-06-06T08:43:21.800 INFO:tasks.workunit.client.0.smithi027.stdout:4/138: symlink d0/d1d/l2c 0 2024-06-06T08:43:21.804 INFO:tasks.workunit.client.0.smithi027.stdout:4/139: getdents d0/d1/d6/d8/d21 0 2024-06-06T08:43:21.806 INFO:tasks.workunit.client.0.smithi027.stdout:5/274: dwrite d0/f44 [0,4194304] 0 2024-06-06T08:43:21.810 INFO:tasks.workunit.client.0.smithi027.stdout:0/148: dwrite d15/f21 [0,4194304] 0 2024-06-06T08:43:21.811 INFO:tasks.workunit.client.0.smithi027.stdout:5/275: link d0/l23 d0/da/d37/l52 0 2024-06-06T08:43:21.812 INFO:tasks.workunit.client.0.smithi027.stdout:5/276: write d0/f4 [1340414,39900] 0 2024-06-06T08:43:21.813 INFO:tasks.workunit.client.0.smithi027.stdout:5/277: symlink d0/da/d37/d2b/l53 0 2024-06-06T08:43:21.813 INFO:tasks.workunit.client.0.smithi027.stdout:5/278: write d0/da/db/f1b [2726794,56135] 0 2024-06-06T08:43:21.813 INFO:tasks.workunit.client.0.smithi027.stdout:5/279: readlink d0/da/db/l33 0 2024-06-06T08:43:21.813 INFO:tasks.workunit.client.0.smithi027.stdout:0/149: dread d15/f16 [0,4194304] 0 2024-06-06T08:43:21.813 INFO:tasks.workunit.client.0.smithi027.stdout:0/150: write fc [3109818,19999] 0 2024-06-06T08:43:21.814 INFO:tasks.workunit.client.0.smithi027.stdout:5/280: mkdir d0/da/db/d46/d54 0 2024-06-06T08:43:21.814 INFO:tasks.workunit.client.0.smithi027.stdout:5/281: write d0/da/d20/f4f [69680,23484] 0 2024-06-06T08:43:21.814 INFO:tasks.workunit.client.0.smithi027.stdout:5/282: fdatasync d0/f29 0 2024-06-06T08:43:21.819 INFO:tasks.workunit.client.0.smithi027.stdout:5/283: symlink d0/da/db/d46/l55 0 2024-06-06T08:43:21.819 INFO:tasks.workunit.client.0.smithi027.stdout:7/279: rename d17/d32/d46/d51/d37 to d17/d32/d61 0 2024-06-06T08:43:21.820 INFO:tasks.workunit.client.0.smithi027.stdout:5/284: mknod d0/da/d20/c56 0 2024-06-06T08:43:21.820 INFO:tasks.workunit.client.0.smithi027.stdout:5/285: truncate d0/da/d20/f4f 202387 0 2024-06-06T08:43:21.820 INFO:tasks.workunit.client.0.smithi027.stdout:7/280: mknod d17/d31/d2a/d3b/c62 0 2024-06-06T08:43:21.820 INFO:tasks.workunit.client.0.smithi027.stdout:7/281: readlink d17/l24 0 2024-06-06T08:43:21.820 INFO:tasks.workunit.client.0.smithi027.stdout:7/282: fdatasync d17/d32/d46/f41 0 2024-06-06T08:43:21.821 INFO:tasks.workunit.client.0.smithi027.stdout:7/283: mknod d17/d32/d46/c63 0 2024-06-06T08:43:21.822 INFO:tasks.workunit.client.0.smithi027.stdout:7/284: mkdir d17/d32/d46/d4f/d5e/d64 0 2024-06-06T08:43:21.824 INFO:tasks.workunit.client.0.smithi027.stdout:4/140: dwrite d0/d1/fb [4194304,4194304] 0 2024-06-06T08:43:21.824 INFO:tasks.workunit.client.0.smithi027.stdout:4/141: stat d0 0 2024-06-06T08:43:21.826 INFO:tasks.workunit.client.0.smithi027.stdout:4/142: creat d0/d1/d6/d8/d21/f2d x:0 0 0 2024-06-06T08:43:21.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/143: mknod d0/d1/d24/c2e 0 2024-06-06T08:43:21.833 INFO:tasks.workunit.client.0.smithi027.stdout:0/151: dwrite d15/d18/d23/f27 [0,4194304] 0 2024-06-06T08:43:21.834 INFO:tasks.workunit.client.0.smithi027.stdout:0/152: creat d15/d18/d23/d28/d2a/d2d/f34 x:0 0 0 2024-06-06T08:43:21.834 INFO:tasks.workunit.client.0.smithi027.stdout:0/153: fsync d15/f16 0 2024-06-06T08:43:21.837 INFO:tasks.workunit.client.0.smithi027.stdout:7/285: link d17/d32/d46/d51/l40 d17/d32/d46/d4f/l65 0 2024-06-06T08:43:21.839 INFO:tasks.workunit.client.0.smithi027.stdout:0/154: rmdir d15/d18/d23 39 2024-06-06T08:43:21.845 INFO:tasks.workunit.client.0.smithi027.stdout:5/286: dwrite d0/fe [4194304,4194304] 0 2024-06-06T08:43:21.846 INFO:tasks.workunit.client.0.smithi027.stdout:5/287: chown d0/l23 1433 1 2024-06-06T08:43:21.847 INFO:tasks.workunit.client.0.smithi027.stdout:5/288: symlink d0/da/d47/l57 0 2024-06-06T08:43:21.848 INFO:tasks.workunit.client.0.smithi027.stdout:0/155: dread d15/f21 [0,4194304] 0 2024-06-06T08:43:21.850 INFO:tasks.workunit.client.0.smithi027.stdout:5/289: rename d0/da/d37/l52 to d0/da/db/l58 0 2024-06-06T08:43:21.853 INFO:tasks.workunit.client.0.smithi027.stdout:0/156: getdents d15/d18/d23/d28 0 2024-06-06T08:43:21.853 INFO:tasks.workunit.client.0.smithi027.stdout:5/290: mknod d0/da/db/c59 0 2024-06-06T08:43:21.855 INFO:tasks.workunit.client.0.smithi027.stdout:5/291: mkdir d0/da/db/d1a/d5a 0 2024-06-06T08:43:21.856 INFO:tasks.workunit.client.0.smithi027.stdout:7/286: dwrite d17/f23 [0,4194304] 0 2024-06-06T08:43:21.856 INFO:tasks.workunit.client.0.smithi027.stdout:5/292: creat d0/da/d37/d2b/f5b x:0 0 0 2024-06-06T08:43:21.858 INFO:tasks.workunit.client.0.smithi027.stdout:7/287: creat d17/d32/d46/d4f/d5e/f66 x:0 0 0 2024-06-06T08:43:21.859 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:43:21.859 INFO:tasks.workunit.client.0.smithi027.stdout:5/293: creat d0/da/d20/f5c x:0 0 0 2024-06-06T08:43:21.860 INFO:tasks.workunit.client.0.smithi027.stdout:7/288: getdents d17/d47 0 2024-06-06T08:43:21.862 INFO:tasks.workunit.client.0.smithi027.stdout:5/294: mkdir d0/da/db/d5d 0 2024-06-06T08:43:21.867 INFO:tasks.workunit.client.0.smithi027.stdout:5/295: dread d0/da/db/f10 [0,4194304] 0 2024-06-06T08:43:21.867 INFO:tasks.workunit.client.0.smithi027.stdout:5/296: write d0/da/db/f1b [3611882,66367] 0 2024-06-06T08:43:21.867 INFO:tasks.workunit.client.0.smithi027.stdout:5/297: write d0/f4e [83299,41353] 0 2024-06-06T08:43:21.868 INFO:tasks.workunit.client.0.smithi027.stdout:5/298: rename d0/da/db/d1a/c34 to d0/da/db/d1a/c5e 0 2024-06-06T08:43:21.877 INFO:tasks.workunit.client.0.smithi027.stdout:0/157: rename lf to d15/d18/d23/l35 0 2024-06-06T08:43:21.878 INFO:tasks.workunit.client.0.smithi027.stdout:0/158: truncate d15/f22 828428 0 2024-06-06T08:43:21.878 INFO:tasks.workunit.client.0.smithi027.stdout:0/159: readlink d15/d18/l1d 0 2024-06-06T08:43:21.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/160: getdents d15/d18/d23/d28/d2a 0 2024-06-06T08:43:21.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/161: chown d15/d18/d23 1 1 2024-06-06T08:43:21.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/162: chown fb 765784 1 2024-06-06T08:43:21.880 INFO:tasks.workunit.client.0.smithi027.stdout:0/163: dread d15/d18/f1c [0,4194304] 0 2024-06-06T08:43:21.886 INFO:tasks.workunit.client.0.smithi027.stdout:5/299: dwrite d0/f4e [0,4194304] 0 2024-06-06T08:43:21.886 INFO:tasks.workunit.client.0.smithi027.stdout:5/300: dread - d0/da/d37/d2b/f5b zero size 2024-06-06T08:43:21.890 INFO:tasks.workunit.client.0.smithi027.stdout:7/289: rename d17/d32/d46/d51/f5c to d17/d32/d46/d4f/d5e/f67 0 2024-06-06T08:43:21.890 INFO:tasks.workunit.client.0.smithi027.stdout:7/290: chown d17/d47/c4b 61770 1 2024-06-06T08:43:21.892 INFO:tasks.workunit.client.0.smithi027.stdout:0/164: rename ld to d15/l36 0 2024-06-06T08:43:21.892 INFO:tasks.workunit.client.0.smithi027.stdout:0/165: stat d15/d18/c1f 0 2024-06-06T08:43:21.893 INFO:tasks.workunit.client.0.smithi027.stdout:5/301: rename d0/da/d37/d2b/l3e to d0/da/db/d1a/d5a/l5f 0 2024-06-06T08:43:21.893 INFO:tasks.workunit.client.0.smithi027.stdout:0/166: creat d15/d18/d23/d28/d2a/d2d/f37 x:0 0 0 2024-06-06T08:43:21.905 INFO:tasks.workunit.client.0.smithi027.stdout:5/302: read d0/da/f13 [356155,13033] 0 2024-06-06T08:43:21.906 INFO:tasks.workunit.client.0.smithi027.stdout:7/291: rename c12 to d17/d31/d2a/c68 0 2024-06-06T08:43:21.907 INFO:tasks.workunit.client.0.smithi027.stdout:5/303: creat d0/da/d20/f60 x:0 0 0 2024-06-06T08:43:21.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/167: dwrite f10 [0,4194304] 0 2024-06-06T08:43:21.908 INFO:tasks.workunit.client.0.smithi027.stdout:7/292: getdents d17 0 2024-06-06T08:43:21.909 INFO:tasks.workunit.client.0.smithi027.stdout:7/293: truncate d17/d31/d2a/d3b/f56 600694 0 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:5/304: rmdir d0/da/db/d5d 0 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:5/305: fdatasync d0/da/d37/d2b/f5b 0 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:5/306: chown d0/da/f26 3 1 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:5/307: chown d0/da/db/lf 5260 1 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:5/308: fdatasync d0/f1 0 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:7/294: creat d17/d31/d2a/f69 x:0 0 0 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:7/295: write d17/d32/f2f [1399551,103288] 0 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:7/296: chown d17/f27 683513 1 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:5/309: getdents d0/da/d47 0 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:5/310: fdatasync d0/da/f26 0 2024-06-06T08:43:21.912 INFO:tasks.workunit.client.0.smithi027.stdout:0/168: mkdir d15/d18/d23/d38 0 2024-06-06T08:43:21.913 INFO:tasks.workunit.client.0.smithi027.stdout:7/297: dread d17/d31/d2a/d3b/f42 [0,4194304] 0 2024-06-06T08:43:21.913 INFO:tasks.workunit.client.0.smithi027.stdout:7/298: dread - d17/f58 zero size 2024-06-06T08:43:21.913 INFO:tasks.workunit.client.0.smithi027.stdout:0/169: mknod d15/d18/d23/d28/d2a/d30/c39 0 2024-06-06T08:43:21.914 INFO:tasks.workunit.client.0.smithi027.stdout:5/311: mknod d0/da/d37/d2b/c61 0 2024-06-06T08:43:21.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/299: creat d17/d32/d61/f6a x:0 0 0 2024-06-06T08:43:21.917 INFO:tasks.workunit.client.0.smithi027.stdout:5/312: dread d0/f4 [0,4194304] 0 2024-06-06T08:43:21.918 INFO:tasks.workunit.client.0.smithi027.stdout:7/300: truncate d17/f21 1834654 0 2024-06-06T08:43:21.918 INFO:tasks.workunit.client.0.smithi027.stdout:7/301: write f11 [3462531,88512] 0 2024-06-06T08:43:21.919 INFO:tasks.workunit.client.0.smithi027.stdout:5/313: symlink d0/da/db/d46/l62 0 2024-06-06T08:43:21.934 INFO:tasks.workunit.client.0.smithi027.stdout:0/170: rename f13 to d15/d18/d23/d28/d2a/d2d/f3a 0 2024-06-06T08:43:21.935 INFO:tasks.workunit.client.0.smithi027.stdout:7/302: link f15 d17/d32/d61/f6b 0 2024-06-06T08:43:21.937 INFO:tasks.workunit.client.0.smithi027.stdout:0/171: creat d15/d18/d23/d28/d2a/d2d/f3b x:0 0 0 2024-06-06T08:43:21.937 INFO:tasks.workunit.client.0.smithi027.stdout:0/172: read - d15/d18/d23/d28/d2a/d2d/f34 zero size 2024-06-06T08:43:21.937 INFO:tasks.workunit.client.0.smithi027.stdout:0/173: truncate d15/d18/d23/d28/d2a/f33 548197 0 2024-06-06T08:43:21.938 INFO:tasks.workunit.client.0.smithi027.stdout:5/314: dwrite d0/da/d37/d2b/f3d [0,4194304] 0 2024-06-06T08:43:21.938 INFO:tasks.workunit.client.0.smithi027.stdout:5/315: readlink d0/l14 0 2024-06-06T08:43:21.938 INFO:tasks.workunit.client.0.smithi027.stdout:5/316: write d0/da/d20/f60 [548473,81599] 0 2024-06-06T08:43:21.938 INFO:tasks.workunit.client.0.smithi027.stdout:7/303: write d17/f21 [415538,77404] 0 2024-06-06T08:43:21.939 INFO:tasks.workunit.client.0.smithi027.stdout:0/174: rmdir d15 39 2024-06-06T08:43:21.940 INFO:tasks.workunit.client.0.smithi027.stdout:5/317: symlink d0/da/d37/d2b/l63 0 2024-06-06T08:43:21.941 INFO:tasks.workunit.client.0.smithi027.stdout:5/318: creat d0/da/db/f64 x:0 0 0 2024-06-06T08:43:21.942 INFO:tasks.workunit.client.0.smithi027.stdout:5/319: chown d0/da/d20 73990 1 2024-06-06T08:43:21.945 INFO:tasks.workunit.client.0.smithi027.stdout:9/148: sync 2024-06-06T08:43:21.945 INFO:tasks.workunit.client.0.smithi027.stdout:2/231: sync 2024-06-06T08:43:21.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/320: dread d0/da/db/f10 [0,4194304] 0 2024-06-06T08:43:21.947 INFO:tasks.workunit.client.0.smithi027.stdout:0/175: mkdir d15/d18/d23/d3c 0 2024-06-06T08:43:21.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/321: stat d0/da/db/d46/d54 0 2024-06-06T08:43:21.947 INFO:tasks.workunit.client.0.smithi027.stdout:0/176: chown d15/d18/d23/d28/d31 666 1 2024-06-06T08:43:21.948 INFO:tasks.workunit.client.0.smithi027.stdout:2/232: link d6/f2a d6/d1e/d2b/f57 0 2024-06-06T08:43:21.950 INFO:tasks.workunit.client.0.smithi027.stdout:0/177: unlink f14 0 2024-06-06T08:43:21.951 INFO:tasks.workunit.client.0.smithi027.stdout:5/322: creat d0/da/d37/d2b/f65 x:0 0 0 2024-06-06T08:43:21.953 INFO:tasks.workunit.client.0.smithi027.stdout:2/233: creat d6/d1e/d2b/f58 x:0 0 0 2024-06-06T08:43:21.953 INFO:tasks.workunit.client.0.smithi027.stdout:2/234: write d6/d1e/f3d [177406,25589] 0 2024-06-06T08:43:21.953 INFO:tasks.workunit.client.0.smithi027.stdout:0/178: mknod d15/d18/d23/c3d 0 2024-06-06T08:43:21.953 INFO:tasks.workunit.client.0.smithi027.stdout:0/179: fdatasync d15/d18/d23/d28/d2a/d2d/f34 0 2024-06-06T08:43:21.954 INFO:tasks.workunit.client.0.smithi027.stdout:5/323: dread d0/da/d20/f4f [0,4194304] 0 2024-06-06T08:43:21.957 INFO:tasks.workunit.client.0.smithi027.stdout:2/235: mkdir d6/dc/d1d/d59 0 2024-06-06T08:43:21.959 INFO:tasks.workunit.client.0.smithi027.stdout:5/324: dread d0/da/d37/f2f [0,4194304] 0 2024-06-06T08:43:21.961 INFO:tasks.workunit.client.0.smithi027.stdout:7/304: dwrite d17/d32/d46/d4f/d5e/f67 [0,4194304] 0 2024-06-06T08:43:21.962 INFO:tasks.workunit.client.0.smithi027.stdout:7/305: dread - d17/d31/f45 zero size 2024-06-06T08:43:21.962 INFO:tasks.workunit.client.0.smithi027.stdout:5/325: creat d0/da/db/d1a/f66 x:0 0 0 2024-06-06T08:43:21.962 INFO:tasks.workunit.client.0.smithi027.stdout:5/326: stat d0/da/db/d46/d4c 0 2024-06-06T08:43:21.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/306: truncate d17/f1d 2856790 0 2024-06-06T08:43:21.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/307: write f11 [4415739,99845] 0 2024-06-06T08:43:21.966 INFO:tasks.workunit.client.0.smithi027.stdout:0/180: write d15/d18/d23/d28/d2a/d2d/f3a [7455222,6959] 0 2024-06-06T08:43:21.967 INFO:tasks.workunit.client.0.smithi027.stdout:7/308: getdents d17/d32/d46/d4f 0 2024-06-06T08:43:21.967 INFO:tasks.workunit.client.0.smithi027.stdout:7/309: dread - d17/f1e zero size 2024-06-06T08:43:21.967 INFO:tasks.workunit.client.0.smithi027.stdout:7/310: fsync d17/d47/f4d 0 2024-06-06T08:43:21.967 INFO:tasks.workunit.client.0.smithi027.stdout:7/311: truncate d17/d31/d2a/d3b/f5b 1000832 0 2024-06-06T08:43:21.967 INFO:tasks.workunit.client.0.smithi027.stdout:0/181: mkdir d15/d18/d23/d38/d3e 0 2024-06-06T08:43:21.967 INFO:tasks.workunit.client.0.smithi027.stdout:0/182: chown d15/d18/d23/l35 2 1 2024-06-06T08:43:21.968 INFO:tasks.workunit.client.0.smithi027.stdout:0/183: write d15/d18/d23/f29 [4569,13706] 0 2024-06-06T08:43:21.968 INFO:tasks.workunit.client.0.smithi027.stdout:0/184: dread d15/d18/d23/d28/d2a/f33 [0,4194304] 0 2024-06-06T08:43:21.969 INFO:tasks.workunit.client.0.smithi027.stdout:9/149: dwrite d3/f15 [0,4194304] 0 2024-06-06T08:43:21.971 INFO:tasks.workunit.client.0.smithi027.stdout:7/312: mkdir d17/d32/d6c 0 2024-06-06T08:43:21.971 INFO:tasks.workunit.client.0.smithi027.stdout:9/150: symlink d3/da/dc/df/d23/l3c 0 2024-06-06T08:43:21.971 INFO:tasks.workunit.client.0.smithi027.stdout:9/151: truncate d3/da/dc/f2d 286630 0 2024-06-06T08:43:21.972 INFO:tasks.workunit.client.0.smithi027.stdout:7/313: creat d17/d32/f6d x:0 0 0 2024-06-06T08:43:21.973 INFO:tasks.workunit.client.0.smithi027.stdout:9/152: rmdir d3/da/dc/df/d23 39 2024-06-06T08:43:21.975 INFO:tasks.workunit.client.0.smithi027.stdout:7/314: symlink d17/d32/l6e 0 2024-06-06T08:43:21.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/153: creat d3/da/dc/df/d23/f3d x:0 0 0 2024-06-06T08:43:21.976 INFO:tasks.workunit.client.0.smithi027.stdout:7/315: truncate d17/d47/f4d 582932 0 2024-06-06T08:43:21.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/154: truncate d3/da/dc/df/f2f 655091 0 2024-06-06T08:43:21.976 INFO:tasks.workunit.client.0.smithi027.stdout:7/316: write d17/f21 [426662,12843] 0 2024-06-06T08:43:21.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/155: write d3/da/dc/df/f2f [821720,47076] 0 2024-06-06T08:43:21.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/156: write d3/da/dc/f2d [662531,107924] 0 2024-06-06T08:43:21.978 INFO:tasks.workunit.client.0.smithi027.stdout:0/185: read d15/d18/d23/f27 [3950013,116060] 0 2024-06-06T08:43:21.980 INFO:tasks.workunit.client.0.smithi027.stdout:0/186: rename l3 to d15/l3f 0 2024-06-06T08:43:21.980 INFO:tasks.workunit.client.0.smithi027.stdout:0/187: chown d15/d18/d23/l35 966893 1 2024-06-06T08:43:21.980 INFO:tasks.workunit.client.0.smithi027.stdout:0/188: write d15/d18/d23/f29 [1045204,14073] 0 2024-06-06T08:43:21.981 INFO:tasks.workunit.client.0.smithi027.stdout:2/236: dwrite d6/dd/f24 [0,4194304] 0 2024-06-06T08:43:21.982 INFO:tasks.workunit.client.0.smithi027.stdout:2/237: mkdir d6/d1e/d2b/d5a 0 2024-06-06T08:43:21.983 INFO:tasks.workunit.client.0.smithi027.stdout:0/189: mkdir d15/d18/d40 0 2024-06-06T08:43:21.983 INFO:tasks.workunit.client.0.smithi027.stdout:0/190: read d15/d18/d23/f29 [821236,68016] 0 2024-06-06T08:43:21.984 INFO:tasks.workunit.client.0.smithi027.stdout:0/191: stat d15/d18/d23/d28/d2a/d2d/c2f 0 2024-06-06T08:43:21.984 INFO:tasks.workunit.client.0.smithi027.stdout:0/192: truncate d15/d18/d23/f25 698673 0 2024-06-06T08:43:21.985 INFO:tasks.workunit.client.0.smithi027.stdout:5/327: dwrite d0/da/d20/f4f [0,4194304] 0 2024-06-06T08:43:21.991 INFO:tasks.workunit.client.0.smithi027.stdout:5/328: rename d0/da/d37/f2f to d0/da/d37/d2b/f67 0 2024-06-06T08:43:21.993 INFO:tasks.workunit.client.0.smithi027.stdout:5/329: rename d0/f25 to d0/da/d47/f68 0 2024-06-06T08:43:21.994 INFO:tasks.workunit.client.0.smithi027.stdout:5/330: mknod d0/da/db/d46/d54/c69 0 2024-06-06T08:43:21.996 INFO:tasks.workunit.client.0.smithi027.stdout:5/331: mkdir d0/d6a 0 2024-06-06T08:43:21.997 INFO:tasks.workunit.client.0.smithi027.stdout:5/332: mknod d0/d6a/c6b 0 2024-06-06T08:43:21.998 INFO:tasks.workunit.client.0.smithi027.stdout:2/238: dwrite d6/dc/d1d/f45 [0,4194304] 0 2024-06-06T08:43:21.998 INFO:tasks.workunit.client.0.smithi027.stdout:0/193: dread fc [4194304,4194304] 0 2024-06-06T08:43:22.005 INFO:tasks.workunit.client.0.smithi027.stdout:9/157: link d3/c26 d3/da/dc/df/d23/d36/d3a/c3e 0 2024-06-06T08:43:22.006 INFO:tasks.workunit.client.0.smithi027.stdout:7/317: link d17/f21 d17/d32/d46/d51/f6f 0 2024-06-06T08:43:22.006 INFO:tasks.workunit.client.0.smithi027.stdout:7/318: dread - d17/d32/d61/f6a zero size 2024-06-06T08:43:22.007 INFO:tasks.workunit.client.0.smithi027.stdout:2/239: dread d6/dd/f17 [0,4194304] 0 2024-06-06T08:43:22.008 INFO:tasks.workunit.client.0.smithi027.stdout:2/240: creat d6/dc/d1d/f5b x:0 0 0 2024-06-06T08:43:22.008 INFO:tasks.workunit.client.0.smithi027.stdout:2/241: fdatasync d6/fe 0 2024-06-06T08:43:22.008 INFO:tasks.workunit.client.0.smithi027.stdout:7/319: mkdir d17/d32/d46/d51/d52/d70 0 2024-06-06T08:43:22.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/194: link d15/d18/d23/d28/d2a/c2b d15/d18/d23/c41 0 2024-06-06T08:43:22.010 INFO:tasks.workunit.client.0.smithi027.stdout:5/333: dread d0/da/db/f19 [0,4194304] 0 2024-06-06T08:43:22.011 INFO:tasks.workunit.client.0.smithi027.stdout:5/334: mknod d0/da/d20/c6c 0 2024-06-06T08:43:22.012 INFO:tasks.workunit.client.0.smithi027.stdout:7/320: creat d17/d32/d46/d51/f71 x:0 0 0 2024-06-06T08:43:22.012 INFO:tasks.workunit.client.0.smithi027.stdout:5/335: creat d0/da/d20/f6d x:0 0 0 2024-06-06T08:43:22.012 INFO:tasks.workunit.client.0.smithi027.stdout:5/336: write d0/da/f35 [2248852,19813] 0 2024-06-06T08:43:22.014 INFO:tasks.workunit.client.0.smithi027.stdout:5/337: unlink d0/da/db/d46/l62 0 2024-06-06T08:43:22.014 INFO:tasks.workunit.client.0.smithi027.stdout:9/158: dread d3/f4 [0,4194304] 0 2024-06-06T08:43:22.015 INFO:tasks.workunit.client.0.smithi027.stdout:0/195: write d15/f22 [555530,84979] 0 2024-06-06T08:43:22.015 INFO:tasks.workunit.client.0.smithi027.stdout:7/321: mkdir d17/d31/d72 0 2024-06-06T08:43:22.015 INFO:tasks.workunit.client.0.smithi027.stdout:5/338: link d0/da/d37/d2b/f3d d0/da/db/d46/d4c/f6e 0 2024-06-06T08:43:22.015 INFO:tasks.workunit.client.0.smithi027.stdout:9/159: creat d3/da/dc/df/f3f x:0 0 0 2024-06-06T08:43:22.016 INFO:tasks.workunit.client.0.smithi027.stdout:0/196: mkdir d15/d18/d23/d28/d2a/d30/d42 0 2024-06-06T08:43:22.016 INFO:tasks.workunit.client.0.smithi027.stdout:5/339: unlink d0/da/f26 0 2024-06-06T08:43:22.017 INFO:tasks.workunit.client.0.smithi027.stdout:0/197: mknod d15/d18/d40/c43 0 2024-06-06T08:43:22.018 INFO:tasks.workunit.client.0.smithi027.stdout:6/255: sync 2024-06-06T08:43:22.018 INFO:tasks.workunit.client.0.smithi027.stdout:8/108: sync 2024-06-06T08:43:22.018 INFO:tasks.workunit.client.0.smithi027.stdout:8/109: stat d7/lf 0 2024-06-06T08:43:22.019 INFO:tasks.workunit.client.0.smithi027.stdout:0/198: unlink d15/c1e 0 2024-06-06T08:43:22.022 INFO:tasks.workunit.client.0.smithi027.stdout:9/160: fdatasync d3/da/dc/df/f2f 0 2024-06-06T08:43:22.025 INFO:tasks.workunit.client.0.smithi027.stdout:9/161: getdents d3 0 2024-06-06T08:43:22.025 INFO:tasks.workunit.client.0.smithi027.stdout:9/162: dread - d3/da/dc/f2e zero size 2024-06-06T08:43:22.026 INFO:tasks.workunit.client.0.smithi027.stdout:2/242: dwrite d6/d1e/d2b/f58 [0,4194304] 0 2024-06-06T08:43:22.027 INFO:tasks.workunit.client.0.smithi027.stdout:5/340: read d0/da/d20/f4f [1907379,23399] 0 2024-06-06T08:43:22.029 INFO:tasks.workunit.client.0.smithi027.stdout:2/243: truncate d6/d1e/d2b/f3e 3931911 0 2024-06-06T08:43:22.029 INFO:tasks.workunit.client.0.smithi027.stdout:2/244: fdatasync d6/d1e/d2b/f2f 0 2024-06-06T08:43:22.032 INFO:tasks.workunit.client.0.smithi027.stdout:5/341: truncate d0/f4e 3942892 0 2024-06-06T08:43:22.035 INFO:tasks.workunit.client.0.smithi027.stdout:8/110: write d7/f8 [247591,26648] 0 2024-06-06T08:43:22.035 INFO:tasks.workunit.client.0.smithi027.stdout:8/111: fdatasync f5 0 2024-06-06T08:43:22.038 INFO:tasks.workunit.client.0.smithi027.stdout:8/112: unlink d7/d13/l16 0 2024-06-06T08:43:22.044 INFO:tasks.workunit.client.0.smithi027.stdout:8/113: getdents d7/d13 0 2024-06-06T08:43:22.044 INFO:tasks.workunit.client.0.smithi027.stdout:6/256: dwrite d4/d9/f26 [0,4194304] 0 2024-06-06T08:43:22.046 INFO:tasks.workunit.client.0.smithi027.stdout:6/257: mknod d4/d1a/d46/c55 0 2024-06-06T08:43:22.047 INFO:tasks.workunit.client.0.smithi027.stdout:0/199: dwrite d15/f16 [0,4194304] 0 2024-06-06T08:43:22.047 INFO:tasks.workunit.client.0.smithi027.stdout:8/114: write d7/f9 [4289250,115080] 0 2024-06-06T08:43:22.048 INFO:tasks.workunit.client.0.smithi027.stdout:6/258: mknod d4/d13/c56 0 2024-06-06T08:43:22.048 INFO:tasks.workunit.client.0.smithi027.stdout:6/259: dread - d4/de/f35 zero size 2024-06-06T08:43:22.048 INFO:tasks.workunit.client.0.smithi027.stdout:6/260: read d4/d13/f21 [2617435,57106] 0 2024-06-06T08:43:22.049 INFO:tasks.workunit.client.0.smithi027.stdout:0/200: mkdir d15/d18/d23/d28/d2a/d2d/d44 0 2024-06-06T08:43:22.050 INFO:tasks.workunit.client.0.smithi027.stdout:9/163: dwrite d3/da/dc/f30 [0,4194304] 0 2024-06-06T08:43:22.055 INFO:tasks.workunit.client.0.smithi027.stdout:9/164: truncate d3/da/dc/df/d23/d36/f37 1509980 0 2024-06-06T08:43:22.055 INFO:tasks.workunit.client.0.smithi027.stdout:0/201: dread d15/f21 [0,4194304] 0 2024-06-06T08:43:22.055 INFO:tasks.workunit.client.0.smithi027.stdout:8/115: truncate d7/f9 656436 0 2024-06-06T08:43:22.055 INFO:tasks.workunit.client.0.smithi027.stdout:8/116: chown d7/f9 65149 1 2024-06-06T08:43:22.056 INFO:tasks.workunit.client.0.smithi027.stdout:2/245: dwrite d6/dc/d1d/f45 [0,4194304] 0 2024-06-06T08:43:22.059 INFO:tasks.workunit.client.0.smithi027.stdout:2/246: mknod d6/dc/d1d/d4d/c5c 0 2024-06-06T08:43:22.060 INFO:tasks.workunit.client.0.smithi027.stdout:0/202: mknod d15/d18/d23/c45 0 2024-06-06T08:43:22.060 INFO:tasks.workunit.client.0.smithi027.stdout:2/247: mknod d6/c5d 0 2024-06-06T08:43:22.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/322: rename ff to d17/d32/d46/d51/d52/f73 0 2024-06-06T08:43:22.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/323: truncate d17/d32/d46/d4f/d5e/f66 680453 0 2024-06-06T08:43:22.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/324: fdatasync d17/d32/d46/d51/f5f 0 2024-06-06T08:43:22.061 INFO:tasks.workunit.client.0.smithi027.stdout:5/342: dread d0/f4e [0,4194304] 0 2024-06-06T08:43:22.062 INFO:tasks.workunit.client.0.smithi027.stdout:2/248: symlink d6/d1e/d2b/l5e 0 2024-06-06T08:43:22.062 INFO:tasks.workunit.client.0.smithi027.stdout:2/249: write d6/f4f [585810,90325] 0 2024-06-06T08:43:22.062 INFO:tasks.workunit.client.0.smithi027.stdout:0/203: mknod d15/d18/d23/d28/d2a/c46 0 2024-06-06T08:43:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:8/117: rename d7/f8 to d7/db/f19 0 2024-06-06T08:43:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:0/204: unlink d15/d18/d23/d28/d2a/d2d/f37 0 2024-06-06T08:43:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:6/261: dread d4/f5 [4194304,4194304] 0 2024-06-06T08:43:22.065 INFO:tasks.workunit.client.0.smithi027.stdout:6/262: truncate d4/d9/f28 298178 0 2024-06-06T08:43:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/165: write d3/f9 [1309841,60759] 0 2024-06-06T08:43:22.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/166: write d3/f11 [520504,111268] 0 2024-06-06T08:43:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:8/118: link f5 d7/d13/d15/f1a 0 2024-06-06T08:43:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:7/325: creat d17/d32/d61/f74 x:0 0 0 2024-06-06T08:43:22.069 INFO:tasks.workunit.client.0.smithi027.stdout:7/326: dread - d17/f58 zero size 2024-06-06T08:43:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:8/119: mknod d7/d13/c1b 0 2024-06-06T08:43:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/343: dread d0/f24 [0,4194304] 0 2024-06-06T08:43:22.073 INFO:tasks.workunit.client.0.smithi027.stdout:2/250: dread f5 [0,4194304] 0 2024-06-06T08:43:22.074 INFO:tasks.workunit.client.0.smithi027.stdout:6/263: stat d4/d9/dc/d18/ca 0 2024-06-06T08:43:22.075 INFO:tasks.workunit.client.0.smithi027.stdout:2/251: creat d6/dc/d1d/d4d/f5f x:0 0 0 2024-06-06T08:43:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:6/264: creat d4/f57 x:0 0 0 2024-06-06T08:43:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:6/265: chown d4/l33 67004 1 2024-06-06T08:43:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:6/266: dread - d4/de/f35 zero size 2024-06-06T08:43:22.076 INFO:tasks.workunit.client.0.smithi027.stdout:6/267: fdatasync d4/de/f35 0 2024-06-06T08:43:22.077 INFO:tasks.workunit.client.0.smithi027.stdout:2/252: rename d6/dc/d1d/f33 to d6/dc/d1d/f60 0 2024-06-06T08:43:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:7/327: mkdir d17/d32/d61/d75 0 2024-06-06T08:43:22.079 INFO:tasks.workunit.client.0.smithi027.stdout:2/253: mkdir d6/d3b/d61 0 2024-06-06T08:43:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:0/205: unlink d15/d18/d23/l35 0 2024-06-06T08:43:22.080 INFO:tasks.workunit.client.0.smithi027.stdout:0/206: chown d15/d18/d23/d28/d2a/d2d/f34 2170 1 2024-06-06T08:43:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:5/344: dread d0/da/db/f17 [0,4194304] 0 2024-06-06T08:43:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:0/207: getdents d15 0 2024-06-06T08:43:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:5/345: readlink d0/da/db/lf 0 2024-06-06T08:43:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:5/346: readlink d0/da/d37/d2b/l63 0 2024-06-06T08:43:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:7/328: mkdir d17/d32/d46/d51/d52/d76 0 2024-06-06T08:43:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:8/120: dread f5 [0,4194304] 0 2024-06-06T08:43:22.086 INFO:tasks.workunit.client.0.smithi027.stdout:6/268: rename d4/d9/d49/f54 to d4/d9/f58 0 2024-06-06T08:43:22.090 INFO:tasks.workunit.client.0.smithi027.stdout:9/167: dwrite d3/da/dc/df/f28 [0,4194304] 0 2024-06-06T08:43:22.092 INFO:tasks.workunit.client.0.smithi027.stdout:2/254: dread f5 [0,4194304] 0 2024-06-06T08:43:22.093 INFO:tasks.workunit.client.0.smithi027.stdout:6/269: unlink d4/d1a/c3b 0 2024-06-06T08:43:22.094 INFO:tasks.workunit.client.0.smithi027.stdout:9/168: unlink d3/da/dc/df/c33 0 2024-06-06T08:43:22.096 INFO:tasks.workunit.client.0.smithi027.stdout:2/255: creat d6/dc/d3f/d48/f62 x:0 0 0 2024-06-06T08:43:22.097 INFO:tasks.workunit.client.0.smithi027.stdout:6/270: creat d4/f59 x:0 0 0 2024-06-06T08:43:22.099 INFO:tasks.workunit.client.0.smithi027.stdout:9/169: unlink d3/l13 0 2024-06-06T08:43:22.099 INFO:tasks.workunit.client.0.smithi027.stdout:9/170: dread - d3/da/dc/f2a zero size 2024-06-06T08:43:22.099 INFO:tasks.workunit.client.0.smithi027.stdout:9/171: readlink d3/l14 0 2024-06-06T08:43:22.099 INFO:tasks.workunit.client.0.smithi027.stdout:2/256: rmdir d6/d1e/d2b 39 2024-06-06T08:43:22.100 INFO:tasks.workunit.client.0.smithi027.stdout:9/172: mknod d3/da/dc/c40 0 2024-06-06T08:43:22.100 INFO:tasks.workunit.client.0.smithi027.stdout:6/271: creat d4/d1a/d2e/d44/f5a x:0 0 0 2024-06-06T08:43:22.101 INFO:tasks.workunit.client.0.smithi027.stdout:9/173: rmdir d3/da/dc/df 39 2024-06-06T08:43:22.102 INFO:tasks.workunit.client.0.smithi027.stdout:9/174: chown d3/da/dc/df 3 1 2024-06-06T08:43:22.103 INFO:tasks.workunit.client.0.smithi027.stdout:6/272: unlink d4/d9/dc/d18/l3a 0 2024-06-06T08:43:22.103 INFO:tasks.workunit.client.0.smithi027.stdout:8/121: dwrite d7/f9 [0,4194304] 0 2024-06-06T08:43:22.103 INFO:tasks.workunit.client.0.smithi027.stdout:8/122: write d7/d13/d15/f1a [287617,19012] 0 2024-06-06T08:43:22.104 INFO:tasks.workunit.client.0.smithi027.stdout:8/123: write d7/d13/f17 [764982,24275] 0 2024-06-06T08:43:22.104 INFO:tasks.workunit.client.0.smithi027.stdout:8/124: fsync d7/f9 0 2024-06-06T08:43:22.105 INFO:tasks.workunit.client.0.smithi027.stdout:8/125: getdents d7/d13 0 2024-06-06T08:43:22.107 INFO:tasks.workunit.client.0.smithi027.stdout:6/273: symlink d4/d9/dc/d1c/l5b 0 2024-06-06T08:43:22.109 INFO:tasks.workunit.client.0.smithi027.stdout:6/274: creat d4/d1a/d2e/f5c x:0 0 0 2024-06-06T08:43:22.110 INFO:tasks.workunit.client.0.smithi027.stdout:6/275: write d4/f25 [669805,123319] 0 2024-06-06T08:43:22.111 INFO:tasks.workunit.client.0.smithi027.stdout:6/276: dread d4/d9/f28 [0,4194304] 0 2024-06-06T08:43:22.114 INFO:tasks.workunit.client.0.smithi027.stdout:6/277: dread d4/de/d45/f48 [0,4194304] 0 2024-06-06T08:43:22.114 INFO:tasks.workunit.client.0.smithi027.stdout:6/278: readlink d4/d9/dc/d1c/l5b 0 2024-06-06T08:43:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:0/208: dwrite f4 [4194304,4194304] 0 2024-06-06T08:43:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:6/279: dread d4/d9/f28 [0,4194304] 0 2024-06-06T08:43:22.116 INFO:tasks.workunit.client.0.smithi027.stdout:6/280: creat d4/de/f5d x:0 0 0 2024-06-06T08:43:22.117 INFO:tasks.workunit.client.0.smithi027.stdout:6/281: chown d4/d9/dc/d1c 511 1 2024-06-06T08:43:22.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/209: symlink d15/d18/d23/d28/d2a/d2d/l47 0 2024-06-06T08:43:22.119 INFO:tasks.workunit.client.0.smithi027.stdout:2/257: dwrite d6/d1e/f1f [0,4194304] 0 2024-06-06T08:43:22.120 INFO:tasks.workunit.client.0.smithi027.stdout:2/258: getdents d6/dd/d3c/d55 0 2024-06-06T08:43:22.120 INFO:tasks.workunit.client.0.smithi027.stdout:0/210: getdents d15/d18/d23/d28/d2a/d30 0 2024-06-06T08:43:22.120 INFO:tasks.workunit.client.0.smithi027.stdout:0/211: write d15/d18/d23/f29 [1618550,104497] 0 2024-06-06T08:43:22.120 INFO:tasks.workunit.client.0.smithi027.stdout:2/259: truncate d6/fe 297269 0 2024-06-06T08:43:22.122 INFO:tasks.workunit.client.0.smithi027.stdout:2/260: link d6/dc/d1d/d4d/c5c d6/dc/d1d/d4d/c63 0 2024-06-06T08:43:22.123 INFO:tasks.workunit.client.0.smithi027.stdout:2/261: truncate d6/d1e/f27 201959 0 2024-06-06T08:43:22.126 INFO:tasks.workunit.client.0.smithi027.stdout:9/175: dwrite d3/da/dc/df/f28 [0,4194304] 0 2024-06-06T08:43:22.128 INFO:tasks.workunit.client.0.smithi027.stdout:9/176: mkdir d3/da/dc/df/d34/d41 0 2024-06-06T08:43:22.130 INFO:tasks.workunit.client.0.smithi027.stdout:9/177: rename d3/da/dc/df/c1b to d3/da/dc/df/d34/d41/c42 0 2024-06-06T08:43:22.130 INFO:tasks.workunit.client.0.smithi027.stdout:9/178: chown d3/da 8479 1 2024-06-06T08:43:22.131 INFO:tasks.workunit.client.0.smithi027.stdout:8/126: dwrite d7/d13/f17 [0,4194304] 0 2024-06-06T08:43:22.133 INFO:tasks.workunit.client.0.smithi027.stdout:8/127: truncate d7/f9 508663 0 2024-06-06T08:43:22.136 INFO:tasks.workunit.client.0.smithi027.stdout:8/128: creat d7/d13/d15/f1c x:0 0 0 2024-06-06T08:43:22.139 INFO:tasks.workunit.client.0.smithi027.stdout:8/129: link d7/d13/f17 d7/db/f1d 0 2024-06-06T08:43:22.140 INFO:tasks.workunit.client.0.smithi027.stdout:6/282: dwrite d4/f29 [0,4194304] 0 2024-06-06T08:43:22.141 INFO:tasks.workunit.client.0.smithi027.stdout:6/283: chown d4/c16 802 1 2024-06-06T08:43:22.141 INFO:tasks.workunit.client.0.smithi027.stdout:8/130: creat d7/d13/f1e x:0 0 0 2024-06-06T08:43:22.142 INFO:tasks.workunit.client.0.smithi027.stdout:9/179: write d3/da/dc/f30 [1651764,54757] 0 2024-06-06T08:43:22.142 INFO:tasks.workunit.client.0.smithi027.stdout:9/180: read - d3/da/dc/f2a zero size 2024-06-06T08:43:22.145 INFO:tasks.workunit.client.0.smithi027.stdout:6/284: getdents d4 0 2024-06-06T08:43:22.148 INFO:tasks.workunit.client.0.smithi027.stdout:8/131: creat d7/d13/d15/f1f x:0 0 0 2024-06-06T08:43:22.148 INFO:tasks.workunit.client.0.smithi027.stdout:6/285: mknod d4/d9/dc/c5e 0 2024-06-06T08:43:22.148 INFO:tasks.workunit.client.0.smithi027.stdout:6/286: readlink d4/d9/dc/l1e 0 2024-06-06T08:43:22.150 INFO:tasks.workunit.client.0.smithi027.stdout:4/144: sync 2024-06-06T08:43:22.150 INFO:tasks.workunit.client.0.smithi027.stdout:1/211: sync 2024-06-06T08:43:22.150 INFO:tasks.workunit.client.0.smithi027.stdout:3/120: sync 2024-06-06T08:43:22.152 INFO:tasks.workunit.client.0.smithi027.stdout:9/181: dread d3/f9 [0,4194304] 0 2024-06-06T08:43:22.152 INFO:tasks.workunit.client.0.smithi027.stdout:3/121: mkdir d1/d1f/d25 0 2024-06-06T08:43:22.152 INFO:tasks.workunit.client.0.smithi027.stdout:8/132: creat d7/dc/f20 x:0 0 0 2024-06-06T08:43:22.153 INFO:tasks.workunit.client.0.smithi027.stdout:4/145: creat d0/d1/d6/f2f x:0 0 0 2024-06-06T08:43:22.153 INFO:tasks.workunit.client.0.smithi027.stdout:4/146: fsync d0/d1/f14 0 2024-06-06T08:43:22.153 INFO:tasks.workunit.client.0.smithi027.stdout:4/147: fdatasync d0/d1/d6/f2f 0 2024-06-06T08:43:22.155 INFO:tasks.workunit.client.0.smithi027.stdout:8/133: mkdir d7/d21 0 2024-06-06T08:43:22.156 INFO:tasks.workunit.client.0.smithi027.stdout:2/262: dwrite d6/dc/f40 [0,4194304] 0 2024-06-06T08:43:22.158 INFO:tasks.workunit.client.0.smithi027.stdout:8/134: unlink d7/dc/l10 0 2024-06-06T08:43:22.158 INFO:tasks.workunit.client.0.smithi027.stdout:8/135: fsync d7/d13/d15/f1f 0 2024-06-06T08:43:22.158 INFO:tasks.workunit.client.0.smithi027.stdout:9/182: dread d3/da/dc/df/d23/d36/f37 [0,4194304] 0 2024-06-06T08:43:22.162 INFO:tasks.workunit.client.0.smithi027.stdout:2/263: creat d6/dc/f64 x:0 0 0 2024-06-06T08:43:22.163 INFO:tasks.workunit.client.0.smithi027.stdout:3/122: write d1/da/f1c [21410,129839] 0 2024-06-06T08:43:22.163 INFO:tasks.workunit.client.0.smithi027.stdout:3/123: write d1/d8/f17 [938736,32124] 0 2024-06-06T08:43:22.164 INFO:tasks.workunit.client.0.smithi027.stdout:4/148: dread d0/d1/f1a [0,4194304] 0 2024-06-06T08:43:22.165 INFO:tasks.workunit.client.0.smithi027.stdout:2/264: mknod d6/dd/d3c/c65 0 2024-06-06T08:43:22.165 INFO:tasks.workunit.client.0.smithi027.stdout:2/265: chown d6/d1e/d2b/f3e 185 1 2024-06-06T08:43:22.165 INFO:tasks.workunit.client.0.smithi027.stdout:6/287: dwrite d4/d1a/d2e/f5c [0,4194304] 0 2024-06-06T08:43:22.166 INFO:tasks.workunit.client.0.smithi027.stdout:9/183: rename d3/da/dc/df/d34/d41/c42 to d3/da/dc/df/d34/c43 0 2024-06-06T08:43:22.166 INFO:tasks.workunit.client.0.smithi027.stdout:9/184: chown d3/da/dc/df/d34/d41 8 1 2024-06-06T08:43:22.167 INFO:tasks.workunit.client.0.smithi027.stdout:6/288: symlink d4/d9/d49/l5f 0 2024-06-06T08:43:22.168 INFO:tasks.workunit.client.0.smithi027.stdout:2/266: symlink d6/dc/d1d/d59/l66 0 2024-06-06T08:43:22.168 INFO:tasks.workunit.client.0.smithi027.stdout:1/212: dwrite dc/de/d22/f3b [0,4194304] 0 2024-06-06T08:43:22.171 INFO:tasks.workunit.client.0.smithi027.stdout:1/213: creat dc/de/d14/d21/d32/f4d x:0 0 0 2024-06-06T08:43:22.171 INFO:tasks.workunit.client.0.smithi027.stdout:1/214: chown dc/l34 788238377 1 2024-06-06T08:43:22.172 INFO:tasks.workunit.client.0.smithi027.stdout:2/267: unlink d6/dc/d3f/c42 0 2024-06-06T08:43:22.173 INFO:tasks.workunit.client.0.smithi027.stdout:1/215: write dc/f1d [780883,72660] 0 2024-06-06T08:43:22.174 INFO:tasks.workunit.client.0.smithi027.stdout:2/268: link d6/dd/d3c/c65 d6/dd/c67 0 2024-06-06T08:43:22.174 INFO:tasks.workunit.client.0.smithi027.stdout:2/269: stat d6/dc/f64 0 2024-06-06T08:43:22.176 INFO:tasks.workunit.client.0.smithi027.stdout:2/270: rename d6/dc/d1d/f47 to d6/d1e/f68 0 2024-06-06T08:43:22.176 INFO:tasks.workunit.client.0.smithi027.stdout:2/271: write d6/dc/f54 [881045,78121] 0 2024-06-06T08:43:22.176 INFO:tasks.workunit.client.0.smithi027.stdout:1/216: mkdir dc/de/d14/d4e 0 2024-06-06T08:43:22.193 INFO:tasks.workunit.client.0.smithi027.stdout:1/217: dwrite dc/de/d22/f3b [0,4194304] 0 2024-06-06T08:43:22.194 INFO:tasks.workunit.client.0.smithi027.stdout:4/149: dwrite d0/d1/f12 [0,4194304] 0 2024-06-06T08:43:22.194 INFO:tasks.workunit.client.0.smithi027.stdout:1/218: mkdir dc/de/d3d/d4f 0 2024-06-06T08:43:22.197 INFO:tasks.workunit.client.0.smithi027.stdout:1/219: creat dc/de/d3d/f50 x:0 0 0 2024-06-06T08:43:22.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/220: unlink dc/de/d14/c2a 0 2024-06-06T08:43:22.202 INFO:tasks.workunit.client.0.smithi027.stdout:2/272: dwrite d6/d1e/d2b/f2f [0,4194304] 0 2024-06-06T08:43:22.207 INFO:tasks.workunit.client.0.smithi027.stdout:1/221: dread dc/de/d14/f15 [0,4194304] 0 2024-06-06T08:43:22.207 INFO:tasks.workunit.client.0.smithi027.stdout:1/222: fdatasync dc/f30 0 2024-06-06T08:43:22.207 INFO:tasks.workunit.client.0.smithi027.stdout:1/223: chown dc/de/d14/d21/l4c 1640013 1 2024-06-06T08:43:22.211 INFO:tasks.workunit.client.0.smithi027.stdout:2/273: read d6/dd/f24 [3147634,88581] 0 2024-06-06T08:43:22.213 INFO:tasks.workunit.client.0.smithi027.stdout:4/150: rmdir d0/d1/d6/d8 39 2024-06-06T08:43:22.213 INFO:tasks.workunit.client.0.smithi027.stdout:4/151: write d0/f13 [11284235,83129] 0 2024-06-06T08:43:22.225 INFO:tasks.workunit.client.0.smithi027.stdout:1/224: dwrite dc/de/d14/f19 [0,4194304] 0 2024-06-06T08:43:22.226 INFO:tasks.workunit.client.0.smithi027.stdout:1/225: mknod dc/de/c51 0 2024-06-06T08:43:22.229 INFO:tasks.workunit.client.0.smithi027.stdout:1/226: rename dc/f1d to dc/de/d14/d21/f52 0 2024-06-06T08:43:22.232 INFO:tasks.workunit.client.0.smithi027.stdout:2/274: dwrite d6/dc/d1d/f36 [0,4194304] 0 2024-06-06T08:43:22.234 INFO:tasks.workunit.client.0.smithi027.stdout:2/275: mkdir d6/dd/d3c/d55/d69 0 2024-06-06T08:43:22.237 INFO:tasks.workunit.client.0.smithi027.stdout:1/227: read f5 [3106043,16118] 0 2024-06-06T08:43:22.259 INFO:tasks.workunit.client.0.smithi027.stdout:2/276: dwrite d6/dc/f29 [0,4194304] 0 2024-06-06T08:43:22.259 INFO:tasks.workunit.client.0.smithi027.stdout:1/228: dread dc/de/d14/d21/f27 [0,4194304] 0 2024-06-06T08:43:22.260 INFO:tasks.workunit.client.0.smithi027.stdout:1/229: read dc/de/d14/f49 [226668,31541] 0 2024-06-06T08:43:22.261 INFO:tasks.workunit.client.0.smithi027.stdout:1/230: creat dc/de/d14/d4e/f53 x:0 0 0 2024-06-06T08:43:22.265 INFO:tasks.workunit.client.0.smithi027.stdout:2/277: read d6/dc/f12 [530513,85270] 0 2024-06-06T08:43:22.276 INFO:tasks.workunit.client.0.smithi027.stdout:1/231: dwrite dc/de/d14/d21/f52 [0,4194304] 0 2024-06-06T08:43:22.279 INFO:tasks.workunit.client.0.smithi027.stdout:1/232: rename dc/de/d22/f2c to dc/de/d14/d4e/f54 0 2024-06-06T08:43:22.487 INFO:tasks.workunit.client.0.smithi027.stdout:5/347: sync 2024-06-06T08:43:22.492 INFO:tasks.workunit.client.0.smithi027.stdout:5/348: dread d0/da/d37/d2b/f67 [0,4194304] 0 2024-06-06T08:43:22.500 INFO:tasks.workunit.client.0.smithi027.stdout:5/349: write d0/f24 [2926765,89366] 0 2024-06-06T08:43:22.501 INFO:tasks.workunit.client.0.smithi027.stdout:5/350: dread - d0/da/d37/f41 zero size 2024-06-06T08:43:22.516 INFO:tasks.workunit.client.0.smithi027.stdout:5/351: dwrite d0/da/f35 [0,4194304] 0 2024-06-06T08:43:22.517 INFO:tasks.workunit.client.0.smithi027.stdout:5/352: creat d0/da/db/d1a/f6f x:0 0 0 2024-06-06T08:43:22.521 INFO:tasks.workunit.client.0.smithi027.stdout:5/353: truncate d0/da/f35 4143382 0 2024-06-06T08:43:22.522 INFO:tasks.workunit.client.0.smithi027.stdout:5/354: mkdir d0/da/db/d46/d4c/d70 0 2024-06-06T08:43:22.522 INFO:tasks.workunit.client.0.smithi027.stdout:5/355: write d0/da/d20/f4f [3542899,6881] 0 2024-06-06T08:43:22.523 INFO:tasks.workunit.client.0.smithi027.stdout:5/356: getdents d0/da/d37/d2b 0 2024-06-06T08:43:22.537 INFO:tasks.workunit.client.0.smithi027.stdout:5/357: dwrite d0/da/d20/f60 [0,4194304] 0 2024-06-06T08:43:22.539 INFO:tasks.workunit.client.0.smithi027.stdout:5/358: rmdir d0/da 39 2024-06-06T08:43:22.586 INFO:tasks.workunit.client.0.smithi027.stdout:7/329: sync 2024-06-06T08:43:22.587 INFO:tasks.workunit.client.0.smithi027.stdout:0/212: sync 2024-06-06T08:43:22.587 INFO:tasks.workunit.client.0.smithi027.stdout:0/213: write d15/f21 [4629509,99692] 0 2024-06-06T08:43:22.590 INFO:tasks.workunit.client.0.smithi027.stdout:7/330: mkdir d17/d31/d77 0 2024-06-06T08:43:22.590 INFO:tasks.workunit.client.0.smithi027.stdout:7/331: chown d17/f2e 31603767 1 2024-06-06T08:43:22.590 INFO:tasks.workunit.client.0.smithi027.stdout:7/332: chown f11 31 1 2024-06-06T08:43:22.592 INFO:tasks.workunit.client.0.smithi027.stdout:7/333: unlink d17/l24 0 2024-06-06T08:43:22.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/214: dwrite fc [0,4194304] 0 2024-06-06T08:43:22.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/215: truncate d15/d18/d23/d28/d2a/d2d/f3a 9199324 0 2024-06-06T08:43:22.605 INFO:tasks.workunit.client.0.smithi027.stdout:0/216: mknod d15/c48 0 2024-06-06T08:43:22.606 INFO:tasks.workunit.client.0.smithi027.stdout:7/334: dwrite d17/d32/f6d [0,4194304] 0 2024-06-06T08:43:22.606 INFO:tasks.workunit.client.0.smithi027.stdout:7/335: read d17/d32/d46/f41 [103212,32243] 0 2024-06-06T08:43:22.607 INFO:tasks.workunit.client.0.smithi027.stdout:7/336: symlink d17/d32/d46/d51/d52/d70/l78 0 2024-06-06T08:43:22.607 INFO:tasks.workunit.client.0.smithi027.stdout:7/337: write d17/f58 [1007286,54897] 0 2024-06-06T08:43:22.608 INFO:tasks.workunit.client.0.smithi027.stdout:7/338: creat d17/d32/d46/d4f/d5e/f79 x:0 0 0 2024-06-06T08:43:22.625 INFO:tasks.workunit.client.0.smithi027.stdout:7/339: dwrite f11 [0,4194304] 0 2024-06-06T08:43:22.627 INFO:tasks.workunit.client.0.smithi027.stdout:7/340: creat d17/d32/d46/d4f/d5e/d64/f7a x:0 0 0 2024-06-06T08:43:22.628 INFO:tasks.workunit.client.0.smithi027.stdout:7/341: mkdir d17/d32/d46/d4f/d5e/d7b 0 2024-06-06T08:43:22.629 INFO:tasks.workunit.client.0.smithi027.stdout:7/342: mkdir d17/d32/d46/d4f/d5e/d7b/d7c 0 2024-06-06T08:43:22.629 INFO:tasks.workunit.client.0.smithi027.stdout:7/343: stat d17/d32/d46/d4f/l65 0 2024-06-06T08:43:22.629 INFO:tasks.workunit.client.0.smithi027.stdout:7/344: chown d17/d32/d61/f6a 2886899 1 2024-06-06T08:43:22.629 INFO:tasks.workunit.client.0.smithi027.stdout:7/345: fdatasync d17/d32/d46/d4f/d5e/f79 0 2024-06-06T08:43:22.630 INFO:tasks.workunit.client.0.smithi027.stdout:0/217: dwrite d15/d18/d23/f25 [0,4194304] 0 2024-06-06T08:43:22.632 INFO:tasks.workunit.client.0.smithi027.stdout:0/218: fsync d15/f21 0 2024-06-06T08:43:22.634 INFO:tasks.workunit.client.0.smithi027.stdout:7/346: rename d17/d32/d61/c44 to d17/d31/c7d 0 2024-06-06T08:43:22.634 INFO:tasks.workunit.client.0.smithi027.stdout:7/347: write d17/d32/d61/f6a [508421,97647] 0 2024-06-06T08:43:22.635 INFO:tasks.workunit.client.0.smithi027.stdout:7/348: mknod d17/d32/d46/d51/d52/c7e 0 2024-06-06T08:43:22.639 INFO:tasks.workunit.client.0.smithi027.stdout:0/219: write f2 [1460634,118195] 0 2024-06-06T08:43:22.641 INFO:tasks.workunit.client.0.smithi027.stdout:7/349: getdents d17/d32/d61 0 2024-06-06T08:43:22.642 INFO:tasks.workunit.client.0.smithi027.stdout:7/350: dread - d17/d31/d2a/f69 zero size 2024-06-06T08:43:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:9/185: sync 2024-06-06T08:43:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:2/278: sync 2024-06-06T08:43:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:8/136: sync 2024-06-06T08:43:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:5/359: sync 2024-06-06T08:43:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:1/233: sync 2024-06-06T08:43:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:6/289: sync 2024-06-06T08:43:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:4/152: sync 2024-06-06T08:43:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:2/279: truncate d6/dc/d1d/f38 874255 0 2024-06-06T08:43:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:1/234: dread - dc/de/d3d/f44 zero size 2024-06-06T08:43:22.647 INFO:tasks.workunit.client.0.smithi027.stdout:2/280: chown d6/d1e/d2b/f37 1974 1 2024-06-06T08:43:22.647 INFO:tasks.workunit.client.0.smithi027.stdout:3/124: sync 2024-06-06T08:43:22.647 INFO:tasks.workunit.client.0.smithi027.stdout:3/125: fsync d1/f13 0 2024-06-06T08:43:22.648 INFO:tasks.workunit.client.0.smithi027.stdout:8/137: creat d7/f22 x:0 0 0 2024-06-06T08:43:22.648 INFO:tasks.workunit.client.0.smithi027.stdout:8/138: write d7/db/f11 [1924101,85019] 0 2024-06-06T08:43:22.648 INFO:tasks.workunit.client.0.smithi027.stdout:8/139: chown d7/dc/f20 30229 1 2024-06-06T08:43:22.648 INFO:tasks.workunit.client.0.smithi027.stdout:8/140: read - d7/d13/f1e zero size 2024-06-06T08:43:22.648 INFO:tasks.workunit.client.0.smithi027.stdout:6/290: symlink d4/d9/dc/l60 0 2024-06-06T08:43:22.650 INFO:tasks.workunit.client.0.smithi027.stdout:5/360: rename d0/da/f1e to d0/da/db/d1a/f71 0 2024-06-06T08:43:22.650 INFO:tasks.workunit.client.0.smithi027.stdout:1/235: symlink dc/de/l55 0 2024-06-06T08:43:22.651 INFO:tasks.workunit.client.0.smithi027.stdout:9/186: symlink d3/da/l44 0 2024-06-06T08:43:22.652 INFO:tasks.workunit.client.0.smithi027.stdout:8/141: mknod d7/c23 0 2024-06-06T08:43:22.653 INFO:tasks.workunit.client.0.smithi027.stdout:5/361: creat d0/da/f72 x:0 0 0 2024-06-06T08:43:22.653 INFO:tasks.workunit.client.0.smithi027.stdout:5/362: chown d0/da/db/f1b 167446 1 2024-06-06T08:43:22.654 INFO:tasks.workunit.client.0.smithi027.stdout:3/126: mknod d1/d8/d12/c26 0 2024-06-06T08:43:22.655 INFO:tasks.workunit.client.0.smithi027.stdout:3/127: fsync d1/d8/f17 0 2024-06-06T08:43:22.655 INFO:tasks.workunit.client.0.smithi027.stdout:9/187: dread d3/da/dc/df/d23/d36/f37 [0,4194304] 0 2024-06-06T08:43:22.656 INFO:tasks.workunit.client.0.smithi027.stdout:0/220: write d15/d18/d23/f25 [1697582,99596] 0 2024-06-06T08:43:22.657 INFO:tasks.workunit.client.0.smithi027.stdout:8/142: creat d7/d13/f24 x:0 0 0 2024-06-06T08:43:22.658 INFO:tasks.workunit.client.0.smithi027.stdout:0/221: mknod d15/d18/d23/d28/d2a/d2d/d44/c49 0 2024-06-06T08:43:22.658 INFO:tasks.workunit.client.0.smithi027.stdout:0/222: write d15/d18/d23/d28/d2a/d2d/f3b [905073,41590] 0 2024-06-06T08:43:22.658 INFO:tasks.workunit.client.0.smithi027.stdout:2/281: dread d6/dc/f25 [0,4194304] 0 2024-06-06T08:43:22.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/363: mknod d0/da/c73 0 2024-06-06T08:43:22.659 INFO:tasks.workunit.client.0.smithi027.stdout:7/351: write d17/d32/f38 [2667230,126142] 0 2024-06-06T08:43:22.659 INFO:tasks.workunit.client.0.smithi027.stdout:7/352: dread - d17/d32/d46/d4f/f55 zero size 2024-06-06T08:43:22.659 INFO:tasks.workunit.client.0.smithi027.stdout:7/353: getdents d17/d32/d61/d75 0 2024-06-06T08:43:22.659 INFO:tasks.workunit.client.0.smithi027.stdout:0/223: creat d15/d18/d23/d28/d2a/d2d/d44/f4a x:0 0 0 2024-06-06T08:43:22.660 INFO:tasks.workunit.client.0.smithi027.stdout:8/143: unlink f5 0 2024-06-06T08:43:22.660 INFO:tasks.workunit.client.0.smithi027.stdout:8/144: write d7/dc/f20 [733501,125404] 0 2024-06-06T08:43:22.661 INFO:tasks.workunit.client.0.smithi027.stdout:2/282: rmdir d6/d1e/d2b 39 2024-06-06T08:43:22.661 INFO:tasks.workunit.client.0.smithi027.stdout:2/283: fdatasync d6/dc/f44 0 2024-06-06T08:43:22.664 INFO:tasks.workunit.client.0.smithi027.stdout:0/224: creat d15/d18/f4b x:0 0 0 2024-06-06T08:43:22.665 INFO:tasks.workunit.client.0.smithi027.stdout:0/225: creat d15/d18/d23/d28/f4c x:0 0 0 2024-06-06T08:43:22.665 INFO:tasks.workunit.client.0.smithi027.stdout:0/226: stat d15/d18/d23/f2e 0 2024-06-06T08:43:22.665 INFO:tasks.workunit.client.0.smithi027.stdout:0/227: readlink d15/l3f 0 2024-06-06T08:43:22.666 INFO:tasks.workunit.client.0.smithi027.stdout:0/228: chown d15/d18/d23/d28/d2a/d2d/f3b 10750460 1 2024-06-06T08:43:22.666 INFO:tasks.workunit.client.0.smithi027.stdout:5/364: unlink d0/da/db/f10 0 2024-06-06T08:43:22.666 INFO:tasks.workunit.client.0.smithi027.stdout:5/365: dread - d0/da/f72 zero size 2024-06-06T08:43:22.666 INFO:tasks.workunit.client.0.smithi027.stdout:8/145: creat d7/d21/f25 x:0 0 0 2024-06-06T08:43:22.667 INFO:tasks.workunit.client.0.smithi027.stdout:0/229: link d15/d18/d23/d28/d2a/d2d/c2f d15/d18/d23/d3c/c4d 0 2024-06-06T08:43:22.667 INFO:tasks.workunit.client.0.smithi027.stdout:0/230: dread - d15/d18/d23/d28/d2a/d2d/f34 zero size 2024-06-06T08:43:22.669 INFO:tasks.workunit.client.0.smithi027.stdout:8/146: truncate d7/d13/f17 1595474 0 2024-06-06T08:43:22.670 INFO:tasks.workunit.client.0.smithi027.stdout:8/147: chown d7/d21 0 1 2024-06-06T08:43:22.670 INFO:tasks.workunit.client.0.smithi027.stdout:8/148: stat d7/d13/f24 0 2024-06-06T08:43:22.671 INFO:tasks.workunit.client.0.smithi027.stdout:8/149: write d7/d13/d15/f1a [3499167,14122] 0 2024-06-06T08:43:22.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/354: write d17/d32/f6d [1142201,118114] 0 2024-06-06T08:43:22.675 INFO:tasks.workunit.client.0.smithi027.stdout:7/355: chown d17/d32/d46/d4f/d5e/d64 42688 1 2024-06-06T08:43:22.676 INFO:tasks.workunit.client.0.smithi027.stdout:8/150: link d7/db/f11 d7/dc/f26 0 2024-06-06T08:43:22.676 INFO:tasks.workunit.client.0.smithi027.stdout:8/151: chown c4 3 1 2024-06-06T08:43:22.676 INFO:tasks.workunit.client.0.smithi027.stdout:8/152: write d7/db/f19 [3263082,53428] 0 2024-06-06T08:43:22.677 INFO:tasks.workunit.client.0.smithi027.stdout:6/291: dwrite d4/d1a/f2c [0,4194304] 0 2024-06-06T08:43:22.678 INFO:tasks.workunit.client.0.smithi027.stdout:9/188: write d3/f20 [3142264,3477] 0 2024-06-06T08:43:22.680 INFO:tasks.workunit.client.0.smithi027.stdout:8/153: creat d7/db/f27 x:0 0 0 2024-06-06T08:43:22.680 INFO:tasks.workunit.client.0.smithi027.stdout:8/154: write d7/f22 [790980,43401] 0 2024-06-06T08:43:22.680 INFO:tasks.workunit.client.0.smithi027.stdout:0/231: dread fb [0,4194304] 0 2024-06-06T08:43:22.681 INFO:tasks.workunit.client.0.smithi027.stdout:3/128: dwrite d1/d8/f1a [0,4194304] 0 2024-06-06T08:43:22.681 INFO:tasks.workunit.client.0.smithi027.stdout:1/236: dwrite dc/de/d3d/f42 [0,4194304] 0 2024-06-06T08:43:22.683 INFO:tasks.workunit.client.0.smithi027.stdout:9/189: truncate d3/f11 536 0 2024-06-06T08:43:22.684 INFO:tasks.workunit.client.0.smithi027.stdout:0/232: creat d15/d18/d23/d28/f4e x:0 0 0 2024-06-06T08:43:22.685 INFO:tasks.workunit.client.0.smithi027.stdout:9/190: getdents d3/da/dc/df 0 2024-06-06T08:43:22.689 INFO:tasks.workunit.client.0.smithi027.stdout:2/284: dwrite d6/dc/f22 [0,4194304] 0 2024-06-06T08:43:22.691 INFO:tasks.workunit.client.0.smithi027.stdout:1/237: mkdir dc/de/d14/d21/d32/d37/d56 0 2024-06-06T08:43:22.691 INFO:tasks.workunit.client.0.smithi027.stdout:1/238: truncate dc/f30 632145 0 2024-06-06T08:43:22.694 INFO:tasks.workunit.client.0.smithi027.stdout:0/233: creat d15/f4f x:0 0 0 2024-06-06T08:43:22.698 INFO:tasks.workunit.client.0.smithi027.stdout:9/191: symlink d3/da/l45 0 2024-06-06T08:43:22.698 INFO:tasks.workunit.client.0.smithi027.stdout:8/155: dwrite d7/d21/f25 [0,4194304] 0 2024-06-06T08:43:22.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/285: stat d6/dc/d3f/f50 0 2024-06-06T08:43:22.706 INFO:tasks.workunit.client.0.smithi027.stdout:1/239: mknod dc/de/d14/d4e/c57 0 2024-06-06T08:43:22.709 INFO:tasks.workunit.client.0.smithi027.stdout:9/192: creat d3/f46 x:0 0 0 2024-06-06T08:43:22.709 INFO:tasks.workunit.client.0.smithi027.stdout:9/193: write d3/da/dc/df/d23/d36/f37 [2263813,6978] 0 2024-06-06T08:43:22.713 INFO:tasks.workunit.client.0.smithi027.stdout:8/156: mknod d7/db/c28 0 2024-06-06T08:43:22.715 INFO:tasks.workunit.client.0.smithi027.stdout:2/286: write d6/dc/d3f/f49 [270549,68722] 0 2024-06-06T08:43:22.720 INFO:tasks.workunit.client.0.smithi027.stdout:8/157: mknod d7/db/c29 0 2024-06-06T08:43:22.723 INFO:tasks.workunit.client.0.smithi027.stdout:1/240: dwrite dc/de/d14/d21/f52 [0,4194304] 0 2024-06-06T08:43:22.723 INFO:tasks.workunit.client.0.smithi027.stdout:1/241: chown dc/de/d3d/d4a 397 1 2024-06-06T08:43:22.727 INFO:tasks.workunit.client.0.smithi027.stdout:8/158: symlink d7/l2a 0 2024-06-06T08:43:22.727 INFO:tasks.workunit.client.0.smithi027.stdout:9/194: dwrite d3/da/dc/f30 [4194304,4194304] 0 2024-06-06T08:43:22.733 INFO:tasks.workunit.client.0.smithi027.stdout:2/287: dread d6/d1e/d2b/f3e [0,4194304] 0 2024-06-06T08:43:22.733 INFO:tasks.workunit.client.0.smithi027.stdout:2/288: fsync d6/d1e/f1f 0 2024-06-06T08:43:22.733 INFO:tasks.workunit.client.0.smithi027.stdout:2/289: write d6/dc/f12 [1005178,57869] 0 2024-06-06T08:43:22.733 INFO:tasks.workunit.client.0.smithi027.stdout:0/234: link c11 d15/d18/d23/d28/d2a/d2d/c50 0 2024-06-06T08:43:22.737 INFO:tasks.workunit.client.0.smithi027.stdout:9/195: dread d3/f9 [0,4194304] 0 2024-06-06T08:43:22.737 INFO:tasks.workunit.client.0.smithi027.stdout:9/196: truncate d3/da/dc/f2a 147701 0 2024-06-06T08:43:22.738 INFO:tasks.workunit.client.0.smithi027.stdout:9/197: mknod d3/da/dc/df/c47 0 2024-06-06T08:43:22.739 INFO:tasks.workunit.client.0.smithi027.stdout:9/198: chown d3/da/dc/df/d34 0 1 2024-06-06T08:43:22.740 INFO:tasks.workunit.client.0.smithi027.stdout:9/199: creat d3/da/dc/df/d34/f48 x:0 0 0 2024-06-06T08:43:22.741 INFO:tasks.workunit.client.0.smithi027.stdout:7/356: rename d17/d32/d46/d4f/d5e/d64 to d17/d32/d46/d51/d7f 0 2024-06-06T08:43:22.741 INFO:tasks.workunit.client.0.smithi027.stdout:7/357: write d17/d32/d61/f74 [635902,95297] 0 2024-06-06T08:43:22.741 INFO:tasks.workunit.client.0.smithi027.stdout:8/159: dread d7/db/f19 [0,4194304] 0 2024-06-06T08:43:22.748 INFO:tasks.workunit.client.0.smithi027.stdout:7/358: creat d17/d32/d61/f80 x:0 0 0 2024-06-06T08:43:22.754 INFO:tasks.workunit.client.0.smithi027.stdout:7/359: symlink d17/d32/d46/d51/d7f/l81 0 2024-06-06T08:43:22.754 INFO:tasks.workunit.client.0.smithi027.stdout:7/360: fsync fd 0 2024-06-06T08:43:22.760 INFO:tasks.workunit.client.0.smithi027.stdout:8/160: dread d7/d21/f25 [0,4194304] 0 2024-06-06T08:43:22.761 INFO:tasks.workunit.client.0.smithi027.stdout:0/235: dwrite d15/f21 [4194304,4194304] 0 2024-06-06T08:43:22.761 INFO:tasks.workunit.client.0.smithi027.stdout:0/236: write d15/d18/d23/f2e [406826,6777] 0 2024-06-06T08:43:22.761 INFO:tasks.workunit.client.0.smithi027.stdout:7/361: getdents d17/d32/d46/d51 0 2024-06-06T08:43:22.761 INFO:tasks.workunit.client.0.smithi027.stdout:2/290: write d6/dc/f40 [2081243,49638] 0 2024-06-06T08:43:22.762 INFO:tasks.workunit.client.0.smithi027.stdout:6/292: rename d4/l6 to d4/d1a/l61 0 2024-06-06T08:43:22.763 INFO:tasks.workunit.client.0.smithi027.stdout:8/161: unlink d7/d13/d15/f1a 0 2024-06-06T08:43:22.763 INFO:tasks.workunit.client.0.smithi027.stdout:9/200: dwrite d3/da/dc/f2a [0,4194304] 0 2024-06-06T08:43:22.764 INFO:tasks.workunit.client.0.smithi027.stdout:9/201: readlink d3/da/l1a 0 2024-06-06T08:43:22.765 INFO:tasks.workunit.client.0.smithi027.stdout:3/129: rename d1/d8/d12 to d1/d8/d22/d27 0 2024-06-06T08:43:22.766 INFO:tasks.workunit.client.0.smithi027.stdout:9/202: symlink d3/da/dc/df/d23/d36/d3a/l49 0 2024-06-06T08:43:22.768 INFO:tasks.workunit.client.0.smithi027.stdout:8/162: mknod d7/d21/c2b 0 2024-06-06T08:43:22.768 INFO:tasks.workunit.client.0.smithi027.stdout:0/237: dread f2 [0,4194304] 0 2024-06-06T08:43:22.770 INFO:tasks.workunit.client.0.smithi027.stdout:0/238: truncate d15/d18/d23/d28/d2a/d2d/f3a 2596108 0 2024-06-06T08:43:22.770 INFO:tasks.workunit.client.0.smithi027.stdout:1/242: rename dc/de/d14/d4e/f54 to dc/de/d3d/d4a/f58 0 2024-06-06T08:43:22.771 INFO:tasks.workunit.client.0.smithi027.stdout:9/203: creat d3/f4a x:0 0 0 2024-06-06T08:43:22.771 INFO:tasks.workunit.client.0.smithi027.stdout:9/204: readlink l0 0 2024-06-06T08:43:22.772 INFO:tasks.workunit.client.0.smithi027.stdout:8/163: creat d7/f2c x:0 0 0 2024-06-06T08:43:22.772 INFO:tasks.workunit.client.0.smithi027.stdout:9/205: mknod d3/da/dc/df/d23/d36/d3a/c4b 0 2024-06-06T08:43:22.774 INFO:tasks.workunit.client.0.smithi027.stdout:1/243: rename l2 to dc/de/l59 0 2024-06-06T08:43:22.774 INFO:tasks.workunit.client.0.smithi027.stdout:1/244: fdatasync dc/de/f13 0 2024-06-06T08:43:22.775 INFO:tasks.workunit.client.0.smithi027.stdout:2/291: write d6/f14 [2116686,24354] 0 2024-06-06T08:43:22.777 INFO:tasks.workunit.client.0.smithi027.stdout:8/164: rename d7/lf to d7/d21/l2d 0 2024-06-06T08:43:22.777 INFO:tasks.workunit.client.0.smithi027.stdout:8/165: write d7/dc/f20 [883241,56185] 0 2024-06-06T08:43:22.780 INFO:tasks.workunit.client.0.smithi027.stdout:2/292: mknod d6/dc/d1d/c6a 0 2024-06-06T08:43:22.781 INFO:tasks.workunit.client.0.smithi027.stdout:8/166: mkdir d7/d13/d15/d2e 0 2024-06-06T08:43:22.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/293: mkdir d6/dd/d6b 0 2024-06-06T08:43:22.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/294: chown d6/l51 1192789 1 2024-06-06T08:43:22.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/295: fsync d6/dd/f26 0 2024-06-06T08:43:22.788 INFO:tasks.workunit.client.0.smithi027.stdout:6/293: rmdir d4/d1a 39 2024-06-06T08:43:22.789 INFO:tasks.workunit.client.0.smithi027.stdout:7/362: read d17/d32/d46/d51/d52/f73 [2151734,90653] 0 2024-06-06T08:43:22.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/130: dwrite d1/f20 [0,4194304] 0 2024-06-06T08:43:22.792 INFO:tasks.workunit.client.0.smithi027.stdout:3/131: symlink d1/d8/l28 0 2024-06-06T08:43:22.794 INFO:tasks.workunit.client.0.smithi027.stdout:6/294: dread d4/d9/dc/f38 [0,4194304] 0 2024-06-06T08:43:22.795 INFO:tasks.workunit.client.0.smithi027.stdout:9/206: dwrite d3/f4a [0,4194304] 0 2024-06-06T08:43:22.795 INFO:tasks.workunit.client.0.smithi027.stdout:9/207: chown d3/da/dc/c40 858207705 1 2024-06-06T08:43:22.796 INFO:tasks.workunit.client.0.smithi027.stdout:6/295: truncate d4/f5 10099226 0 2024-06-06T08:43:22.799 INFO:tasks.workunit.client.0.smithi027.stdout:9/208: symlink d3/l4c 0 2024-06-06T08:43:22.799 INFO:tasks.workunit.client.0.smithi027.stdout:9/209: write d3/da/dc/df/d23/d36/f37 [3220083,23722] 0 2024-06-06T08:43:22.801 INFO:tasks.workunit.client.0.smithi027.stdout:6/296: unlink d4/d9/c4a 0 2024-06-06T08:43:22.803 INFO:tasks.workunit.client.0.smithi027.stdout:6/297: creat d4/d9/d49/f62 x:0 0 0 2024-06-06T08:43:22.803 INFO:tasks.workunit.client.0.smithi027.stdout:6/298: write d4/d9/dc/f43 [517862,67673] 0 2024-06-06T08:43:22.803 INFO:tasks.workunit.client.0.smithi027.stdout:8/167: dwrite d7/db/f1d [0,4194304] 0 2024-06-06T08:43:22.804 INFO:tasks.workunit.client.0.smithi027.stdout:0/239: link f4 d15/d18/d23/d38/f51 0 2024-06-06T08:43:22.807 INFO:tasks.workunit.client.0.smithi027.stdout:8/168: mkdir d7/d2f 0 2024-06-06T08:43:22.807 INFO:tasks.workunit.client.0.smithi027.stdout:8/169: fsync d7/f2c 0 2024-06-06T08:43:22.810 INFO:tasks.workunit.client.0.smithi027.stdout:2/296: write d6/d3b/f43 [2720607,62627] 0 2024-06-06T08:43:22.810 INFO:tasks.workunit.client.0.smithi027.stdout:1/245: dwrite dc/de/d14/f49 [0,4194304] 0 2024-06-06T08:43:22.810 INFO:tasks.workunit.client.0.smithi027.stdout:1/246: fsync dc/de/d14/d4e/f53 0 2024-06-06T08:43:22.810 INFO:tasks.workunit.client.0.smithi027.stdout:8/170: unlink d7/l2a 0 2024-06-06T08:43:22.813 INFO:tasks.workunit.client.0.smithi027.stdout:7/363: dwrite f14 [0,4194304] 0 2024-06-06T08:43:22.813 INFO:tasks.workunit.client.0.smithi027.stdout:7/364: dread - d17/d31/f45 zero size 2024-06-06T08:43:22.813 INFO:tasks.workunit.client.0.smithi027.stdout:7/365: chown d17/d31/d2a/d3b/f5b 874 1 2024-06-06T08:43:22.813 INFO:tasks.workunit.client.0.smithi027.stdout:7/366: write d17/d32/d46/d4f/f55 [909190,36344] 0 2024-06-06T08:43:22.820 INFO:tasks.workunit.client.0.smithi027.stdout:2/297: creat d6/dd/d3c/d55/d69/f6c x:0 0 0 2024-06-06T08:43:22.820 INFO:tasks.workunit.client.0.smithi027.stdout:2/298: fdatasync d6/f14 0 2024-06-06T08:43:22.821 INFO:tasks.workunit.client.0.smithi027.stdout:9/210: dwrite d3/da/dc/df/f2f [0,4194304] 0 2024-06-06T08:43:22.822 INFO:tasks.workunit.client.0.smithi027.stdout:1/247: mknod dc/de/d14/d21/d32/d37/d56/c5a 0 2024-06-06T08:43:22.824 INFO:tasks.workunit.client.0.smithi027.stdout:8/171: rmdir d7/d13/d15/d2e 0 2024-06-06T08:43:22.824 INFO:tasks.workunit.client.0.smithi027.stdout:8/172: chown d7/c23 118060 1 2024-06-06T08:43:22.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/240: write d15/d18/d23/d38/f51 [3233618,15675] 0 2024-06-06T08:43:22.825 INFO:tasks.workunit.client.0.smithi027.stdout:7/367: mknod d17/d47/c82 0 2024-06-06T08:43:22.825 INFO:tasks.workunit.client.0.smithi027.stdout:0/241: getdents d15/d18/d23/d28/d2a/d2d 0 2024-06-06T08:43:22.828 INFO:tasks.workunit.client.0.smithi027.stdout:0/242: symlink d15/d18/d23/d38/l52 0 2024-06-06T08:43:22.828 INFO:tasks.workunit.client.0.smithi027.stdout:2/299: creat d6/dd/d3c/d55/f6d x:0 0 0 2024-06-06T08:43:22.830 INFO:tasks.workunit.client.0.smithi027.stdout:1/248: creat dc/de/d14/d21/d32/f5b x:0 0 0 2024-06-06T08:43:22.833 INFO:tasks.workunit.client.0.smithi027.stdout:8/173: rmdir d7/dc 39 2024-06-06T08:43:22.834 INFO:tasks.workunit.client.0.smithi027.stdout:0/243: mknod d15/c53 0 2024-06-06T08:43:22.836 INFO:tasks.workunit.client.0.smithi027.stdout:0/244: rmdir d15/d18/d40 39 2024-06-06T08:43:22.836 INFO:tasks.workunit.client.0.smithi027.stdout:0/245: read f2 [2064060,56707] 0 2024-06-06T08:43:22.836 INFO:tasks.workunit.client.0.smithi027.stdout:0/246: truncate d15/d18/d23/d28/f4e 659056 0 2024-06-06T08:43:22.836 INFO:tasks.workunit.client.0.smithi027.stdout:0/247: truncate d15/d18/d23/d28/d2a/d2d/f3b 1181418 0 2024-06-06T08:43:22.838 INFO:tasks.workunit.client.0.smithi027.stdout:0/248: creat d15/d18/d23/d28/d2a/d2d/f54 x:0 0 0 2024-06-06T08:43:22.840 INFO:tasks.workunit.client.0.smithi027.stdout:0/249: creat d15/d18/d23/d28/f55 x:0 0 0 2024-06-06T08:43:22.840 INFO:tasks.workunit.client.0.smithi027.stdout:0/250: dread - d15/d18/d23/d28/d2a/d2d/f54 zero size 2024-06-06T08:43:22.840 INFO:tasks.workunit.client.0.smithi027.stdout:0/251: chown d15/d18/d23/d38/f51 204 1 2024-06-06T08:43:22.842 INFO:tasks.workunit.client.0.smithi027.stdout:0/252: creat d15/d18/d23/d28/d2a/d2d/f56 x:0 0 0 2024-06-06T08:43:22.847 INFO:tasks.workunit.client.0.smithi027.stdout:6/299: rename d4/d9/d49/l5f to d4/de/l63 0 2024-06-06T08:43:22.847 INFO:tasks.workunit.client.0.smithi027.stdout:6/300: chown d4/f29 422 1 2024-06-06T08:43:22.847 INFO:tasks.workunit.client.0.smithi027.stdout:6/301: write d4/d9/f28 [823838,125541] 0 2024-06-06T08:43:22.850 INFO:tasks.workunit.client.0.smithi027.stdout:6/302: unlink d4/de/f35 0 2024-06-06T08:43:22.850 INFO:tasks.workunit.client.0.smithi027.stdout:8/174: dwrite d7/d13/f24 [0,4194304] 0 2024-06-06T08:43:22.850 INFO:tasks.workunit.client.0.smithi027.stdout:1/249: dwrite dc/f36 [0,4194304] 0 2024-06-06T08:43:22.851 INFO:tasks.workunit.client.0.smithi027.stdout:6/303: mknod d4/c64 0 2024-06-06T08:43:22.854 INFO:tasks.workunit.client.0.smithi027.stdout:8/175: creat d7/f30 x:0 0 0 2024-06-06T08:43:22.857 INFO:tasks.workunit.client.0.smithi027.stdout:1/250: symlink dc/de/d3d/d4f/l5c 0 2024-06-06T08:43:22.859 INFO:tasks.workunit.client.0.smithi027.stdout:0/253: dwrite fb [0,4194304] 0 2024-06-06T08:43:22.859 INFO:tasks.workunit.client.0.smithi027.stdout:0/254: fsync d15/d18/d23/d28/d2a/f33 0 2024-06-06T08:43:22.859 INFO:tasks.workunit.client.0.smithi027.stdout:0/255: fdatasync f2 0 2024-06-06T08:43:22.859 INFO:tasks.workunit.client.0.smithi027.stdout:0/256: write d15/d18/d23/f29 [1938455,10006] 0 2024-06-06T08:43:22.860 INFO:tasks.workunit.client.0.smithi027.stdout:1/251: link dc/de/d14/d21/f52 dc/de/d14/d4e/f5d 0 2024-06-06T08:43:22.861 INFO:tasks.workunit.client.0.smithi027.stdout:8/176: getdents d7 0 2024-06-06T08:43:22.864 INFO:tasks.workunit.client.0.smithi027.stdout:6/304: creat d4/d1a/d2e/f65 x:0 0 0 2024-06-06T08:43:22.864 INFO:tasks.workunit.client.0.smithi027.stdout:6/305: read - d4/d1a/d2e/d44/f5a zero size 2024-06-06T08:43:22.866 INFO:tasks.workunit.client.0.smithi027.stdout:1/252: mknod dc/de/d14/d21/d32/d37/d56/c5e 0 2024-06-06T08:43:22.867 INFO:tasks.workunit.client.0.smithi027.stdout:6/306: creat d4/d9/f66 x:0 0 0 2024-06-06T08:43:22.867 INFO:tasks.workunit.client.0.smithi027.stdout:7/368: rename d17/d31 to d17/d32/d46/d83 0 2024-06-06T08:43:22.869 INFO:tasks.workunit.client.0.smithi027.stdout:6/307: unlink d4/f10 0 2024-06-06T08:43:22.873 INFO:tasks.workunit.client.0.smithi027.stdout:5/366: sync 2024-06-06T08:43:22.874 INFO:tasks.workunit.client.0.smithi027.stdout:5/367: chown d0/l49 0 1 2024-06-06T08:43:22.876 INFO:tasks.workunit.client.0.smithi027.stdout:4/153: sync 2024-06-06T08:43:22.880 INFO:tasks.workunit.client.0.smithi027.stdout:4/154: creat d0/d1/d6/d8/d21/f30 x:0 0 0 2024-06-06T08:43:22.881 INFO:tasks.workunit.client.0.smithi027.stdout:4/155: mknod d0/d1/d29/c31 0 2024-06-06T08:43:22.882 INFO:tasks.workunit.client.0.smithi027.stdout:0/257: dwrite d15/f22 [0,4194304] 0 2024-06-06T08:43:22.883 INFO:tasks.workunit.client.0.smithi027.stdout:8/177: dwrite d7/db/f27 [0,4194304] 0 2024-06-06T08:43:22.886 INFO:tasks.workunit.client.0.smithi027.stdout:8/178: creat d7/d2f/f31 x:0 0 0 2024-06-06T08:43:22.887 INFO:tasks.workunit.client.0.smithi027.stdout:1/253: dwrite dc/f30 [0,4194304] 0 2024-06-06T08:43:22.888 INFO:tasks.workunit.client.0.smithi027.stdout:8/179: symlink d7/db/l32 0 2024-06-06T08:43:22.893 INFO:tasks.workunit.client.0.smithi027.stdout:8/180: unlink d7/d13/d15/f1f 0 2024-06-06T08:43:22.893 INFO:tasks.workunit.client.0.smithi027.stdout:6/308: rename d4/d9/dc/l1e to d4/d1a/d46/l67 0 2024-06-06T08:43:22.895 INFO:tasks.workunit.client.0.smithi027.stdout:8/181: link d7/c18 d7/d21/c33 0 2024-06-06T08:43:22.896 INFO:tasks.workunit.client.0.smithi027.stdout:6/309: symlink d4/l68 0 2024-06-06T08:43:22.897 INFO:tasks.workunit.client.0.smithi027.stdout:1/254: dread dc/de/d14/d21/f27 [0,4194304] 0 2024-06-06T08:43:22.897 INFO:tasks.workunit.client.0.smithi027.stdout:1/255: fsync dc/de/d14/f1a 0 2024-06-06T08:43:22.908 INFO:tasks.workunit.client.0.smithi027.stdout:4/156: rename d0/d1/d6/l18 to d0/d1/d6/d8/l32 0 2024-06-06T08:43:22.908 INFO:tasks.workunit.client.0.smithi027.stdout:4/157: truncate d0/d1/d6/d8/d21/f2d 705501 0 2024-06-06T08:43:22.908 INFO:tasks.workunit.client.0.smithi027.stdout:4/158: write d0/d1/d6/d8/d21/f2d [1458026,70137] 0 2024-06-06T08:43:22.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/258: rename d15/d18/d23/d28/d2a/d2d/d44/c49 to d15/d18/d40/c57 0 2024-06-06T08:43:22.911 INFO:tasks.workunit.client.0.smithi027.stdout:8/182: rename d7/dc/f26 to d7/d21/f34 0 2024-06-06T08:43:22.912 INFO:tasks.workunit.client.0.smithi027.stdout:8/183: symlink d7/d13/l35 0 2024-06-06T08:43:22.913 INFO:tasks.workunit.client.0.smithi027.stdout:8/184: mkdir d7/dc/d36 0 2024-06-06T08:43:22.914 INFO:tasks.workunit.client.0.smithi027.stdout:8/185: truncate d7/d21/f25 4297749 0 2024-06-06T08:43:22.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/310: dread d4/f25 [0,4194304] 0 2024-06-06T08:43:22.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/311: readlink d4/l47 0 2024-06-06T08:43:22.922 INFO:tasks.workunit.client.0.smithi027.stdout:4/159: read d0/d1/fb [7252396,44563] 0 2024-06-06T08:43:22.927 INFO:tasks.workunit.client.0.smithi027.stdout:1/256: dread dc/f30 [0,4194304] 0 2024-06-06T08:43:22.928 INFO:tasks.workunit.client.0.smithi027.stdout:1/257: mknod dc/de/d22/c5f 0 2024-06-06T08:43:22.928 INFO:tasks.workunit.client.0.smithi027.stdout:0/259: dwrite fb [0,4194304] 0 2024-06-06T08:43:22.930 INFO:tasks.workunit.client.0.smithi027.stdout:1/258: rename dc/de/d22/c5f to dc/de/d3d/d4f/c60 0 2024-06-06T08:43:22.930 INFO:tasks.workunit.client.0.smithi027.stdout:0/260: creat d15/d18/d23/d28/d2a/d2d/f58 x:0 0 0 2024-06-06T08:43:22.933 INFO:tasks.workunit.client.0.smithi027.stdout:0/261: dread d15/d18/d23/d28/d2a/d2d/f3b [0,4194304] 0 2024-06-06T08:43:22.935 INFO:tasks.workunit.client.0.smithi027.stdout:0/262: dread f9 [0,4194304] 0 2024-06-06T08:43:22.936 INFO:tasks.workunit.client.0.smithi027.stdout:0/263: mknod d15/d18/d23/d28/c59 0 2024-06-06T08:43:22.940 INFO:tasks.workunit.client.0.smithi027.stdout:1/259: dread dc/de/d22/f3b [0,4194304] 0 2024-06-06T08:43:22.941 INFO:tasks.workunit.client.0.smithi027.stdout:1/260: truncate dc/de/d14/f15 1330758 0 2024-06-06T08:43:22.942 INFO:tasks.workunit.client.0.smithi027.stdout:4/160: dwrite d0/d1/d6/d8/d21/f2d [0,4194304] 0 2024-06-06T08:43:22.944 INFO:tasks.workunit.client.0.smithi027.stdout:4/161: creat d0/d1/d6/f33 x:0 0 0 2024-06-06T08:43:22.944 INFO:tasks.workunit.client.0.smithi027.stdout:4/162: chown d0/d1 1326 1 2024-06-06T08:43:22.945 INFO:tasks.workunit.client.0.smithi027.stdout:4/163: getdents d0/d1/d6/d8/d21 0 2024-06-06T08:43:22.946 INFO:tasks.workunit.client.0.smithi027.stdout:4/164: mkdir d0/d1d/d34 0 2024-06-06T08:43:22.946 INFO:tasks.workunit.client.0.smithi027.stdout:4/165: chown d0/d1/c28 176 1 2024-06-06T08:43:22.947 INFO:tasks.workunit.client.0.smithi027.stdout:4/166: chown d0/d1/d6/d8/d21 28370168 1 2024-06-06T08:43:22.951 INFO:tasks.workunit.client.0.smithi027.stdout:1/261: read dc/de/d14/f19 [2718942,98355] 0 2024-06-06T08:43:22.952 INFO:tasks.workunit.client.0.smithi027.stdout:0/264: dwrite d15/d18/d23/d28/d2a/d2d/f34 [0,4194304] 0 2024-06-06T08:43:22.953 INFO:tasks.workunit.client.0.smithi027.stdout:1/262: symlink dc/de/d14/d21/d32/d37/l61 0 2024-06-06T08:43:22.954 INFO:tasks.workunit.client.0.smithi027.stdout:1/263: creat dc/de/d14/d4e/f62 x:0 0 0 2024-06-06T08:43:22.955 INFO:tasks.workunit.client.0.smithi027.stdout:0/265: link d15/d18/d23/d28/d2a/d2d/f58 d15/d18/d23/d38/d3e/f5a 0 2024-06-06T08:43:22.955 INFO:tasks.workunit.client.0.smithi027.stdout:0/266: truncate d15/d18/d23/d28/d2a/d2d/d44/f4a 370209 0 2024-06-06T08:43:22.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:22 smithi027 ceph-mon[32335]: pgmap v96: 129 pgs: 129 active+clean; 398 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 46 MiB/s wr, 847 op/s 2024-06-06T08:43:22.958 INFO:tasks.workunit.client.0.smithi027.stdout:4/167: dread d0/d1/fb [0,4194304] 0 2024-06-06T08:43:22.959 INFO:tasks.workunit.client.0.smithi027.stdout:4/168: chown d0/d1/d6/d8/c26 3526 1 2024-06-06T08:43:22.959 INFO:tasks.workunit.client.0.smithi027.stdout:0/267: getdents d15/d18/d23/d38/d3e 0 2024-06-06T08:43:22.959 INFO:tasks.workunit.client.0.smithi027.stdout:0/268: dread - d15/d18/d23/d28/d2a/d2d/f54 zero size 2024-06-06T08:43:22.960 INFO:tasks.workunit.client.0.smithi027.stdout:4/169: mknod d0/d1/d29/c35 0 2024-06-06T08:43:22.961 INFO:tasks.workunit.client.0.smithi027.stdout:4/170: link d0/d1/d6/l10 d0/l36 0 2024-06-06T08:43:22.961 INFO:tasks.workunit.client.0.smithi027.stdout:4/171: fsync d0/d1/ff 0 2024-06-06T08:43:22.962 INFO:tasks.workunit.client.0.smithi027.stdout:4/172: mknod d0/d1d/d2a/c37 0 2024-06-06T08:43:22.962 INFO:tasks.workunit.client.0.smithi027.stdout:4/173: write d0/d1/d6/f2f [958209,77015] 0 2024-06-06T08:43:22.970 INFO:tasks.workunit.client.0.smithi027.stdout:4/174: dread d0/f13 [8388608,4194304] 0 2024-06-06T08:43:22.970 INFO:tasks.workunit.client.0.smithi027.stdout:4/175: write d0/f1c [269954,73941] 0 2024-06-06T08:43:22.970 INFO:tasks.workunit.client.0.smithi027.stdout:1/264: dwrite dc/de/f47 [0,4194304] 0 2024-06-06T08:43:22.976 INFO:tasks.workunit.client.0.smithi027.stdout:4/176: rename d0/d1/d29 to d0/d1d/d38 0 2024-06-06T08:43:22.979 INFO:tasks.workunit.client.0.smithi027.stdout:1/265: dread dc/f30 [0,4194304] 0 2024-06-06T08:43:22.987 INFO:tasks.workunit.client.0.smithi027.stdout:4/177: rename d0/c16 to d0/d1/d6/c39 0 2024-06-06T08:43:22.988 INFO:tasks.workunit.client.0.smithi027.stdout:4/178: link d0/d1/l17 d0/d1d/d38/l3a 0 2024-06-06T08:43:22.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:22 smithi203 ceph-mon[36579]: pgmap v96: 129 pgs: 129 active+clean; 398 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 46 MiB/s wr, 847 op/s 2024-06-06T08:43:22.990 INFO:tasks.workunit.client.0.smithi027.stdout:4/179: rename d0/d1/d24 to d0/d1d/d38/d3b 0 2024-06-06T08:43:22.990 INFO:tasks.workunit.client.0.smithi027.stdout:4/180: chown d0/l36 26 1 2024-06-06T08:43:22.990 INFO:tasks.workunit.client.0.smithi027.stdout:4/181: chown d0/d1/f12 6 1 2024-06-06T08:43:22.999 INFO:tasks.workunit.client.0.smithi027.stdout:1/266: dwrite dc/de/d14/d4e/f62 [0,4194304] 0 2024-06-06T08:43:23.001 INFO:tasks.workunit.client.0.smithi027.stdout:1/267: truncate dc/de/f47 241847 0 2024-06-06T08:43:23.001 INFO:tasks.workunit.client.0.smithi027.stdout:1/268: write dc/de/d3d/f50 [19020,20648] 0 2024-06-06T08:43:23.003 INFO:tasks.workunit.client.0.smithi027.stdout:1/269: creat dc/de/d14/f63 x:0 0 0 2024-06-06T08:43:23.003 INFO:tasks.workunit.client.0.smithi027.stdout:1/270: chown dc/de/d14/d21/d32/d37/c40 26512483 1 2024-06-06T08:43:23.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:22 smithi176 ceph-mon[36786]: pgmap v96: 129 pgs: 129 active+clean; 398 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 46 MiB/s wr, 847 op/s 2024-06-06T08:43:23.008 INFO:tasks.workunit.client.0.smithi027.stdout:4/182: dwrite d0/d1/f1a [4194304,4194304] 0 2024-06-06T08:43:23.008 INFO:tasks.workunit.client.0.smithi027.stdout:4/183: chown d0/f1c 23563048 1 2024-06-06T08:43:23.010 INFO:tasks.workunit.client.0.smithi027.stdout:4/184: creat d0/d1d/d38/f3c x:0 0 0 2024-06-06T08:43:23.017 INFO:tasks.workunit.client.0.smithi027.stdout:4/185: write d0/d1/f14 [2889814,118797] 0 2024-06-06T08:43:23.018 INFO:tasks.workunit.client.0.smithi027.stdout:4/186: rmdir d0/d1d 39 2024-06-06T08:43:23.018 INFO:tasks.workunit.client.0.smithi027.stdout:4/187: chown d0/l36 15935 1 2024-06-06T08:43:23.019 INFO:tasks.workunit.client.0.smithi027.stdout:4/188: mknod d0/d1/c3d 0 2024-06-06T08:43:23.035 INFO:tasks.workunit.client.0.smithi027.stdout:4/189: dwrite d0/d1/d6/f33 [0,4194304] 0 2024-06-06T08:43:23.036 INFO:tasks.workunit.client.0.smithi027.stdout:4/190: write d0/d1/fb [8977733,125947] 0 2024-06-06T08:43:23.045 INFO:tasks.workunit.client.0.smithi027.stdout:4/191: write d0/d1/d6/d8/d21/f2d [2554843,89892] 0 2024-06-06T08:43:23.045 INFO:tasks.workunit.client.0.smithi027.stdout:4/192: truncate d0/d1/d6/f2f 1415409 0 2024-06-06T08:43:23.045 INFO:tasks.workunit.client.0.smithi027.stdout:4/193: write d0/d1/fb [6559504,62070] 0 2024-06-06T08:43:23.045 INFO:tasks.workunit.client.0.smithi027.stdout:4/194: fdatasync d0/d1/f1a 0 2024-06-06T08:43:23.046 INFO:tasks.workunit.client.0.smithi027.stdout:4/195: symlink d0/d1/d6/d8/d21/l3e 0 2024-06-06T08:43:23.060 INFO:tasks.workunit.client.0.smithi027.stdout:4/196: dwrite d0/d1/d6/f25 [0,4194304] 0 2024-06-06T08:43:23.062 INFO:tasks.workunit.client.0.smithi027.stdout:4/197: symlink d0/d1d/d34/l3f 0 2024-06-06T08:43:23.066 INFO:tasks.workunit.client.0.smithi027.stdout:4/198: truncate d0/d1/f14 10028955 0 2024-06-06T08:43:23.068 INFO:tasks.workunit.client.0.smithi027.stdout:4/199: creat d0/d1d/d34/f40 x:0 0 0 2024-06-06T08:43:23.068 INFO:tasks.workunit.client.0.smithi027.stdout:4/200: write d0/d1d/d34/f40 [541479,14957] 0 2024-06-06T08:43:23.069 INFO:tasks.workunit.client.0.smithi027.stdout:4/201: rmdir d0 39 2024-06-06T08:43:23.070 INFO:tasks.workunit.client.0.smithi027.stdout:4/202: rename d0/d1/d6/l10 to d0/d1/d6/l41 0 2024-06-06T08:43:23.071 INFO:tasks.workunit.client.0.smithi027.stdout:4/203: write d0/fd [7503028,125712] 0 2024-06-06T08:43:23.089 INFO:tasks.workunit.client.0.smithi027.stdout:4/204: dwrite d0/f20 [0,4194304] 0 2024-06-06T08:43:23.092 INFO:tasks.workunit.client.0.smithi027.stdout:4/205: creat d0/d1d/d38/f42 x:0 0 0 2024-06-06T08:43:23.100 INFO:tasks.workunit.client.0.smithi027.stdout:4/206: dread d0/d1/f1a [4194304,4194304] 0 2024-06-06T08:43:23.113 INFO:tasks.workunit.client.0.smithi027.stdout:4/207: dwrite d0/f1c [0,4194304] 0 2024-06-06T08:43:23.115 INFO:tasks.workunit.client.0.smithi027.stdout:4/208: symlink d0/d1d/d38/d3b/l43 0 2024-06-06T08:43:23.116 INFO:tasks.workunit.client.0.smithi027.stdout:4/209: fsync d0/fd 0 2024-06-06T08:43:23.125 INFO:tasks.workunit.client.0.smithi027.stdout:3/132: sync 2024-06-06T08:43:23.125 INFO:tasks.workunit.client.0.smithi027.stdout:3/133: readlink d1/da/l11 0 2024-06-06T08:43:23.125 INFO:tasks.workunit.client.0.smithi027.stdout:3/134: readlink d1/l4 0 2024-06-06T08:43:23.126 INFO:tasks.workunit.client.0.smithi027.stdout:3/135: readlink d1/da/l11 0 2024-06-06T08:43:23.127 INFO:tasks.workunit.client.0.smithi027.stdout:3/136: creat d1/f29 x:0 0 0 2024-06-06T08:43:23.128 INFO:tasks.workunit.client.0.smithi027.stdout:3/137: creat d1/f2a x:0 0 0 2024-06-06T08:43:23.129 INFO:tasks.workunit.client.0.smithi027.stdout:3/138: unlink d1/d8/cd 0 2024-06-06T08:43:23.129 INFO:tasks.workunit.client.0.smithi027.stdout:3/139: stat d1/d1f 0 2024-06-06T08:43:23.140 INFO:tasks.workunit.client.0.smithi027.stdout:3/140: dread d1/d8/f1a [0,4194304] 0 2024-06-06T08:43:23.141 INFO:tasks.workunit.client.0.smithi027.stdout:3/141: rename d1/l10 to d1/d1f/l2b 0 2024-06-06T08:43:23.143 INFO:tasks.workunit.client.0.smithi027.stdout:3/142: rename d1/d8/f1d to d1/da/f2c 0 2024-06-06T08:43:23.143 INFO:tasks.workunit.client.0.smithi027.stdout:3/143: readlink d1/da/l19 0 2024-06-06T08:43:23.143 INFO:tasks.workunit.client.0.smithi027.stdout:3/144: readlink l0 0 2024-06-06T08:43:23.143 INFO:tasks.workunit.client.0.smithi027.stdout:3/145: write d1/da/f1c [878911,93782] 0 2024-06-06T08:43:23.144 INFO:tasks.workunit.client.0.smithi027.stdout:3/146: mkdir d1/d8/d22/d2d 0 2024-06-06T08:43:23.145 INFO:tasks.workunit.client.0.smithi027.stdout:3/147: truncate d1/d8/f17 221640 0 2024-06-06T08:43:23.145 INFO:tasks.workunit.client.0.smithi027.stdout:3/148: write d1/da/f1c [1909610,72968] 0 2024-06-06T08:43:23.153 INFO:tasks.workunit.client.0.smithi027.stdout:3/149: truncate d1/da/f1c 640759 0 2024-06-06T08:43:23.153 INFO:tasks.workunit.client.0.smithi027.stdout:3/150: write d1/da/f2c [459807,105570] 0 2024-06-06T08:43:23.155 INFO:tasks.workunit.client.0.smithi027.stdout:3/151: rename d1/da/c21 to d1/d8/d22/c2e 0 2024-06-06T08:43:23.155 INFO:tasks.workunit.client.0.smithi027.stdout:3/152: chown d1/d8/c23 23924 1 2024-06-06T08:43:23.155 INFO:tasks.workunit.client.0.smithi027.stdout:3/153: chown d1/f13 821137 1 2024-06-06T08:43:23.162 INFO:tasks.workunit.client.0.smithi027.stdout:3/154: dread d1/f6 [0,4194304] 0 2024-06-06T08:43:23.162 INFO:tasks.workunit.client.0.smithi027.stdout:3/155: write d1/da/f1c [360109,80514] 0 2024-06-06T08:43:23.162 INFO:tasks.workunit.client.0.smithi027.stdout:3/156: truncate d1/d8/f16 14832 0 2024-06-06T08:43:23.165 INFO:tasks.workunit.client.0.smithi027.stdout:3/157: unlink d1/d8/d22/d27/c26 0 2024-06-06T08:43:23.167 INFO:tasks.workunit.client.0.smithi027.stdout:3/158: rename d1/d8/d22/c2e to d1/d1f/c2f 0 2024-06-06T08:43:23.167 INFO:tasks.workunit.client.0.smithi027.stdout:3/159: stat d1/f20 0 2024-06-06T08:43:23.169 INFO:tasks.workunit.client.0.smithi027.stdout:3/160: unlink d1/d8/f1a 0 2024-06-06T08:43:23.185 INFO:tasks.workunit.client.0.smithi027.stdout:3/161: dwrite d1/da/f18 [0,4194304] 0 2024-06-06T08:43:23.185 INFO:tasks.workunit.client.0.smithi027.stdout:3/162: readlink d1/da/l19 0 2024-06-06T08:43:23.187 INFO:tasks.workunit.client.0.smithi027.stdout:3/163: rename d1/da/l24 to d1/d1f/l30 0 2024-06-06T08:43:23.187 INFO:tasks.workunit.client.0.smithi027.stdout:3/164: readlink d1/d1f/l30 0 2024-06-06T08:43:23.202 INFO:tasks.workunit.client.0.smithi027.stdout:3/165: dwrite d1/f29 [0,4194304] 0 2024-06-06T08:43:23.202 INFO:tasks.workunit.client.0.smithi027.stdout:3/166: fdatasync d1/d8/f17 0 2024-06-06T08:43:23.205 INFO:tasks.workunit.client.0.smithi027.stdout:3/167: creat d1/da/f31 x:0 0 0 2024-06-06T08:43:23.206 INFO:tasks.workunit.client.0.smithi027.stdout:3/168: creat d1/d8/d22/d27/f32 x:0 0 0 2024-06-06T08:43:23.206 INFO:tasks.workunit.client.0.smithi027.stdout:3/169: readlink d1/d1f/l2b 0 2024-06-06T08:43:23.208 INFO:tasks.workunit.client.0.smithi027.stdout:3/170: rename d1/da/f1c to d1/f33 0 2024-06-06T08:43:23.208 INFO:tasks.workunit.client.0.smithi027.stdout:3/171: write d1/f6 [768497,93824] 0 2024-06-06T08:43:23.208 INFO:tasks.workunit.client.0.smithi027.stdout:3/172: write d1/f2a [557405,47246] 0 2024-06-06T08:43:23.209 INFO:tasks.workunit.client.0.smithi027.stdout:3/173: creat d1/f34 x:0 0 0 2024-06-06T08:43:23.209 INFO:tasks.workunit.client.0.smithi027.stdout:3/174: write d1/da/f31 [297240,2381] 0 2024-06-06T08:43:23.210 INFO:tasks.workunit.client.0.smithi027.stdout:3/175: symlink d1/d1f/l35 0 2024-06-06T08:43:23.211 INFO:tasks.workunit.client.0.smithi027.stdout:3/176: unlink d1/d8/f17 0 2024-06-06T08:43:23.213 INFO:tasks.workunit.client.0.smithi027.stdout:3/177: rename d1/f34 to d1/d8/d22/d27/f36 0 2024-06-06T08:43:23.213 INFO:tasks.workunit.client.0.smithi027.stdout:3/178: fsync d1/d8/f15 0 2024-06-06T08:43:23.215 INFO:tasks.workunit.client.0.smithi027.stdout:3/179: rmdir d1/da 39 2024-06-06T08:43:23.219 INFO:tasks.workunit.client.0.smithi027.stdout:3/180: rename d1/f2a to d1/d8/d22/f37 0 2024-06-06T08:43:23.220 INFO:tasks.workunit.client.0.smithi027.stdout:1/271: sync 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:6/312: sync 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:9/211: sync 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:1/272: readlink dc/de/d14/d21/l25 0 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:5/368: sync 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:0/269: sync 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:2/300: sync 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:8/186: sync 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:7/369: sync 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:0/270: write d15/d18/d23/d38/d3e/f5a [969097,94972] 0 2024-06-06T08:43:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:6/313: write d4/de/d45/f48 [1116421,125712] 0 2024-06-06T08:43:23.222 INFO:tasks.workunit.client.0.smithi027.stdout:1/273: creat dc/de/d3d/d4f/f64 x:0 0 0 2024-06-06T08:43:23.223 INFO:tasks.workunit.client.0.smithi027.stdout:2/301: creat d6/f6e x:0 0 0 2024-06-06T08:43:23.224 INFO:tasks.workunit.client.0.smithi027.stdout:2/302: readlink d6/d3b/l53 0 2024-06-06T08:43:23.225 INFO:tasks.workunit.client.0.smithi027.stdout:5/369: symlink d0/da/d37/l74 0 2024-06-06T08:43:23.225 INFO:tasks.workunit.client.0.smithi027.stdout:5/370: chown d0/da/d47 10602457 1 2024-06-06T08:43:23.225 INFO:tasks.workunit.client.0.smithi027.stdout:5/371: stat d0/l49 0 2024-06-06T08:43:23.225 INFO:tasks.workunit.client.0.smithi027.stdout:9/212: mknod d3/da/c4d 0 2024-06-06T08:43:23.226 INFO:tasks.workunit.client.0.smithi027.stdout:6/314: creat d4/f69 x:0 0 0 2024-06-06T08:43:23.226 INFO:tasks.workunit.client.0.smithi027.stdout:6/315: chown d4/f5 245905131 1 2024-06-06T08:43:23.226 INFO:tasks.workunit.client.0.smithi027.stdout:4/210: sync 2024-06-06T08:43:23.227 INFO:tasks.workunit.client.0.smithi027.stdout:7/370: mkdir d17/d32/d46/d4f/d5e/d7b/d84 0 2024-06-06T08:43:23.227 INFO:tasks.workunit.client.0.smithi027.stdout:3/181: getdents d1/d8/d22 0 2024-06-06T08:43:23.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/274: symlink dc/de/l65 0 2024-06-06T08:43:23.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/275: dread - dc/de/d3d/f3e zero size 2024-06-06T08:43:23.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/276: write dc/de/f47 [564328,111888] 0 2024-06-06T08:43:23.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/277: write dc/de/d3d/f50 [661406,31980] 0 2024-06-06T08:43:23.230 INFO:tasks.workunit.client.0.smithi027.stdout:2/303: truncate d6/fe 129873 0 2024-06-06T08:43:23.231 INFO:tasks.workunit.client.0.smithi027.stdout:9/213: unlink d3/cb 0 2024-06-06T08:43:23.232 INFO:tasks.workunit.client.0.smithi027.stdout:3/182: truncate d1/d8/f15 249623 0 2024-06-06T08:43:23.233 INFO:tasks.workunit.client.0.smithi027.stdout:8/187: mkdir d7/dc/d37 0 2024-06-06T08:43:23.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/188: rename d7/d21 to d7/d2f/d38 0 2024-06-06T08:43:23.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/189: chown d7/dc/d36 12881989 1 2024-06-06T08:43:23.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/190: write d7/f2c [188993,61048] 0 2024-06-06T08:43:23.236 INFO:tasks.workunit.client.0.smithi027.stdout:2/304: symlink d6/dc/d3f/l6f 0 2024-06-06T08:43:23.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/211: dread d0/d1/d6/f25 [0,4194304] 0 2024-06-06T08:43:23.237 INFO:tasks.workunit.client.0.smithi027.stdout:5/372: read d0/da/d37/d2b/f3d [3109558,88388] 0 2024-06-06T08:43:23.238 INFO:tasks.workunit.client.0.smithi027.stdout:0/271: read d15/f21 [8088138,126805] 0 2024-06-06T08:43:23.239 INFO:tasks.workunit.client.0.smithi027.stdout:6/316: write d4/d9/dc/f38 [436965,103528] 0 2024-06-06T08:43:23.239 INFO:tasks.workunit.client.0.smithi027.stdout:6/317: truncate d4/d9/f66 174568 0 2024-06-06T08:43:23.239 INFO:tasks.workunit.client.0.smithi027.stdout:0/272: getdents d15/d18/d23 0 2024-06-06T08:43:23.240 INFO:tasks.workunit.client.0.smithi027.stdout:2/305: rename d6/d1e/f3d to d6/d3b/d61/f70 0 2024-06-06T08:43:23.240 INFO:tasks.workunit.client.0.smithi027.stdout:2/306: chown d6/dd/f26 188419 1 2024-06-06T08:43:23.242 INFO:tasks.workunit.client.0.smithi027.stdout:5/373: link d0/c4d d0/da/d47/c75 0 2024-06-06T08:43:23.242 INFO:tasks.workunit.client.0.smithi027.stdout:5/374: chown d0/da/d37/l74 7 1 2024-06-06T08:43:23.242 INFO:tasks.workunit.client.0.smithi027.stdout:5/375: chown d0/f4 2176968 1 2024-06-06T08:43:23.245 INFO:tasks.workunit.client.0.smithi027.stdout:0/273: mkdir d15/d18/d23/d5b 0 2024-06-06T08:43:23.247 INFO:tasks.workunit.client.0.smithi027.stdout:2/307: rmdir d6/dd/d3c/d55/d69 39 2024-06-06T08:43:23.249 INFO:tasks.workunit.client.0.smithi027.stdout:0/274: mkdir d15/d18/d23/d28/d2a/d30/d5c 0 2024-06-06T08:43:23.250 INFO:tasks.workunit.client.0.smithi027.stdout:9/214: dread d3/da/dc/df/f2f [0,4194304] 0 2024-06-06T08:43:23.250 INFO:tasks.workunit.client.0.smithi027.stdout:2/308: read d6/dc/d1d/f38 [599605,105829] 0 2024-06-06T08:43:23.251 INFO:tasks.workunit.client.0.smithi027.stdout:0/275: readlink d15/d18/l1b 0 2024-06-06T08:43:23.252 INFO:tasks.workunit.client.0.smithi027.stdout:8/191: getdents d7 0 2024-06-06T08:43:23.252 INFO:tasks.workunit.client.0.smithi027.stdout:8/192: chown d7/c18 53 1 2024-06-06T08:43:23.253 INFO:tasks.workunit.client.0.smithi027.stdout:7/371: dwrite d17/d32/f3a [4194304,4194304] 0 2024-06-06T08:43:23.257 INFO:tasks.workunit.client.0.smithi027.stdout:1/278: dwrite dc/f36 [0,4194304] 0 2024-06-06T08:43:23.257 INFO:tasks.workunit.client.0.smithi027.stdout:1/279: write dc/de/d14/f1a [599457,117738] 0 2024-06-06T08:43:23.257 INFO:tasks.workunit.client.0.smithi027.stdout:1/280: chown dc/de/d14/f63 21657 1 2024-06-06T08:43:23.257 INFO:tasks.workunit.client.0.smithi027.stdout:7/372: creat d17/d32/d46/d51/d52/f85 x:0 0 0 2024-06-06T08:43:23.259 INFO:tasks.workunit.client.0.smithi027.stdout:9/215: mkdir d3/d4e 0 2024-06-06T08:43:23.259 INFO:tasks.workunit.client.0.smithi027.stdout:9/216: fsync d3/da/f2c 0 2024-06-06T08:43:23.259 INFO:tasks.workunit.client.0.smithi027.stdout:9/217: write d3/da/dc/df/d34/f48 [193958,123164] 0 2024-06-06T08:43:23.259 INFO:tasks.workunit.client.0.smithi027.stdout:9/218: dread - d3/da/dc/df/f39 zero size 2024-06-06T08:43:23.262 INFO:tasks.workunit.client.0.smithi027.stdout:2/309: creat d6/dc/d3f/f71 x:0 0 0 2024-06-06T08:43:23.262 INFO:tasks.workunit.client.0.smithi027.stdout:9/219: creat d3/da/dc/df/f4f x:0 0 0 2024-06-06T08:43:23.262 INFO:tasks.workunit.client.0.smithi027.stdout:9/220: fsync d3/f9 0 2024-06-06T08:43:23.262 INFO:tasks.workunit.client.0.smithi027.stdout:8/193: mknod d7/d2f/d38/c39 0 2024-06-06T08:43:23.262 INFO:tasks.workunit.client.0.smithi027.stdout:9/221: truncate d3/da/dc/df/f28 1525287 0 2024-06-06T08:43:23.262 INFO:tasks.workunit.client.0.smithi027.stdout:9/222: write d3/f20 [843568,404] 0 2024-06-06T08:43:23.264 INFO:tasks.workunit.client.0.smithi027.stdout:2/310: dread d6/d1e/f27 [0,4194304] 0 2024-06-06T08:43:23.264 INFO:tasks.workunit.client.0.smithi027.stdout:2/311: readlink d6/d3b/l53 0 2024-06-06T08:43:23.264 INFO:tasks.workunit.client.0.smithi027.stdout:7/373: rmdir d17/d47 39 2024-06-06T08:43:23.265 INFO:tasks.workunit.client.0.smithi027.stdout:2/312: mknod d6/dc/d1d/c72 0 2024-06-06T08:43:23.265 INFO:tasks.workunit.client.0.smithi027.stdout:4/212: dwrite d0/f20 [4194304,4194304] 0 2024-06-06T08:43:23.266 INFO:tasks.workunit.client.0.smithi027.stdout:5/376: dwrite d0/da/d37/d2b/f65 [0,4194304] 0 2024-06-06T08:43:23.266 INFO:tasks.workunit.client.0.smithi027.stdout:5/377: write d0/da/db/d1a/f66 [105914,23049] 0 2024-06-06T08:43:23.268 INFO:tasks.workunit.client.0.smithi027.stdout:2/313: creat d6/d1e/f73 x:0 0 0 2024-06-06T08:43:23.268 INFO:tasks.workunit.client.0.smithi027.stdout:8/194: dread d7/db/f11 [0,4194304] 0 2024-06-06T08:43:23.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/223: creat d3/da/f50 x:0 0 0 2024-06-06T08:43:23.272 INFO:tasks.workunit.client.0.smithi027.stdout:4/213: symlink d0/d1d/d38/l44 0 2024-06-06T08:43:23.273 INFO:tasks.workunit.client.0.smithi027.stdout:4/214: fsync d0/fd 0 2024-06-06T08:43:23.273 INFO:tasks.workunit.client.0.smithi027.stdout:1/281: dread dc/de/d14/f49 [0,4194304] 0 2024-06-06T08:43:23.273 INFO:tasks.workunit.client.0.smithi027.stdout:1/282: dread - dc/de/d14/d21/d32/f5b zero size 2024-06-06T08:43:23.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/374: mknod d17/d32/d61/c86 0 2024-06-06T08:43:23.277 INFO:tasks.workunit.client.0.smithi027.stdout:3/183: dwrite d1/d8/f16 [0,4194304] 0 2024-06-06T08:43:23.277 INFO:tasks.workunit.client.0.smithi027.stdout:7/375: mkdir d17/d32/d46/d51/d52/d70/d87 0 2024-06-06T08:43:23.279 INFO:tasks.workunit.client.0.smithi027.stdout:1/283: dread dc/de/f13 [0,4194304] 0 2024-06-06T08:43:23.279 INFO:tasks.workunit.client.0.smithi027.stdout:7/376: write d17/d32/d46/d4f/f55 [510777,26355] 0 2024-06-06T08:43:23.280 INFO:tasks.workunit.client.0.smithi027.stdout:5/378: mknod d0/da/d47/c76 0 2024-06-06T08:43:23.282 INFO:tasks.workunit.client.0.smithi027.stdout:4/215: dread d0/f20 [0,4194304] 0 2024-06-06T08:43:23.282 INFO:tasks.workunit.client.0.smithi027.stdout:2/314: creat d6/dd/d3c/d55/f74 x:0 0 0 2024-06-06T08:43:23.283 INFO:tasks.workunit.client.0.smithi027.stdout:8/195: truncate d7/f9 327419 0 2024-06-06T08:43:23.285 INFO:tasks.workunit.client.0.smithi027.stdout:3/184: creat d1/da/f38 x:0 0 0 2024-06-06T08:43:23.287 INFO:tasks.workunit.client.0.smithi027.stdout:5/379: unlink d0/da/db/l42 0 2024-06-06T08:43:23.288 INFO:tasks.workunit.client.0.smithi027.stdout:3/185: write d1/f13 [4436704,63340] 0 2024-06-06T08:43:23.288 INFO:tasks.workunit.client.0.smithi027.stdout:3/186: dread - d1/da/f38 zero size 2024-06-06T08:43:23.288 INFO:tasks.workunit.client.0.smithi027.stdout:3/187: write d1/da/f38 [115247,11470] 0 2024-06-06T08:43:23.289 INFO:tasks.workunit.client.0.smithi027.stdout:1/284: dread dc/de/d14/d21/f27 [0,4194304] 0 2024-06-06T08:43:23.290 INFO:tasks.workunit.client.0.smithi027.stdout:2/315: truncate d6/d1e/d2b/f58 3443720 0 2024-06-06T08:43:23.290 INFO:tasks.workunit.client.0.smithi027.stdout:2/316: getdents d6/dd/d6b 0 2024-06-06T08:43:23.290 INFO:tasks.workunit.client.0.smithi027.stdout:2/317: fsync d6/d3b/f43 0 2024-06-06T08:43:23.290 INFO:tasks.workunit.client.0.smithi027.stdout:2/318: write d6/dd/d3c/d55/d69/f6c [91076,74278] 0 2024-06-06T08:43:23.293 INFO:tasks.workunit.client.0.smithi027.stdout:5/380: mkdir d0/da/db/d46/d77 0 2024-06-06T08:43:23.301 INFO:tasks.workunit.client.0.smithi027.stdout:6/318: rename d4/d1a/d46 to d4/de/d45/d6a 0 2024-06-06T08:43:23.302 INFO:tasks.workunit.client.0.smithi027.stdout:8/196: dwrite d7/f22 [0,4194304] 0 2024-06-06T08:43:23.304 INFO:tasks.workunit.client.0.smithi027.stdout:9/224: rename d3/da/dc/f21 to d3/da/dc/df/d34/f51 0 2024-06-06T08:43:23.304 INFO:tasks.workunit.client.0.smithi027.stdout:9/225: chown d3/f46 1494535679 1 2024-06-06T08:43:23.304 INFO:tasks.workunit.client.0.smithi027.stdout:9/226: readlink d3/l14 0 2024-06-06T08:43:23.306 INFO:tasks.workunit.client.0.smithi027.stdout:5/381: rmdir d0/da/d37/d2b 39 2024-06-06T08:43:23.307 INFO:tasks.workunit.client.0.smithi027.stdout:1/285: rename dc/de/l55 to dc/de/d3d/l66 0 2024-06-06T08:43:23.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/188: dread d1/d8/d22/d27/f14 [0,4194304] 0 2024-06-06T08:43:23.308 INFO:tasks.workunit.client.0.smithi027.stdout:8/197: write d7/db/f11 [843028,122418] 0 2024-06-06T08:43:23.308 INFO:tasks.workunit.client.0.smithi027.stdout:6/319: rmdir d4/d9/d49 39 2024-06-06T08:43:23.309 INFO:tasks.workunit.client.0.smithi027.stdout:5/382: creat d0/da/db/d1a/f78 x:0 0 0 2024-06-06T08:43:23.310 INFO:tasks.workunit.client.0.smithi027.stdout:1/286: mkdir dc/d67 0 2024-06-06T08:43:23.310 INFO:tasks.workunit.client.0.smithi027.stdout:1/287: fdatasync dc/de/d14/d21/f52 0 2024-06-06T08:43:23.311 INFO:tasks.workunit.client.0.smithi027.stdout:4/216: rmdir d0/d1/d6/d8 39 2024-06-06T08:43:23.313 INFO:tasks.workunit.client.0.smithi027.stdout:3/189: creat d1/d8/d22/d27/f39 x:0 0 0 2024-06-06T08:43:23.314 INFO:tasks.workunit.client.0.smithi027.stdout:9/227: dread d3/da/dc/df/d23/d36/f37 [0,4194304] 0 2024-06-06T08:43:23.314 INFO:tasks.workunit.client.0.smithi027.stdout:9/228: write d3/da/dc/df/f3f [601398,79902] 0 2024-06-06T08:43:23.314 INFO:tasks.workunit.client.0.smithi027.stdout:8/198: mknod d7/db/c3a 0 2024-06-06T08:43:23.315 INFO:tasks.workunit.client.0.smithi027.stdout:5/383: getdents d0/da/db/d1a/d5a 0 2024-06-06T08:43:23.315 INFO:tasks.workunit.client.0.smithi027.stdout:5/384: fsync d0/da/db/d46/d4c/f6e 0 2024-06-06T08:43:23.315 INFO:tasks.workunit.client.0.smithi027.stdout:2/319: dread d6/d1e/d2b/f58 [0,4194304] 0 2024-06-06T08:43:23.317 INFO:tasks.workunit.client.0.smithi027.stdout:1/288: truncate dc/fd 1208161 0 2024-06-06T08:43:23.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/217: rename d0/d1d/d38/d3b/l43 to d0/d1d/d38/d3b/l45 0 2024-06-06T08:43:23.320 INFO:tasks.workunit.client.0.smithi027.stdout:8/199: creat d7/d2f/f3b x:0 0 0 2024-06-06T08:43:23.321 INFO:tasks.workunit.client.0.smithi027.stdout:5/385: creat d0/da/db/d46/d77/f79 x:0 0 0 2024-06-06T08:43:23.321 INFO:tasks.workunit.client.0.smithi027.stdout:5/386: chown d0/da/db/d1a/l50 1705653 1 2024-06-06T08:43:23.323 INFO:tasks.workunit.client.0.smithi027.stdout:4/218: rmdir d0/d1d 39 2024-06-06T08:43:23.323 INFO:tasks.workunit.client.0.smithi027.stdout:4/219: truncate d0/d1/fb 9810588 0 2024-06-06T08:43:23.323 INFO:tasks.workunit.client.0.smithi027.stdout:1/289: write dc/de/d14/f1a [2547672,122612] 0 2024-06-06T08:43:23.324 INFO:tasks.workunit.client.0.smithi027.stdout:2/320: rmdir d6/d1e 39 2024-06-06T08:43:23.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/321: symlink d6/d1e/d2b/d5a/l75 0 2024-06-06T08:43:23.328 INFO:tasks.workunit.client.0.smithi027.stdout:3/190: write d1/f2 [5861641,6532] 0 2024-06-06T08:43:23.330 INFO:tasks.workunit.client.0.smithi027.stdout:7/377: link d17/d32/d46/d51/f71 d17/d32/d46/d4f/d5e/d7b/f88 0 2024-06-06T08:43:23.332 INFO:tasks.workunit.client.0.smithi027.stdout:6/320: dread d4/f29 [0,4194304] 0 2024-06-06T08:43:23.333 INFO:tasks.workunit.client.0.smithi027.stdout:6/321: truncate d4/d1a/d2e/d44/f5a 1010249 0 2024-06-06T08:43:23.333 INFO:tasks.workunit.client.0.smithi027.stdout:6/322: stat d4/d9/dc/l3f 0 2024-06-06T08:43:23.336 INFO:tasks.workunit.client.0.smithi027.stdout:8/200: dread d7/f2c [0,4194304] 0 2024-06-06T08:43:23.336 INFO:tasks.workunit.client.0.smithi027.stdout:8/201: fdatasync d7/f9 0 2024-06-06T08:43:23.336 INFO:tasks.workunit.client.0.smithi027.stdout:8/202: chown d7/d2f/f3b 8 1 2024-06-06T08:43:23.338 INFO:tasks.workunit.client.0.smithi027.stdout:8/203: mkdir d7/dc/d36/d3c 0 2024-06-06T08:43:23.338 INFO:tasks.workunit.client.0.smithi027.stdout:5/387: dread d0/da/f35 [0,4194304] 0 2024-06-06T08:43:23.338 INFO:tasks.workunit.client.0.smithi027.stdout:7/378: rmdir d17/d32/d46/d83/d2a 39 2024-06-06T08:43:23.339 INFO:tasks.workunit.client.0.smithi027.stdout:9/229: dread d3/da/dc/df/d23/d36/f37 [0,4194304] 0 2024-06-06T08:43:23.339 INFO:tasks.workunit.client.0.smithi027.stdout:9/230: dread - d3/da/dc/df/f2b zero size 2024-06-06T08:43:23.340 INFO:tasks.workunit.client.0.smithi027.stdout:5/388: rename d0/da/d47/c76 to d0/da/db/d46/d54/c7a 0 2024-06-06T08:43:23.340 INFO:tasks.workunit.client.0.smithi027.stdout:5/389: fsync d0/da/f43 0 2024-06-06T08:43:23.341 INFO:tasks.workunit.client.0.smithi027.stdout:5/390: truncate d0/da/d20/f4f 4389360 0 2024-06-06T08:43:23.341 INFO:tasks.workunit.client.0.smithi027.stdout:5/391: write d0/f24 [1611183,53427] 0 2024-06-06T08:43:23.341 INFO:tasks.workunit.client.0.smithi027.stdout:5/392: stat d0/fe 0 2024-06-06T08:43:23.341 INFO:tasks.workunit.client.0.smithi027.stdout:8/204: mknod d7/d13/d15/c3d 0 2024-06-06T08:43:23.342 INFO:tasks.workunit.client.0.smithi027.stdout:9/231: link d3/f46 d3/f52 0 2024-06-06T08:43:23.343 INFO:tasks.workunit.client.0.smithi027.stdout:7/379: mknod d17/d32/d61/c89 0 2024-06-06T08:43:23.343 INFO:tasks.workunit.client.0.smithi027.stdout:7/380: chown d17/d32/d61/f6a 51165825 1 2024-06-06T08:43:23.344 INFO:tasks.workunit.client.0.smithi027.stdout:8/205: rename d7/d13/d15/c3d to d7/d2f/c3e 0 2024-06-06T08:43:23.345 INFO:tasks.workunit.client.0.smithi027.stdout:9/232: mknod d3/d4e/c53 0 2024-06-06T08:43:23.345 INFO:tasks.workunit.client.0.smithi027.stdout:9/233: truncate d3/da/dc/df/f39 464861 0 2024-06-06T08:43:23.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/234: symlink d3/da/dc/df/d34/l54 0 2024-06-06T08:43:23.350 INFO:tasks.workunit.client.0.smithi027.stdout:7/381: read d17/d32/d46/d83/d2a/d3b/f56 [485880,6575] 0 2024-06-06T08:43:23.350 INFO:tasks.workunit.client.0.smithi027.stdout:8/206: unlink d7/dc/f20 0 2024-06-06T08:43:23.351 INFO:tasks.workunit.client.0.smithi027.stdout:4/220: dwrite d0/d1/f1a [0,4194304] 0 2024-06-06T08:43:23.353 INFO:tasks.workunit.client.0.smithi027.stdout:4/221: creat d0/d1d/d38/d3b/f46 x:0 0 0 2024-06-06T08:43:23.355 INFO:tasks.workunit.client.0.smithi027.stdout:4/222: mkdir d0/d1/d6/d8/d21/d47 0 2024-06-06T08:43:23.358 INFO:tasks.workunit.client.0.smithi027.stdout:2/322: dwrite d6/d1e/d2b/f57 [0,4194304] 0 2024-06-06T08:43:23.358 INFO:tasks.workunit.client.0.smithi027.stdout:2/323: chown d6/dc/d1d/d4d/c4e 15 1 2024-06-06T08:43:23.358 INFO:tasks.workunit.client.0.smithi027.stdout:2/324: write d6/dc/d1d/f45 [5232433,80157] 0 2024-06-06T08:43:23.358 INFO:tasks.workunit.client.0.smithi027.stdout:2/325: write d6/f2e [590354,70203] 0 2024-06-06T08:43:23.361 INFO:tasks.workunit.client.0.smithi027.stdout:8/207: dread d7/d13/f24 [0,4194304] 0 2024-06-06T08:43:23.366 INFO:tasks.workunit.client.0.smithi027.stdout:8/208: mknod d7/dc/d36/d3c/c3f 0 2024-06-06T08:43:23.366 INFO:tasks.workunit.client.0.smithi027.stdout:4/223: rename d0/d1/d6/d8/d21/c22 to d0/d1/d6/d8/c48 0 2024-06-06T08:43:23.367 INFO:tasks.workunit.client.0.smithi027.stdout:6/323: read d4/d1a/d2e/f5c [2458804,66810] 0 2024-06-06T08:43:23.370 INFO:tasks.workunit.client.0.smithi027.stdout:7/382: dwrite d17/f1e [0,4194304] 0 2024-06-06T08:43:23.370 INFO:tasks.workunit.client.0.smithi027.stdout:9/235: dwrite d3/da/dc/df/f2f [0,4194304] 0 2024-06-06T08:43:23.371 INFO:tasks.workunit.client.0.smithi027.stdout:5/393: dread d0/f29 [0,4194304] 0 2024-06-06T08:43:23.372 INFO:tasks.workunit.client.0.smithi027.stdout:1/290: dwrite dc/de/d22/f35 [0,4194304] 0 2024-06-06T08:43:23.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/236: creat d3/da/f55 x:0 0 0 2024-06-06T08:43:23.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/326: write d6/d1e/f27 [877350,58306] 0 2024-06-06T08:43:23.382 INFO:tasks.workunit.client.0.smithi027.stdout:9/237: dread d3/da/dc/df/f28 [0,4194304] 0 2024-06-06T08:43:23.382 INFO:tasks.workunit.client.0.smithi027.stdout:9/238: chown d3/f11 0 1 2024-06-06T08:43:23.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/327: symlink d6/dd/d3c/d55/d69/l76 0 2024-06-06T08:43:23.388 INFO:tasks.workunit.client.0.smithi027.stdout:1/291: dread dc/de/d3d/d4a/f58 [0,4194304] 0 2024-06-06T08:43:23.388 INFO:tasks.workunit.client.0.smithi027.stdout:9/239: symlink d3/l56 0 2024-06-06T08:43:23.388 INFO:tasks.workunit.client.0.smithi027.stdout:9/240: dread - d3/da/dc/df/f4f zero size 2024-06-06T08:43:23.390 INFO:tasks.workunit.client.0.smithi027.stdout:1/292: creat dc/de/d14/d21/d32/d37/d56/f68 x:0 0 0 2024-06-06T08:43:23.390 INFO:tasks.workunit.client.0.smithi027.stdout:7/383: dwrite d17/f26 [0,4194304] 0 2024-06-06T08:43:23.391 INFO:tasks.workunit.client.0.smithi027.stdout:7/384: write d17/d32/d46/d51/d52/f73 [5165748,92994] 0 2024-06-06T08:43:23.392 INFO:tasks.workunit.client.0.smithi027.stdout:7/385: symlink d17/d32/d46/d51/d52/d76/l8a 0 2024-06-06T08:43:23.393 INFO:tasks.workunit.client.0.smithi027.stdout:4/224: dwrite d0/d1/d6/f33 [0,4194304] 0 2024-06-06T08:43:23.394 INFO:tasks.workunit.client.0.smithi027.stdout:7/386: link d17/d32/d46/d4f/d5e/f66 d17/d32/d46/d4f/d5e/f8b 0 2024-06-06T08:43:23.395 INFO:tasks.workunit.client.0.smithi027.stdout:7/387: mkdir d17/d32/d46/d83/d77/d8c 0 2024-06-06T08:43:23.396 INFO:tasks.workunit.client.0.smithi027.stdout:1/293: dread dc/fd [0,4194304] 0 2024-06-06T08:43:23.397 INFO:tasks.workunit.client.0.smithi027.stdout:9/241: dread d3/da/dc/df/d34/f48 [0,4194304] 0 2024-06-06T08:43:23.404 INFO:tasks.workunit.client.0.smithi027.stdout:2/328: dwrite d6/dc/d3f/f71 [0,4194304] 0 2024-06-06T08:43:23.405 INFO:tasks.workunit.client.0.smithi027.stdout:6/324: rename d4/lf to d4/d9/dc/l6b 0 2024-06-06T08:43:23.409 INFO:tasks.workunit.client.0.smithi027.stdout:5/394: rename d0/da/d20/f6d to d0/da/db/d1a/f7b 0 2024-06-06T08:43:23.409 INFO:tasks.workunit.client.0.smithi027.stdout:5/395: read - d0/da/db/d1a/f78 zero size 2024-06-06T08:43:23.409 INFO:tasks.workunit.client.0.smithi027.stdout:5/396: write d0/fe [3043159,97921] 0 2024-06-06T08:43:23.410 INFO:tasks.workunit.client.0.smithi027.stdout:6/325: truncate d4/d13/f21 542664 0 2024-06-06T08:43:23.412 INFO:tasks.workunit.client.0.smithi027.stdout:6/326: creat d4/de/f6c x:0 0 0 2024-06-06T08:43:23.412 INFO:tasks.workunit.client.0.smithi027.stdout:6/327: stat d4/de/f6c 0 2024-06-06T08:43:23.412 INFO:tasks.workunit.client.0.smithi027.stdout:2/329: write f5 [2773520,20331] 0 2024-06-06T08:43:23.413 INFO:tasks.workunit.client.0.smithi027.stdout:6/328: mknod d4/c6d 0 2024-06-06T08:43:23.414 INFO:tasks.workunit.client.0.smithi027.stdout:7/388: dwrite d17/f23 [0,4194304] 0 2024-06-06T08:43:23.414 INFO:tasks.workunit.client.0.smithi027.stdout:7/389: truncate d17/f27 237282 0 2024-06-06T08:43:23.415 INFO:tasks.workunit.client.0.smithi027.stdout:9/242: dwrite d3/f46 [0,4194304] 0 2024-06-06T08:43:23.415 INFO:tasks.workunit.client.0.smithi027.stdout:9/243: write d3/da/dc/df/d34/f48 [853197,55986] 0 2024-06-06T08:43:23.417 INFO:tasks.workunit.client.0.smithi027.stdout:7/390: symlink d17/l8d 0 2024-06-06T08:43:23.421 INFO:tasks.workunit.client.0.smithi027.stdout:7/391: unlink d17/d32/d61/f80 0 2024-06-06T08:43:23.421 INFO:tasks.workunit.client.0.smithi027.stdout:7/392: dread - d17/f60 zero size 2024-06-06T08:43:23.427 INFO:tasks.workunit.client.0.smithi027.stdout:9/244: dread d3/f4a [0,4194304] 0 2024-06-06T08:43:23.428 INFO:tasks.workunit.client.0.smithi027.stdout:6/329: dread d4/f5 [0,4194304] 0 2024-06-06T08:43:23.429 INFO:tasks.workunit.client.0.smithi027.stdout:9/245: unlink d3/f4a 0 2024-06-06T08:43:23.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/225: rename d0/d1d/d38/f42 to d0/d1/d6/d8/f49 0 2024-06-06T08:43:23.433 INFO:tasks.workunit.client.0.smithi027.stdout:5/397: dwrite d0/da/d20/f36 [0,4194304] 0 2024-06-06T08:43:23.440 INFO:tasks.workunit.client.0.smithi027.stdout:9/246: read d3/f46 [1607284,78146] 0 2024-06-06T08:43:23.440 INFO:tasks.workunit.client.0.smithi027.stdout:7/393: rename d17/d32/d46/d51/f5f to d17/d32/d46/d51/f8e 0 2024-06-06T08:43:23.440 INFO:tasks.workunit.client.0.smithi027.stdout:7/394: chown d17/d32/d61/f6a 1846 1 2024-06-06T08:43:23.440 INFO:tasks.workunit.client.0.smithi027.stdout:1/294: rename dc/de/d14/d21 to dc/d67/d69 0 2024-06-06T08:43:23.440 INFO:tasks.workunit.client.0.smithi027.stdout:6/330: dread d4/d9/dc/d18/f2f [0,4194304] 0 2024-06-06T08:43:23.441 INFO:tasks.workunit.client.0.smithi027.stdout:9/247: creat d3/da/dc/df/f57 x:0 0 0 2024-06-06T08:43:23.441 INFO:tasks.workunit.client.0.smithi027.stdout:9/248: fdatasync d3/da/dc/f30 0 2024-06-06T08:43:23.441 INFO:tasks.workunit.client.0.smithi027.stdout:1/295: symlink dc/d67/d69/d32/d37/d56/l6a 0 2024-06-06T08:43:23.442 INFO:tasks.workunit.client.0.smithi027.stdout:1/296: getdents dc/d67/d69/d32 0 2024-06-06T08:43:23.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/297: fsync dc/de/d3d/f42 0 2024-06-06T08:43:23.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/298: stat dc/l2d 0 2024-06-06T08:43:23.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/299: write dc/de/d14/d4e/f53 [1024026,69010] 0 2024-06-06T08:43:23.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/300: stat dc/d67/d69/d32/d37 0 2024-06-06T08:43:23.445 INFO:tasks.workunit.client.0.smithi027.stdout:6/331: getdents d4/de/d45/d6a 0 2024-06-06T08:43:23.446 INFO:tasks.workunit.client.0.smithi027.stdout:1/301: symlink dc/d67/d69/d32/d37/l6b 0 2024-06-06T08:43:23.449 INFO:tasks.workunit.client.0.smithi027.stdout:2/330: rename d6/dd/d3c to d6/dc/d3f/d77 0 2024-06-06T08:43:23.452 INFO:tasks.workunit.client.0.smithi027.stdout:5/398: dread d0/da/d37/d2b/f67 [0,4194304] 0 2024-06-06T08:43:23.460 INFO:tasks.workunit.client.0.smithi027.stdout:4/226: dwrite d0/d1/f14 [0,4194304] 0 2024-06-06T08:43:23.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/332: dread d4/d9/f26 [0,4194304] 0 2024-06-06T08:43:23.466 INFO:tasks.workunit.client.0.smithi027.stdout:6/333: unlink d4/d9/dc/l27 0 2024-06-06T08:43:23.468 INFO:tasks.workunit.client.0.smithi027.stdout:9/249: dwrite d3/f52 [0,4194304] 0 2024-06-06T08:43:23.468 INFO:tasks.workunit.client.0.smithi027.stdout:9/250: chown d3/da/dc/df/f2b 7208 1 2024-06-06T08:43:23.473 INFO:tasks.workunit.client.0.smithi027.stdout:9/251: rename d3/da/dc/df/f3f to d3/da/dc/df/d34/d41/f58 0 2024-06-06T08:43:23.473 INFO:tasks.workunit.client.0.smithi027.stdout:7/395: dwrite d17/d32/f38 [0,4194304] 0 2024-06-06T08:43:23.473 INFO:tasks.workunit.client.0.smithi027.stdout:1/302: dwrite dc/de/d14/d4e/f62 [0,4194304] 0 2024-06-06T08:43:23.473 INFO:tasks.workunit.client.0.smithi027.stdout:1/303: chown f5 1877 1 2024-06-06T08:43:23.477 INFO:tasks.workunit.client.0.smithi027.stdout:9/252: symlink d3/da/dc/df/d34/d41/l59 0 2024-06-06T08:43:23.477 INFO:tasks.workunit.client.0.smithi027.stdout:9/253: chown d3/da/dc/df/f39 2789 1 2024-06-06T08:43:23.477 INFO:tasks.workunit.client.0.smithi027.stdout:7/396: creat d17/d32/d46/d4f/f8f x:0 0 0 2024-06-06T08:43:23.477 INFO:tasks.workunit.client.0.smithi027.stdout:7/397: dread - d17/f60 zero size 2024-06-06T08:43:23.477 INFO:tasks.workunit.client.0.smithi027.stdout:7/398: fdatasync d17/d32/d46/d51/f35 0 2024-06-06T08:43:23.477 INFO:tasks.workunit.client.0.smithi027.stdout:1/304: mknod dc/c6c 0 2024-06-06T08:43:23.479 INFO:tasks.workunit.client.0.smithi027.stdout:7/399: getdents d17/d32/d46/d83/d77 0 2024-06-06T08:43:23.479 INFO:tasks.workunit.client.0.smithi027.stdout:7/400: write d17/d32/d46/f41 [161863,51873] 0 2024-06-06T08:43:23.479 INFO:tasks.workunit.client.0.smithi027.stdout:1/305: truncate dc/de/d3d/f42 935908 0 2024-06-06T08:43:23.479 INFO:tasks.workunit.client.0.smithi027.stdout:1/306: dread - dc/de/d3d/f3e zero size 2024-06-06T08:43:23.479 INFO:tasks.workunit.client.0.smithi027.stdout:5/399: dwrite d0/da/d37/d2b/f5b [0,4194304] 0 2024-06-06T08:43:23.479 INFO:tasks.workunit.client.0.smithi027.stdout:4/227: dread d0/d1/d6/f33 [0,4194304] 0 2024-06-06T08:43:23.481 INFO:tasks.workunit.client.0.smithi027.stdout:4/228: creat d0/d1/d6/d8/d21/f4a x:0 0 0 2024-06-06T08:43:23.481 INFO:tasks.workunit.client.0.smithi027.stdout:1/307: mkdir dc/d6d 0 2024-06-06T08:43:23.481 INFO:tasks.workunit.client.0.smithi027.stdout:1/308: dread - dc/d67/d69/d32/f5b zero size 2024-06-06T08:43:23.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/400: creat d0/da/db/d1a/f7c x:0 0 0 2024-06-06T08:43:23.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/401: truncate d0/da/f13 5108449 0 2024-06-06T08:43:23.488 INFO:tasks.workunit.client.0.smithi027.stdout:2/331: dwrite f5 [0,4194304] 0 2024-06-06T08:43:23.492 INFO:tasks.workunit.client.0.smithi027.stdout:7/401: dread f11 [0,4194304] 0 2024-06-06T08:43:23.493 INFO:tasks.workunit.client.0.smithi027.stdout:9/254: dread d3/f46 [0,4194304] 0 2024-06-06T08:43:23.494 INFO:tasks.workunit.client.0.smithi027.stdout:7/402: mkdir d17/d32/d46/d83/d77/d8c/d90 0 2024-06-06T08:43:23.494 INFO:tasks.workunit.client.0.smithi027.stdout:9/255: creat d3/da/dc/df/d23/d36/f5a x:0 0 0 2024-06-06T08:43:23.497 INFO:tasks.workunit.client.0.smithi027.stdout:4/229: dread d0/d1/f14 [0,4194304] 0 2024-06-06T08:43:23.499 INFO:tasks.workunit.client.0.smithi027.stdout:6/334: dwrite d4/f57 [0,4194304] 0 2024-06-06T08:43:23.500 INFO:tasks.workunit.client.0.smithi027.stdout:4/230: mkdir d0/d1/d6/d4b 0 2024-06-06T08:43:23.502 INFO:tasks.workunit.client.0.smithi027.stdout:2/332: symlink d6/dc/d3f/d77/l78 0 2024-06-06T08:43:23.506 INFO:tasks.workunit.client.0.smithi027.stdout:4/231: mknod d0/d1/c4c 0 2024-06-06T08:43:23.508 INFO:tasks.workunit.client.0.smithi027.stdout:4/232: rmdir d0/d1d/d38/d3b 39 2024-06-06T08:43:23.508 INFO:tasks.workunit.client.0.smithi027.stdout:4/233: dread - d0/d1/d6/d8/d21/f4a zero size 2024-06-06T08:43:23.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/402: write d0/da/d37/d2b/f5b [2896740,120629] 0 2024-06-06T08:43:23.509 INFO:tasks.workunit.client.0.smithi027.stdout:1/309: dwrite dc/de/f13 [0,4194304] 0 2024-06-06T08:43:23.509 INFO:tasks.workunit.client.0.smithi027.stdout:1/310: chown ca 11 1 2024-06-06T08:43:23.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/403: fdatasync d0/da/db/d1a/f71 0 2024-06-06T08:43:23.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/404: chown d0/da/db/d1a/c45 7 1 2024-06-06T08:43:23.511 INFO:tasks.workunit.client.0.smithi027.stdout:4/234: creat d0/d1d/d34/f4d x:0 0 0 2024-06-06T08:43:23.511 INFO:tasks.workunit.client.0.smithi027.stdout:4/235: chown d0/d1d/d38/d3b/c2e 60949098 1 2024-06-06T08:43:23.513 INFO:tasks.workunit.client.0.smithi027.stdout:7/403: read d17/d32/f3a [3266488,117456] 0 2024-06-06T08:43:23.516 INFO:tasks.workunit.client.0.smithi027.stdout:7/404: mkdir d17/d32/d46/d83/d77/d8c/d91 0 2024-06-06T08:43:23.516 INFO:tasks.workunit.client.0.smithi027.stdout:1/311: symlink dc/de/l6e 0 2024-06-06T08:43:23.516 INFO:tasks.workunit.client.0.smithi027.stdout:1/312: dread - dc/de/d3d/f3f zero size 2024-06-06T08:43:23.517 INFO:tasks.workunit.client.0.smithi027.stdout:7/405: mkdir d17/d32/d46/d4f/d5e/d7b/d7c/d92 0 2024-06-06T08:43:23.519 INFO:tasks.workunit.client.0.smithi027.stdout:1/313: unlink dc/de/d22/l26 0 2024-06-06T08:43:23.521 INFO:tasks.workunit.client.0.smithi027.stdout:9/256: dwrite d3/da/dc/df/d34/f51 [0,4194304] 0 2024-06-06T08:43:23.532 INFO:tasks.workunit.client.0.smithi027.stdout:4/236: write d0/f20 [5240987,60794] 0 2024-06-06T08:43:23.535 INFO:tasks.workunit.client.0.smithi027.stdout:4/237: symlink d0/d1/d6/d8/l4e 0 2024-06-06T08:43:23.536 INFO:tasks.workunit.client.0.smithi027.stdout:2/333: dwrite d6/f4f [0,4194304] 0 2024-06-06T08:43:23.536 INFO:tasks.workunit.client.0.smithi027.stdout:2/334: chown d6/dc/d1d 19513 1 2024-06-06T08:43:23.536 INFO:tasks.workunit.client.0.smithi027.stdout:2/335: read - d6/f6e zero size 2024-06-06T08:43:23.536 INFO:tasks.workunit.client.0.smithi027.stdout:6/335: dwrite d4/d9/dc/d18/f2f [4194304,4194304] 0 2024-06-06T08:43:23.537 INFO:tasks.workunit.client.0.smithi027.stdout:4/238: unlink d0/d1d/d38/l3a 0 2024-06-06T08:43:23.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/336: creat d6/dc/d1d/d59/f79 x:0 0 0 2024-06-06T08:43:23.542 INFO:tasks.workunit.client.0.smithi027.stdout:6/336: mkdir d4/d9/d6e 0 2024-06-06T08:43:23.542 INFO:tasks.workunit.client.0.smithi027.stdout:6/337: read d4/d9/dc/d1c/f30 [1903,95536] 0 2024-06-06T08:43:23.542 INFO:tasks.workunit.client.0.smithi027.stdout:6/338: fsync d4/de/f5d 0 2024-06-06T08:43:23.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/337: getdents d6/d1e/d2b 0 2024-06-06T08:43:23.543 INFO:tasks.workunit.client.0.smithi027.stdout:2/338: chown d6/dc/c34 120403 1 2024-06-06T08:43:23.546 INFO:tasks.workunit.client.0.smithi027.stdout:5/405: dwrite d0/da/d37/d2b/f4a [0,4194304] 0 2024-06-06T08:43:23.546 INFO:tasks.workunit.client.0.smithi027.stdout:5/406: fsync d0/da/db/d1a/f7c 0 2024-06-06T08:43:23.546 INFO:tasks.workunit.client.0.smithi027.stdout:1/314: write dc/de/d3d/d4a/f58 [2534201,93860] 0 2024-06-06T08:43:23.546 INFO:tasks.workunit.client.0.smithi027.stdout:5/407: write d0/da/db/d46/d4c/f6e [4374362,81227] 0 2024-06-06T08:43:23.546 INFO:tasks.workunit.client.0.smithi027.stdout:5/408: fdatasync d0/da/db/d1a/f71 0 2024-06-06T08:43:23.547 INFO:tasks.workunit.client.0.smithi027.stdout:5/409: chown d0/c4b 40287 1 2024-06-06T08:43:23.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/315: write dc/de/d14/f63 [789066,90405] 0 2024-06-06T08:43:23.547 INFO:tasks.workunit.client.0.smithi027.stdout:9/257: dread d3/da/dc/f30 [4194304,4194304] 0 2024-06-06T08:43:23.547 INFO:tasks.workunit.client.0.smithi027.stdout:7/406: dwrite d17/d32/d46/d51/f71 [0,4194304] 0 2024-06-06T08:43:23.552 INFO:tasks.workunit.client.0.smithi027.stdout:5/410: symlink d0/da/db/d46/d4c/d70/l7d 0 2024-06-06T08:43:23.553 INFO:tasks.workunit.client.0.smithi027.stdout:2/339: dread d6/f14 [0,4194304] 0 2024-06-06T08:43:23.556 INFO:tasks.workunit.client.0.smithi027.stdout:9/258: mknod d3/c5b 0 2024-06-06T08:43:23.556 INFO:tasks.workunit.client.0.smithi027.stdout:9/259: dread - d3/da/dc/df/f2b zero size 2024-06-06T08:43:23.556 INFO:tasks.workunit.client.0.smithi027.stdout:9/260: write d3/da/dc/df/f39 [1162563,25821] 0 2024-06-06T08:43:23.557 INFO:tasks.workunit.client.0.smithi027.stdout:9/261: mknod d3/da/dc/df/d23/c5c 0 2024-06-06T08:43:23.559 INFO:tasks.workunit.client.0.smithi027.stdout:0/276: sync 2024-06-06T08:43:23.559 INFO:tasks.workunit.client.0.smithi027.stdout:3/191: sync 2024-06-06T08:43:23.559 INFO:tasks.workunit.client.0.smithi027.stdout:8/209: sync 2024-06-06T08:43:23.559 INFO:tasks.workunit.client.0.smithi027.stdout:3/192: chown d1/d1f/l2b 0 1 2024-06-06T08:43:23.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/262: creat d3/da/dc/df/d23/d36/f5d x:0 0 0 2024-06-06T08:43:23.559 INFO:tasks.workunit.client.0.smithi027.stdout:5/411: mkdir d0/da/db/d1a/d7e 0 2024-06-06T08:43:23.560 INFO:tasks.workunit.client.0.smithi027.stdout:0/277: rename d15/d18/d23/d28/d2a/d2d/d44 to d15/d18/d23/d28/d2a/d30/d5d 0 2024-06-06T08:43:23.560 INFO:tasks.workunit.client.0.smithi027.stdout:8/210: truncate d7/db/f27 2115705 0 2024-06-06T08:43:23.561 INFO:tasks.workunit.client.0.smithi027.stdout:9/263: creat d3/da/dc/df/f5e x:0 0 0 2024-06-06T08:43:23.562 INFO:tasks.workunit.client.0.smithi027.stdout:5/412: unlink d0/da/db/f28 0 2024-06-06T08:43:23.564 INFO:tasks.workunit.client.0.smithi027.stdout:8/211: symlink d7/d13/l40 0 2024-06-06T08:43:23.564 INFO:tasks.workunit.client.0.smithi027.stdout:8/212: truncate d7/d2f/f3b 694781 0 2024-06-06T08:43:23.564 INFO:tasks.workunit.client.0.smithi027.stdout:8/213: chown d7/d2f/d38/l2d 14896 1 2024-06-06T08:43:23.564 INFO:tasks.workunit.client.0.smithi027.stdout:8/214: write d7/db/f19 [4499072,115047] 0 2024-06-06T08:43:23.564 INFO:tasks.workunit.client.0.smithi027.stdout:8/215: readlink d7/d13/l40 0 2024-06-06T08:43:23.564 INFO:tasks.workunit.client.0.smithi027.stdout:6/339: write d4/d9/dc/d1c/f50 [881570,115585] 0 2024-06-06T08:43:23.564 INFO:tasks.workunit.client.0.smithi027.stdout:6/340: write d4/d9/f66 [1138904,1821] 0 2024-06-06T08:43:23.565 INFO:tasks.workunit.client.0.smithi027.stdout:4/239: dwrite d0/f20 [4194304,4194304] 0 2024-06-06T08:43:23.565 INFO:tasks.workunit.client.0.smithi027.stdout:7/407: read d17/f58 [665741,43387] 0 2024-06-06T08:43:23.566 INFO:tasks.workunit.client.0.smithi027.stdout:7/408: rename d17/d32/d46/d4f/d5e to d17/d32/d46/d4f/d5e/d7b/d84/d93 22 2024-06-06T08:43:23.568 INFO:tasks.workunit.client.0.smithi027.stdout:4/240: creat d0/d1/f4f x:0 0 0 2024-06-06T08:43:23.568 INFO:tasks.workunit.client.0.smithi027.stdout:1/316: read dc/de/d14/f49 [2306059,52411] 0 2024-06-06T08:43:23.569 INFO:tasks.workunit.client.0.smithi027.stdout:4/241: symlink d0/d1d/d34/l50 0 2024-06-06T08:43:23.570 INFO:tasks.workunit.client.0.smithi027.stdout:1/317: mknod dc/de/d14/d4e/c6f 0 2024-06-06T08:43:23.571 INFO:tasks.workunit.client.0.smithi027.stdout:4/242: rename d0/d1d/d38/l44 to d0/d1d/d38/l51 0 2024-06-06T08:43:23.572 INFO:tasks.workunit.client.0.smithi027.stdout:6/341: write d4/d9/f58 [928956,106601] 0 2024-06-06T08:43:23.572 INFO:tasks.workunit.client.0.smithi027.stdout:6/342: chown d4/d1a/d2e/f65 85149 1 2024-06-06T08:43:23.572 INFO:tasks.workunit.client.0.smithi027.stdout:0/278: write d15/f21 [1652578,3403] 0 2024-06-06T08:43:23.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/318: rename dc/c6c to dc/d67/d69/c70 0 2024-06-06T08:43:23.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/319: fdatasync dc/de/d14/f1a 0 2024-06-06T08:43:23.573 INFO:tasks.workunit.client.0.smithi027.stdout:0/279: creat d15/d18/d23/d28/d31/f5e x:0 0 0 2024-06-06T08:43:23.573 INFO:tasks.workunit.client.0.smithi027.stdout:8/216: read d7/d13/f17 [3576933,20112] 0 2024-06-06T08:43:23.574 INFO:tasks.workunit.client.0.smithi027.stdout:6/343: symlink d4/d13/l6f 0 2024-06-06T08:43:23.574 INFO:tasks.workunit.client.0.smithi027.stdout:8/217: write d7/d2f/d38/f34 [2236005,111003] 0 2024-06-06T08:43:23.574 INFO:tasks.workunit.client.0.smithi027.stdout:1/320: truncate dc/f30 685586 0 2024-06-06T08:43:23.575 INFO:tasks.workunit.client.0.smithi027.stdout:1/321: write dc/d67/d69/d32/d37/f41 [407567,4105] 0 2024-06-06T08:43:23.578 INFO:tasks.workunit.client.0.smithi027.stdout:0/280: symlink d15/d18/d23/l5f 0 2024-06-06T08:43:23.579 INFO:tasks.workunit.client.0.smithi027.stdout:4/243: rename d0/d1/d6/d8/c15 to d0/d1/d6/d8/c52 0 2024-06-06T08:43:23.584 INFO:tasks.workunit.client.0.smithi027.stdout:4/244: mknod d0/d1/d6/d8/c53 0 2024-06-06T08:43:23.584 INFO:tasks.workunit.client.0.smithi027.stdout:2/340: dwrite f5 [4194304,4194304] 0 2024-06-06T08:43:23.589 INFO:tasks.workunit.client.0.smithi027.stdout:4/245: unlink d0/d1/c28 0 2024-06-06T08:43:23.591 INFO:tasks.workunit.client.0.smithi027.stdout:6/344: dread d4/d9/f2d [0,4194304] 0 2024-06-06T08:43:23.591 INFO:tasks.workunit.client.0.smithi027.stdout:6/345: dread - d4/f69 zero size 2024-06-06T08:43:23.591 INFO:tasks.workunit.client.0.smithi027.stdout:9/264: dwrite d3/da/dc/df/d34/d41/f58 [0,4194304] 0 2024-06-06T08:43:23.593 INFO:tasks.workunit.client.0.smithi027.stdout:3/193: dwrite d1/d8/d22/d27/f36 [0,4194304] 0 2024-06-06T08:43:23.595 INFO:tasks.workunit.client.0.smithi027.stdout:4/246: creat d0/d1d/f54 x:0 0 0 2024-06-06T08:43:23.603 INFO:tasks.workunit.client.0.smithi027.stdout:9/265: link d3/f46 d3/f5f 0 2024-06-06T08:43:23.604 INFO:tasks.workunit.client.0.smithi027.stdout:6/346: creat d4/d1a/d2e/f70 x:0 0 0 2024-06-06T08:43:23.604 INFO:tasks.workunit.client.0.smithi027.stdout:6/347: write d4/d9/dc/d1c/f50 [2223930,54103] 0 2024-06-06T08:43:23.604 INFO:tasks.workunit.client.0.smithi027.stdout:2/341: dwrite d6/d1e/d2b/f57 [0,4194304] 0 2024-06-06T08:43:23.605 INFO:tasks.workunit.client.0.smithi027.stdout:3/194: creat d1/d1f/d25/f3a x:0 0 0 2024-06-06T08:43:23.606 INFO:tasks.workunit.client.0.smithi027.stdout:3/195: fsync d1/da/f18 0 2024-06-06T08:43:23.606 INFO:tasks.workunit.client.0.smithi027.stdout:3/196: dread - d1/d8/d22/d27/f39 zero size 2024-06-06T08:43:23.606 INFO:tasks.workunit.client.0.smithi027.stdout:3/197: write d1/da/f38 [535032,130409] 0 2024-06-06T08:43:23.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/348: dread d4/de/d45/f48 [0,4194304] 0 2024-06-06T08:43:23.609 INFO:tasks.workunit.client.0.smithi027.stdout:7/409: dread d17/d32/f38 [0,4194304] 0 2024-06-06T08:43:23.609 INFO:tasks.workunit.client.0.smithi027.stdout:0/281: dwrite d15/d18/d23/f2e [0,4194304] 0 2024-06-06T08:43:23.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/342: dread d6/dc/f54 [0,4194304] 0 2024-06-06T08:43:23.610 INFO:tasks.workunit.client.0.smithi027.stdout:9/266: fsync d3/f52 0 2024-06-06T08:43:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:9/267: chown d3/l4c 28160173 1 2024-06-06T08:43:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:9/268: chown d3/da/dc/df/l31 1 1 2024-06-06T08:43:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:9/269: dread - d3/da/dc/df/f5e zero size 2024-06-06T08:43:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:0/282: dread d15/d18/d23/d28/d2a/d30/d5d/f4a [0,4194304] 0 2024-06-06T08:43:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:0/283: truncate d15/f4f 581426 0 2024-06-06T08:43:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:5/413: dwrite d0/da/db/d46/d4c/f6e [0,4194304] 0 2024-06-06T08:43:23.611 INFO:tasks.workunit.client.0.smithi027.stdout:7/410: mknod d17/d32/d46/d51/d52/d70/c94 0 2024-06-06T08:43:23.612 INFO:tasks.workunit.client.0.smithi027.stdout:3/198: symlink d1/d8/d22/d27/l3b 0 2024-06-06T08:43:23.613 INFO:tasks.workunit.client.0.smithi027.stdout:5/414: mkdir d0/da/db/d46/d7f 0 2024-06-06T08:43:23.613 INFO:tasks.workunit.client.0.smithi027.stdout:6/349: creat d4/d9/d49/f71 x:0 0 0 2024-06-06T08:43:23.614 INFO:tasks.workunit.client.0.smithi027.stdout:7/411: symlink d17/d32/d46/l95 0 2024-06-06T08:43:23.615 INFO:tasks.workunit.client.0.smithi027.stdout:9/270: creat d3/da/dc/df/d34/f60 x:0 0 0 2024-06-06T08:43:23.616 INFO:tasks.workunit.client.0.smithi027.stdout:0/284: link d15/d18/l1b d15/d18/d23/d28/d31/l60 0 2024-06-06T08:43:23.617 INFO:tasks.workunit.client.0.smithi027.stdout:6/350: symlink d4/de/d45/l72 0 2024-06-06T08:43:23.618 INFO:tasks.workunit.client.0.smithi027.stdout:7/412: mknod d17/d32/d61/d75/c96 0 2024-06-06T08:43:23.618 INFO:tasks.workunit.client.0.smithi027.stdout:7/413: write d17/d32/d46/d51/d52/f85 [944270,19531] 0 2024-06-06T08:43:23.618 INFO:tasks.workunit.client.0.smithi027.stdout:3/199: mkdir d1/d8/d3c 0 2024-06-06T08:43:23.620 INFO:tasks.workunit.client.0.smithi027.stdout:2/343: rename d6/dc/f64 to d6/dc/d3f/f7a 0 2024-06-06T08:43:23.622 INFO:tasks.workunit.client.0.smithi027.stdout:6/351: symlink d4/d9/dc/d1c/d52/l73 0 2024-06-06T08:43:23.623 INFO:tasks.workunit.client.0.smithi027.stdout:0/285: mknod d15/d18/c61 0 2024-06-06T08:43:23.623 INFO:tasks.workunit.client.0.smithi027.stdout:0/286: readlink d15/d18/d23/d28/d31/l60 0 2024-06-06T08:43:23.623 INFO:tasks.workunit.client.0.smithi027.stdout:7/414: unlink d17/d32/c4a 0 2024-06-06T08:43:23.623 INFO:tasks.workunit.client.0.smithi027.stdout:7/415: stat d17/d32/d46/d83/f45 0 2024-06-06T08:43:23.624 INFO:tasks.workunit.client.0.smithi027.stdout:5/415: dread d0/da/d37/d2b/f3d [0,4194304] 0 2024-06-06T08:43:23.625 INFO:tasks.workunit.client.0.smithi027.stdout:3/200: creat d1/d8/d22/d2d/f3d x:0 0 0 2024-06-06T08:43:23.627 INFO:tasks.workunit.client.0.smithi027.stdout:2/344: mkdir d6/d1e/d2b/d5a/d7b 0 2024-06-06T08:43:23.627 INFO:tasks.workunit.client.0.smithi027.stdout:3/201: dread d1/da/f2c [0,4194304] 0 2024-06-06T08:43:23.628 INFO:tasks.workunit.client.0.smithi027.stdout:3/202: write d1/d8/f9 [310983,98939] 0 2024-06-06T08:43:23.629 INFO:tasks.workunit.client.0.smithi027.stdout:3/203: read d1/da/f31 [192593,81788] 0 2024-06-06T08:43:23.629 INFO:tasks.workunit.client.0.smithi027.stdout:9/271: stat d3/da 0 2024-06-06T08:43:23.630 INFO:tasks.workunit.client.0.smithi027.stdout:9/272: creat d3/da/dc/df/d23/d36/f61 x:0 0 0 2024-06-06T08:43:23.632 INFO:tasks.workunit.client.0.smithi027.stdout:5/416: truncate d0/da/d37/d2b/f5b 2595537 0 2024-06-06T08:43:23.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/417: chown d0/da/db/d46/d54/c7a 5554 1 2024-06-06T08:43:23.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/418: truncate d0/da/db/d1a/f6f 391898 0 2024-06-06T08:43:23.634 INFO:tasks.workunit.client.0.smithi027.stdout:3/204: truncate d1/da/f31 8711 0 2024-06-06T08:43:23.636 INFO:tasks.workunit.client.0.smithi027.stdout:5/419: creat d0/da/db/d46/d4c/f80 x:0 0 0 2024-06-06T08:43:23.637 INFO:tasks.workunit.client.0.smithi027.stdout:2/345: dread f1 [0,4194304] 0 2024-06-06T08:43:23.638 INFO:tasks.workunit.client.0.smithi027.stdout:3/205: dread d1/f6 [0,4194304] 0 2024-06-06T08:43:23.638 INFO:tasks.workunit.client.0.smithi027.stdout:3/206: getdents d1/d8/d22 0 2024-06-06T08:43:23.639 INFO:tasks.workunit.client.0.smithi027.stdout:2/346: read d6/d3b/d61/f70 [40912,97774] 0 2024-06-06T08:43:23.639 INFO:tasks.workunit.client.0.smithi027.stdout:2/347: write d6/dc/d1d/f5b [887839,9126] 0 2024-06-06T08:43:23.639 INFO:tasks.workunit.client.0.smithi027.stdout:2/348: chown d6/l51 50747514 1 2024-06-06T08:43:23.639 INFO:tasks.workunit.client.0.smithi027.stdout:3/207: dread d1/d8/d22/d27/f14 [4194304,4194304] 0 2024-06-06T08:43:23.640 INFO:tasks.workunit.client.0.smithi027.stdout:3/208: fsync d1/d8/d22/d27/f39 0 2024-06-06T08:43:23.640 INFO:tasks.workunit.client.0.smithi027.stdout:3/209: chown d1/d8/f15 28 1 2024-06-06T08:43:23.640 INFO:tasks.workunit.client.0.smithi027.stdout:5/420: mkdir d0/da/db/d46/d7f/d81 0 2024-06-06T08:43:23.642 INFO:tasks.workunit.client.0.smithi027.stdout:4/247: link d0/d1/d6/d8/d21/f30 d0/f55 0 2024-06-06T08:43:23.642 INFO:tasks.workunit.client.0.smithi027.stdout:4/248: read - d0/d1d/d38/f3c zero size 2024-06-06T08:43:23.645 INFO:tasks.workunit.client.0.smithi027.stdout:2/349: getdents d6/dc/d1d/d4d 0 2024-06-06T08:43:23.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/210: mknod d1/c3e 0 2024-06-06T08:43:23.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/211: write d1/d8/d22/d2d/f3d [311837,27168] 0 2024-06-06T08:43:23.647 INFO:tasks.workunit.client.0.smithi027.stdout:3/212: write d1/d8/d22/d2d/f3d [938978,29226] 0 2024-06-06T08:43:23.647 INFO:tasks.workunit.client.0.smithi027.stdout:3/213: truncate d1/d8/f15 650267 0 2024-06-06T08:43:23.649 INFO:tasks.workunit.client.0.smithi027.stdout:2/350: getdents d6/dc 0 2024-06-06T08:43:23.652 INFO:tasks.workunit.client.0.smithi027.stdout:5/421: dread d0/da/f13 [0,4194304] 0 2024-06-06T08:43:23.652 INFO:tasks.workunit.client.0.smithi027.stdout:2/351: symlink d6/d3b/d61/l7c 0 2024-06-06T08:43:23.654 INFO:tasks.workunit.client.0.smithi027.stdout:3/214: symlink d1/d8/d22/d27/l3f 0 2024-06-06T08:43:23.654 INFO:tasks.workunit.client.0.smithi027.stdout:3/215: chown d1/f2 1395 1 2024-06-06T08:43:23.656 INFO:tasks.workunit.client.0.smithi027.stdout:5/422: getdents d0/da/db/d46/d77 0 2024-06-06T08:43:23.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/352: creat d6/d1e/d2b/d5a/f7d x:0 0 0 2024-06-06T08:43:23.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/353: chown d6/dd/c67 2465 1 2024-06-06T08:43:23.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/354: write d6/d3b/f43 [3533046,113944] 0 2024-06-06T08:43:23.662 INFO:tasks.workunit.client.0.smithi027.stdout:5/423: rename d0/da/db/d46/d4c/f80 to d0/da/db/d46/d4c/f82 0 2024-06-06T08:43:23.662 INFO:tasks.workunit.client.0.smithi027.stdout:7/416: dwrite d17/f27 [0,4194304] 0 2024-06-06T08:43:23.665 INFO:tasks.workunit.client.0.smithi027.stdout:5/424: unlink d0/da/d47/c75 0 2024-06-06T08:43:23.665 INFO:tasks.workunit.client.0.smithi027.stdout:5/425: chown d0/da/db/d46/l55 1285534187 1 2024-06-06T08:43:23.668 INFO:tasks.workunit.client.0.smithi027.stdout:5/426: write d0/da/db/f1b [1472712,31201] 0 2024-06-06T08:43:23.674 INFO:tasks.workunit.client.0.smithi027.stdout:6/352: rename d4/d9/f66 to d4/d1a/d2e/d44/f74 0 2024-06-06T08:43:23.675 INFO:tasks.workunit.client.0.smithi027.stdout:6/353: chown d4/d9/d6e 450 1 2024-06-06T08:43:23.675 INFO:tasks.workunit.client.0.smithi027.stdout:6/354: write d4/d13/f21 [715999,85630] 0 2024-06-06T08:43:23.677 INFO:tasks.workunit.client.0.smithi027.stdout:6/355: creat d4/de/d45/f75 x:0 0 0 2024-06-06T08:43:23.677 INFO:tasks.workunit.client.0.smithi027.stdout:0/287: rename d15/d18/d23/d28/c59 to d15/d18/d23/d28/d2a/d30/d5d/c62 0 2024-06-06T08:43:23.677 INFO:tasks.workunit.client.0.smithi027.stdout:6/356: write d4/d1a/d2e/f5c [2437545,9030] 0 2024-06-06T08:43:23.680 INFO:tasks.workunit.client.0.smithi027.stdout:4/249: rename d0/d1d/d38/d3b/f46 to d0/d1/d6/d4b/f56 0 2024-06-06T08:43:23.680 INFO:tasks.workunit.client.0.smithi027.stdout:4/250: read d0/d1/f2 [1406641,87893] 0 2024-06-06T08:43:23.680 INFO:tasks.workunit.client.0.smithi027.stdout:4/251: stat d0/d1d/d38/c35 0 2024-06-06T08:43:23.681 INFO:tasks.workunit.client.0.smithi027.stdout:2/355: dwrite d6/d1e/f68 [0,4194304] 0 2024-06-06T08:43:23.681 INFO:tasks.workunit.client.0.smithi027.stdout:6/357: creat d4/de/d45/d6a/f76 x:0 0 0 2024-06-06T08:43:23.681 INFO:tasks.workunit.client.0.smithi027.stdout:6/358: fdatasync d4/d9/dc/d18/f24 0 2024-06-06T08:43:23.682 INFO:tasks.workunit.client.0.smithi027.stdout:3/216: dwrite d1/d8/d22/d2d/f3d [0,4194304] 0 2024-06-06T08:43:23.691 INFO:tasks.workunit.client.0.smithi027.stdout:2/356: creat d6/d1e/d2b/d5a/d7b/f7e x:0 0 0 2024-06-06T08:43:23.692 INFO:tasks.workunit.client.0.smithi027.stdout:7/417: dwrite d17/d32/d46/d51/d7f/f7a [0,4194304] 0 2024-06-06T08:43:23.692 INFO:tasks.workunit.client.0.smithi027.stdout:7/418: chown d17/d32/d46/d4f/l65 60053909 1 2024-06-06T08:43:23.700 INFO:tasks.workunit.client.0.smithi027.stdout:0/288: dwrite d15/d18/f4b [0,4194304] 0 2024-06-06T08:43:23.700 INFO:tasks.workunit.client.0.smithi027.stdout:0/289: chown d15/d18/d23/d28/d2a/c46 724179 1 2024-06-06T08:43:23.701 INFO:tasks.workunit.client.0.smithi027.stdout:4/252: write d0/d1/ff [689382,105032] 0 2024-06-06T08:43:23.702 INFO:tasks.workunit.client.0.smithi027.stdout:0/290: mknod d15/d18/d40/c63 0 2024-06-06T08:43:23.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/253: creat d0/d1d/d2a/f57 x:0 0 0 2024-06-06T08:43:23.704 INFO:tasks.workunit.client.0.smithi027.stdout:0/291: dread d15/d18/d23/d28/d2a/f33 [0,4194304] 0 2024-06-06T08:43:23.704 INFO:tasks.workunit.client.0.smithi027.stdout:0/292: write d15/f4f [1407945,102539] 0 2024-06-06T08:43:23.705 INFO:tasks.workunit.client.0.smithi027.stdout:7/419: write d17/d32/f38 [3963012,53619] 0 2024-06-06T08:43:23.707 INFO:tasks.workunit.client.0.smithi027.stdout:0/293: mkdir d15/d18/d64 0 2024-06-06T08:43:23.707 INFO:tasks.workunit.client.0.smithi027.stdout:7/420: dread d17/f2e [0,4194304] 0 2024-06-06T08:43:23.707 INFO:tasks.workunit.client.0.smithi027.stdout:7/421: chown d17/d32/d46/d51/f8e 39 1 2024-06-06T08:43:23.708 INFO:tasks.workunit.client.0.smithi027.stdout:6/359: dwrite d4/d1a/f2c [0,4194304] 0 2024-06-06T08:43:23.708 INFO:tasks.workunit.client.0.smithi027.stdout:6/360: dread - d4/f69 zero size 2024-06-06T08:43:23.711 INFO:tasks.workunit.client.0.smithi027.stdout:3/217: dwrite d1/d8/d22/d27/f36 [0,4194304] 0 2024-06-06T08:43:23.711 INFO:tasks.workunit.client.0.smithi027.stdout:3/218: write d1/f2 [8811505,3353] 0 2024-06-06T08:43:23.711 INFO:tasks.workunit.client.0.smithi027.stdout:7/422: mknod d17/d32/d46/d51/d52/d70/c97 0 2024-06-06T08:43:23.711 INFO:tasks.workunit.client.0.smithi027.stdout:7/423: readlink l16 0 2024-06-06T08:43:23.712 INFO:tasks.workunit.client.0.smithi027.stdout:6/361: creat d4/d9/d49/f77 x:0 0 0 2024-06-06T08:43:23.712 INFO:tasks.workunit.client.0.smithi027.stdout:6/362: fsync d4/d1a/d2e/f65 0 2024-06-06T08:43:23.713 INFO:tasks.workunit.client.0.smithi027.stdout:0/294: unlink d15/d18/d23/f25 0 2024-06-06T08:43:23.713 INFO:tasks.workunit.client.0.smithi027.stdout:0/295: write f4 [8688997,127226] 0 2024-06-06T08:43:23.714 INFO:tasks.workunit.client.0.smithi027.stdout:4/254: dread d0/d1/f14 [4194304,4194304] 0 2024-06-06T08:43:23.714 INFO:tasks.workunit.client.0.smithi027.stdout:7/424: unlink d17/d32/d46/d83/d2a/c36 0 2024-06-06T08:43:23.717 INFO:tasks.workunit.client.0.smithi027.stdout:2/357: dwrite d6/dc/d3f/f50 [0,4194304] 0 2024-06-06T08:43:23.717 INFO:tasks.workunit.client.0.smithi027.stdout:4/255: dread d0/d1/d6/f2f [0,4194304] 0 2024-06-06T08:43:23.719 INFO:tasks.workunit.client.0.smithi027.stdout:2/358: creat d6/dc/d3f/d77/d55/f7f x:0 0 0 2024-06-06T08:43:23.721 INFO:tasks.workunit.client.0.smithi027.stdout:2/359: unlink d6/d1e/d2b/f37 0 2024-06-06T08:43:23.728 INFO:tasks.workunit.client.0.smithi027.stdout:0/296: read d15/d18/d23/d28/d2a/d2d/f3b [556462,15031] 0 2024-06-06T08:43:23.728 INFO:tasks.workunit.client.0.smithi027.stdout:0/297: chown d15/d18/d23/d28/d31/l60 866894 1 2024-06-06T08:43:23.730 INFO:tasks.workunit.client.0.smithi027.stdout:0/298: truncate f9 200671 0 2024-06-06T08:43:23.733 INFO:tasks.workunit.client.0.smithi027.stdout:0/299: mknod d15/d18/d23/c65 0 2024-06-06T08:43:23.733 INFO:tasks.workunit.client.0.smithi027.stdout:2/360: write d6/d1e/d2b/f57 [1089364,106011] 0 2024-06-06T08:43:23.733 INFO:tasks.workunit.client.0.smithi027.stdout:3/219: dwrite d1/da/f18 [0,4194304] 0 2024-06-06T08:43:23.735 INFO:tasks.workunit.client.0.smithi027.stdout:7/425: dwrite d17/d32/d46/d4f/d5e/f66 [0,4194304] 0 2024-06-06T08:43:23.735 INFO:tasks.workunit.client.0.smithi027.stdout:2/361: mkdir d6/d3b/d80 0 2024-06-06T08:43:23.735 INFO:tasks.workunit.client.0.smithi027.stdout:2/362: write d6/d3b/d61/f70 [251304,118019] 0 2024-06-06T08:43:23.737 INFO:tasks.workunit.client.0.smithi027.stdout:6/363: dwrite d4/d9/dc/f43 [0,4194304] 0 2024-06-06T08:43:23.737 INFO:tasks.workunit.client.0.smithi027.stdout:3/220: mkdir d1/d40 0 2024-06-06T08:43:23.738 INFO:tasks.workunit.client.0.smithi027.stdout:4/256: dread d0/d1/ff [0,4194304] 0 2024-06-06T08:43:23.741 INFO:tasks.workunit.client.0.smithi027.stdout:7/426: read d17/d32/d61/f6b [435680,29353] 0 2024-06-06T08:43:23.741 INFO:tasks.workunit.client.0.smithi027.stdout:7/427: read - d17/d47/f49 zero size 2024-06-06T08:43:23.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/428: creat d17/d32/d46/d83/d72/f98 x:0 0 0 2024-06-06T08:43:23.743 INFO:tasks.workunit.client.0.smithi027.stdout:6/364: creat d4/d9/f78 x:0 0 0 2024-06-06T08:43:23.744 INFO:tasks.workunit.client.0.smithi027.stdout:2/363: dread - d6/dc/d3f/f7a zero size 2024-06-06T08:43:23.744 INFO:tasks.workunit.client.0.smithi027.stdout:2/364: write d6/dc/f54 [947557,19316] 0 2024-06-06T08:43:23.747 INFO:tasks.workunit.client.0.smithi027.stdout:6/365: unlink d4/c16 0 2024-06-06T08:43:23.749 INFO:tasks.workunit.client.0.smithi027.stdout:2/365: rename d6/d1e/d2b/l5e to d6/d1e/d2b/d5a/l81 0 2024-06-06T08:43:23.749 INFO:tasks.workunit.client.0.smithi027.stdout:3/221: dread d1/d8/d22/d27/f36 [0,4194304] 0 2024-06-06T08:43:23.751 INFO:tasks.workunit.client.0.smithi027.stdout:6/366: creat d4/f79 x:0 0 0 2024-06-06T08:43:23.751 INFO:tasks.workunit.client.0.smithi027.stdout:6/367: dread - d4/de/d45/d6a/f76 zero size 2024-06-06T08:43:23.756 INFO:tasks.workunit.client.0.smithi027.stdout:2/366: rename d6/dc/d1d/d59 to d6/dc/d3f/d77/d55/d69/d82 0 2024-06-06T08:43:23.756 INFO:tasks.workunit.client.0.smithi027.stdout:7/429: write d17/d32/d46/d4f/d5e/f8b [652133,32529] 0 2024-06-06T08:43:23.759 INFO:tasks.workunit.client.0.smithi027.stdout:7/430: dread d17/f3f [0,4194304] 0 2024-06-06T08:43:23.763 INFO:tasks.workunit.client.0.smithi027.stdout:4/257: dwrite d0/d1d/d2a/f57 [0,4194304] 0 2024-06-06T08:43:23.763 INFO:tasks.workunit.client.0.smithi027.stdout:4/258: write d0/d1/d6/d8/d21/f2d [1631028,57434] 0 2024-06-06T08:43:23.768 INFO:tasks.workunit.client.0.smithi027.stdout:0/300: dwrite d15/d18/d23/d38/f51 [8388608,4194304] 0 2024-06-06T08:43:23.770 INFO:tasks.workunit.client.0.smithi027.stdout:0/301: getdents d15/d18/d23/d28/d31 0 2024-06-06T08:43:23.771 INFO:tasks.workunit.client.0.smithi027.stdout:2/367: mkdir d6/dc/d3f/d48/d83 0 2024-06-06T08:43:23.774 INFO:tasks.workunit.client.0.smithi027.stdout:3/222: dwrite d1/d8/d22/f37 [0,4194304] 0 2024-06-06T08:43:23.775 INFO:tasks.workunit.client.0.smithi027.stdout:4/259: read d0/d1/d6/f33 [2531024,60663] 0 2024-06-06T08:43:23.775 INFO:tasks.workunit.client.0.smithi027.stdout:2/368: creat d6/d1e/d2b/d5a/f84 x:0 0 0 2024-06-06T08:43:23.776 INFO:tasks.workunit.client.0.smithi027.stdout:0/302: rmdir d15/d18/d23/d38 39 2024-06-06T08:43:23.777 INFO:tasks.workunit.client.0.smithi027.stdout:0/303: write d15/d18/d23/f27 [2444618,107684] 0 2024-06-06T08:43:23.791 INFO:tasks.workunit.client.0.smithi027.stdout:6/368: rename d4/d9/dc/d18/c4f to d4/d9/dc/d1c/d52/c7a 0 2024-06-06T08:43:23.794 INFO:tasks.workunit.client.0.smithi027.stdout:3/223: rename d1/da/f38 to d1/f41 0 2024-06-06T08:43:23.794 INFO:tasks.workunit.client.0.smithi027.stdout:3/224: write d1/f41 [1040391,61116] 0 2024-06-06T08:43:23.794 INFO:tasks.workunit.client.0.smithi027.stdout:3/225: truncate d1/f1e 929345 0 2024-06-06T08:43:23.794 INFO:tasks.workunit.client.0.smithi027.stdout:3/226: truncate d1/f41 1232043 0 2024-06-06T08:43:23.795 INFO:tasks.workunit.client.0.smithi027.stdout:2/369: dwrite d6/dd/f17 [0,4194304] 0 2024-06-06T08:43:23.796 INFO:tasks.workunit.client.0.smithi027.stdout:3/227: rename d1/d8/d22/d27/f39 to d1/da/f42 0 2024-06-06T08:43:23.798 INFO:tasks.workunit.client.0.smithi027.stdout:2/370: creat d6/dc/f85 x:0 0 0 2024-06-06T08:43:23.798 INFO:tasks.workunit.client.0.smithi027.stdout:2/371: chown d6/dc/d3f/l6f 0 1 2024-06-06T08:43:23.798 INFO:tasks.workunit.client.0.smithi027.stdout:2/372: getdents d6/d1e/d2b/d5a 0 2024-06-06T08:43:23.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/373: rename d6/f4f to d6/dc/d1d/f86 0 2024-06-06T08:43:23.800 INFO:tasks.workunit.client.0.smithi027.stdout:4/260: dwrite d0/fd [8388608,4194304] 0 2024-06-06T08:43:23.800 INFO:tasks.workunit.client.0.smithi027.stdout:4/261: chown d0/d1/d6/d8/d21 1810347 1 2024-06-06T08:43:23.801 INFO:tasks.workunit.client.0.smithi027.stdout:2/374: mknod d6/dd/d6b/c87 0 2024-06-06T08:43:23.801 INFO:tasks.workunit.client.0.smithi027.stdout:2/375: truncate d6/f6e 466096 0 2024-06-06T08:43:23.802 INFO:tasks.workunit.client.0.smithi027.stdout:2/376: creat d6/dd/d6b/f88 x:0 0 0 2024-06-06T08:43:23.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/304: dwrite d15/d18/d23/d28/d2a/d30/d5d/f4a [0,4194304] 0 2024-06-06T08:43:23.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/377: truncate f1 208676 0 2024-06-06T08:43:23.806 INFO:tasks.workunit.client.0.smithi027.stdout:0/305: symlink d15/d18/d40/l66 0 2024-06-06T08:43:23.806 INFO:tasks.workunit.client.0.smithi027.stdout:3/228: write d1/d8/d22/d27/f14 [1873345,113785] 0 2024-06-06T08:43:23.807 INFO:tasks.workunit.client.0.smithi027.stdout:4/262: write d0/d1/f14 [917064,116731] 0 2024-06-06T08:43:23.808 INFO:tasks.workunit.client.0.smithi027.stdout:3/229: symlink d1/l43 0 2024-06-06T08:43:23.808 INFO:tasks.workunit.client.0.smithi027.stdout:3/230: truncate d1/f13 5238558 0 2024-06-06T08:43:23.808 INFO:tasks.workunit.client.0.smithi027.stdout:3/231: fdatasync d1/f20 0 2024-06-06T08:43:23.810 INFO:tasks.workunit.client.0.smithi027.stdout:3/232: mkdir d1/da/d44 0 2024-06-06T08:43:23.812 INFO:tasks.workunit.client.0.smithi027.stdout:3/233: unlink d1/da/l19 0 2024-06-06T08:43:23.815 INFO:tasks.workunit.client.0.smithi027.stdout:3/234: mkdir d1/d1f/d45 0 2024-06-06T08:43:23.815 INFO:tasks.workunit.client.0.smithi027.stdout:3/235: chown d1/d1f/l2b 421122579 1 2024-06-06T08:43:23.816 INFO:tasks.workunit.client.0.smithi027.stdout:3/236: truncate d1/d8/d22/d27/f36 137383 0 2024-06-06T08:43:23.816 INFO:tasks.workunit.client.0.smithi027.stdout:3/237: chown d1/d8/d22/d27/c1b 1 1 2024-06-06T08:43:23.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/378: read d6/dd/f23 [4097815,83380] 0 2024-06-06T08:43:23.818 INFO:tasks.workunit.client.0.smithi027.stdout:3/238: creat d1/d8/d22/f46 x:0 0 0 2024-06-06T08:43:23.819 INFO:tasks.workunit.client.0.smithi027.stdout:2/379: getdents d6/dc/d1d 0 2024-06-06T08:43:23.850 INFO:tasks.workunit.client.0.smithi027.stdout:3/239: dwrite d1/d8/d22/f37 [0,4194304] 0 2024-06-06T08:43:23.850 INFO:tasks.workunit.client.0.smithi027.stdout:4/263: rename d0/d1/d6/f25 to d0/d1/d6/d8/f58 0 2024-06-06T08:43:23.851 INFO:tasks.workunit.client.0.smithi027.stdout:4/264: creat d0/d1d/f59 x:0 0 0 2024-06-06T08:43:23.852 INFO:tasks.workunit.client.0.smithi027.stdout:2/380: dread d6/f2a [0,4194304] 0 2024-06-06T08:43:23.853 INFO:tasks.workunit.client.0.smithi027.stdout:0/306: dwrite d15/f21 [0,4194304] 0 2024-06-06T08:43:23.853 INFO:tasks.workunit.client.0.smithi027.stdout:3/240: rename d1/d1f/l30 to d1/da/l47 0 2024-06-06T08:43:23.854 INFO:tasks.workunit.client.0.smithi027.stdout:0/307: creat d15/d18/d23/d3c/f67 x:0 0 0 2024-06-06T08:43:23.855 INFO:tasks.workunit.client.0.smithi027.stdout:4/265: rmdir d0/d1/d6/d8/d21/d47 0 2024-06-06T08:43:23.855 INFO:tasks.workunit.client.0.smithi027.stdout:3/241: rename d1/da/f2c to d1/d8/d22/d2d/f48 0 2024-06-06T08:43:23.856 INFO:tasks.workunit.client.0.smithi027.stdout:4/266: mkdir d0/d1d/d34/d5a 0 2024-06-06T08:43:23.858 INFO:tasks.workunit.client.0.smithi027.stdout:3/242: creat d1/da/d44/f49 x:0 0 0 2024-06-06T08:43:23.860 INFO:tasks.workunit.client.0.smithi027.stdout:4/267: creat d0/d1/f5b x:0 0 0 2024-06-06T08:43:23.862 INFO:tasks.workunit.client.0.smithi027.stdout:2/381: rename d6/dd/c67 to d6/dc/d3f/c89 0 2024-06-06T08:43:23.862 INFO:tasks.workunit.client.0.smithi027.stdout:0/308: dread d15/d18/d23/f27 [0,4194304] 0 2024-06-06T08:43:23.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/309: mkdir d15/d18/d23/d38/d68 0 2024-06-06T08:43:23.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/268: dread d0/d1/fb [4194304,4194304] 0 2024-06-06T08:43:23.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/269: chown d0/c27 4188525 1 2024-06-06T08:43:23.870 INFO:tasks.workunit.client.0.smithi027.stdout:4/270: rmdir d0/d1d/d38 39 2024-06-06T08:43:23.872 INFO:tasks.workunit.client.0.smithi027.stdout:0/310: dread d15/d18/d23/f29 [0,4194304] 0 2024-06-06T08:43:23.874 INFO:tasks.workunit.client.0.smithi027.stdout:0/311: symlink d15/d18/d64/l69 0 2024-06-06T08:43:23.875 INFO:tasks.workunit.client.0.smithi027.stdout:3/243: dwrite d1/f6 [0,4194304] 0 2024-06-06T08:43:23.875 INFO:tasks.workunit.client.0.smithi027.stdout:3/244: fsync d1/f2 0 2024-06-06T08:43:23.875 INFO:tasks.workunit.client.0.smithi027.stdout:3/245: fsync d1/f20 0 2024-06-06T08:43:23.875 INFO:tasks.workunit.client.0.smithi027.stdout:3/246: readlink d1/d8/d22/d27/l3b 0 2024-06-06T08:43:23.876 INFO:tasks.workunit.client.0.smithi027.stdout:0/312: creat d15/d18/d23/d38/d68/f6a x:0 0 0 2024-06-06T08:43:23.877 INFO:tasks.workunit.client.0.smithi027.stdout:0/313: creat d15/d18/d23/d5b/f6b x:0 0 0 2024-06-06T08:43:23.877 INFO:tasks.workunit.client.0.smithi027.stdout:0/314: write d15/d18/d23/d28/d2a/d2d/f3b [1671983,89044] 0 2024-06-06T08:43:23.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/315: unlink d15/d18/d23/f29 0 2024-06-06T08:43:23.881 INFO:tasks.workunit.client.0.smithi027.stdout:4/271: write d0/fd [11875466,33615] 0 2024-06-06T08:43:23.882 INFO:tasks.workunit.client.0.smithi027.stdout:4/272: chown d0/d1d/d38/l51 407 1 2024-06-06T08:43:23.883 INFO:tasks.workunit.client.0.smithi027.stdout:0/316: dread d15/d18/d23/d28/d2a/d2d/f58 [0,4194304] 0 2024-06-06T08:43:23.884 INFO:tasks.workunit.client.0.smithi027.stdout:0/317: write d15/d18/d23/d38/d68/f6a [370950,68446] 0 2024-06-06T08:43:23.884 INFO:tasks.workunit.client.0.smithi027.stdout:0/318: dread - d15/d18/d23/d28/f4c zero size 2024-06-06T08:43:23.884 INFO:tasks.workunit.client.0.smithi027.stdout:2/382: dwrite d6/d3b/f52 [0,4194304] 0 2024-06-06T08:43:23.893 INFO:tasks.workunit.client.0.smithi027.stdout:4/273: dread d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:23.894 INFO:tasks.workunit.client.0.smithi027.stdout:4/274: fsync d0/f13 0 2024-06-06T08:43:23.894 INFO:tasks.workunit.client.0.smithi027.stdout:4/275: fdatasync d0/d1/f2 0 2024-06-06T08:43:23.894 INFO:tasks.workunit.client.0.smithi027.stdout:4/276: read d0/fd [1242838,66901] 0 2024-06-06T08:43:23.896 INFO:tasks.workunit.client.0.smithi027.stdout:4/277: creat d0/f5c x:0 0 0 2024-06-06T08:43:23.896 INFO:tasks.workunit.client.0.smithi027.stdout:4/278: write d0/d1/f5b [258529,16323] 0 2024-06-06T08:43:23.897 INFO:tasks.workunit.client.0.smithi027.stdout:4/279: getdents d0/d1/d6/d4b 0 2024-06-06T08:43:23.897 INFO:tasks.workunit.client.0.smithi027.stdout:3/247: dwrite d1/d8/d22/d27/f32 [0,4194304] 0 2024-06-06T08:43:23.898 INFO:tasks.workunit.client.0.smithi027.stdout:4/280: creat d0/d1/d6/d8/d21/f5d x:0 0 0 2024-06-06T08:43:23.898 INFO:tasks.workunit.client.0.smithi027.stdout:4/281: write d0/d1/f4f [422773,110784] 0 2024-06-06T08:43:23.901 INFO:tasks.workunit.client.0.smithi027.stdout:4/282: unlink d0/d1d/d38/f3c 0 2024-06-06T08:43:23.902 INFO:tasks.workunit.client.0.smithi027.stdout:4/283: creat d0/d1d/d2a/f5e x:0 0 0 2024-06-06T08:43:23.904 INFO:tasks.workunit.client.0.smithi027.stdout:4/284: unlink d0/d1d/d38/d3b/c2e 0 2024-06-06T08:43:23.905 INFO:tasks.workunit.client.0.smithi027.stdout:4/285: symlink d0/d1d/d34/d5a/l5f 0 2024-06-06T08:43:23.906 INFO:tasks.workunit.client.0.smithi027.stdout:2/383: dwrite d6/d3b/f52 [0,4194304] 0 2024-06-06T08:43:23.906 INFO:tasks.workunit.client.0.smithi027.stdout:4/286: mkdir d0/d1d/d60 0 2024-06-06T08:43:23.910 INFO:tasks.workunit.client.0.smithi027.stdout:3/248: dread d1/d8/f9 [0,4194304] 0 2024-06-06T08:43:23.910 INFO:tasks.workunit.client.0.smithi027.stdout:4/287: dread d0/d1d/d34/f40 [0,4194304] 0 2024-06-06T08:43:23.910 INFO:tasks.workunit.client.0.smithi027.stdout:2/384: dread d6/f2e [0,4194304] 0 2024-06-06T08:43:23.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/249: unlink d1/f33 0 2024-06-06T08:43:23.912 INFO:tasks.workunit.client.0.smithi027.stdout:2/385: mknod d6/d3b/d80/c8a 0 2024-06-06T08:43:23.914 INFO:tasks.workunit.client.0.smithi027.stdout:2/386: rmdir d6/dc/d1d/d4d 39 2024-06-06T08:43:23.914 INFO:tasks.workunit.client.0.smithi027.stdout:2/387: chown d6/dd/l1a 5 1 2024-06-06T08:43:23.914 INFO:tasks.workunit.client.0.smithi027.stdout:2/388: fsync d6/dc/d1d/f4b 0 2024-06-06T08:43:23.915 INFO:tasks.workunit.client.0.smithi027.stdout:2/389: creat d6/f8b x:0 0 0 2024-06-06T08:43:23.915 INFO:tasks.workunit.client.0.smithi027.stdout:2/390: write d6/dc/d1d/d4d/f5f [793889,113383] 0 2024-06-06T08:43:23.915 INFO:tasks.workunit.client.0.smithi027.stdout:2/391: readlink d6/dc/d3f/l6f 0 2024-06-06T08:43:23.918 INFO:tasks.workunit.client.0.smithi027.stdout:2/392: symlink d6/dd/l8c 0 2024-06-06T08:43:23.919 INFO:tasks.workunit.client.0.smithi027.stdout:2/393: mkdir d6/d3b/d80/d8d 0 2024-06-06T08:43:23.919 INFO:tasks.workunit.client.0.smithi027.stdout:2/394: chown d6/dc/d3f/d77/d55/d69/d82/l66 4 1 2024-06-06T08:43:23.919 INFO:tasks.workunit.client.0.smithi027.stdout:2/395: stat d6/d1e/d2b/d5a 0 2024-06-06T08:43:23.919 INFO:tasks.workunit.client.0.smithi027.stdout:2/396: chown d6/d1e 1129089 1 2024-06-06T08:43:23.920 INFO:tasks.workunit.client.0.smithi027.stdout:4/288: dread d0/d1/ff [0,4194304] 0 2024-06-06T08:43:23.928 INFO:tasks.workunit.client.0.smithi027.stdout:3/250: dwrite d1/d8/d22/d27/f14 [4194304,4194304] 0 2024-06-06T08:43:23.941 INFO:tasks.workunit.client.0.smithi027.stdout:4/289: rename d0/d1/d6/d8/c26 to d0/d1d/c61 0 2024-06-06T08:43:23.943 INFO:tasks.workunit.client.0.smithi027.stdout:4/290: symlink d0/d1/d6/d4b/l62 0 2024-06-06T08:43:23.943 INFO:tasks.workunit.client.0.smithi027.stdout:3/251: dwrite d1/da/f18 [0,4194304] 0 2024-06-06T08:43:23.943 INFO:tasks.workunit.client.0.smithi027.stdout:4/291: mkdir d0/d63 0 2024-06-06T08:43:23.944 INFO:tasks.workunit.client.0.smithi027.stdout:4/292: mknod d0/d63/c64 0 2024-06-06T08:43:23.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/252: dread d1/d8/d22/d2d/f3d [0,4194304] 0 2024-06-06T08:43:23.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/253: chown d1/f13 47734641 1 2024-06-06T08:43:23.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/254: fsync d1/f20 0 2024-06-06T08:43:23.956 INFO:tasks.workunit.client.0.smithi027.stdout:3/255: rmdir d1/d40 0 2024-06-06T08:43:23.956 INFO:tasks.workunit.client.0.smithi027.stdout:3/256: chown d1/d8/d22/d27/l3b 5227 1 2024-06-06T08:43:23.961 INFO:tasks.workunit.client.0.smithi027.stdout:4/293: dwrite d0/f13 [0,4194304] 0 2024-06-06T08:43:23.961 INFO:tasks.workunit.client.0.smithi027.stdout:4/294: write d0/d1d/d34/f4d [654761,85382] 0 2024-06-06T08:43:23.961 INFO:tasks.workunit.client.0.smithi027.stdout:4/295: stat d0/d1d/c61 0 2024-06-06T08:43:23.962 INFO:tasks.workunit.client.0.smithi027.stdout:4/296: unlink d0/d1d/d38/d3b/l45 0 2024-06-06T08:43:23.963 INFO:tasks.workunit.client.0.smithi027.stdout:4/297: truncate d0/d1/f4f 63068 0 2024-06-06T08:43:23.964 INFO:tasks.workunit.client.0.smithi027.stdout:4/298: symlink d0/d1/d6/l65 0 2024-06-06T08:43:23.965 INFO:tasks.workunit.client.0.smithi027.stdout:4/299: creat d0/f66 x:0 0 0 2024-06-06T08:43:23.975 INFO:tasks.workunit.client.0.smithi027.stdout:4/300: link d0/d1/d6/d8/c1e d0/d1d/d34/d5a/c67 0 2024-06-06T08:43:23.982 INFO:tasks.workunit.client.0.smithi027.stdout:4/301: dread d0/f13 [0,4194304] 0 2024-06-06T08:43:23.982 INFO:tasks.workunit.client.0.smithi027.stdout:4/302: fsync d0/d1/f4f 0 2024-06-06T08:43:24.044 INFO:tasks.workunit.client.0.smithi027.stdout:8/218: sync 2024-06-06T08:43:24.060 INFO:tasks.workunit.client.0.smithi027.stdout:8/219: dwrite d7/f2c [0,4194304] 0 2024-06-06T08:43:24.062 INFO:tasks.workunit.client.0.smithi027.stdout:8/220: rmdir d7/d13/d15 39 2024-06-06T08:43:24.079 INFO:tasks.workunit.client.0.smithi027.stdout:8/221: dwrite d7/db/f27 [0,4194304] 0 2024-06-06T08:43:24.081 INFO:tasks.workunit.client.0.smithi027.stdout:8/222: rename d7/d2f/d38 to d7/dc/d36/d41 0 2024-06-06T08:43:24.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/223: creat d7/f42 x:0 0 0 2024-06-06T08:43:24.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/224: creat d7/f43 x:0 0 0 2024-06-06T08:43:24.084 INFO:tasks.workunit.client.0.smithi027.stdout:8/225: unlink d7/f43 0 2024-06-06T08:43:24.085 INFO:tasks.workunit.client.0.smithi027.stdout:8/226: symlink d7/l44 0 2024-06-06T08:43:24.086 INFO:tasks.workunit.client.0.smithi027.stdout:8/227: rename d7/db/c3a to d7/d13/c45 0 2024-06-06T08:43:24.086 INFO:tasks.workunit.client.0.smithi027.stdout:8/228: truncate d7/f30 908137 0 2024-06-06T08:43:24.087 INFO:tasks.workunit.client.0.smithi027.stdout:8/229: stat d7/d13/d15/f1c 0 2024-06-06T08:43:24.087 INFO:tasks.workunit.client.0.smithi027.stdout:8/230: fsync d7/db/f27 0 2024-06-06T08:43:24.088 INFO:tasks.workunit.client.0.smithi027.stdout:8/231: mkdir d7/dc/d37/d46 0 2024-06-06T08:43:24.088 INFO:tasks.workunit.client.0.smithi027.stdout:8/232: chown d7/d13/c45 1 1 2024-06-06T08:43:24.089 INFO:tasks.workunit.client.0.smithi027.stdout:8/233: getdents d7/dc/d36/d3c 0 2024-06-06T08:43:24.090 INFO:tasks.workunit.client.0.smithi027.stdout:8/234: symlink d7/d13/l47 0 2024-06-06T08:43:24.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/235: truncate d7/db/f1d 679436 0 2024-06-06T08:43:24.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/236: fsync d7/f30 0 2024-06-06T08:43:24.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/237: stat d7/d13/f1e 0 2024-06-06T08:43:24.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/238: chown d7/f22 1555 1 2024-06-06T08:43:24.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/239: dread - d7/f42 zero size 2024-06-06T08:43:24.097 INFO:tasks.workunit.client.0.smithi027.stdout:8/240: rmdir d7/dc/d36 39 2024-06-06T08:43:24.099 INFO:tasks.workunit.client.0.smithi027.stdout:8/241: symlink d7/dc/d36/d3c/l48 0 2024-06-06T08:43:24.100 INFO:tasks.workunit.client.0.smithi027.stdout:8/242: link d7/l44 d7/dc/d37/l49 0 2024-06-06T08:43:24.107 INFO:tasks.workunit.client.0.smithi027.stdout:8/243: dread d7/db/f1d [0,4194304] 0 2024-06-06T08:43:24.107 INFO:tasks.workunit.client.0.smithi027.stdout:8/244: chown d7/l44 9 1 2024-06-06T08:43:24.109 INFO:tasks.workunit.client.0.smithi027.stdout:8/245: dread d7/f9 [0,4194304] 0 2024-06-06T08:43:24.109 INFO:tasks.workunit.client.0.smithi027.stdout:8/246: dread - d7/d2f/f31 zero size 2024-06-06T08:43:24.110 INFO:tasks.workunit.client.0.smithi027.stdout:8/247: unlink d7/dc/d36/d41/c39 0 2024-06-06T08:43:24.110 INFO:tasks.workunit.client.0.smithi027.stdout:8/248: dread - d7/d13/f1e zero size 2024-06-06T08:43:24.111 INFO:tasks.workunit.client.0.smithi027.stdout:8/249: mknod d7/dc/d36/c4a 0 2024-06-06T08:43:24.113 INFO:tasks.workunit.client.0.smithi027.stdout:8/250: link d7/dc/d36/d41/c2b d7/dc/c4b 0 2024-06-06T08:43:24.123 INFO:tasks.workunit.client.0.smithi027.stdout:8/251: dread d7/db/f27 [0,4194304] 0 2024-06-06T08:43:24.134 INFO:tasks.workunit.client.0.smithi027.stdout:8/252: dread d7/db/f19 [0,4194304] 0 2024-06-06T08:43:24.134 INFO:tasks.workunit.client.0.smithi027.stdout:8/253: rmdir d7/d2f 39 2024-06-06T08:43:24.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/254: dread - d7/d13/f1e zero size 2024-06-06T08:43:24.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/255: read d7/f30 [63267,130221] 0 2024-06-06T08:43:24.136 INFO:tasks.workunit.client.0.smithi027.stdout:8/256: rename d7/d13/l35 to d7/dc/d36/d3c/l4c 0 2024-06-06T08:43:24.138 INFO:tasks.workunit.client.0.smithi027.stdout:8/257: mknod d7/d13/d15/c4d 0 2024-06-06T08:43:24.139 INFO:tasks.workunit.client.0.smithi027.stdout:8/258: creat d7/f4e x:0 0 0 2024-06-06T08:43:24.140 INFO:tasks.workunit.client.0.smithi027.stdout:8/259: symlink d7/dc/d36/d3c/l4f 0 2024-06-06T08:43:24.141 INFO:tasks.workunit.client.0.smithi027.stdout:8/260: mkdir d7/dc/d37/d46/d50 0 2024-06-06T08:43:24.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/261: creat d7/dc/d36/d3c/f51 x:0 0 0 2024-06-06T08:43:24.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/262: dread - d7/d2f/f31 zero size 2024-06-06T08:43:24.143 INFO:tasks.workunit.client.0.smithi027.stdout:8/263: rmdir d7/dc/d36 39 2024-06-06T08:43:24.144 INFO:tasks.workunit.client.0.smithi027.stdout:8/264: creat d7/dc/d37/d46/f52 x:0 0 0 2024-06-06T08:43:24.146 INFO:tasks.workunit.client.0.smithi027.stdout:8/265: link d7/d13/d15/c4d d7/dc/d36/d3c/c53 0 2024-06-06T08:43:24.157 INFO:tasks.workunit.client.0.smithi027.stdout:8/266: dread d7/f2c [0,4194304] 0 2024-06-06T08:43:24.158 INFO:tasks.workunit.client.0.smithi027.stdout:8/267: mknod d7/dc/d36/d3c/c54 0 2024-06-06T08:43:24.159 INFO:tasks.workunit.client.0.smithi027.stdout:8/268: symlink d7/l55 0 2024-06-06T08:43:24.160 INFO:tasks.workunit.client.0.smithi027.stdout:8/269: rename d7/dc/d36/d3c/l4f to d7/dc/d36/d41/l56 0 2024-06-06T08:43:24.235 INFO:tasks.workunit.client.0.smithi027.stdout:1/322: sync 2024-06-06T08:43:24.236 INFO:tasks.workunit.client.0.smithi027.stdout:1/323: rmdir dc/d67/d69/d32 39 2024-06-06T08:43:24.247 INFO:tasks.workunit.client.0.smithi027.stdout:1/324: dread dc/de/d22/f35 [0,4194304] 0 2024-06-06T08:43:24.248 INFO:tasks.workunit.client.0.smithi027.stdout:1/325: mkdir dc/d67/d69/d32/d71 0 2024-06-06T08:43:24.248 INFO:tasks.workunit.client.0.smithi027.stdout:1/326: write dc/de/d3d/f44 [171794,60808] 0 2024-06-06T08:43:24.248 INFO:tasks.workunit.client.0.smithi027.stdout:1/327: readlink dc/de/d3d/d4f/l5c 0 2024-06-06T08:43:24.249 INFO:tasks.workunit.client.0.smithi027.stdout:1/328: creat dc/de/d22/f72 x:0 0 0 2024-06-06T08:43:24.250 INFO:tasks.workunit.client.0.smithi027.stdout:1/329: creat dc/de/d14/f73 x:0 0 0 2024-06-06T08:43:24.251 INFO:tasks.workunit.client.0.smithi027.stdout:1/330: creat dc/d67/d69/d32/d71/f74 x:0 0 0 2024-06-06T08:43:24.251 INFO:tasks.workunit.client.0.smithi027.stdout:1/331: stat dc/de/l65 0 2024-06-06T08:43:24.252 INFO:tasks.workunit.client.0.smithi027.stdout:1/332: creat dc/de/f75 x:0 0 0 2024-06-06T08:43:24.252 INFO:tasks.workunit.client.0.smithi027.stdout:1/333: readlink dc/de/l65 0 2024-06-06T08:43:24.261 INFO:tasks.workunit.client.0.smithi027.stdout:1/334: write dc/de/d14/d4e/f62 [223375,97344] 0 2024-06-06T08:43:24.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/335: mkdir dc/de/d3d/d4a/d76 0 2024-06-06T08:43:24.264 INFO:tasks.workunit.client.0.smithi027.stdout:1/336: creat dc/de/d3d/d4f/f77 x:0 0 0 2024-06-06T08:43:24.264 INFO:tasks.workunit.client.0.smithi027.stdout:1/337: truncate dc/de/d3d/f3f 432954 0 2024-06-06T08:43:24.265 INFO:tasks.workunit.client.0.smithi027.stdout:1/338: mkdir dc/d67/d69/d32/d37/d56/d78 0 2024-06-06T08:43:24.310 INFO:tasks.workunit.client.0.smithi027.stdout:2/397: sync 2024-06-06T08:43:24.311 INFO:tasks.workunit.client.0.smithi027.stdout:3/257: sync 2024-06-06T08:43:24.311 INFO:tasks.workunit.client.0.smithi027.stdout:6/369: sync 2024-06-06T08:43:24.311 INFO:tasks.workunit.client.0.smithi027.stdout:5/427: sync 2024-06-06T08:43:24.311 INFO:tasks.workunit.client.0.smithi027.stdout:3/258: chown d1/d8/f9 65154 1 2024-06-06T08:43:24.311 INFO:tasks.workunit.client.0.smithi027.stdout:3/259: fsync d1/d8/f9 0 2024-06-06T08:43:24.311 INFO:tasks.workunit.client.0.smithi027.stdout:2/398: write d6/dd/f23 [957808,10000] 0 2024-06-06T08:43:24.311 INFO:tasks.workunit.client.0.smithi027.stdout:2/399: fsync d6/dd/f18 0 2024-06-06T08:43:24.311 INFO:tasks.workunit.client.0.smithi027.stdout:2/400: dread - d6/dc/d3f/d48/f62 zero size 2024-06-06T08:43:24.313 INFO:tasks.workunit.client.0.smithi027.stdout:0/319: sync 2024-06-06T08:43:24.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/431: sync 2024-06-06T08:43:24.313 INFO:tasks.workunit.client.0.smithi027.stdout:4/303: sync 2024-06-06T08:43:24.313 INFO:tasks.workunit.client.0.smithi027.stdout:4/304: write d0/d1/d6/f33 [327597,119920] 0 2024-06-06T08:43:24.313 INFO:tasks.workunit.client.0.smithi027.stdout:4/305: write d0/d1/d6/d8/d21/f4a [605382,35176] 0 2024-06-06T08:43:24.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/306: write d0/d1/f2 [4061434,100452] 0 2024-06-06T08:43:24.314 INFO:tasks.workunit.client.0.smithi027.stdout:9/273: sync 2024-06-06T08:43:24.315 INFO:tasks.workunit.client.0.smithi027.stdout:6/370: rename d4/d1a/d2e/l37 to d4/de/d45/l7b 0 2024-06-06T08:43:24.317 INFO:tasks.workunit.client.0.smithi027.stdout:4/307: creat d0/d1d/d38/d3b/f68 x:0 0 0 2024-06-06T08:43:24.329 INFO:tasks.workunit.client.0.smithi027.stdout:4/308: getdents d0/d1d/d38/d3b 0 2024-06-06T08:43:24.331 INFO:tasks.workunit.client.0.smithi027.stdout:7/432: dread d17/d32/d46/d4f/d5e/f8b [0,4194304] 0 2024-06-06T08:43:24.334 INFO:tasks.workunit.client.0.smithi027.stdout:7/433: symlink d17/d32/d46/d4f/d5e/l99 0 2024-06-06T08:43:24.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/320: dread d15/d18/d23/f2e [0,4194304] 0 2024-06-06T08:43:24.338 INFO:tasks.workunit.client.0.smithi027.stdout:9/274: dwrite d3/f5f [4194304,4194304] 0 2024-06-06T08:43:24.339 INFO:tasks.workunit.client.0.smithi027.stdout:2/401: dwrite d6/dc/d3f/f71 [4194304,4194304] 0 2024-06-06T08:43:24.339 INFO:tasks.workunit.client.0.smithi027.stdout:2/402: dread - d6/d1e/d2b/d5a/d7b/f7e zero size 2024-06-06T08:43:24.339 INFO:tasks.workunit.client.0.smithi027.stdout:6/371: rename d4/d9/dc/l60 to d4/de/d45/d6a/l7c 0 2024-06-06T08:43:24.339 INFO:tasks.workunit.client.0.smithi027.stdout:6/372: chown d4/d1a/d2e 34880 1 2024-06-06T08:43:24.340 INFO:tasks.workunit.client.0.smithi027.stdout:7/434: truncate d17/d32/d46/d51/f8e 768848 0 2024-06-06T08:43:24.340 INFO:tasks.workunit.client.0.smithi027.stdout:7/435: dread - d17/d32/d46/d4f/f8f zero size 2024-06-06T08:43:24.340 INFO:tasks.workunit.client.0.smithi027.stdout:7/436: write d17/d32/d46/d4f/d5e/f66 [4922149,105706] 0 2024-06-06T08:43:24.340 INFO:tasks.workunit.client.0.smithi027.stdout:9/275: symlink d3/da/dc/df/l62 0 2024-06-06T08:43:24.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/260: dwrite d1/f6 [0,4194304] 0 2024-06-06T08:43:24.341 INFO:tasks.workunit.client.0.smithi027.stdout:5/428: dwrite d0/da/d37/d2b/f4a [0,4194304] 0 2024-06-06T08:43:24.342 INFO:tasks.workunit.client.0.smithi027.stdout:5/429: stat d0/da/db/f19 0 2024-06-06T08:43:24.342 INFO:tasks.workunit.client.0.smithi027.stdout:0/321: symlink d15/d18/d23/l6c 0 2024-06-06T08:43:24.342 INFO:tasks.workunit.client.0.smithi027.stdout:7/437: creat d17/d32/d46/d83/d77/d8c/d90/f9a x:0 0 0 2024-06-06T08:43:24.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/403: mkdir d6/dc/d1d/d4d/d8e 0 2024-06-06T08:43:24.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/404: fdatasync d6/dc/d1d/f4b 0 2024-06-06T08:43:24.344 INFO:tasks.workunit.client.0.smithi027.stdout:4/309: rename d0/d1/f5b to d0/d1d/f69 0 2024-06-06T08:43:24.345 INFO:tasks.workunit.client.0.smithi027.stdout:4/310: read d0/d1/f4f [22644,115651] 0 2024-06-06T08:43:24.346 INFO:tasks.workunit.client.0.smithi027.stdout:6/373: mknod d4/d9/dc/d1c/d52/c7d 0 2024-06-06T08:43:24.350 INFO:tasks.workunit.client.0.smithi027.stdout:5/430: getdents d0/da/db/d46/d7f 0 2024-06-06T08:43:24.350 INFO:tasks.workunit.client.0.smithi027.stdout:5/431: fsync d0/da/db/d46/d77/f79 0 2024-06-06T08:43:24.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/322: unlink d15/f16 0 2024-06-06T08:43:24.354 INFO:tasks.workunit.client.0.smithi027.stdout:7/438: creat d17/f9b x:0 0 0 2024-06-06T08:43:24.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/311: write d0/d1d/d2a/f57 [3480234,126306] 0 2024-06-06T08:43:24.354 INFO:tasks.workunit.client.0.smithi027.stdout:9/276: write d3/da/dc/f2a [641559,23591] 0 2024-06-06T08:43:24.356 INFO:tasks.workunit.client.0.smithi027.stdout:2/405: rename d6/dc/d3f/f7a to d6/f8f 0 2024-06-06T08:43:24.356 INFO:tasks.workunit.client.0.smithi027.stdout:2/406: fdatasync d6/dc/f29 0 2024-06-06T08:43:24.357 INFO:tasks.workunit.client.0.smithi027.stdout:9/277: symlink d3/da/dc/df/l63 0 2024-06-06T08:43:24.357 INFO:tasks.workunit.client.0.smithi027.stdout:9/278: chown d3/da/f2c 465408 1 2024-06-06T08:43:24.358 INFO:tasks.workunit.client.0.smithi027.stdout:3/261: dread d1/f6 [0,4194304] 0 2024-06-06T08:43:24.363 INFO:tasks.workunit.client.0.smithi027.stdout:5/432: creat d0/da/db/d1a/d5a/f83 x:0 0 0 2024-06-06T08:43:24.365 INFO:tasks.workunit.client.0.smithi027.stdout:6/374: dwrite d4/d9/dc/d18/f2f [4194304,4194304] 0 2024-06-06T08:43:24.371 INFO:tasks.workunit.client.0.smithi027.stdout:2/407: symlink d6/d3b/l90 0 2024-06-06T08:43:24.371 INFO:tasks.workunit.client.0.smithi027.stdout:9/279: unlink c2 0 2024-06-06T08:43:24.372 INFO:tasks.workunit.client.0.smithi027.stdout:2/408: dread - d6/dc/d3f/d77/d55/f6d zero size 2024-06-06T08:43:24.372 INFO:tasks.workunit.client.0.smithi027.stdout:0/323: dwrite d15/f22 [0,4194304] 0 2024-06-06T08:43:24.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/262: symlink d1/d1f/d25/l4a 0 2024-06-06T08:43:24.374 INFO:tasks.workunit.client.0.smithi027.stdout:8/270: sync 2024-06-06T08:43:24.374 INFO:tasks.workunit.client.0.smithi027.stdout:8/271: write d7/db/f1d [1636899,53859] 0 2024-06-06T08:43:24.374 INFO:tasks.workunit.client.0.smithi027.stdout:8/272: chown d7/dc/d37/d46 2561858 1 2024-06-06T08:43:24.377 INFO:tasks.workunit.client.0.smithi027.stdout:5/433: rename d0/da/db/d1a/f7c to d0/da/db/d46/d54/f84 0 2024-06-06T08:43:24.377 INFO:tasks.workunit.client.0.smithi027.stdout:5/434: chown d0/da/d37/d2b/f4a 1 1 2024-06-06T08:43:24.381 INFO:tasks.workunit.client.0.smithi027.stdout:8/273: unlink d7/dc/d36/d3c/c3f 0 2024-06-06T08:43:24.381 INFO:tasks.workunit.client.0.smithi027.stdout:8/274: fsync d7/db/f27 0 2024-06-06T08:43:24.382 INFO:tasks.workunit.client.0.smithi027.stdout:9/280: write d3/f5f [3610954,19586] 0 2024-06-06T08:43:24.384 INFO:tasks.workunit.client.0.smithi027.stdout:5/435: link d0/l14 d0/da/d47/l85 0 2024-06-06T08:43:24.384 INFO:tasks.workunit.client.0.smithi027.stdout:5/436: chown d0/l14 17 1 2024-06-06T08:43:24.384 INFO:tasks.workunit.client.0.smithi027.stdout:5/437: write d0/da/d37/f41 [2873,116092] 0 2024-06-06T08:43:24.394 INFO:tasks.workunit.client.0.smithi027.stdout:2/409: dwrite d6/dc/f44 [0,4194304] 0 2024-06-06T08:43:24.394 INFO:tasks.workunit.client.0.smithi027.stdout:2/410: dread - d6/f8b zero size 2024-06-06T08:43:24.394 INFO:tasks.workunit.client.0.smithi027.stdout:2/411: chown d6/d3b/d61/l7c 5203010 1 2024-06-06T08:43:24.395 INFO:tasks.workunit.client.0.smithi027.stdout:5/438: creat d0/da/db/d46/d4c/d70/f86 x:0 0 0 2024-06-06T08:43:24.395 INFO:tasks.workunit.client.0.smithi027.stdout:5/439: dread - d0/da/f72 zero size 2024-06-06T08:43:24.395 INFO:tasks.workunit.client.0.smithi027.stdout:7/439: link d17/d32/d46/d4f/d5e/d7b/f88 d17/d32/d46/d83/f9c 0 2024-06-06T08:43:24.397 INFO:tasks.workunit.client.0.smithi027.stdout:0/324: dwrite d15/d18/d23/d38/f51 [8388608,4194304] 0 2024-06-06T08:43:24.401 INFO:tasks.workunit.client.0.smithi027.stdout:4/312: rmdir d0/d1/d6/d8 39 2024-06-06T08:43:24.401 INFO:tasks.workunit.client.0.smithi027.stdout:4/313: read d0/d1/f4f [48,2248] 0 2024-06-06T08:43:24.401 INFO:tasks.workunit.client.0.smithi027.stdout:4/314: chown d0/c1f 33 1 2024-06-06T08:43:24.401 INFO:tasks.workunit.client.0.smithi027.stdout:4/315: write d0/d1d/f59 [91235,976] 0 2024-06-06T08:43:24.401 INFO:tasks.workunit.client.0.smithi027.stdout:4/316: chown d0/d1/d6/f33 6872874 1 2024-06-06T08:43:24.401 INFO:tasks.workunit.client.0.smithi027.stdout:7/440: rename d17/d32/d46/d51/d52/d70/c97 to d17/d32/d46/d51/c9d 0 2024-06-06T08:43:24.403 INFO:tasks.workunit.client.0.smithi027.stdout:3/263: dwrite d1/f20 [4194304,4194304] 0 2024-06-06T08:43:24.403 INFO:tasks.workunit.client.0.smithi027.stdout:3/264: fdatasync d1/d8/f16 0 2024-06-06T08:43:24.404 INFO:tasks.workunit.client.0.smithi027.stdout:9/281: dread d3/da/dc/f30 [0,4194304] 0 2024-06-06T08:43:24.404 INFO:tasks.workunit.client.0.smithi027.stdout:7/441: link d17/d32/d61/c57 d17/d32/d46/d83/d2a/c9e 0 2024-06-06T08:43:24.405 INFO:tasks.workunit.client.0.smithi027.stdout:1/339: sync 2024-06-06T08:43:24.405 INFO:tasks.workunit.client.0.smithi027.stdout:1/340: dread - dc/d67/d69/d32/f5b zero size 2024-06-06T08:43:24.405 INFO:tasks.workunit.client.0.smithi027.stdout:3/265: symlink d1/da/d44/l4b 0 2024-06-06T08:43:24.407 INFO:tasks.workunit.client.0.smithi027.stdout:4/317: creat d0/d1/d6/d8/f6a x:0 0 0 2024-06-06T08:43:24.408 INFO:tasks.workunit.client.0.smithi027.stdout:5/440: read d0/f44 [1638370,23398] 0 2024-06-06T08:43:24.409 INFO:tasks.workunit.client.0.smithi027.stdout:8/275: dwrite d7/db/f19 [0,4194304] 0 2024-06-06T08:43:24.411 INFO:tasks.workunit.client.0.smithi027.stdout:5/441: symlink d0/da/db/d46/d4c/l87 0 2024-06-06T08:43:24.412 INFO:tasks.workunit.client.0.smithi027.stdout:5/442: getdents d0/da/db/d46/d77 0 2024-06-06T08:43:24.412 INFO:tasks.workunit.client.0.smithi027.stdout:8/276: mknod d7/dc/d36/c57 0 2024-06-06T08:43:24.412 INFO:tasks.workunit.client.0.smithi027.stdout:8/277: stat d7/dc/d36/c4a 0 2024-06-06T08:43:24.412 INFO:tasks.workunit.client.0.smithi027.stdout:8/278: write d7/db/f1d [403962,93397] 0 2024-06-06T08:43:24.412 INFO:tasks.workunit.client.0.smithi027.stdout:9/282: rmdir d3/da/dc 39 2024-06-06T08:43:24.413 INFO:tasks.workunit.client.0.smithi027.stdout:5/443: dread d0/da/f13 [4194304,4194304] 0 2024-06-06T08:43:24.413 INFO:tasks.workunit.client.0.smithi027.stdout:8/279: rmdir d7/dc/d37 39 2024-06-06T08:43:24.414 INFO:tasks.workunit.client.0.smithi027.stdout:8/280: fsync d7/f4e 0 2024-06-06T08:43:24.414 INFO:tasks.workunit.client.0.smithi027.stdout:5/444: mknod d0/da/db/d46/d4c/c88 0 2024-06-06T08:43:24.416 INFO:tasks.workunit.client.0.smithi027.stdout:9/283: creat d3/da/dc/df/d34/f64 x:0 0 0 2024-06-06T08:43:24.416 INFO:tasks.workunit.client.0.smithi027.stdout:2/412: write d6/d3b/f52 [2397086,54297] 0 2024-06-06T08:43:24.417 INFO:tasks.workunit.client.0.smithi027.stdout:9/284: creat d3/da/dc/df/d34/f65 x:0 0 0 2024-06-06T08:43:24.419 INFO:tasks.workunit.client.0.smithi027.stdout:2/413: rename d6/dd/d6b/f88 to d6/d1e/d2b/f91 0 2024-06-06T08:43:24.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/414: symlink d6/d3b/d80/l92 0 2024-06-06T08:43:24.424 INFO:tasks.workunit.client.0.smithi027.stdout:2/415: creat d6/dc/d3f/d48/f93 x:0 0 0 2024-06-06T08:43:24.424 INFO:tasks.workunit.client.0.smithi027.stdout:2/416: write d6/dc/d3f/d77/f4c [265972,62658] 0 2024-06-06T08:43:24.425 INFO:tasks.workunit.client.0.smithi027.stdout:7/442: unlink d17/d32/d46/d4f/l65 0 2024-06-06T08:43:24.435 INFO:tasks.workunit.client.0.smithi027.stdout:3/266: dwrite d1/d8/d22/d2d/f3d [0,4194304] 0 2024-06-06T08:43:24.436 INFO:tasks.workunit.client.0.smithi027.stdout:1/341: dwrite dc/f36 [4194304,4194304] 0 2024-06-06T08:43:24.437 INFO:tasks.workunit.client.0.smithi027.stdout:4/318: dwrite d0/f1c [0,4194304] 0 2024-06-06T08:43:24.438 INFO:tasks.workunit.client.0.smithi027.stdout:3/267: link d1/d8/d22/d27/f32 d1/d1f/d25/f4c 0 2024-06-06T08:43:24.442 INFO:tasks.workunit.client.0.smithi027.stdout:5/445: dwrite d0/da/d37/d2b/f4a [0,4194304] 0 2024-06-06T08:43:24.442 INFO:tasks.workunit.client.0.smithi027.stdout:5/446: readlink d0/l23 0 2024-06-06T08:43:24.442 INFO:tasks.workunit.client.0.smithi027.stdout:5/447: fsync d0/da/db/f1b 0 2024-06-06T08:43:24.442 INFO:tasks.workunit.client.0.smithi027.stdout:4/319: creat d0/d1d/d34/f6b x:0 0 0 2024-06-06T08:43:24.442 INFO:tasks.workunit.client.0.smithi027.stdout:3/268: mkdir d1/d1f/d4d 0 2024-06-06T08:43:24.442 INFO:tasks.workunit.client.0.smithi027.stdout:7/443: dread d17/d32/d46/f41 [0,4194304] 0 2024-06-06T08:43:24.443 INFO:tasks.workunit.client.0.smithi027.stdout:9/285: dwrite d3/da/dc/df/d34/f51 [0,4194304] 0 2024-06-06T08:43:24.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/286: mknod d3/da/dc/df/d23/c66 0 2024-06-06T08:43:24.446 INFO:tasks.workunit.client.0.smithi027.stdout:5/448: link d0/da/db/l58 d0/d6a/l89 0 2024-06-06T08:43:24.447 INFO:tasks.workunit.client.0.smithi027.stdout:4/320: truncate d0/d1/f12 620019 0 2024-06-06T08:43:24.448 INFO:tasks.workunit.client.0.smithi027.stdout:3/269: mknod d1/da/d44/c4e 0 2024-06-06T08:43:24.450 INFO:tasks.workunit.client.0.smithi027.stdout:5/449: rename d0/da/d20/f60 to d0/d6a/f8a 0 2024-06-06T08:43:24.451 INFO:tasks.workunit.client.0.smithi027.stdout:4/321: mknod d0/c6c 0 2024-06-06T08:43:24.453 INFO:tasks.workunit.client.0.smithi027.stdout:4/322: creat d0/d63/f6d x:0 0 0 2024-06-06T08:43:24.454 INFO:tasks.workunit.client.0.smithi027.stdout:4/323: write d0/d63/f6d [821181,78844] 0 2024-06-06T08:43:24.454 INFO:tasks.workunit.client.0.smithi027.stdout:4/324: fsync d0/d1d/d2a/f5e 0 2024-06-06T08:43:24.454 INFO:tasks.workunit.client.0.smithi027.stdout:7/444: dread d17/f1b [0,4194304] 0 2024-06-06T08:43:24.457 INFO:tasks.workunit.client.0.smithi027.stdout:7/445: symlink d17/d32/d61/l9f 0 2024-06-06T08:43:24.464 INFO:tasks.workunit.client.0.smithi027.stdout:9/287: dwrite d3/f15 [4194304,4194304] 0 2024-06-06T08:43:24.465 INFO:tasks.workunit.client.0.smithi027.stdout:9/288: creat d3/da/dc/df/d23/d36/d3a/f67 x:0 0 0 2024-06-06T08:43:24.466 INFO:tasks.workunit.client.0.smithi027.stdout:5/450: dread d0/da/db/f51 [0,4194304] 0 2024-06-06T08:43:24.467 INFO:tasks.workunit.client.0.smithi027.stdout:5/451: chown d0/d6a/l89 93 1 2024-06-06T08:43:24.468 INFO:tasks.workunit.client.0.smithi027.stdout:3/270: dwrite d1/f41 [0,4194304] 0 2024-06-06T08:43:24.468 INFO:tasks.workunit.client.0.smithi027.stdout:9/289: creat d3/da/dc/df/d23/d36/d3a/f68 x:0 0 0 2024-06-06T08:43:24.468 INFO:tasks.workunit.client.0.smithi027.stdout:9/290: chown d3/da/dc/f2e 375669 1 2024-06-06T08:43:24.468 INFO:tasks.workunit.client.0.smithi027.stdout:5/452: rmdir d0/da/db/d46/d4c/d70 39 2024-06-06T08:43:24.468 INFO:tasks.workunit.client.0.smithi027.stdout:5/453: dread - d0/da/db/f64 zero size 2024-06-06T08:43:24.468 INFO:tasks.workunit.client.0.smithi027.stdout:9/291: getdents d3/da/dc/df 0 2024-06-06T08:43:24.479 INFO:tasks.workunit.client.0.smithi027.stdout:5/454: read d0/da/d37/d2b/f65 [2087877,23094] 0 2024-06-06T08:43:24.480 INFO:tasks.workunit.client.0.smithi027.stdout:3/271: write d1/f13 [979916,92565] 0 2024-06-06T08:43:24.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/455: symlink d0/da/d47/l8b 0 2024-06-06T08:43:24.481 INFO:tasks.workunit.client.0.smithi027.stdout:3/272: truncate d1/f1e 138599 0 2024-06-06T08:43:24.486 INFO:tasks.workunit.client.0.smithi027.stdout:7/446: write d17/d32/d46/d4f/f50 [3359481,21666] 0 2024-06-06T08:43:24.486 INFO:tasks.workunit.client.0.smithi027.stdout:4/325: dwrite d0/d1/d6/d8/f58 [4194304,4194304] 0 2024-06-06T08:43:24.486 INFO:tasks.workunit.client.0.smithi027.stdout:4/326: stat d0/d1d/d38/d3b/f68 0 2024-06-06T08:43:24.486 INFO:tasks.workunit.client.0.smithi027.stdout:7/447: write d17/d32/d46/d83/d2a/f69 [792461,26478] 0 2024-06-06T08:43:24.487 INFO:tasks.workunit.client.0.smithi027.stdout:4/327: chown d0/d1d/d34/d5a 47 1 2024-06-06T08:43:24.487 INFO:tasks.workunit.client.0.smithi027.stdout:7/448: chown d17/d32/d46/d83/d77/d8c/d90/f9a 76236465 1 2024-06-06T08:43:24.487 INFO:tasks.workunit.client.0.smithi027.stdout:7/449: chown d17/d32/d46/d51/f8e 29210 1 2024-06-06T08:43:24.487 INFO:tasks.workunit.client.0.smithi027.stdout:4/328: mkdir d0/d1d/d34/d5a/d6e 0 2024-06-06T08:43:24.489 INFO:tasks.workunit.client.0.smithi027.stdout:4/329: mknod d0/d63/c6f 0 2024-06-06T08:43:24.490 INFO:tasks.workunit.client.0.smithi027.stdout:4/330: creat d0/d1d/d38/d3b/f70 x:0 0 0 2024-06-06T08:43:24.490 INFO:tasks.workunit.client.0.smithi027.stdout:4/331: readlink d0/l9 0 2024-06-06T08:43:24.490 INFO:tasks.workunit.client.0.smithi027.stdout:4/332: chown d0/d1d/d34/d5a/c67 3 1 2024-06-06T08:43:24.490 INFO:tasks.workunit.client.0.smithi027.stdout:4/333: write d0/d1d/d34/f6b [665499,56221] 0 2024-06-06T08:43:24.491 INFO:tasks.workunit.client.0.smithi027.stdout:4/334: symlink d0/d1d/d60/l71 0 2024-06-06T08:43:24.492 INFO:tasks.workunit.client.0.smithi027.stdout:9/292: dwrite d3/da/dc/df/f4f [0,4194304] 0 2024-06-06T08:43:24.492 INFO:tasks.workunit.client.0.smithi027.stdout:4/335: unlink d0/f20 0 2024-06-06T08:43:24.492 INFO:tasks.workunit.client.0.smithi027.stdout:4/336: truncate d0/d1d/d2a/f57 4920877 0 2024-06-06T08:43:24.493 INFO:tasks.workunit.client.0.smithi027.stdout:9/293: mknod d3/d4e/c69 0 2024-06-06T08:43:24.494 INFO:tasks.workunit.client.0.smithi027.stdout:9/294: readlink d3/da/l1e 0 2024-06-06T08:43:24.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/295: rmdir d3 39 2024-06-06T08:43:24.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/296: write d3/f5f [3233197,24231] 0 2024-06-06T08:43:24.500 INFO:tasks.workunit.client.0.smithi027.stdout:9/297: read d3/da/dc/df/d23/d36/f37 [3054435,47202] 0 2024-06-06T08:43:24.504 INFO:tasks.workunit.client.0.smithi027.stdout:3/273: write d1/f6 [3390574,755] 0 2024-06-06T08:43:24.504 INFO:tasks.workunit.client.0.smithi027.stdout:7/450: dwrite d17/d32/d46/d83/d2a/d3b/f5b [0,4194304] 0 2024-06-06T08:43:24.505 INFO:tasks.workunit.client.0.smithi027.stdout:5/456: dwrite d0/da/db/f64 [0,4194304] 0 2024-06-06T08:43:24.510 INFO:tasks.workunit.client.0.smithi027.stdout:3/274: rmdir d1/d8 39 2024-06-06T08:43:24.513 INFO:tasks.workunit.client.0.smithi027.stdout:3/275: unlink d1/d8/d22/d2d/f48 0 2024-06-06T08:43:24.515 INFO:tasks.workunit.client.0.smithi027.stdout:3/276: mknod d1/d1f/d25/c4f 0 2024-06-06T08:43:24.515 INFO:tasks.workunit.client.0.smithi027.stdout:3/277: fdatasync d1/f2 0 2024-06-06T08:43:24.519 INFO:tasks.workunit.client.0.smithi027.stdout:9/298: dwrite d3/da/dc/df/d23/d36/f37 [0,4194304] 0 2024-06-06T08:43:24.520 INFO:tasks.workunit.client.0.smithi027.stdout:9/299: symlink d3/da/dc/df/d34/l6a 0 2024-06-06T08:43:24.523 INFO:tasks.workunit.client.0.smithi027.stdout:7/451: dwrite f11 [0,4194304] 0 2024-06-06T08:43:24.523 INFO:tasks.workunit.client.0.smithi027.stdout:7/452: chown d17/d32/d46/d4f/f55 192 1 2024-06-06T08:43:24.523 INFO:tasks.workunit.client.0.smithi027.stdout:7/453: write d17/d32/d46/d83/f45 [204844,2071] 0 2024-06-06T08:43:24.524 INFO:tasks.workunit.client.0.smithi027.stdout:7/454: rmdir d17/d32/d46/d4f/d5e 39 2024-06-06T08:43:24.526 INFO:tasks.workunit.client.0.smithi027.stdout:7/455: symlink d17/d32/d6c/la0 0 2024-06-06T08:43:24.526 INFO:tasks.workunit.client.0.smithi027.stdout:7/456: read d17/d32/d46/d51/d52/f73 [4062083,101543] 0 2024-06-06T08:43:24.526 INFO:tasks.workunit.client.0.smithi027.stdout:7/457: dread - d17/f9b zero size 2024-06-06T08:43:24.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/300: write d3/da/dc/df/f4f [2430365,41410] 0 2024-06-06T08:43:24.529 INFO:tasks.workunit.client.0.smithi027.stdout:4/337: rename d0/d1d/d38/c31 to d0/d1/d6/d8/c72 0 2024-06-06T08:43:24.529 INFO:tasks.workunit.client.0.smithi027.stdout:4/338: fdatasync d0/d1/d6/d8/d21/f30 0 2024-06-06T08:43:24.530 INFO:tasks.workunit.client.0.smithi027.stdout:4/339: write d0/d63/f6d [796306,13703] 0 2024-06-06T08:43:24.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/301: creat d3/da/dc/df/d23/d36/f6b x:0 0 0 2024-06-06T08:43:24.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/302: chown d3/da/dc/df/f28 12035 1 2024-06-06T08:43:24.531 INFO:tasks.workunit.client.0.smithi027.stdout:5/457: rename d0/da/db/f19 to d0/da/db/d46/d77/f8c 0 2024-06-06T08:43:24.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/303: unlink d3/da/dc/l22 0 2024-06-06T08:43:24.533 INFO:tasks.workunit.client.0.smithi027.stdout:5/458: symlink d0/da/db/d46/l8d 0 2024-06-06T08:43:24.535 INFO:tasks.workunit.client.0.smithi027.stdout:3/278: dwrite d1/f29 [0,4194304] 0 2024-06-06T08:43:24.535 INFO:tasks.workunit.client.0.smithi027.stdout:3/279: dread - d1/d8/d22/f46 zero size 2024-06-06T08:43:24.536 INFO:tasks.workunit.client.0.smithi027.stdout:5/459: creat d0/da/db/d46/d7f/f8e x:0 0 0 2024-06-06T08:43:24.538 INFO:tasks.workunit.client.0.smithi027.stdout:5/460: link d0/da/d37/d2b/f65 d0/da/db/d46/d77/f8f 0 2024-06-06T08:43:24.538 INFO:tasks.workunit.client.0.smithi027.stdout:5/461: write d0/da/db/d46/d54/f84 [837040,100308] 0 2024-06-06T08:43:24.540 INFO:tasks.workunit.client.0.smithi027.stdout:5/462: getdents d0/da/db/d46/d54 0 2024-06-06T08:43:24.541 INFO:tasks.workunit.client.0.smithi027.stdout:7/458: dwrite d17/d32/f3a [8388608,4194304] 0 2024-06-06T08:43:24.541 INFO:tasks.workunit.client.0.smithi027.stdout:7/459: truncate d17/f60 27645 0 2024-06-06T08:43:24.541 INFO:tasks.workunit.client.0.smithi027.stdout:7/460: write d17/d32/d46/d83/f45 [728707,77478] 0 2024-06-06T08:43:24.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/304: write d3/f5f [6537569,32851] 0 2024-06-06T08:43:24.544 INFO:tasks.workunit.client.0.smithi027.stdout:7/461: unlink d17/d32/d46/d83/d2a/c30 0 2024-06-06T08:43:24.544 INFO:tasks.workunit.client.0.smithi027.stdout:3/280: write d1/d8/d22/d27/f32 [3622322,29639] 0 2024-06-06T08:43:24.545 INFO:tasks.workunit.client.0.smithi027.stdout:3/281: read d1/f2 [8364886,6559] 0 2024-06-06T08:43:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:9/305: creat d3/da/dc/df/d23/d36/f6c x:0 0 0 2024-06-06T08:43:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:9/306: truncate d3/da/dc/df/d23/d36/f61 705287 0 2024-06-06T08:43:24.546 INFO:tasks.workunit.client.0.smithi027.stdout:7/462: rmdir d17/d32/d46/d4f/d5e/d7b 39 2024-06-06T08:43:24.547 INFO:tasks.workunit.client.0.smithi027.stdout:9/307: link d3/da/dc/df/d23/d36/f5a d3/da/dc/df/d23/d36/d3a/f6d 0 2024-06-06T08:43:24.549 INFO:tasks.workunit.client.0.smithi027.stdout:9/308: symlink d3/d4e/l6e 0 2024-06-06T08:43:24.553 INFO:tasks.workunit.client.0.smithi027.stdout:4/340: rename d0/d1/d6/d8/c1e to d0/d1d/d38/d3b/c73 0 2024-06-06T08:43:24.556 INFO:tasks.workunit.client.0.smithi027.stdout:3/282: dread d1/f20 [0,4194304] 0 2024-06-06T08:43:24.567 INFO:tasks.workunit.client.0.smithi027.stdout:5/463: dwrite d0/da/db/d46/d54/f84 [0,4194304] 0 2024-06-06T08:43:24.567 INFO:tasks.workunit.client.0.smithi027.stdout:5/464: fsync d0/da/db/d46/d54/f84 0 2024-06-06T08:43:24.568 INFO:tasks.workunit.client.0.smithi027.stdout:5/465: chown d0/da/d20/c2e 16300435 1 2024-06-06T08:43:24.571 INFO:tasks.workunit.client.0.smithi027.stdout:5/466: creat d0/da/db/d46/d54/f90 x:0 0 0 2024-06-06T08:43:24.580 INFO:tasks.workunit.client.0.smithi027.stdout:5/467: write d0/da/db/f51 [3622432,129483] 0 2024-06-06T08:43:24.583 INFO:tasks.workunit.client.0.smithi027.stdout:7/463: rename d17/d32/d61/c86 to d17/d32/d6c/ca1 0 2024-06-06T08:43:24.585 INFO:tasks.workunit.client.0.smithi027.stdout:7/464: getdents d17/d47 0 2024-06-06T08:43:24.587 INFO:tasks.workunit.client.0.smithi027.stdout:7/465: link d17/d32/d46/d51/f54 d17/d32/d46/d83/d77/fa2 0 2024-06-06T08:43:24.589 INFO:tasks.workunit.client.0.smithi027.stdout:5/468: dread d0/da/d37/f48 [0,4194304] 0 2024-06-06T08:43:24.589 INFO:tasks.workunit.client.0.smithi027.stdout:5/469: chown d0/da/db/d1a/d7e 600873364 1 2024-06-06T08:43:24.597 INFO:tasks.workunit.client.0.smithi027.stdout:4/341: rename d0/d1/f1a to d0/d1/d6/d8/d21/f74 0 2024-06-06T08:43:24.598 INFO:tasks.workunit.client.0.smithi027.stdout:4/342: mkdir d0/d1/d75 0 2024-06-06T08:43:24.599 INFO:tasks.workunit.client.0.smithi027.stdout:5/470: dread d0/f4e [0,4194304] 0 2024-06-06T08:43:24.599 INFO:tasks.workunit.client.0.smithi027.stdout:5/471: chown d0/da/db/f17 40788 1 2024-06-06T08:43:24.599 INFO:tasks.workunit.client.0.smithi027.stdout:4/343: dread - d0/f55 zero size 2024-06-06T08:43:24.602 INFO:tasks.workunit.client.0.smithi027.stdout:3/283: rename d1/d8/d3c to d1/d8/d22/d27/d50 0 2024-06-06T08:43:24.604 INFO:tasks.workunit.client.0.smithi027.stdout:4/344: rmdir d0/d1/d6 39 2024-06-06T08:43:24.604 INFO:tasks.workunit.client.0.smithi027.stdout:4/345: chown d0/d1d/d34/f40 324135 1 2024-06-06T08:43:24.608 INFO:tasks.workunit.client.0.smithi027.stdout:7/466: dwrite f15 [0,4194304] 0 2024-06-06T08:43:24.610 INFO:tasks.workunit.client.0.smithi027.stdout:7/467: creat d17/d32/d46/d51/d52/d70/d87/fa3 x:0 0 0 2024-06-06T08:43:24.614 INFO:tasks.workunit.client.0.smithi027.stdout:5/472: dread d0/da/db/f51 [0,4194304] 0 2024-06-06T08:43:24.615 INFO:tasks.workunit.client.0.smithi027.stdout:5/473: creat d0/da/d37/d2b/f91 x:0 0 0 2024-06-06T08:43:24.620 INFO:tasks.workunit.client.0.smithi027.stdout:7/468: write d17/f26 [40592,53289] 0 2024-06-06T08:43:24.620 INFO:tasks.workunit.client.0.smithi027.stdout:7/469: write d17/d32/d61/f74 [1555624,47706] 0 2024-06-06T08:43:24.620 INFO:tasks.workunit.client.0.smithi027.stdout:7/470: truncate d17/d32/d61/f74 2566552 0 2024-06-06T08:43:24.624 INFO:tasks.workunit.client.0.smithi027.stdout:7/471: dread d17/d32/d46/d83/d2a/d3b/f56 [0,4194304] 0 2024-06-06T08:43:24.625 INFO:tasks.workunit.client.0.smithi027.stdout:7/472: chown d17/f60 3743 1 2024-06-06T08:43:24.625 INFO:tasks.workunit.client.0.smithi027.stdout:7/473: fsync d17/d32/f2f 0 2024-06-06T08:43:24.626 INFO:tasks.workunit.client.0.smithi027.stdout:7/474: truncate d17/d32/d46/d51/d52/f85 415201 0 2024-06-06T08:43:24.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/284: dwrite d1/f2 [8388608,4194304] 0 2024-06-06T08:43:24.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/285: fdatasync d1/d8/d22/d2d/f3d 0 2024-06-06T08:43:24.627 INFO:tasks.workunit.client.0.smithi027.stdout:7/475: creat d17/fa4 x:0 0 0 2024-06-06T08:43:24.628 INFO:tasks.workunit.client.0.smithi027.stdout:3/286: creat d1/d8/d22/d27/d50/f51 x:0 0 0 2024-06-06T08:43:24.630 INFO:tasks.workunit.client.0.smithi027.stdout:3/287: rmdir d1/d8/d22/d27 39 2024-06-06T08:43:24.630 INFO:tasks.workunit.client.0.smithi027.stdout:3/288: fdatasync d1/d8/d22/f37 0 2024-06-06T08:43:24.631 INFO:tasks.workunit.client.0.smithi027.stdout:5/474: dread d0/f1 [0,4194304] 0 2024-06-06T08:43:24.631 INFO:tasks.workunit.client.0.smithi027.stdout:3/289: truncate d1/f13 4218315 0 2024-06-06T08:43:24.631 INFO:tasks.workunit.client.0.smithi027.stdout:3/290: fsync d1/f29 0 2024-06-06T08:43:24.632 INFO:tasks.workunit.client.0.smithi027.stdout:4/346: dwrite d0/d1/fb [0,4194304] 0 2024-06-06T08:43:24.632 INFO:tasks.workunit.client.0.smithi027.stdout:4/347: chown d0/d1/d6/d8/l4e 48875 1 2024-06-06T08:43:24.632 INFO:tasks.workunit.client.0.smithi027.stdout:4/348: stat d0/d1d/d2a/f57 0 2024-06-06T08:43:24.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/475: creat d0/da/f92 x:0 0 0 2024-06-06T08:43:24.633 INFO:tasks.workunit.client.0.smithi027.stdout:3/291: rmdir d1/d1f 39 2024-06-06T08:43:24.633 INFO:tasks.workunit.client.0.smithi027.stdout:3/292: chown d1/d8/d22/d2d 1038145832 1 2024-06-06T08:43:24.633 INFO:tasks.workunit.client.0.smithi027.stdout:3/293: truncate d1/da/d44/f49 91913 0 2024-06-06T08:43:24.633 INFO:tasks.workunit.client.0.smithi027.stdout:3/294: chown d1/f20 842803345 1 2024-06-06T08:43:24.637 INFO:tasks.workunit.client.0.smithi027.stdout:7/476: dread d17/f1e [0,4194304] 0 2024-06-06T08:43:24.640 INFO:tasks.workunit.client.0.smithi027.stdout:7/477: link d17/d32/d61/l9f d17/d32/d46/d83/d2a/la5 0 2024-06-06T08:43:24.640 INFO:tasks.workunit.client.0.smithi027.stdout:7/478: stat d17/d32/d46/d4f/d5e/f8b 0 2024-06-06T08:43:24.640 INFO:tasks.workunit.client.0.smithi027.stdout:7/479: read d17/d32/d46/d4f/f55 [347449,88699] 0 2024-06-06T08:43:24.642 INFO:tasks.workunit.client.0.smithi027.stdout:5/476: write d0/d6a/f8a [3554619,111451] 0 2024-06-06T08:43:24.642 INFO:tasks.workunit.client.0.smithi027.stdout:5/477: write d0/da/db/d1a/f7b [643431,63544] 0 2024-06-06T08:43:24.642 INFO:tasks.workunit.client.0.smithi027.stdout:5/478: write d0/f4 [1430476,72806] 0 2024-06-06T08:43:24.643 INFO:tasks.workunit.client.0.smithi027.stdout:7/480: read d17/d32/d46/d51/d52/f85 [170790,73292] 0 2024-06-06T08:43:24.644 INFO:tasks.workunit.client.0.smithi027.stdout:5/479: mkdir d0/d6a/d93 0 2024-06-06T08:43:24.646 INFO:tasks.workunit.client.0.smithi027.stdout:4/349: read d0/f1c [318346,67833] 0 2024-06-06T08:43:24.646 INFO:tasks.workunit.client.0.smithi027.stdout:4/350: chown d0/d1d/d60/l71 53077 1 2024-06-06T08:43:24.646 INFO:tasks.workunit.client.0.smithi027.stdout:4/351: write d0/f13 [10452684,29241] 0 2024-06-06T08:43:24.646 INFO:tasks.workunit.client.0.smithi027.stdout:4/352: dread - d0/f55 zero size 2024-06-06T08:43:24.647 INFO:tasks.workunit.client.0.smithi027.stdout:5/480: creat d0/da/d47/f94 x:0 0 0 2024-06-06T08:43:24.647 INFO:tasks.workunit.client.0.smithi027.stdout:5/481: stat d0/da/cd 0 2024-06-06T08:43:24.648 INFO:tasks.workunit.client.0.smithi027.stdout:4/353: mknod d0/d1/d6/d8/d21/c76 0 2024-06-06T08:43:24.648 INFO:tasks.workunit.client.0.smithi027.stdout:5/482: mknod d0/da/db/d46/d54/c95 0 2024-06-06T08:43:24.651 INFO:tasks.workunit.client.0.smithi027.stdout:4/354: rename d0/d1d/d34/d5a to d0/d1d/d77 0 2024-06-06T08:43:24.652 INFO:tasks.workunit.client.0.smithi027.stdout:5/483: symlink d0/da/db/d1a/d5a/l96 0 2024-06-06T08:43:24.652 INFO:tasks.workunit.client.0.smithi027.stdout:5/484: truncate d0/da/db/d1a/f78 122975 0 2024-06-06T08:43:24.654 INFO:tasks.workunit.client.0.smithi027.stdout:3/295: dwrite d1/da/d44/f49 [0,4194304] 0 2024-06-06T08:43:24.655 INFO:tasks.workunit.client.0.smithi027.stdout:5/485: truncate d0/da/d47/f68 571553 0 2024-06-06T08:43:24.655 INFO:tasks.workunit.client.0.smithi027.stdout:5/486: stat d0/d6a/c6b 0 2024-06-06T08:43:24.655 INFO:tasks.workunit.client.0.smithi027.stdout:5/487: readlink d0/da/db/d1a/d5a/l5f 0 2024-06-06T08:43:24.655 INFO:tasks.workunit.client.0.smithi027.stdout:4/355: rmdir d0/d1d/d38/d3b 39 2024-06-06T08:43:24.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/356: dread - d0/d1/d6/d4b/f56 zero size 2024-06-06T08:43:24.662 INFO:tasks.workunit.client.0.smithi027.stdout:7/481: dwrite d17/d32/d46/d4f/d5e/d7b/f88 [0,4194304] 0 2024-06-06T08:43:24.662 INFO:tasks.workunit.client.0.smithi027.stdout:7/482: fsync d17/d32/d46/d4f/d5e/f67 0 2024-06-06T08:43:24.662 INFO:tasks.workunit.client.0.smithi027.stdout:4/357: write d0/f13 [2216742,32668] 0 2024-06-06T08:43:24.664 INFO:tasks.workunit.client.0.smithi027.stdout:7/483: mknod d17/d32/d46/d83/d77/d8c/d91/ca6 0 2024-06-06T08:43:24.664 INFO:tasks.workunit.client.0.smithi027.stdout:7/484: readlink d17/d32/d46/d4f/d5e/l99 0 2024-06-06T08:43:24.664 INFO:tasks.workunit.client.0.smithi027.stdout:5/488: mkdir d0/da/db/d46/d7f/d81/d97 0 2024-06-06T08:43:24.664 INFO:tasks.workunit.client.0.smithi027.stdout:7/485: write d17/d32/f3a [1743541,60036] 0 2024-06-06T08:43:24.664 INFO:tasks.workunit.client.0.smithi027.stdout:7/486: stat d17/d32/d46/d83/d77/d8c/d91/ca6 0 2024-06-06T08:43:24.664 INFO:tasks.workunit.client.0.smithi027.stdout:7/487: fdatasync d17/d32/d46/d83/d2a/d3b/f5b 0 2024-06-06T08:43:24.665 INFO:tasks.workunit.client.0.smithi027.stdout:6/375: sync 2024-06-06T08:43:24.666 INFO:tasks.workunit.client.0.smithi027.stdout:4/358: symlink d0/d1/d6/d4b/l78 0 2024-06-06T08:43:24.670 INFO:tasks.workunit.client.0.smithi027.stdout:4/359: creat d0/d1d/f79 x:0 0 0 2024-06-06T08:43:24.670 INFO:tasks.workunit.client.0.smithi027.stdout:4/360: stat d0/d1 0 2024-06-06T08:43:24.673 INFO:tasks.workunit.client.0.smithi027.stdout:4/361: rmdir d0/d1d/d2a 39 2024-06-06T08:43:24.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/362: link d0/d63/c64 d0/d1/c7a 0 2024-06-06T08:43:24.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/363: chown d0/d1d/d77 66399513 1 2024-06-06T08:43:24.676 INFO:tasks.workunit.client.0.smithi027.stdout:6/376: getdents d4 0 2024-06-06T08:43:24.678 INFO:tasks.workunit.client.0.smithi027.stdout:6/377: creat d4/d1a/d2e/f7e x:0 0 0 2024-06-06T08:43:24.679 INFO:tasks.workunit.client.0.smithi027.stdout:6/378: creat d4/f7f x:0 0 0 2024-06-06T08:43:24.679 INFO:tasks.workunit.client.0.smithi027.stdout:6/379: chown d4/de/d45/d6a 59574 1 2024-06-06T08:43:24.686 INFO:tasks.workunit.client.0.smithi027.stdout:5/489: dwrite d0/f24 [4194304,4194304] 0 2024-06-06T08:43:24.686 INFO:tasks.workunit.client.0.smithi027.stdout:4/364: dread d0/d1d/d34/f6b [0,4194304] 0 2024-06-06T08:43:24.686 INFO:tasks.workunit.client.0.smithi027.stdout:4/365: chown d0/fd 202648419 1 2024-06-06T08:43:24.689 INFO:tasks.workunit.client.0.smithi027.stdout:5/490: mknod d0/da/db/d1a/d7e/c98 0 2024-06-06T08:43:24.691 INFO:tasks.workunit.client.0.smithi027.stdout:7/488: dwrite d17/d32/d61/f6b [4194304,4194304] 0 2024-06-06T08:43:24.691 INFO:tasks.workunit.client.0.smithi027.stdout:7/489: fdatasync d17/d32/d46/d4f/f8f 0 2024-06-06T08:43:24.692 INFO:tasks.workunit.client.0.smithi027.stdout:5/491: creat d0/da/db/d46/f99 x:0 0 0 2024-06-06T08:43:24.693 INFO:tasks.workunit.client.0.smithi027.stdout:5/492: write d0/d6a/f8a [3128111,66668] 0 2024-06-06T08:43:24.695 INFO:tasks.workunit.client.0.smithi027.stdout:7/490: mknod d17/d32/d46/d4f/d5e/ca7 0 2024-06-06T08:43:24.695 INFO:tasks.workunit.client.0.smithi027.stdout:7/491: fsync d17/f1e 0 2024-06-06T08:43:24.696 INFO:tasks.workunit.client.0.smithi027.stdout:5/493: symlink d0/da/d37/d2b/l9a 0 2024-06-06T08:43:24.696 INFO:tasks.workunit.client.0.smithi027.stdout:5/494: read - d0/da/db/d46/d7f/f8e zero size 2024-06-06T08:43:24.697 INFO:tasks.workunit.client.0.smithi027.stdout:6/380: dwrite d4/d9/d49/f62 [0,4194304] 0 2024-06-06T08:43:24.698 INFO:tasks.workunit.client.0.smithi027.stdout:5/495: creat d0/da/db/d1a/d5a/f9b x:0 0 0 2024-06-06T08:43:24.699 INFO:tasks.workunit.client.0.smithi027.stdout:5/496: creat d0/da/db/d46/d7f/d81/d97/f9c x:0 0 0 2024-06-06T08:43:24.699 INFO:tasks.workunit.client.0.smithi027.stdout:5/497: write d0/d6a/f8a [1750337,118578] 0 2024-06-06T08:43:24.699 INFO:tasks.workunit.client.0.smithi027.stdout:5/498: write d0/da/f43 [3721498,115471] 0 2024-06-06T08:43:24.700 INFO:tasks.workunit.client.0.smithi027.stdout:6/381: creat d4/d9/d6e/f80 x:0 0 0 2024-06-06T08:43:24.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/499: creat d0/da/db/d46/d77/f9d x:0 0 0 2024-06-06T08:43:24.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/382: dread d4/d1a/d2e/f3c [0,4194304] 0 2024-06-06T08:43:24.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/383: chown d4/d9/dc/d1c/f50 151319 1 2024-06-06T08:43:24.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/384: symlink d4/d9/dc/d18/l81 0 2024-06-06T08:43:24.706 INFO:tasks.workunit.client.0.smithi027.stdout:6/385: mknod d4/d1a/c82 0 2024-06-06T08:43:24.707 INFO:tasks.workunit.client.0.smithi027.stdout:6/386: dread d4/d1a/d2e/f3c [0,4194304] 0 2024-06-06T08:43:24.708 INFO:tasks.workunit.client.0.smithi027.stdout:4/366: dwrite d0/d1/d6/d8/f6a [0,4194304] 0 2024-06-06T08:43:24.710 INFO:tasks.workunit.client.0.smithi027.stdout:4/367: symlink d0/l7b 0 2024-06-06T08:43:24.710 INFO:tasks.workunit.client.0.smithi027.stdout:4/368: read d0/d1/f2 [1787721,72351] 0 2024-06-06T08:43:24.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/369: creat d0/d1/f7c x:0 0 0 2024-06-06T08:43:24.712 INFO:tasks.workunit.client.0.smithi027.stdout:4/370: chown d0/d1/d6/d4b/l78 152162258 1 2024-06-06T08:43:24.712 INFO:tasks.workunit.client.0.smithi027.stdout:4/371: truncate d0/f5c 199781 0 2024-06-06T08:43:24.713 INFO:tasks.workunit.client.0.smithi027.stdout:4/372: creat d0/d1d/d2a/f7d x:0 0 0 2024-06-06T08:43:24.718 INFO:tasks.workunit.client.0.smithi027.stdout:4/373: rename d0/d1d/d38 to d0/d63/d7e 0 2024-06-06T08:43:24.718 INFO:tasks.workunit.client.0.smithi027.stdout:4/374: write d0/d1d/f54 [714528,23099] 0 2024-06-06T08:43:24.725 INFO:tasks.workunit.client.0.smithi027.stdout:4/375: dread d0/f1c [0,4194304] 0 2024-06-06T08:43:24.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/492: dwrite d17/d32/d46/d83/d2a/d3b/f56 [0,4194304] 0 2024-06-06T08:43:24.744 INFO:tasks.workunit.client.0.smithi027.stdout:4/376: dwrite d0/f66 [0,4194304] 0 2024-06-06T08:43:24.764 INFO:tasks.workunit.client.0.smithi027.stdout:7/493: rename d17/f9b to d17/d32/d46/d51/fa8 0 2024-06-06T08:43:24.780 INFO:tasks.workunit.client.0.smithi027.stdout:7/494: dwrite d17/d32/d46/d83/d2a/f69 [0,4194304] 0 2024-06-06T08:43:24.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/495: getdents d17/d32/d46/d4f/d5e 0 2024-06-06T08:43:24.786 INFO:tasks.workunit.client.0.smithi027.stdout:7/496: read d17/d32/d46/d51/f6f [1156098,10255] 0 2024-06-06T08:43:24.800 INFO:tasks.workunit.client.0.smithi027.stdout:7/497: dwrite d17/fa4 [0,4194304] 0 2024-06-06T08:43:24.803 INFO:tasks.workunit.client.0.smithi027.stdout:7/498: creat d17/d47/fa9 x:0 0 0 2024-06-06T08:43:24.807 INFO:tasks.workunit.client.0.smithi027.stdout:7/499: write d17/f58 [128314,10646] 0 2024-06-06T08:43:24.808 INFO:tasks.workunit.client.0.smithi027.stdout:7/500: mkdir d17/d32/d46/d83/d77/d8c/d91/daa 0 2024-06-06T08:43:24.808 INFO:tasks.workunit.client.0.smithi027.stdout:7/501: chown d17/d32/d46/d51/d52/d76 1237720 1 2024-06-06T08:43:24.817 INFO:tasks.workunit.client.0.smithi027.stdout:7/502: dread d17/f1d [0,4194304] 0 2024-06-06T08:43:24.817 INFO:tasks.workunit.client.0.smithi027.stdout:7/503: chown d17/d32/d46/d51/d52/d70/c94 163 1 2024-06-06T08:43:24.817 INFO:tasks.workunit.client.0.smithi027.stdout:7/504: write d17/d32/d46/d4f/d5e/f8b [870212,54758] 0 2024-06-06T08:43:24.819 INFO:tasks.workunit.client.0.smithi027.stdout:7/505: symlink d17/d32/d46/d51/d7f/lab 0 2024-06-06T08:43:24.820 INFO:tasks.workunit.client.0.smithi027.stdout:7/506: creat d17/fac x:0 0 0 2024-06-06T08:43:24.821 INFO:tasks.workunit.client.0.smithi027.stdout:7/507: getdents d17/d32/d46/d4f/d5e/d7b/d7c 0 2024-06-06T08:43:24.827 INFO:tasks.workunit.client.0.smithi027.stdout:7/508: dread d17/d32/f6d [0,4194304] 0 2024-06-06T08:43:24.828 INFO:tasks.workunit.client.0.smithi027.stdout:7/509: stat d17/d32/d46/d83/d2a/c3c 0 2024-06-06T08:43:24.829 INFO:tasks.workunit.client.0.smithi027.stdout:7/510: mkdir d17/d32/d46/d83/d2a/d3b/dad 0 2024-06-06T08:43:24.830 INFO:tasks.workunit.client.0.smithi027.stdout:7/511: mkdir d17/d32/d46/d83/d77/d8c/dae 0 2024-06-06T08:43:24.831 INFO:tasks.workunit.client.0.smithi027.stdout:7/512: creat d17/d32/d46/d51/d52/d76/faf x:0 0 0 2024-06-06T08:43:24.847 INFO:tasks.workunit.client.0.smithi027.stdout:7/513: dwrite f14 [0,4194304] 0 2024-06-06T08:43:24.861 INFO:tasks.workunit.client.0.smithi027.stdout:7/514: dwrite d17/fac [0,4194304] 0 2024-06-06T08:43:24.878 INFO:tasks.workunit.client.0.smithi027.stdout:7/515: dwrite d17/d32/d46/d4f/d5e/f67 [0,4194304] 0 2024-06-06T08:43:24.892 INFO:tasks.workunit.client.0.smithi027.stdout:7/516: dwrite d17/d32/d46/d83/d77/d8c/d90/f9a [0,4194304] 0 2024-06-06T08:43:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:24 smithi027 ceph-mon[32335]: pgmap v97: 129 pgs: 129 active+clean; 425 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 52 MiB/s wr, 967 op/s 2024-06-06T08:43:24.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:24 smithi203 ceph-mon[36579]: pgmap v97: 129 pgs: 129 active+clean; 425 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 52 MiB/s wr, 967 op/s 2024-06-06T08:43:25.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:24 smithi176 ceph-mon[36786]: pgmap v97: 129 pgs: 129 active+clean; 425 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 52 MiB/s wr, 967 op/s 2024-06-06T08:43:25.372 INFO:tasks.workunit.client.0.smithi027.stdout:5/500: sync 2024-06-06T08:43:25.372 INFO:tasks.workunit.client.0.smithi027.stdout:6/387: sync 2024-06-06T08:43:25.372 INFO:tasks.workunit.client.0.smithi027.stdout:9/309: sync 2024-06-06T08:43:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:3/296: sync 2024-06-06T08:43:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/342: sync 2024-06-06T08:43:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/281: sync 2024-06-06T08:43:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:0/325: sync 2024-06-06T08:43:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/343: chown dc/d67/d69/d32/d37/d56 410 1 2024-06-06T08:43:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:4/377: sync 2024-06-06T08:43:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/517: sync 2024-06-06T08:43:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:2/417: sync 2024-06-06T08:43:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/518: write d17/d32/d46/d51/f54 [318468,122557] 0 2024-06-06T08:43:25.374 INFO:tasks.workunit.client.0.smithi027.stdout:6/388: read d4/d1a/d2e/d44/f5a [173986,127237] 0 2024-06-06T08:43:25.374 INFO:tasks.workunit.client.0.smithi027.stdout:5/501: creat d0/da/db/d46/d4c/d70/f9e x:0 0 0 2024-06-06T08:43:25.374 INFO:tasks.workunit.client.0.smithi027.stdout:9/310: mknod d3/da/dc/df/d23/c6f 0 2024-06-06T08:43:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:4/378: write d0/d1/f12 [558300,93257] 0 2024-06-06T08:43:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:4/379: chown d0/d1d/l2c 32 1 2024-06-06T08:43:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:4/380: fdatasync d0/d1d/d34/f4d 0 2024-06-06T08:43:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:4/381: truncate d0/f5c 323076 0 2024-06-06T08:43:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:7/519: getdents d17/d32/d46/d51/d52 0 2024-06-06T08:43:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:1/344: dread dc/d67/d69/d32/d37/f41 [0,4194304] 0 2024-06-06T08:43:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:3/297: rename d1/d8/d22/d2d/f3d to d1/d1f/d4d/f52 0 2024-06-06T08:43:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:3/298: write d1/f2 [7839508,118826] 0 2024-06-06T08:43:25.376 INFO:tasks.workunit.client.0.smithi027.stdout:5/502: creat d0/da/db/d46/d7f/d81/d97/f9f x:0 0 0 2024-06-06T08:43:25.381 INFO:tasks.workunit.client.0.smithi027.stdout:8/282: mkdir d7/db/d58 0 2024-06-06T08:43:25.381 INFO:tasks.workunit.client.0.smithi027.stdout:8/283: chown d7/d2f/f3b 188 1 2024-06-06T08:43:25.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/389: creat d4/f83 x:0 0 0 2024-06-06T08:43:25.381 INFO:tasks.workunit.client.0.smithi027.stdout:1/345: rmdir dc/de/d14/d4e 39 2024-06-06T08:43:25.381 INFO:tasks.workunit.client.0.smithi027.stdout:1/346: chown dc/de/d14/d4e/c6f 14904 1 2024-06-06T08:43:25.381 INFO:tasks.workunit.client.0.smithi027.stdout:1/347: dread - dc/de/d22/f72 zero size 2024-06-06T08:43:25.382 INFO:tasks.workunit.client.0.smithi027.stdout:3/299: symlink d1/d1f/d4d/l53 0 2024-06-06T08:43:25.383 INFO:tasks.workunit.client.0.smithi027.stdout:5/503: creat d0/da/db/fa0 x:0 0 0 2024-06-06T08:43:25.383 INFO:tasks.workunit.client.0.smithi027.stdout:5/504: chown d0/fe 1090427437 1 2024-06-06T08:43:25.384 INFO:tasks.workunit.client.0.smithi027.stdout:9/311: mknod d3/da/c70 0 2024-06-06T08:43:25.384 INFO:tasks.workunit.client.0.smithi027.stdout:9/312: dread - d3/da/dc/df/d23/d36/f5a zero size 2024-06-06T08:43:25.384 INFO:tasks.workunit.client.0.smithi027.stdout:9/313: stat d3/da/dc/df/d34/f64 0 2024-06-06T08:43:25.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/390: getdents d4/de/d45 0 2024-06-06T08:43:25.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/391: chown d4/de/f5d 4032 1 2024-06-06T08:43:25.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/392: stat d4/d9/f78 0 2024-06-06T08:43:25.385 INFO:tasks.workunit.client.0.smithi027.stdout:9/314: mkdir d3/da/dc/d71 0 2024-06-06T08:43:25.386 INFO:tasks.workunit.client.0.smithi027.stdout:8/284: symlink d7/dc/d37/d46/l59 0 2024-06-06T08:43:25.386 INFO:tasks.workunit.client.0.smithi027.stdout:8/285: write d7/dc/d37/d46/f52 [48593,41640] 0 2024-06-06T08:43:25.386 INFO:tasks.workunit.client.0.smithi027.stdout:7/520: write d17/d32/d46/d83/f9c [3260361,48274] 0 2024-06-06T08:43:25.387 INFO:tasks.workunit.client.0.smithi027.stdout:1/348: creat dc/de/d22/f79 x:0 0 0 2024-06-06T08:43:25.391 INFO:tasks.workunit.client.0.smithi027.stdout:8/286: rename d7/dc/d36/d3c/c54 to d7/db/d58/c5a 0 2024-06-06T08:43:25.394 INFO:tasks.workunit.client.0.smithi027.stdout:2/418: dwrite d6/d1e/f73 [0,4194304] 0 2024-06-06T08:43:25.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/419: chown d6/f2a 0 1 2024-06-06T08:43:25.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/420: rename d6/d1e to d6/d1e/d94 22 2024-06-06T08:43:25.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/421: fsync d6/dc/d1d/d4d/f5f 0 2024-06-06T08:43:25.397 INFO:tasks.workunit.client.0.smithi027.stdout:3/300: getdents d1/d1f/d4d 0 2024-06-06T08:43:25.399 INFO:tasks.workunit.client.0.smithi027.stdout:1/349: dread f5 [0,4194304] 0 2024-06-06T08:43:25.400 INFO:tasks.workunit.client.0.smithi027.stdout:4/382: dwrite d0/d1d/f54 [0,4194304] 0 2024-06-06T08:43:25.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/422: symlink d6/dc/d3f/d77/d55/d69/l95 0 2024-06-06T08:43:25.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/423: read - d6/d1e/d2b/d5a/d7b/f7e zero size 2024-06-06T08:43:25.408 INFO:tasks.workunit.client.0.smithi027.stdout:8/287: mknod d7/d13/d15/c5b 0 2024-06-06T08:43:25.409 INFO:tasks.workunit.client.0.smithi027.stdout:3/301: mkdir d1/da/d44/d54 0 2024-06-06T08:43:25.413 INFO:tasks.workunit.client.0.smithi027.stdout:7/521: write d17/d32/d46/d83/d2a/f69 [3302148,128487] 0 2024-06-06T08:43:25.414 INFO:tasks.workunit.client.0.smithi027.stdout:4/383: mkdir d0/d1d/d34/d7f 0 2024-06-06T08:43:25.414 INFO:tasks.workunit.client.0.smithi027.stdout:4/384: readlink d0/d1d/l2c 0 2024-06-06T08:43:25.414 INFO:tasks.workunit.client.0.smithi027.stdout:9/315: dwrite d3/da/dc/df/d23/d36/f5a [0,4194304] 0 2024-06-06T08:43:25.414 INFO:tasks.workunit.client.0.smithi027.stdout:3/302: creat d1/d8/f55 x:0 0 0 2024-06-06T08:43:25.418 INFO:tasks.workunit.client.0.smithi027.stdout:5/505: dwrite d0/da/d47/f94 [0,4194304] 0 2024-06-06T08:43:25.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/385: dread d0/d1/f12 [0,4194304] 0 2024-06-06T08:43:25.419 INFO:tasks.workunit.client.0.smithi027.stdout:1/350: rename dc/d67/d69/d32/d37/d56/f68 to dc/de/d14/d4e/f7a 0 2024-06-06T08:43:25.419 INFO:tasks.workunit.client.0.smithi027.stdout:5/506: dread d0/da/d20/f4f [4194304,4194304] 0 2024-06-06T08:43:25.419 INFO:tasks.workunit.client.0.smithi027.stdout:5/507: dread - d0/da/db/fa0 zero size 2024-06-06T08:43:25.422 INFO:tasks.workunit.client.0.smithi027.stdout:2/424: dread d6/dc/d3f/f71 [4194304,4194304] 0 2024-06-06T08:43:25.426 INFO:tasks.workunit.client.0.smithi027.stdout:4/386: creat d0/d1/d6/d4b/f80 x:0 0 0 2024-06-06T08:43:25.426 INFO:tasks.workunit.client.0.smithi027.stdout:4/387: chown d0/d63/c6f 85695735 1 2024-06-06T08:43:25.430 INFO:tasks.workunit.client.0.smithi027.stdout:5/508: dread d0/da/d20/f4f [0,4194304] 0 2024-06-06T08:43:25.430 INFO:tasks.workunit.client.0.smithi027.stdout:5/509: read - d0/da/db/d46/d77/f79 zero size 2024-06-06T08:43:25.430 INFO:tasks.workunit.client.0.smithi027.stdout:4/388: mknod d0/d1/d75/c81 0 2024-06-06T08:43:25.430 INFO:tasks.workunit.client.0.smithi027.stdout:4/389: chown d0/d1/c7a 3039 1 2024-06-06T08:43:25.430 INFO:tasks.workunit.client.0.smithi027.stdout:4/390: write d0/d1/d6/d8/d21/f2d [1444910,10863] 0 2024-06-06T08:43:25.433 INFO:tasks.workunit.client.0.smithi027.stdout:1/351: symlink dc/d67/d69/d32/d71/l7b 0 2024-06-06T08:43:25.433 INFO:tasks.workunit.client.0.smithi027.stdout:0/326: link d15/d18/d23/f27 d15/d18/d64/f6d 0 2024-06-06T08:43:25.434 INFO:tasks.workunit.client.0.smithi027.stdout:9/316: write d3/da/dc/df/d34/f51 [3990135,29556] 0 2024-06-06T08:43:25.434 INFO:tasks.workunit.client.0.smithi027.stdout:7/522: dwrite f14 [0,4194304] 0 2024-06-06T08:43:25.435 INFO:tasks.workunit.client.0.smithi027.stdout:4/391: mkdir d0/d1/d6/d8/d82 0 2024-06-06T08:43:25.436 INFO:tasks.workunit.client.0.smithi027.stdout:5/510: creat d0/da/d20/fa1 x:0 0 0 2024-06-06T08:43:25.436 INFO:tasks.workunit.client.0.smithi027.stdout:5/511: readlink d0/da/l30 0 2024-06-06T08:43:25.438 INFO:tasks.workunit.client.0.smithi027.stdout:3/303: dwrite d1/da/d44/f49 [0,4194304] 0 2024-06-06T08:43:25.438 INFO:tasks.workunit.client.0.smithi027.stdout:3/304: getdents d1/da/d44/d54 0 2024-06-06T08:43:25.439 INFO:tasks.workunit.client.0.smithi027.stdout:2/425: write d6/dc/d1d/f38 [1397119,77878] 0 2024-06-06T08:43:25.440 INFO:tasks.workunit.client.0.smithi027.stdout:7/523: creat d17/d32/d46/d4f/d5e/fb0 x:0 0 0 2024-06-06T08:43:25.441 INFO:tasks.workunit.client.0.smithi027.stdout:4/392: symlink d0/d1/l83 0 2024-06-06T08:43:25.441 INFO:tasks.workunit.client.0.smithi027.stdout:4/393: chown d0/d63/d7e/l51 8979 1 2024-06-06T08:43:25.442 INFO:tasks.workunit.client.0.smithi027.stdout:5/512: creat d0/da/fa2 x:0 0 0 2024-06-06T08:43:25.442 INFO:tasks.workunit.client.0.smithi027.stdout:9/317: dread d3/f4 [4194304,4194304] 0 2024-06-06T08:43:25.443 INFO:tasks.workunit.client.0.smithi027.stdout:3/305: rename d1/d1f/d25/c4f to d1/da/d44/d54/c56 0 2024-06-06T08:43:25.443 INFO:tasks.workunit.client.0.smithi027.stdout:9/318: symlink d3/da/dc/l72 0 2024-06-06T08:43:25.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/319: dread - d3/da/dc/df/f57 zero size 2024-06-06T08:43:25.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/320: chown d3/da/dc/df/d34/f51 354394631 1 2024-06-06T08:43:25.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/321: dread - d3/da/dc/df/d23/d36/f5d zero size 2024-06-06T08:43:25.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/513: mkdir d0/da/db/d1a/da3 0 2024-06-06T08:43:25.446 INFO:tasks.workunit.client.0.smithi027.stdout:0/327: symlink d15/d18/d23/d28/d31/l6e 0 2024-06-06T08:43:25.447 INFO:tasks.workunit.client.0.smithi027.stdout:2/426: mknod d6/d3b/d80/c96 0 2024-06-06T08:43:25.448 INFO:tasks.workunit.client.0.smithi027.stdout:5/514: creat d0/da/db/fa4 x:0 0 0 2024-06-06T08:43:25.448 INFO:tasks.workunit.client.0.smithi027.stdout:4/394: rmdir d0/d1/d6/d4b 39 2024-06-06T08:43:25.450 INFO:tasks.workunit.client.0.smithi027.stdout:2/427: read d6/d1e/f27 [174960,26770] 0 2024-06-06T08:43:25.450 INFO:tasks.workunit.client.0.smithi027.stdout:0/328: rmdir d15/d18/d23/d28/d2a/d30/d42 0 2024-06-06T08:43:25.452 INFO:tasks.workunit.client.0.smithi027.stdout:5/515: symlink d0/da/db/d46/d7f/d81/d97/la5 0 2024-06-06T08:43:25.452 INFO:tasks.workunit.client.0.smithi027.stdout:5/516: write d0/da/db/d1a/d5a/f9b [791984,12176] 0 2024-06-06T08:43:25.452 INFO:tasks.workunit.client.0.smithi027.stdout:5/517: dread - d0/da/f72 zero size 2024-06-06T08:43:25.453 INFO:tasks.workunit.client.0.smithi027.stdout:2/428: mknod d6/d1e/d2b/d5a/c97 0 2024-06-06T08:43:25.453 INFO:tasks.workunit.client.0.smithi027.stdout:2/429: readlink d6/dc/l39 0 2024-06-06T08:43:25.453 INFO:tasks.workunit.client.0.smithi027.stdout:2/430: write d6/d1e/f1f [3515079,44782] 0 2024-06-06T08:43:25.455 INFO:tasks.workunit.client.0.smithi027.stdout:0/329: mkdir d15/d18/d23/d6f 0 2024-06-06T08:43:25.455 INFO:tasks.workunit.client.0.smithi027.stdout:0/330: truncate d15/d18/d23/d38/d68/f6a 1283250 0 2024-06-06T08:43:25.457 INFO:tasks.workunit.client.0.smithi027.stdout:1/352: dwrite dc/d67/d69/d32/d71/f74 [0,4194304] 0 2024-06-06T08:43:25.460 INFO:tasks.workunit.client.0.smithi027.stdout:1/353: mknod dc/de/c7c 0 2024-06-06T08:43:25.462 INFO:tasks.workunit.client.0.smithi027.stdout:0/331: unlink c11 0 2024-06-06T08:43:25.465 INFO:tasks.workunit.client.0.smithi027.stdout:1/354: dread dc/de/d3d/f50 [0,4194304] 0 2024-06-06T08:43:25.466 INFO:tasks.workunit.client.0.smithi027.stdout:1/355: truncate dc/de/d22/f35 1541391 0 2024-06-06T08:43:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/518: dwrite d0/da/d47/f68 [0,4194304] 0 2024-06-06T08:43:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/519: fdatasync d0/da/db/d46/d7f/d81/d97/f9c 0 2024-06-06T08:43:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/520: write d0/f4 [4178510,116755] 0 2024-06-06T08:43:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/521: chown d0/da/db/d46/d54/c69 3591 1 2024-06-06T08:43:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:1/356: getdents dc/d67/d69/d32/d37/d56 0 2024-06-06T08:43:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:3/306: dwrite d1/d8/d22/f46 [0,4194304] 0 2024-06-06T08:43:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:2/431: dread d6/dc/d3f/f71 [4194304,4194304] 0 2024-06-06T08:43:25.470 INFO:tasks.workunit.client.0.smithi027.stdout:4/395: dwrite d0/d1/d6/f2f [0,4194304] 0 2024-06-06T08:43:25.470 INFO:tasks.workunit.client.0.smithi027.stdout:4/396: chown d0/d63/c6f 1476 1 2024-06-06T08:43:25.470 INFO:tasks.workunit.client.0.smithi027.stdout:5/522: truncate d0/da/db/f64 3266924 0 2024-06-06T08:43:25.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/523: truncate d0/da/db/f1b 4509257 0 2024-06-06T08:43:25.476 INFO:tasks.workunit.client.0.smithi027.stdout:7/524: link d17/d32/d61/c89 d17/d32/d46/d83/d2a/d3b/cb1 0 2024-06-06T08:43:25.476 INFO:tasks.workunit.client.0.smithi027.stdout:7/525: truncate d17/d32/f2f 2070081 0 2024-06-06T08:43:25.476 INFO:tasks.workunit.client.0.smithi027.stdout:7/526: truncate d17/d32/d46/d4f/f8f 774379 0 2024-06-06T08:43:25.476 INFO:tasks.workunit.client.0.smithi027.stdout:7/527: stat d17/d32/d46/d83/d72 0 2024-06-06T08:43:25.476 INFO:tasks.workunit.client.0.smithi027.stdout:0/332: write fb [1531742,48095] 0 2024-06-06T08:43:25.476 INFO:tasks.workunit.client.0.smithi027.stdout:0/333: truncate d15/d18/d23/d28/f4e 1388831 0 2024-06-06T08:43:25.477 INFO:tasks.workunit.client.0.smithi027.stdout:2/432: symlink d6/d1e/d2b/d5a/d7b/l98 0 2024-06-06T08:43:25.480 INFO:tasks.workunit.client.0.smithi027.stdout:3/307: unlink d1/d1f/c2f 0 2024-06-06T08:43:25.480 INFO:tasks.workunit.client.0.smithi027.stdout:3/308: chown d1/d1f/l35 38518832 1 2024-06-06T08:43:25.481 INFO:tasks.workunit.client.0.smithi027.stdout:7/528: dread fd [0,4194304] 0 2024-06-06T08:43:25.481 INFO:tasks.workunit.client.0.smithi027.stdout:7/529: fsync f11 0 2024-06-06T08:43:25.483 INFO:tasks.workunit.client.0.smithi027.stdout:1/357: mknod dc/de/d14/d4e/c7d 0 2024-06-06T08:43:25.483 INFO:tasks.workunit.client.0.smithi027.stdout:1/358: fsync dc/de/f13 0 2024-06-06T08:43:25.487 INFO:tasks.workunit.client.0.smithi027.stdout:5/524: getdents d0/da/db/d46/d7f/d81 0 2024-06-06T08:43:25.487 INFO:tasks.workunit.client.0.smithi027.stdout:5/525: write d0/da/db/d46/d7f/d81/d97/f9f [1011617,92087] 0 2024-06-06T08:43:25.487 INFO:tasks.workunit.client.0.smithi027.stdout:5/526: fdatasync d0/da/db/d46/d77/f8f 0 2024-06-06T08:43:25.487 INFO:tasks.workunit.client.0.smithi027.stdout:2/433: truncate d6/dc/d3f/d77/f4c 312560 0 2024-06-06T08:43:25.487 INFO:tasks.workunit.client.0.smithi027.stdout:2/434: write d6/d1e/f1f [5008445,61004] 0 2024-06-06T08:43:25.488 INFO:tasks.workunit.client.0.smithi027.stdout:4/397: dwrite d0/f5c [0,4194304] 0 2024-06-06T08:43:25.488 INFO:tasks.workunit.client.0.smithi027.stdout:4/398: truncate d0/d1/f7c 869437 0 2024-06-06T08:43:25.491 INFO:tasks.workunit.client.0.smithi027.stdout:1/359: mknod dc/de/d3d/c7e 0 2024-06-06T08:43:25.491 INFO:tasks.workunit.client.0.smithi027.stdout:1/360: fdatasync dc/de/f13 0 2024-06-06T08:43:25.493 INFO:tasks.workunit.client.0.smithi027.stdout:4/399: read d0/d1d/f69 [96442,1609] 0 2024-06-06T08:43:25.494 INFO:tasks.workunit.client.0.smithi027.stdout:4/400: chown d0/d1/d6/d8/c48 15155018 1 2024-06-06T08:43:25.494 INFO:tasks.workunit.client.0.smithi027.stdout:7/530: write d17/d32/d46/d4f/d5e/f67 [310355,87999] 0 2024-06-06T08:43:25.496 INFO:tasks.workunit.client.0.smithi027.stdout:5/527: creat d0/da/db/d1a/da3/fa6 x:0 0 0 2024-06-06T08:43:25.496 INFO:tasks.workunit.client.0.smithi027.stdout:5/528: chown d0/da/db/d46 133769862 1 2024-06-06T08:43:25.496 INFO:tasks.workunit.client.0.smithi027.stdout:5/529: readlink d0/da/d47/l57 0 2024-06-06T08:43:25.502 INFO:tasks.workunit.client.0.smithi027.stdout:4/401: mknod d0/d1d/d77/c84 0 2024-06-06T08:43:25.502 INFO:tasks.workunit.client.0.smithi027.stdout:4/402: write d0/d1/d6/d4b/f56 [629143,81383] 0 2024-06-06T08:43:25.502 INFO:tasks.workunit.client.0.smithi027.stdout:4/403: chown d0/d1/d6/d8/d21/f30 6200 1 2024-06-06T08:43:25.502 INFO:tasks.workunit.client.0.smithi027.stdout:3/309: dwrite d1/f29 [0,4194304] 0 2024-06-06T08:43:25.503 INFO:tasks.workunit.client.0.smithi027.stdout:1/361: rmdir dc/d67/d69 39 2024-06-06T08:43:25.503 INFO:tasks.workunit.client.0.smithi027.stdout:1/362: fsync dc/de/d14/f19 0 2024-06-06T08:43:25.507 INFO:tasks.workunit.client.0.smithi027.stdout:3/310: creat d1/da/f57 x:0 0 0 2024-06-06T08:43:25.508 INFO:tasks.workunit.client.0.smithi027.stdout:1/363: rename dc/l2f to dc/de/d3d/d4f/l7f 0 2024-06-06T08:43:25.509 INFO:tasks.workunit.client.0.smithi027.stdout:4/404: rmdir d0/d1/d6/d4b 39 2024-06-06T08:43:25.515 INFO:tasks.workunit.client.0.smithi027.stdout:3/311: rename l0 to d1/d1f/d25/l58 0 2024-06-06T08:43:25.515 INFO:tasks.workunit.client.0.smithi027.stdout:1/364: chown dc/de/d14/d4e/f5d 231013 1 2024-06-06T08:43:25.515 INFO:tasks.workunit.client.0.smithi027.stdout:7/531: mkdir d17/d32/d46/d51/d7f/db2 0 2024-06-06T08:43:25.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/334: rmdir d15/d18 39 2024-06-06T08:43:25.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/335: stat fc 0 2024-06-06T08:43:25.516 INFO:tasks.workunit.client.0.smithi027.stdout:3/312: write d1/d8/d22/d27/f36 [968138,116305] 0 2024-06-06T08:43:25.516 INFO:tasks.workunit.client.0.smithi027.stdout:0/336: write fb [4909576,37944] 0 2024-06-06T08:43:25.516 INFO:tasks.workunit.client.0.smithi027.stdout:5/530: dwrite d0/fe [4194304,4194304] 0 2024-06-06T08:43:25.516 INFO:tasks.workunit.client.0.smithi027.stdout:7/532: mknod d17/d32/d46/d83/d77/d8c/d91/cb3 0 2024-06-06T08:43:25.516 INFO:tasks.workunit.client.0.smithi027.stdout:5/531: creat d0/da/db/d46/d4c/fa7 x:0 0 0 2024-06-06T08:43:25.517 INFO:tasks.workunit.client.0.smithi027.stdout:5/532: chown d0/da/db/d46/d54/c69 2036 1 2024-06-06T08:43:25.517 INFO:tasks.workunit.client.0.smithi027.stdout:3/313: mkdir d1/d8/d59 0 2024-06-06T08:43:25.517 INFO:tasks.workunit.client.0.smithi027.stdout:7/533: creat d17/d32/d46/d83/d77/fb4 x:0 0 0 2024-06-06T08:43:25.518 INFO:tasks.workunit.client.0.smithi027.stdout:7/534: symlink d17/d32/d46/d51/d52/d70/d87/lb5 0 2024-06-06T08:43:25.518 INFO:tasks.workunit.client.0.smithi027.stdout:1/365: mkdir dc/de/d14/d80 0 2024-06-06T08:43:25.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/533: creat d0/da/d47/fa8 x:0 0 0 2024-06-06T08:43:25.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/534: write d0/da/d37/f41 [273702,29547] 0 2024-06-06T08:43:25.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/366: creat dc/f81 x:0 0 0 2024-06-06T08:43:25.521 INFO:tasks.workunit.client.0.smithi027.stdout:3/314: truncate d1/f1e 776043 0 2024-06-06T08:43:25.522 INFO:tasks.workunit.client.0.smithi027.stdout:3/315: chown d1/d8/d22/d27/d50/f51 52 1 2024-06-06T08:43:25.522 INFO:tasks.workunit.client.0.smithi027.stdout:3/316: fsync d1/d1f/d25/f3a 0 2024-06-06T08:43:25.522 INFO:tasks.workunit.client.0.smithi027.stdout:5/535: mkdir d0/da/db/d1a/da9 0 2024-06-06T08:43:25.524 INFO:tasks.workunit.client.0.smithi027.stdout:5/536: dread d0/da/db/d1a/f6f [0,4194304] 0 2024-06-06T08:43:25.526 INFO:tasks.workunit.client.0.smithi027.stdout:0/337: dread d15/d18/d23/d28/d2a/d2d/f34 [0,4194304] 0 2024-06-06T08:43:25.527 INFO:tasks.workunit.client.0.smithi027.stdout:0/338: symlink d15/l70 0 2024-06-06T08:43:25.529 INFO:tasks.workunit.client.0.smithi027.stdout:0/339: unlink d15/d18/d23/d28/d2a/d2d/f3b 0 2024-06-06T08:43:25.531 INFO:tasks.workunit.client.0.smithi027.stdout:0/340: creat d15/d18/d23/d28/d2a/d30/f71 x:0 0 0 2024-06-06T08:43:25.532 INFO:tasks.workunit.client.0.smithi027.stdout:4/405: dwrite d0/fd [8388608,4194304] 0 2024-06-06T08:43:25.533 INFO:tasks.workunit.client.0.smithi027.stdout:0/341: mknod d15/d18/d23/d38/d68/c72 0 2024-06-06T08:43:25.533 INFO:tasks.workunit.client.0.smithi027.stdout:0/342: write d15/d18/d23/d28/d2a/d30/f71 [92266,128143] 0 2024-06-06T08:43:25.533 INFO:tasks.workunit.client.0.smithi027.stdout:0/343: write d15/d18/d23/d28/d2a/d2d/f56 [242025,112469] 0 2024-06-06T08:43:25.534 INFO:tasks.workunit.client.0.smithi027.stdout:3/317: write d1/da/d44/f49 [2526445,33778] 0 2024-06-06T08:43:25.534 INFO:tasks.workunit.client.0.smithi027.stdout:3/318: chown d1/d1f/l2b 339373 1 2024-06-06T08:43:25.534 INFO:tasks.workunit.client.0.smithi027.stdout:0/344: creat d15/d18/f73 x:0 0 0 2024-06-06T08:43:25.536 INFO:tasks.workunit.client.0.smithi027.stdout:0/345: mkdir d15/d18/d23/d28/d2a/d74 0 2024-06-06T08:43:25.538 INFO:tasks.workunit.client.0.smithi027.stdout:0/346: creat d15/d18/d23/d28/d2a/d30/d5d/f75 x:0 0 0 2024-06-06T08:43:25.538 INFO:tasks.workunit.client.0.smithi027.stdout:0/347: chown d15/d18/c1f 9 1 2024-06-06T08:43:25.541 INFO:tasks.workunit.client.0.smithi027.stdout:0/348: rename d15/d18/d23/d28/d2a/d30/d5d/f75 to d15/d18/d40/f76 0 2024-06-06T08:43:25.541 INFO:tasks.workunit.client.0.smithi027.stdout:7/535: dwrite d17/d32/d46/d51/d52/d70/d87/fa3 [0,4194304] 0 2024-06-06T08:43:25.541 INFO:tasks.workunit.client.0.smithi027.stdout:4/406: dread d0/d1/d6/d8/f58 [0,4194304] 0 2024-06-06T08:43:25.542 INFO:tasks.workunit.client.0.smithi027.stdout:7/536: mkdir d17/d32/d46/d4f/db6 0 2024-06-06T08:43:25.542 INFO:tasks.workunit.client.0.smithi027.stdout:4/407: symlink d0/d1d/d60/l85 0 2024-06-06T08:43:25.544 INFO:tasks.workunit.client.0.smithi027.stdout:7/537: symlink d17/d32/d46/d4f/d5e/d7b/d84/lb7 0 2024-06-06T08:43:25.544 INFO:tasks.workunit.client.0.smithi027.stdout:7/538: chown d17/d32/d46/d51/d52/d70/c94 1 1 2024-06-06T08:43:25.544 INFO:tasks.workunit.client.0.smithi027.stdout:7/539: readlink d17/d32/d46/l95 0 2024-06-06T08:43:25.544 INFO:tasks.workunit.client.0.smithi027.stdout:4/408: mknod d0/d63/d7e/d3b/c86 0 2024-06-06T08:43:25.546 INFO:tasks.workunit.client.0.smithi027.stdout:0/349: dread d15/d18/d23/d28/d2a/d2d/f34 [0,4194304] 0 2024-06-06T08:43:25.546 INFO:tasks.workunit.client.0.smithi027.stdout:7/540: creat d17/d32/d46/d51/d52/d76/fb8 x:0 0 0 2024-06-06T08:43:25.546 INFO:tasks.workunit.client.0.smithi027.stdout:7/541: stat d17/f27 0 2024-06-06T08:43:25.547 INFO:tasks.workunit.client.0.smithi027.stdout:4/409: unlink d0/d1/d6/d8/l4e 0 2024-06-06T08:43:25.547 INFO:tasks.workunit.client.0.smithi027.stdout:4/410: fsync d0/d1d/f79 0 2024-06-06T08:43:25.547 INFO:tasks.workunit.client.0.smithi027.stdout:4/411: dread - d0/d1/d6/d8/f49 zero size 2024-06-06T08:43:25.547 INFO:tasks.workunit.client.0.smithi027.stdout:4/412: rename d0 to d0/d1d/d2a/d87 22 2024-06-06T08:43:25.548 INFO:tasks.workunit.client.0.smithi027.stdout:0/350: creat d15/d18/d23/d38/f77 x:0 0 0 2024-06-06T08:43:25.548 INFO:tasks.workunit.client.0.smithi027.stdout:4/413: truncate d0/d1/d6/f2f 1691375 0 2024-06-06T08:43:25.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/542: readlink d17/d32/d46/d83/l2b 0 2024-06-06T08:43:25.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/543: write d17/f1b [3526015,44177] 0 2024-06-06T08:43:25.549 INFO:tasks.workunit.client.0.smithi027.stdout:0/351: mknod d15/c78 0 2024-06-06T08:43:25.551 INFO:tasks.workunit.client.0.smithi027.stdout:7/544: rename d17/d47/c82 to d17/d47/cb9 0 2024-06-06T08:43:25.554 INFO:tasks.workunit.client.0.smithi027.stdout:3/319: dwrite d1/d8/f9 [0,4194304] 0 2024-06-06T08:43:25.554 INFO:tasks.workunit.client.0.smithi027.stdout:4/414: unlink d0/d1/d6/d8/l32 0 2024-06-06T08:43:25.555 INFO:tasks.workunit.client.0.smithi027.stdout:4/415: mknod d0/d1d/c88 0 2024-06-06T08:43:25.557 INFO:tasks.workunit.client.0.smithi027.stdout:4/416: creat d0/d63/d7e/f89 x:0 0 0 2024-06-06T08:43:25.562 INFO:tasks.workunit.client.0.smithi027.stdout:7/545: dread d17/d32/d46/d4f/d5e/f8b [0,4194304] 0 2024-06-06T08:43:25.563 INFO:tasks.workunit.client.0.smithi027.stdout:3/320: write d1/f41 [1166007,48469] 0 2024-06-06T08:43:25.567 INFO:tasks.workunit.client.0.smithi027.stdout:0/352: dwrite d15/d18/d23/d5b/f6b [0,4194304] 0 2024-06-06T08:43:25.567 INFO:tasks.workunit.client.0.smithi027.stdout:0/353: readlink d15/d18/l1b 0 2024-06-06T08:43:25.567 INFO:tasks.workunit.client.0.smithi027.stdout:0/354: fsync d15/d18/d23/d28/d2a/d2d/f58 0 2024-06-06T08:43:25.567 INFO:tasks.workunit.client.0.smithi027.stdout:0/355: write d15/d18/d23/d28/d2a/d2d/f54 [676158,100313] 0 2024-06-06T08:43:25.575 INFO:tasks.workunit.client.0.smithi027.stdout:7/546: write d17/d32/d46/d83/d77/d8c/d90/f9a [284821,20686] 0 2024-06-06T08:43:25.581 INFO:tasks.workunit.client.0.smithi027.stdout:4/417: dwrite d0/d63/d7e/f89 [0,4194304] 0 2024-06-06T08:43:25.581 INFO:tasks.workunit.client.0.smithi027.stdout:4/418: write d0/d1/d6/d8/d21/f74 [8800435,34811] 0 2024-06-06T08:43:25.584 INFO:tasks.workunit.client.0.smithi027.stdout:4/419: unlink d0/d1/l83 0 2024-06-06T08:43:25.587 INFO:tasks.workunit.client.0.smithi027.stdout:4/420: dread d0/d63/f6d [0,4194304] 0 2024-06-06T08:43:25.587 INFO:tasks.workunit.client.0.smithi027.stdout:4/421: fdatasync d0/f66 0 2024-06-06T08:43:25.587 INFO:tasks.workunit.client.0.smithi027.stdout:4/422: write d0/d1/d6/f2f [2104793,125293] 0 2024-06-06T08:43:25.594 INFO:tasks.workunit.client.0.smithi027.stdout:0/356: dread f10 [0,4194304] 0 2024-06-06T08:43:25.602 INFO:tasks.workunit.client.0.smithi027.stdout:7/547: write d17/d32/f3a [4666515,31564] 0 2024-06-06T08:43:25.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/423: dwrite d0/d1/d6/f2f [0,4194304] 0 2024-06-06T08:43:25.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/424: read d0/d1/d6/d8/d21/f2d [3345425,27413] 0 2024-06-06T08:43:25.615 INFO:tasks.workunit.client.0.smithi027.stdout:7/548: write f14 [3197850,18949] 0 2024-06-06T08:43:25.616 INFO:tasks.workunit.client.0.smithi027.stdout:0/357: dwrite d15/d18/d23/d28/d2a/d30/d5d/f4a [0,4194304] 0 2024-06-06T08:43:25.619 INFO:tasks.workunit.client.0.smithi027.stdout:0/358: mknod d15/d18/d23/d28/d2a/d30/d5c/c79 0 2024-06-06T08:43:25.619 INFO:tasks.workunit.client.0.smithi027.stdout:0/359: truncate d15/d18/d23/d28/f4c 404826 0 2024-06-06T08:43:25.624 INFO:tasks.workunit.client.0.smithi027.stdout:4/425: rename d0/d1/d6/d8/c52 to d0/d63/c8a 0 2024-06-06T08:43:25.630 INFO:tasks.workunit.client.0.smithi027.stdout:4/426: dread d0/d63/f6d [0,4194304] 0 2024-06-06T08:43:25.632 INFO:tasks.workunit.client.0.smithi027.stdout:4/427: mkdir d0/d1/d6/d8/d82/d8b 0 2024-06-06T08:43:25.632 INFO:tasks.workunit.client.0.smithi027.stdout:4/428: chown d0/d63/d7e/d3b/f70 82 1 2024-06-06T08:43:25.632 INFO:tasks.workunit.client.0.smithi027.stdout:4/429: readlink d0/l36 0 2024-06-06T08:43:25.632 INFO:tasks.workunit.client.0.smithi027.stdout:4/430: stat d0/d1/d6/d8/d82/d8b 0 2024-06-06T08:43:25.633 INFO:tasks.workunit.client.0.smithi027.stdout:0/360: dread d15/d18/d23/f2e [0,4194304] 0 2024-06-06T08:43:25.633 INFO:tasks.workunit.client.0.smithi027.stdout:7/549: dwrite d17/f3f [0,4194304] 0 2024-06-06T08:43:25.637 INFO:tasks.workunit.client.0.smithi027.stdout:0/361: dread d15/d18/d23/d28/d2a/d2d/f58 [0,4194304] 0 2024-06-06T08:43:25.637 INFO:tasks.workunit.client.0.smithi027.stdout:0/362: chown f10 62 1 2024-06-06T08:43:25.639 INFO:tasks.workunit.client.0.smithi027.stdout:0/363: symlink d15/d18/d23/d38/d68/l7a 0 2024-06-06T08:43:25.641 INFO:tasks.workunit.client.0.smithi027.stdout:0/364: creat d15/d18/d23/d38/d68/f7b x:0 0 0 2024-06-06T08:43:25.641 INFO:tasks.workunit.client.0.smithi027.stdout:4/431: write d0/f13 [10049303,302] 0 2024-06-06T08:43:25.648 INFO:tasks.workunit.client.0.smithi027.stdout:7/550: dread d17/d32/d46/d4f/f50 [0,4194304] 0 2024-06-06T08:43:25.656 INFO:tasks.workunit.client.0.smithi027.stdout:7/551: dread d17/d32/d46/d51/f6f [0,4194304] 0 2024-06-06T08:43:25.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/432: dread d0/d1d/f54 [0,4194304] 0 2024-06-06T08:43:25.662 INFO:tasks.workunit.client.0.smithi027.stdout:0/365: dwrite d15/d18/d23/f2e [0,4194304] 0 2024-06-06T08:43:25.662 INFO:tasks.workunit.client.0.smithi027.stdout:4/433: dread d0/d1d/f69 [0,4194304] 0 2024-06-06T08:43:25.662 INFO:tasks.workunit.client.0.smithi027.stdout:0/366: fdatasync d15/d18/d23/d28/d2a/d2d/f34 0 2024-06-06T08:43:25.662 INFO:tasks.workunit.client.0.smithi027.stdout:4/434: stat d0/d1/d6/d8/d21/c2b 0 2024-06-06T08:43:25.662 INFO:tasks.workunit.client.0.smithi027.stdout:4/435: read d0/d1/ff [2634582,36476] 0 2024-06-06T08:43:25.662 INFO:tasks.workunit.client.0.smithi027.stdout:4/436: write d0/d1/f12 [1245819,38922] 0 2024-06-06T08:43:25.664 INFO:tasks.workunit.client.0.smithi027.stdout:4/437: creat d0/d63/d7e/d3b/f8c x:0 0 0 2024-06-06T08:43:25.664 INFO:tasks.workunit.client.0.smithi027.stdout:4/438: fsync d0/d63/d7e/d3b/f68 0 2024-06-06T08:43:25.664 INFO:tasks.workunit.client.0.smithi027.stdout:4/439: write d0/d1/d6/d8/f49 [1017183,116240] 0 2024-06-06T08:43:25.664 INFO:tasks.workunit.client.0.smithi027.stdout:4/440: write d0/d1/fb [4527698,43180] 0 2024-06-06T08:43:25.665 INFO:tasks.workunit.client.0.smithi027.stdout:4/441: rename d0/d63/c64 to d0/c8d 0 2024-06-06T08:43:25.666 INFO:tasks.workunit.client.0.smithi027.stdout:0/367: symlink d15/d18/d23/d28/d31/l7c 0 2024-06-06T08:43:25.666 INFO:tasks.workunit.client.0.smithi027.stdout:4/442: read d0/d1/f7c [18271,18116] 0 2024-06-06T08:43:25.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/368: symlink d15/d18/l7d 0 2024-06-06T08:43:25.669 INFO:tasks.workunit.client.0.smithi027.stdout:4/443: fsync d0/d1/fb 0 2024-06-06T08:43:25.669 INFO:tasks.workunit.client.0.smithi027.stdout:4/444: stat d0/d1/c4 0 2024-06-06T08:43:25.670 INFO:tasks.workunit.client.0.smithi027.stdout:0/369: mknod d15/d18/d23/d38/d3e/c7e 0 2024-06-06T08:43:25.672 INFO:tasks.workunit.client.0.smithi027.stdout:4/445: write d0/d1/d6/d8/d21/f4a [262325,72069] 0 2024-06-06T08:43:25.673 INFO:tasks.workunit.client.0.smithi027.stdout:7/552: dwrite d17/f58 [0,4194304] 0 2024-06-06T08:43:25.673 INFO:tasks.workunit.client.0.smithi027.stdout:0/370: fsync d15/d18/d23/d38/d68/f7b 0 2024-06-06T08:43:25.676 INFO:tasks.workunit.client.0.smithi027.stdout:7/553: dread d17/d32/d46/d83/d2a/d3b/f42 [0,4194304] 0 2024-06-06T08:43:25.677 INFO:tasks.workunit.client.0.smithi027.stdout:7/554: mknod d17/d32/d46/d83/d77/d8c/dae/cba 0 2024-06-06T08:43:25.678 INFO:tasks.workunit.client.0.smithi027.stdout:7/555: read - d17/d32/d46/d51/d52/d76/fb8 zero size 2024-06-06T08:43:25.691 INFO:tasks.workunit.client.0.smithi027.stdout:4/446: dwrite d0/d1/d6/d8/d21/f2d [0,4194304] 0 2024-06-06T08:43:25.691 INFO:tasks.workunit.client.0.smithi027.stdout:4/447: stat d0/d1d/d2a/c37 0 2024-06-06T08:43:25.691 INFO:tasks.workunit.client.0.smithi027.stdout:4/448: truncate d0/f55 771957 0 2024-06-06T08:43:25.691 INFO:tasks.workunit.client.0.smithi027.stdout:4/449: fdatasync d0/f1c 0 2024-06-06T08:43:25.693 INFO:tasks.workunit.client.0.smithi027.stdout:4/450: dread d0/d1d/d34/f40 [0,4194304] 0 2024-06-06T08:43:25.694 INFO:tasks.workunit.client.0.smithi027.stdout:7/556: dwrite d17/d32/d46/d4f/d5e/f8b [0,4194304] 0 2024-06-06T08:43:25.694 INFO:tasks.workunit.client.0.smithi027.stdout:7/557: fdatasync d17/fa4 0 2024-06-06T08:43:25.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/558: truncate d17/f27 850031 0 2024-06-06T08:43:25.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/559: write d17/d32/d46/d83/d2a/f69 [3011151,60411] 0 2024-06-06T08:43:25.698 INFO:tasks.workunit.client.0.smithi027.stdout:4/451: unlink d0/d63/d7e/d3b/f70 0 2024-06-06T08:43:25.700 INFO:tasks.workunit.client.0.smithi027.stdout:4/452: unlink d0/d63/d7e/l51 0 2024-06-06T08:43:25.702 INFO:tasks.workunit.client.0.smithi027.stdout:4/453: rename d0/c1f to d0/d1d/d34/d7f/c8e 0 2024-06-06T08:43:25.713 INFO:tasks.workunit.client.0.smithi027.stdout:7/560: read d17/f58 [3804628,43746] 0 2024-06-06T08:43:25.718 INFO:tasks.workunit.client.0.smithi027.stdout:4/454: dwrite d0/d1/d6/d8/f49 [0,4194304] 0 2024-06-06T08:43:25.718 INFO:tasks.workunit.client.0.smithi027.stdout:7/561: rmdir d17/d32/d46/d83/d2a 39 2024-06-06T08:43:25.719 INFO:tasks.workunit.client.0.smithi027.stdout:4/455: dread d0/d1d/d2a/f57 [4194304,4194304] 0 2024-06-06T08:43:25.720 INFO:tasks.workunit.client.0.smithi027.stdout:4/456: mknod d0/d63/d7e/d3b/c8f 0 2024-06-06T08:43:25.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/562: creat d17/d32/d6c/fbb x:0 0 0 2024-06-06T08:43:25.722 INFO:tasks.workunit.client.0.smithi027.stdout:7/563: dread d17/f60 [0,4194304] 0 2024-06-06T08:43:25.722 INFO:tasks.workunit.client.0.smithi027.stdout:4/457: mknod d0/d1d/d77/d6e/c90 0 2024-06-06T08:43:25.722 INFO:tasks.workunit.client.0.smithi027.stdout:4/458: chown d0/d1d/l2c 1 1 2024-06-06T08:43:25.722 INFO:tasks.workunit.client.0.smithi027.stdout:4/459: fdatasync d0/d1/f12 0 2024-06-06T08:43:25.723 INFO:tasks.workunit.client.0.smithi027.stdout:7/564: chown d17/d32/d46/d51/l4e 207616 1 2024-06-06T08:43:25.723 INFO:tasks.workunit.client.0.smithi027.stdout:4/460: mknod d0/d63/d7e/d3b/c91 0 2024-06-06T08:43:25.724 INFO:tasks.workunit.client.0.smithi027.stdout:4/461: write d0/d1d/f59 [119534,90425] 0 2024-06-06T08:43:25.724 INFO:tasks.workunit.client.0.smithi027.stdout:7/565: dread d17/d32/d46/d83/d2a/d3b/f42 [0,4194304] 0 2024-06-06T08:43:25.725 INFO:tasks.workunit.client.0.smithi027.stdout:7/566: read - d17/d32/d46/d83/d77/fb4 zero size 2024-06-06T08:43:25.726 INFO:tasks.workunit.client.0.smithi027.stdout:7/567: rmdir d17/d32/d46/d4f/d5e/d7b 39 2024-06-06T08:43:25.727 INFO:tasks.workunit.client.0.smithi027.stdout:7/568: creat d17/d32/d46/d4f/d5e/d7b/d84/fbc x:0 0 0 2024-06-06T08:43:25.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/569: read - d17/d32/d46/d51/d52/d76/faf zero size 2024-06-06T08:43:25.740 INFO:tasks.workunit.client.0.smithi027.stdout:4/462: dwrite d0/d1d/d34/f6b [0,4194304] 0 2024-06-06T08:43:25.742 INFO:tasks.workunit.client.0.smithi027.stdout:4/463: mknod d0/d1d/d60/c92 0 2024-06-06T08:43:25.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/570: dwrite d17/d32/d46/d51/f6f [0,4194304] 0 2024-06-06T08:43:25.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/571: chown d17/d32/d46/d83/d2a/d3b/c62 612145113 1 2024-06-06T08:43:25.744 INFO:tasks.workunit.client.0.smithi027.stdout:4/464: creat d0/d1/d6/d8/f93 x:0 0 0 2024-06-06T08:43:25.745 INFO:tasks.workunit.client.0.smithi027.stdout:4/465: getdents d0/d1d/d77 0 2024-06-06T08:43:25.746 INFO:tasks.workunit.client.0.smithi027.stdout:4/466: fdatasync d0/d1d/d2a/f7d 0 2024-06-06T08:43:25.746 INFO:tasks.workunit.client.0.smithi027.stdout:7/572: link d17/d32/d46/d51/l4e d17/d32/d46/d83/d77/d8c/dae/lbd 0 2024-06-06T08:43:25.746 INFO:tasks.workunit.client.0.smithi027.stdout:7/573: symlink d17/d32/d46/d83/d2a/lbe 0 2024-06-06T08:43:25.747 INFO:tasks.workunit.client.0.smithi027.stdout:7/574: creat d17/d32/d46/d83/d2a/fbf x:0 0 0 2024-06-06T08:43:25.748 INFO:tasks.workunit.client.0.smithi027.stdout:4/467: creat d0/d1d/d34/d7f/f94 x:0 0 0 2024-06-06T08:43:25.753 INFO:tasks.workunit.client.0.smithi027.stdout:7/575: dread d17/d32/d61/f74 [0,4194304] 0 2024-06-06T08:43:25.753 INFO:tasks.workunit.client.0.smithi027.stdout:7/576: truncate d17/d32/d46/d4f/d5e/f79 353054 0 2024-06-06T08:43:25.758 INFO:tasks.workunit.client.0.smithi027.stdout:4/468: write d0/f66 [534986,52352] 0 2024-06-06T08:43:25.758 INFO:tasks.workunit.client.0.smithi027.stdout:4/469: stat d0/d1/d6/d8/c53 0 2024-06-06T08:43:25.758 INFO:tasks.workunit.client.0.smithi027.stdout:4/470: dread - d0/d1d/d2a/f5e zero size 2024-06-06T08:43:25.758 INFO:tasks.workunit.client.0.smithi027.stdout:4/471: fsync d0/d63/d7e/d3b/f68 0 2024-06-06T08:43:25.759 INFO:tasks.workunit.client.0.smithi027.stdout:4/472: mknod d0/d1/d6/d4b/c95 0 2024-06-06T08:43:25.760 INFO:tasks.workunit.client.0.smithi027.stdout:4/473: fdatasync d0/d1d/d2a/f57 0 2024-06-06T08:43:25.772 INFO:tasks.workunit.client.0.smithi027.stdout:7/577: dwrite d17/f26 [4194304,4194304] 0 2024-06-06T08:43:25.773 INFO:tasks.workunit.client.0.smithi027.stdout:7/578: write d17/d32/d46/d4f/f55 [1737717,48500] 0 2024-06-06T08:43:25.774 INFO:tasks.workunit.client.0.smithi027.stdout:7/579: mkdir d17/d32/d46/d51/d52/d70/d87/dc0 0 2024-06-06T08:43:25.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/474: dwrite d0/d1/d6/d8/f6a [0,4194304] 0 2024-06-06T08:43:25.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/475: dread - d0/d1d/d2a/f5e zero size 2024-06-06T08:43:25.785 INFO:tasks.workunit.client.0.smithi027.stdout:7/580: dread d17/d32/d46/d4f/d5e/f67 [0,4194304] 0 2024-06-06T08:43:25.786 INFO:tasks.workunit.client.0.smithi027.stdout:7/581: chown d17/d32/d46/d4f/f50 8 1 2024-06-06T08:43:25.786 INFO:tasks.workunit.client.0.smithi027.stdout:7/582: chown d17/l8d 57142742 1 2024-06-06T08:43:25.794 INFO:tasks.workunit.client.0.smithi027.stdout:4/476: dwrite d0/d1d/d34/d7f/f94 [0,4194304] 0 2024-06-06T08:43:25.795 INFO:tasks.workunit.client.0.smithi027.stdout:7/583: dread d17/f1b [0,4194304] 0 2024-06-06T08:43:25.795 INFO:tasks.workunit.client.0.smithi027.stdout:7/584: fsync d17/d32/d46/d51/d52/d76/faf 0 2024-06-06T08:43:25.795 INFO:tasks.workunit.client.0.smithi027.stdout:7/585: write d17/d32/d46/d4f/d5e/f67 [5156810,119834] 0 2024-06-06T08:43:25.795 INFO:tasks.workunit.client.0.smithi027.stdout:7/586: write d17/f60 [707613,48339] 0 2024-06-06T08:43:25.799 INFO:tasks.workunit.client.0.smithi027.stdout:7/587: mkdir d17/d32/d46/d4f/d5e/d7b/dc1 0 2024-06-06T08:43:25.805 INFO:tasks.workunit.client.0.smithi027.stdout:0/371: sync 2024-06-06T08:43:25.805 INFO:tasks.workunit.client.0.smithi027.stdout:9/322: sync 2024-06-06T08:43:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:8/288: sync 2024-06-06T08:43:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:5/537: sync 2024-06-06T08:43:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:1/367: sync 2024-06-06T08:43:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:8/289: stat d7/d13/d15/f1c 0 2024-06-06T08:43:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:6/393: sync 2024-06-06T08:43:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:0/372: write f2 [2548961,108323] 0 2024-06-06T08:43:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:3/321: sync 2024-06-06T08:43:25.806 INFO:tasks.workunit.client.0.smithi027.stdout:2/435: sync 2024-06-06T08:43:25.807 INFO:tasks.workunit.client.0.smithi027.stdout:1/368: creat dc/de/d22/f82 x:0 0 0 2024-06-06T08:43:25.807 INFO:tasks.workunit.client.0.smithi027.stdout:9/323: mkdir d3/da/dc/df/d34/d73 0 2024-06-06T08:43:25.807 INFO:tasks.workunit.client.0.smithi027.stdout:9/324: readlink d3/l4c 0 2024-06-06T08:43:25.808 INFO:tasks.workunit.client.0.smithi027.stdout:6/394: chown d4/d1a/c1d 8060 1 2024-06-06T08:43:25.808 INFO:tasks.workunit.client.0.smithi027.stdout:2/436: dread d6/f2e [0,4194304] 0 2024-06-06T08:43:25.810 INFO:tasks.workunit.client.0.smithi027.stdout:3/322: unlink d1/da/f31 0 2024-06-06T08:43:25.812 INFO:tasks.workunit.client.0.smithi027.stdout:2/437: symlink d6/dc/d3f/l99 0 2024-06-06T08:43:25.812 INFO:tasks.workunit.client.0.smithi027.stdout:1/369: rmdir dc/d67/d69/d32/d37 39 2024-06-06T08:43:25.813 INFO:tasks.workunit.client.0.smithi027.stdout:1/370: fsync dc/de/d14/f15 0 2024-06-06T08:43:25.815 INFO:tasks.workunit.client.0.smithi027.stdout:2/438: creat d6/d1e/f9a x:0 0 0 2024-06-06T08:43:25.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/371: truncate dc/de/d14/d4e/f7a 621267 0 2024-06-06T08:43:25.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/372: readlink dc/d67/d69/l25 0 2024-06-06T08:43:25.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/439: symlink d6/d1e/d2b/l9b 0 2024-06-06T08:43:25.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/440: fdatasync d6/dc/d3f/d48/f93 0 2024-06-06T08:43:25.819 INFO:tasks.workunit.client.0.smithi027.stdout:1/373: creat dc/d67/d69/d32/d37/d56/d78/f83 x:0 0 0 2024-06-06T08:43:25.819 INFO:tasks.workunit.client.0.smithi027.stdout:1/374: read - dc/f81 zero size 2024-06-06T08:43:25.819 INFO:tasks.workunit.client.0.smithi027.stdout:1/375: write dc/de/d3d/f3e [236467,99585] 0 2024-06-06T08:43:25.820 INFO:tasks.workunit.client.0.smithi027.stdout:2/441: write d6/dc/d3f/d77/f4c [232574,94300] 0 2024-06-06T08:43:25.820 INFO:tasks.workunit.client.0.smithi027.stdout:2/442: fdatasync d6/d1e/d2b/d5a/d7b/f7e 0 2024-06-06T08:43:25.822 INFO:tasks.workunit.client.0.smithi027.stdout:7/588: dwrite f15 [0,4194304] 0 2024-06-06T08:43:25.827 INFO:tasks.workunit.client.0.smithi027.stdout:6/395: dread d4/de/f11 [0,4194304] 0 2024-06-06T08:43:25.829 INFO:tasks.workunit.client.0.smithi027.stdout:8/290: link d7/dc/d36/d41/c2b d7/db/c5c 0 2024-06-06T08:43:25.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/396: truncate d4/d9/f58 304382 0 2024-06-06T08:43:25.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/397: chown d4/de 198 1 2024-06-06T08:43:25.833 INFO:tasks.workunit.client.0.smithi027.stdout:5/538: dwrite d0/da/db/d1a/f71 [4194304,4194304] 0 2024-06-06T08:43:25.837 INFO:tasks.workunit.client.0.smithi027.stdout:5/539: link d0/da/d37/d2b/f91 d0/da/db/d1a/da9/faa 0 2024-06-06T08:43:25.837 INFO:tasks.workunit.client.0.smithi027.stdout:5/540: fsync d0/da/d37/f48 0 2024-06-06T08:43:25.837 INFO:tasks.workunit.client.0.smithi027.stdout:5/541: dread - d0/da/d20/fa1 zero size 2024-06-06T08:43:25.837 INFO:tasks.workunit.client.0.smithi027.stdout:5/542: chown d0/f1 200737376 1 2024-06-06T08:43:25.837 INFO:tasks.workunit.client.0.smithi027.stdout:1/376: write dc/d67/d69/d32/d71/f74 [2604137,113748] 0 2024-06-06T08:43:25.837 INFO:tasks.workunit.client.0.smithi027.stdout:6/398: dread d4/d9/f28 [0,4194304] 0 2024-06-06T08:43:25.837 INFO:tasks.workunit.client.0.smithi027.stdout:3/323: dwrite d1/f29 [0,4194304] 0 2024-06-06T08:43:25.838 INFO:tasks.workunit.client.0.smithi027.stdout:2/443: write d6/dc/d1d/f45 [445391,129377] 0 2024-06-06T08:43:25.838 INFO:tasks.workunit.client.0.smithi027.stdout:2/444: write d6/dc/f40 [4851018,121280] 0 2024-06-06T08:43:25.839 INFO:tasks.workunit.client.0.smithi027.stdout:3/324: symlink d1/d8/d22/l5a 0 2024-06-06T08:43:25.839 INFO:tasks.workunit.client.0.smithi027.stdout:3/325: fsync d1/d8/f16 0 2024-06-06T08:43:25.840 INFO:tasks.workunit.client.0.smithi027.stdout:5/543: rmdir d0/d6a/d93 0 2024-06-06T08:43:25.841 INFO:tasks.workunit.client.0.smithi027.stdout:0/373: rename le to d15/d18/d23/d28/d2a/d74/l7f 0 2024-06-06T08:43:25.841 INFO:tasks.workunit.client.0.smithi027.stdout:0/374: read d15/d18/d23/d38/f51 [524857,12546] 0 2024-06-06T08:43:25.841 INFO:tasks.workunit.client.0.smithi027.stdout:6/399: rmdir d4/d1a/d2e/d44 39 2024-06-06T08:43:25.841 INFO:tasks.workunit.client.0.smithi027.stdout:2/445: mknod d6/d1e/d2b/d5a/c9c 0 2024-06-06T08:43:25.842 INFO:tasks.workunit.client.0.smithi027.stdout:9/325: rename d3/da/dc/df/d34/d41/f58 to d3/da/dc/df/d23/d36/f74 0 2024-06-06T08:43:25.842 INFO:tasks.workunit.client.0.smithi027.stdout:3/326: mkdir d1/d1f/d5b 0 2024-06-06T08:43:25.844 INFO:tasks.workunit.client.0.smithi027.stdout:2/446: creat d6/f9d x:0 0 0 2024-06-06T08:43:25.844 INFO:tasks.workunit.client.0.smithi027.stdout:0/375: symlink d15/d18/d23/d28/d2a/d30/d5c/l80 0 2024-06-06T08:43:25.845 INFO:tasks.workunit.client.0.smithi027.stdout:2/447: symlink d6/dc/d3f/d48/d83/l9e 0 2024-06-06T08:43:25.848 INFO:tasks.workunit.client.0.smithi027.stdout:7/589: rename d17/d32/d46/d4f/d5e/d7b/d84 to d17/d32/d6c/dc2 0 2024-06-06T08:43:25.850 INFO:tasks.workunit.client.0.smithi027.stdout:0/376: creat d15/d18/f81 x:0 0 0 2024-06-06T08:43:25.850 INFO:tasks.workunit.client.0.smithi027.stdout:0/377: chown d15/d18/d23/d38 0 1 2024-06-06T08:43:25.850 INFO:tasks.workunit.client.0.smithi027.stdout:0/378: fsync d15/f21 0 2024-06-06T08:43:25.853 INFO:tasks.workunit.client.0.smithi027.stdout:2/448: unlink d6/dc/d1d/f38 0 2024-06-06T08:43:25.854 INFO:tasks.workunit.client.0.smithi027.stdout:8/291: dwrite d7/f30 [0,4194304] 0 2024-06-06T08:43:25.856 INFO:tasks.workunit.client.0.smithi027.stdout:3/327: rename d1/da/f57 to d1/d8/f5c 0 2024-06-06T08:43:25.857 INFO:tasks.workunit.client.0.smithi027.stdout:7/590: fsync d17/f60 0 2024-06-06T08:43:25.857 INFO:tasks.workunit.client.0.smithi027.stdout:7/591: write d17/d32/d46/d51/f71 [3047813,624] 0 2024-06-06T08:43:25.857 INFO:tasks.workunit.client.0.smithi027.stdout:7/592: chown d17/d47/fa9 1 1 2024-06-06T08:43:25.858 INFO:tasks.workunit.client.0.smithi027.stdout:2/449: getdents d6/d1e/d2b/d5a/d7b 0 2024-06-06T08:43:25.860 INFO:tasks.workunit.client.0.smithi027.stdout:3/328: dread d1/d8/f15 [0,4194304] 0 2024-06-06T08:43:25.862 INFO:tasks.workunit.client.0.smithi027.stdout:2/450: symlink d6/dc/d3f/d77/d55/l9f 0 2024-06-06T08:43:25.867 INFO:tasks.workunit.client.0.smithi027.stdout:1/377: dwrite dc/de/d22/f35 [0,4194304] 0 2024-06-06T08:43:25.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/329: dread d1/f13 [0,4194304] 0 2024-06-06T08:43:25.871 INFO:tasks.workunit.client.0.smithi027.stdout:2/451: rmdir d6/d3b/d80/d8d 0 2024-06-06T08:43:25.871 INFO:tasks.workunit.client.0.smithi027.stdout:2/452: fdatasync d6/d3b/d61/f70 0 2024-06-06T08:43:25.873 INFO:tasks.workunit.client.0.smithi027.stdout:3/330: creat d1/d1f/d25/f5d x:0 0 0 2024-06-06T08:43:25.874 INFO:tasks.workunit.client.0.smithi027.stdout:2/453: mknod d6/dc/ca0 0 2024-06-06T08:43:25.874 INFO:tasks.workunit.client.0.smithi027.stdout:2/454: fdatasync d6/d3b/f43 0 2024-06-06T08:43:25.874 INFO:tasks.workunit.client.0.smithi027.stdout:9/326: dwrite d3/da/dc/f30 [0,4194304] 0 2024-06-06T08:43:25.874 INFO:tasks.workunit.client.0.smithi027.stdout:9/327: dread - d3/da/dc/df/d23/d36/d3a/f68 zero size 2024-06-06T08:43:25.874 INFO:tasks.workunit.client.0.smithi027.stdout:8/292: dread d7/dc/d37/d46/f52 [0,4194304] 0 2024-06-06T08:43:25.875 INFO:tasks.workunit.client.0.smithi027.stdout:5/544: dwrite d0/da/db/d46/d4c/f6e [0,4194304] 0 2024-06-06T08:43:25.875 INFO:tasks.workunit.client.0.smithi027.stdout:5/545: write d0/da/f92 [777790,111260] 0 2024-06-06T08:43:25.875 INFO:tasks.workunit.client.0.smithi027.stdout:2/455: mknod d6/d1e/d2b/ca1 0 2024-06-06T08:43:25.875 INFO:tasks.workunit.client.0.smithi027.stdout:8/293: mknod d7/dc/d37/d46/c5d 0 2024-06-06T08:43:25.876 INFO:tasks.workunit.client.0.smithi027.stdout:5/546: stat d0/d6a/l89 0 2024-06-06T08:43:25.877 INFO:tasks.workunit.client.0.smithi027.stdout:2/456: mknod d6/d3b/d80/ca2 0 2024-06-06T08:43:25.880 INFO:tasks.workunit.client.0.smithi027.stdout:8/294: creat d7/d2f/f5e x:0 0 0 2024-06-06T08:43:25.882 INFO:tasks.workunit.client.0.smithi027.stdout:6/400: dwrite d4/de/f11 [0,4194304] 0 2024-06-06T08:43:25.882 INFO:tasks.workunit.client.0.smithi027.stdout:2/457: creat d6/dc/d3f/d77/d55/d69/d82/fa3 x:0 0 0 2024-06-06T08:43:25.882 INFO:tasks.workunit.client.0.smithi027.stdout:2/458: readlink d6/dc/d3f/d77/d55/d69/l76 0 2024-06-06T08:43:25.883 INFO:tasks.workunit.client.0.smithi027.stdout:6/401: symlink d4/d9/d49/l84 0 2024-06-06T08:43:25.884 INFO:tasks.workunit.client.0.smithi027.stdout:8/295: mknod d7/dc/c5f 0 2024-06-06T08:43:25.886 INFO:tasks.workunit.client.0.smithi027.stdout:8/296: mkdir d7/db/d60 0 2024-06-06T08:43:25.887 INFO:tasks.workunit.client.0.smithi027.stdout:1/378: dwrite dc/de/f75 [0,4194304] 0 2024-06-06T08:43:25.887 INFO:tasks.workunit.client.0.smithi027.stdout:3/331: dread d1/d8/d22/d27/f14 [4194304,4194304] 0 2024-06-06T08:43:25.889 INFO:tasks.workunit.client.0.smithi027.stdout:8/297: link d7/d2f/f31 d7/dc/d36/d3c/f61 0 2024-06-06T08:43:25.897 INFO:tasks.workunit.client.0.smithi027.stdout:6/402: write d4/d9/dc/d18/f2f [8093002,3807] 0 2024-06-06T08:43:25.902 INFO:tasks.workunit.client.0.smithi027.stdout:8/298: getdents d7/dc 0 2024-06-06T08:43:25.902 INFO:tasks.workunit.client.0.smithi027.stdout:8/299: write d7/f4e [169255,28898] 0 2024-06-06T08:43:25.904 INFO:tasks.workunit.client.0.smithi027.stdout:8/300: creat d7/dc/d37/d46/d50/f62 x:0 0 0 2024-06-06T08:43:25.904 INFO:tasks.workunit.client.0.smithi027.stdout:8/301: fdatasync d7/f22 0 2024-06-06T08:43:25.905 INFO:tasks.workunit.client.0.smithi027.stdout:3/332: dread d1/d1f/d4d/f52 [0,4194304] 0 2024-06-06T08:43:25.906 INFO:tasks.workunit.client.0.smithi027.stdout:8/302: link d7/l44 d7/l63 0 2024-06-06T08:43:25.906 INFO:tasks.workunit.client.0.smithi027.stdout:1/379: dread dc/de/f13 [0,4194304] 0 2024-06-06T08:43:25.906 INFO:tasks.workunit.client.0.smithi027.stdout:5/547: dwrite d0/da/d37/d2b/f5b [0,4194304] 0 2024-06-06T08:43:25.906 INFO:tasks.workunit.client.0.smithi027.stdout:7/593: rename d17/d32/d46/d51/f8e to d17/d32/d46/d83/d72/fc3 0 2024-06-06T08:43:25.906 INFO:tasks.workunit.client.0.smithi027.stdout:7/594: stat c13 0 2024-06-06T08:43:25.906 INFO:tasks.workunit.client.0.smithi027.stdout:7/595: chown d17/d47 16119170 1 2024-06-06T08:43:25.907 INFO:tasks.workunit.client.0.smithi027.stdout:7/596: chown d17/d32/f2f 82 1 2024-06-06T08:43:25.907 INFO:tasks.workunit.client.0.smithi027.stdout:3/333: creat d1/da/f5e x:0 0 0 2024-06-06T08:43:25.907 INFO:tasks.workunit.client.0.smithi027.stdout:3/334: chown d1/d8/d22/d27/f14 135 1 2024-06-06T08:43:25.908 INFO:tasks.workunit.client.0.smithi027.stdout:7/597: symlink d17/d32/d46/d83/d77/d8c/d90/lc4 0 2024-06-06T08:43:25.908 INFO:tasks.workunit.client.0.smithi027.stdout:7/598: write d17/d32/d46/d4f/f8f [1027150,86455] 0 2024-06-06T08:43:25.909 INFO:tasks.workunit.client.0.smithi027.stdout:1/380: creat dc/f84 x:0 0 0 2024-06-06T08:43:25.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/335: mkdir d1/d1f/d45/d5f 0 2024-06-06T08:43:25.911 INFO:tasks.workunit.client.0.smithi027.stdout:6/403: dread d4/f57 [0,4194304] 0 2024-06-06T08:43:25.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/336: chown d1/d1f/d45/d5f 2454672 1 2024-06-06T08:43:25.912 INFO:tasks.workunit.client.0.smithi027.stdout:1/381: creat dc/de/d3d/f85 x:0 0 0 2024-06-06T08:43:25.913 INFO:tasks.workunit.client.0.smithi027.stdout:2/459: dwrite d6/f14 [0,4194304] 0 2024-06-06T08:43:25.914 INFO:tasks.workunit.client.0.smithi027.stdout:6/404: unlink d4/d1a/c82 0 2024-06-06T08:43:25.915 INFO:tasks.workunit.client.0.smithi027.stdout:3/337: getdents d1/d8/d22/d2d 0 2024-06-06T08:43:25.919 INFO:tasks.workunit.client.0.smithi027.stdout:6/405: rmdir d4/d1a/d2e 39 2024-06-06T08:43:25.926 INFO:tasks.workunit.client.0.smithi027.stdout:7/599: dwrite d17/d32/d46/d4f/d5e/f8b [4194304,4194304] 0 2024-06-06T08:43:25.926 INFO:tasks.workunit.client.0.smithi027.stdout:8/303: dread d7/f4e [0,4194304] 0 2024-06-06T08:43:25.927 INFO:tasks.workunit.client.0.smithi027.stdout:2/460: creat d6/dc/d3f/d77/d55/fa4 x:0 0 0 2024-06-06T08:43:25.927 INFO:tasks.workunit.client.0.smithi027.stdout:3/338: mknod d1/da/d44/c60 0 2024-06-06T08:43:25.927 INFO:tasks.workunit.client.0.smithi027.stdout:1/382: mknod dc/de/d14/c86 0 2024-06-06T08:43:25.930 INFO:tasks.workunit.client.0.smithi027.stdout:1/383: creat dc/de/d14/f87 x:0 0 0 2024-06-06T08:43:25.931 INFO:tasks.workunit.client.0.smithi027.stdout:2/461: unlink d6/d1e/d2b/l3a 0 2024-06-06T08:43:25.933 INFO:tasks.workunit.client.0.smithi027.stdout:3/339: link d1/d8/f15 d1/d8/f61 0 2024-06-06T08:43:25.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/462: getdents d6/dc/d3f/d48 0 2024-06-06T08:43:25.935 INFO:tasks.workunit.client.0.smithi027.stdout:7/600: dread d17/f1e [0,4194304] 0 2024-06-06T08:43:25.935 INFO:tasks.workunit.client.0.smithi027.stdout:8/304: dread d7/f30 [0,4194304] 0 2024-06-06T08:43:25.937 INFO:tasks.workunit.client.0.smithi027.stdout:2/463: creat d6/d3b/d61/fa5 x:0 0 0 2024-06-06T08:43:25.938 INFO:tasks.workunit.client.0.smithi027.stdout:8/305: creat d7/db/d58/f64 x:0 0 0 2024-06-06T08:43:25.940 INFO:tasks.workunit.client.0.smithi027.stdout:8/306: rename d7/db/c29 to d7/db/c65 0 2024-06-06T08:43:25.941 INFO:tasks.workunit.client.0.smithi027.stdout:8/307: mknod d7/dc/d37/d46/d50/c66 0 2024-06-06T08:43:25.942 INFO:tasks.workunit.client.0.smithi027.stdout:9/328: rename d3/da to d3/d4e/d75 0 2024-06-06T08:43:25.943 INFO:tasks.workunit.client.0.smithi027.stdout:8/308: symlink d7/db/d58/l67 0 2024-06-06T08:43:25.944 INFO:tasks.workunit.client.0.smithi027.stdout:8/309: fsync d7/dc/d36/d41/f25 0 2024-06-06T08:43:25.944 INFO:tasks.workunit.client.0.smithi027.stdout:9/329: mkdir d3/d76 0 2024-06-06T08:43:25.944 INFO:tasks.workunit.client.0.smithi027.stdout:9/330: fdatasync d3/d4e/d75/dc/df/f5e 0 2024-06-06T08:43:25.946 INFO:tasks.workunit.client.0.smithi027.stdout:8/310: dread d7/dc/d37/d46/f52 [0,4194304] 0 2024-06-06T08:43:25.947 INFO:tasks.workunit.client.0.smithi027.stdout:8/311: rmdir d7/db/d58 39 2024-06-06T08:43:25.952 INFO:tasks.workunit.client.0.smithi027.stdout:3/340: dwrite d1/d8/d22/d27/d50/f51 [0,4194304] 0 2024-06-06T08:43:25.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:25 smithi027 ceph-mon[32335]: scrub summary: idle 2024-06-06T08:43:25.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:25 smithi027 ceph-mon[32335]: scrub summary: idle 2024-06-06T08:43:25.956 INFO:tasks.workunit.client.0.smithi027.stdout:2/464: dwrite d6/d1e/d2b/f58 [0,4194304] 0 2024-06-06T08:43:25.956 INFO:tasks.workunit.client.0.smithi027.stdout:2/465: write d6/dc/d3f/d77/d55/d69/d82/f79 [167421,117921] 0 2024-06-06T08:43:25.959 INFO:tasks.workunit.client.0.smithi027.stdout:3/341: dread d1/d8/d22/d27/f32 [0,4194304] 0 2024-06-06T08:43:25.961 INFO:tasks.workunit.client.0.smithi027.stdout:2/466: mknod d6/dc/d3f/ca6 0 2024-06-06T08:43:25.963 INFO:tasks.workunit.client.0.smithi027.stdout:3/342: symlink d1/d8/d22/l62 0 2024-06-06T08:43:25.966 INFO:tasks.workunit.client.0.smithi027.stdout:3/343: creat d1/da/d44/d54/f63 x:0 0 0 2024-06-06T08:43:25.966 INFO:tasks.workunit.client.0.smithi027.stdout:3/344: write d1/f6 [903923,66036] 0 2024-06-06T08:43:25.971 INFO:tasks.workunit.client.0.smithi027.stdout:3/345: dread d1/f41 [0,4194304] 0 2024-06-06T08:43:25.972 INFO:tasks.workunit.client.0.smithi027.stdout:6/406: rename d4/de/d45/d6a/l67 to d4/de/d45/l85 0 2024-06-06T08:43:25.972 INFO:tasks.workunit.client.0.smithi027.stdout:6/407: fsync d4/d9/dc/f43 0 2024-06-06T08:43:25.974 INFO:tasks.workunit.client.0.smithi027.stdout:3/346: symlink d1/d1f/d45/d5f/l64 0 2024-06-06T08:43:25.974 INFO:tasks.workunit.client.0.smithi027.stdout:9/331: rename d3/d4e/d75/dc/c25 to d3/d4e/d75/dc/df/c77 0 2024-06-06T08:43:25.974 INFO:tasks.workunit.client.0.smithi027.stdout:9/332: dread - d3/d4e/d75/dc/df/f57 zero size 2024-06-06T08:43:25.975 INFO:tasks.workunit.client.0.smithi027.stdout:8/312: rename d7/d2f/f5e to d7/db/d60/f68 0 2024-06-06T08:43:25.976 INFO:tasks.workunit.client.0.smithi027.stdout:2/467: dwrite d6/dc/f25 [0,4194304] 0 2024-06-06T08:43:25.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/333: mkdir d3/d4e/d75/dc/df/d23/d78 0 2024-06-06T08:43:25.976 INFO:tasks.workunit.client.0.smithi027.stdout:8/313: creat d7/dc/f69 x:0 0 0 2024-06-06T08:43:25.978 INFO:tasks.workunit.client.0.smithi027.stdout:9/334: rmdir d3/d4e/d75/dc/df/d23 39 2024-06-06T08:43:25.978 INFO:tasks.workunit.client.0.smithi027.stdout:9/335: fsync d3/f5f 0 2024-06-06T08:43:25.978 INFO:tasks.workunit.client.0.smithi027.stdout:8/314: rename d7/d13/d15/c4d to d7/dc/d36/c6a 0 2024-06-06T08:43:25.979 INFO:tasks.workunit.client.0.smithi027.stdout:8/315: mknod d7/d13/c6b 0 2024-06-06T08:43:25.981 INFO:tasks.workunit.client.0.smithi027.stdout:8/316: mknod d7/dc/d37/d46/d50/c6c 0 2024-06-06T08:43:25.982 INFO:tasks.workunit.client.0.smithi027.stdout:3/347: dread d1/d8/d22/d27/f32 [0,4194304] 0 2024-06-06T08:43:25.982 INFO:tasks.workunit.client.0.smithi027.stdout:3/348: write d1/da/f18 [4286975,3377] 0 2024-06-06T08:43:25.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:25 smithi203 ceph-mon[36579]: scrub summary: idle 2024-06-06T08:43:25.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:25 smithi203 ceph-mon[36579]: scrub summary: idle 2024-06-06T08:43:25.998 INFO:tasks.workunit.client.0.smithi027.stdout:2/468: dwrite d6/dc/f85 [0,4194304] 0 2024-06-06T08:43:26.002 INFO:tasks.workunit.client.0.smithi027.stdout:9/336: dwrite d3/d4e/d75/dc/df/d23/d36/f6c [0,4194304] 0 2024-06-06T08:43:26.002 INFO:tasks.workunit.client.0.smithi027.stdout:6/408: dwrite d4/d9/f2d [0,4194304] 0 2024-06-06T08:43:26.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:25 smithi176 ceph-mon[36786]: scrub summary: idle 2024-06-06T08:43:26.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:25 smithi176 ceph-mon[36786]: scrub summary: idle 2024-06-06T08:43:26.010 INFO:tasks.workunit.client.0.smithi027.stdout:6/409: creat d4/d9/d49/f86 x:0 0 0 2024-06-06T08:43:26.010 INFO:tasks.workunit.client.0.smithi027.stdout:8/317: dwrite d7/d13/f17 [0,4194304] 0 2024-06-06T08:43:26.010 INFO:tasks.workunit.client.0.smithi027.stdout:8/318: readlink d7/dc/d37/d46/l59 0 2024-06-06T08:43:26.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/349: dwrite d1/f2 [0,4194304] 0 2024-06-06T08:43:26.010 INFO:tasks.workunit.client.0.smithi027.stdout:6/410: chown d4/d1a/d2e/f3c 244776 1 2024-06-06T08:43:26.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/411: creat d4/d9/dc/f87 x:0 0 0 2024-06-06T08:43:26.012 INFO:tasks.workunit.client.0.smithi027.stdout:8/319: link d7/dc/d37/d46/c5d d7/db/d60/c6d 0 2024-06-06T08:43:26.013 INFO:tasks.workunit.client.0.smithi027.stdout:8/320: readlink d7/db/d58/l67 0 2024-06-06T08:43:26.014 INFO:tasks.workunit.client.0.smithi027.stdout:9/337: write d3/d4e/d75/dc/df/d23/d36/d3a/f6d [1112242,84807] 0 2024-06-06T08:43:26.014 INFO:tasks.workunit.client.0.smithi027.stdout:8/321: creat d7/db/f6e x:0 0 0 2024-06-06T08:43:26.014 INFO:tasks.workunit.client.0.smithi027.stdout:9/338: fdatasync d3/f9 0 2024-06-06T08:43:26.015 INFO:tasks.workunit.client.0.smithi027.stdout:3/350: rename d1/d8/d22/l62 to d1/d8/l65 0 2024-06-06T08:43:26.015 INFO:tasks.workunit.client.0.smithi027.stdout:9/339: creat d3/d4e/d75/dc/df/d34/d41/f79 x:0 0 0 2024-06-06T08:43:26.016 INFO:tasks.workunit.client.0.smithi027.stdout:6/412: dread d4/d1a/d2e/d44/f74 [0,4194304] 0 2024-06-06T08:43:26.016 INFO:tasks.workunit.client.0.smithi027.stdout:3/351: mknod d1/c66 0 2024-06-06T08:43:26.016 INFO:tasks.workunit.client.0.smithi027.stdout:9/340: mkdir d3/d4e/d75/dc/df/d34/d41/d7a 0 2024-06-06T08:43:26.016 INFO:tasks.workunit.client.0.smithi027.stdout:9/341: write d3/d4e/d75/dc/df/d23/d36/f6b [1029967,116695] 0 2024-06-06T08:43:26.016 INFO:tasks.workunit.client.0.smithi027.stdout:9/342: truncate d3/d4e/d75/dc/df/f57 128595 0 2024-06-06T08:43:26.017 INFO:tasks.workunit.client.0.smithi027.stdout:6/413: link d4/d13/c22 d4/c88 0 2024-06-06T08:43:26.018 INFO:tasks.workunit.client.0.smithi027.stdout:6/414: creat d4/de/d45/f89 x:0 0 0 2024-06-06T08:43:26.027 INFO:tasks.workunit.client.0.smithi027.stdout:9/343: write d3/d4e/d75/dc/df/d23/d36/f74 [3373427,88811] 0 2024-06-06T08:43:26.039 INFO:tasks.workunit.client.0.smithi027.stdout:6/415: write d4/d9/f26 [2427326,49094] 0 2024-06-06T08:43:26.040 INFO:tasks.workunit.client.0.smithi027.stdout:8/322: dwrite d7/f4e [0,4194304] 0 2024-06-06T08:43:26.043 INFO:tasks.workunit.client.0.smithi027.stdout:3/352: dwrite d1/d8/d22/f46 [0,4194304] 0 2024-06-06T08:43:26.047 INFO:tasks.workunit.client.0.smithi027.stdout:9/344: dwrite d3/d4e/d75/dc/df/f2b [0,4194304] 0 2024-06-06T08:43:26.048 INFO:tasks.workunit.client.0.smithi027.stdout:6/416: unlink d4/d9/dc/l6b 0 2024-06-06T08:43:26.048 INFO:tasks.workunit.client.0.smithi027.stdout:6/417: chown d4/d1a/f2c 1556866147 1 2024-06-06T08:43:26.053 INFO:tasks.workunit.client.0.smithi027.stdout:9/345: rmdir d3/d4e/d75/dc/df/d34/d41/d7a 0 2024-06-06T08:43:26.054 INFO:tasks.workunit.client.0.smithi027.stdout:6/418: symlink d4/d9/d49/l8a 0 2024-06-06T08:43:26.054 INFO:tasks.workunit.client.0.smithi027.stdout:6/419: dread - d4/f59 zero size 2024-06-06T08:43:26.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/420: symlink d4/d13/l8b 0 2024-06-06T08:43:26.055 INFO:tasks.workunit.client.0.smithi027.stdout:9/346: mknod d3/d4e/d75/dc/df/d23/c7b 0 2024-06-06T08:43:26.056 INFO:tasks.workunit.client.0.smithi027.stdout:9/347: creat d3/f7c x:0 0 0 2024-06-06T08:43:26.057 INFO:tasks.workunit.client.0.smithi027.stdout:6/421: unlink d4/d9/dc/f38 0 2024-06-06T08:43:26.057 INFO:tasks.workunit.client.0.smithi027.stdout:6/422: readlink d4/d9/d49/l84 0 2024-06-06T08:43:26.057 INFO:tasks.workunit.client.0.smithi027.stdout:6/423: stat d4/l42 0 2024-06-06T08:43:26.057 INFO:tasks.workunit.client.0.smithi027.stdout:6/424: write d4/d9/d49/f86 [291012,85831] 0 2024-06-06T08:43:26.061 INFO:tasks.workunit.client.0.smithi027.stdout:8/323: dwrite d7/f42 [0,4194304] 0 2024-06-06T08:43:26.068 INFO:tasks.workunit.client.0.smithi027.stdout:9/348: dread d3/d4e/d75/dc/f30 [4194304,4194304] 0 2024-06-06T08:43:26.068 INFO:tasks.workunit.client.0.smithi027.stdout:9/349: dread - d3/f7c zero size 2024-06-06T08:43:26.069 INFO:tasks.workunit.client.0.smithi027.stdout:9/350: mknod d3/d76/c7d 0 2024-06-06T08:43:26.069 INFO:tasks.workunit.client.0.smithi027.stdout:9/351: write d3/d4e/d75/f55 [1039840,13240] 0 2024-06-06T08:43:26.069 INFO:tasks.workunit.client.0.smithi027.stdout:9/352: write d3/d4e/d75/dc/f2e [541001,22503] 0 2024-06-06T08:43:26.070 INFO:tasks.workunit.client.0.smithi027.stdout:9/353: getdents d3/d4e/d75/dc/df/d34/d41 0 2024-06-06T08:43:26.070 INFO:tasks.workunit.client.0.smithi027.stdout:9/354: fsync d3/d4e/d75/dc/df/d23/d36/f74 0 2024-06-06T08:43:26.070 INFO:tasks.workunit.client.0.smithi027.stdout:9/355: stat d3/d4e/d75/dc/df/d23/c5c 0 2024-06-06T08:43:26.072 INFO:tasks.workunit.client.0.smithi027.stdout:9/356: creat d3/d4e/d75/dc/f7e x:0 0 0 2024-06-06T08:43:26.074 INFO:tasks.workunit.client.0.smithi027.stdout:9/357: mknod d3/d4e/d75/dc/df/d23/d78/c7f 0 2024-06-06T08:43:26.074 INFO:tasks.workunit.client.0.smithi027.stdout:9/358: chown d3/d4e/d75/dc/df/d34/l6a 1 1 2024-06-06T08:43:26.076 INFO:tasks.workunit.client.0.smithi027.stdout:9/359: rename d3/d4e/d75/dc/f7e to d3/d4e/d75/dc/f80 0 2024-06-06T08:43:26.076 INFO:tasks.workunit.client.0.smithi027.stdout:8/324: dread d7/d13/f17 [0,4194304] 0 2024-06-06T08:43:26.078 INFO:tasks.workunit.client.0.smithi027.stdout:6/425: dwrite d4/d9/dc/d18/f24 [0,4194304] 0 2024-06-06T08:43:26.078 INFO:tasks.workunit.client.0.smithi027.stdout:6/426: fdatasync d4/f7f 0 2024-06-06T08:43:26.091 INFO:tasks.workunit.client.0.smithi027.stdout:8/325: write d7/f22 [1091971,86818] 0 2024-06-06T08:43:26.092 INFO:tasks.workunit.client.0.smithi027.stdout:8/326: write d7/db/d60/f68 [227456,27108] 0 2024-06-06T08:43:26.093 INFO:tasks.workunit.client.0.smithi027.stdout:9/360: rename d3/f7c to d3/d4e/d75/dc/df/d34/d41/f81 0 2024-06-06T08:43:26.098 INFO:tasks.workunit.client.0.smithi027.stdout:6/427: dwrite d4/f7f [0,4194304] 0 2024-06-06T08:43:26.098 INFO:tasks.workunit.client.0.smithi027.stdout:6/428: readlink d4/d9/dc/d1c/l5b 0 2024-06-06T08:43:26.101 INFO:tasks.workunit.client.0.smithi027.stdout:6/429: creat d4/de/f8c x:0 0 0 2024-06-06T08:43:26.101 INFO:tasks.workunit.client.0.smithi027.stdout:6/430: chown d4/d9/f2d 182655620 1 2024-06-06T08:43:26.103 INFO:tasks.workunit.client.0.smithi027.stdout:6/431: link d4/d9/f28 d4/d9/d49/f8d 0 2024-06-06T08:43:26.104 INFO:tasks.workunit.client.0.smithi027.stdout:6/432: readlink d4/d9/dc/d18/l53 0 2024-06-06T08:43:26.104 INFO:tasks.workunit.client.0.smithi027.stdout:6/433: chown d4/c40 123 1 2024-06-06T08:43:26.105 INFO:tasks.workunit.client.0.smithi027.stdout:6/434: mkdir d4/d9/d8e 0 2024-06-06T08:43:26.106 INFO:tasks.workunit.client.0.smithi027.stdout:6/435: fsync d4/d9/d49/f71 0 2024-06-06T08:43:26.110 INFO:tasks.workunit.client.0.smithi027.stdout:9/361: dwrite d3/d4e/d75/dc/df/d23/d36/d3a/f6d [0,4194304] 0 2024-06-06T08:43:26.118 INFO:tasks.workunit.client.0.smithi027.stdout:9/362: write d3/d4e/d75/dc/df/d23/d36/f6c [688367,7012] 0 2024-06-06T08:43:26.120 INFO:tasks.workunit.client.0.smithi027.stdout:8/327: dwrite d7/dc/d36/d41/f34 [0,4194304] 0 2024-06-06T08:43:26.120 INFO:tasks.workunit.client.0.smithi027.stdout:8/328: readlink d7/l63 0 2024-06-06T08:43:26.123 INFO:tasks.workunit.client.0.smithi027.stdout:6/436: dwrite d4/d1a/d2e/f7e [0,4194304] 0 2024-06-06T08:43:26.138 INFO:tasks.workunit.client.0.smithi027.stdout:9/363: dwrite d3/d4e/d75/dc/df/d34/d41/f79 [0,4194304] 0 2024-06-06T08:43:26.138 INFO:tasks.workunit.client.0.smithi027.stdout:9/364: chown d3/c5b 11 1 2024-06-06T08:43:26.141 INFO:tasks.workunit.client.0.smithi027.stdout:9/365: symlink d3/d4e/d75/dc/l82 0 2024-06-06T08:43:26.143 INFO:tasks.workunit.client.0.smithi027.stdout:9/366: mknod d3/d4e/d75/dc/df/d34/d73/c83 0 2024-06-06T08:43:26.143 INFO:tasks.workunit.client.0.smithi027.stdout:6/437: dwrite d4/d9/f2d [4194304,4194304] 0 2024-06-06T08:43:26.143 INFO:tasks.workunit.client.0.smithi027.stdout:8/329: dwrite d7/db/f6e [0,4194304] 0 2024-06-06T08:43:26.146 INFO:tasks.workunit.client.0.smithi027.stdout:9/367: mkdir d3/d4e/d75/dc/df/d23/d36/d3a/d84 0 2024-06-06T08:43:26.147 INFO:tasks.workunit.client.0.smithi027.stdout:8/330: rmdir d7/dc 39 2024-06-06T08:43:26.148 INFO:tasks.workunit.client.0.smithi027.stdout:8/331: mknod d7/d13/c6f 0 2024-06-06T08:43:26.165 INFO:tasks.workunit.client.0.smithi027.stdout:9/368: dwrite d3/d4e/d75/dc/df/d23/d36/f6b [0,4194304] 0 2024-06-06T08:43:26.166 INFO:tasks.workunit.client.0.smithi027.stdout:9/369: write d3/d4e/d75/f2c [566170,32757] 0 2024-06-06T08:43:26.166 INFO:tasks.workunit.client.0.smithi027.stdout:9/370: chown d3/d4e/d75/dc/df/d23/d78 8943 1 2024-06-06T08:43:26.166 INFO:tasks.workunit.client.0.smithi027.stdout:8/332: dwrite d7/d2f/f3b [0,4194304] 0 2024-06-06T08:43:26.169 INFO:tasks.workunit.client.0.smithi027.stdout:8/333: creat d7/d2f/f70 x:0 0 0 2024-06-06T08:43:26.170 INFO:tasks.workunit.client.0.smithi027.stdout:8/334: symlink d7/dc/d36/l71 0 2024-06-06T08:43:26.171 INFO:tasks.workunit.client.0.smithi027.stdout:8/335: getdents d7/dc/d37/d46/d50 0 2024-06-06T08:43:26.173 INFO:tasks.workunit.client.0.smithi027.stdout:8/336: creat d7/dc/d36/d3c/f72 x:0 0 0 2024-06-06T08:43:26.175 INFO:tasks.workunit.client.0.smithi027.stdout:8/337: mkdir d7/dc/d37/d46/d50/d73 0 2024-06-06T08:43:26.178 INFO:tasks.workunit.client.0.smithi027.stdout:8/338: unlink d7/dc/d36/d3c/f51 0 2024-06-06T08:43:26.178 INFO:tasks.workunit.client.0.smithi027.stdout:8/339: fsync d7/db/f11 0 2024-06-06T08:43:26.178 INFO:tasks.workunit.client.0.smithi027.stdout:8/340: stat d7/dc/c4b 0 2024-06-06T08:43:26.178 INFO:tasks.workunit.client.0.smithi027.stdout:8/341: fsync d7/db/d60/f68 0 2024-06-06T08:43:26.179 INFO:tasks.workunit.client.0.smithi027.stdout:8/342: link d7/dc/d36/d3c/l4c d7/dc/d37/d46/l74 0 2024-06-06T08:43:26.183 INFO:tasks.workunit.client.0.smithi027.stdout:9/371: dwrite d3/d4e/d75/dc/df/d34/d41/f79 [0,4194304] 0 2024-06-06T08:43:26.185 INFO:tasks.workunit.client.0.smithi027.stdout:9/372: creat d3/d4e/d75/dc/df/d34/d73/f85 x:0 0 0 2024-06-06T08:43:26.190 INFO:tasks.workunit.client.0.smithi027.stdout:8/343: write d7/f4e [613735,99390] 0 2024-06-06T08:43:26.191 INFO:tasks.workunit.client.0.smithi027.stdout:8/344: creat d7/dc/d36/f75 x:0 0 0 2024-06-06T08:43:26.199 INFO:tasks.workunit.client.0.smithi027.stdout:8/345: dread d7/f30 [0,4194304] 0 2024-06-06T08:43:26.200 INFO:tasks.workunit.client.0.smithi027.stdout:9/373: dwrite d3/d4e/d75/dc/df/d34/d73/f85 [0,4194304] 0 2024-06-06T08:43:26.201 INFO:tasks.workunit.client.0.smithi027.stdout:8/346: link d7/dc/d37/d46/f52 d7/dc/d37/d46/d50/f76 0 2024-06-06T08:43:26.201 INFO:tasks.workunit.client.0.smithi027.stdout:9/374: creat d3/d76/f86 x:0 0 0 2024-06-06T08:43:26.202 INFO:tasks.workunit.client.0.smithi027.stdout:9/375: fdatasync d3/d4e/d75/dc/df/d23/d36/f74 0 2024-06-06T08:43:26.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/347: rmdir d7/dc 39 2024-06-06T08:43:26.203 INFO:tasks.workunit.client.0.smithi027.stdout:8/348: truncate d7/d13/f1e 830621 0 2024-06-06T08:43:26.216 INFO:tasks.workunit.client.0.smithi027.stdout:9/376: dwrite d3/d4e/d75/dc/df/d34/f60 [0,4194304] 0 2024-06-06T08:43:26.216 INFO:tasks.workunit.client.0.smithi027.stdout:9/377: read - d3/d4e/d75/f50 zero size 2024-06-06T08:43:26.220 INFO:tasks.workunit.client.0.smithi027.stdout:8/349: dwrite d7/d2f/f31 [0,4194304] 0 2024-06-06T08:43:26.230 INFO:tasks.workunit.client.0.smithi027.stdout:8/350: dread d7/dc/d36/d3c/f61 [0,4194304] 0 2024-06-06T08:43:26.231 INFO:tasks.workunit.client.0.smithi027.stdout:8/351: chown d7/f30 5 1 2024-06-06T08:43:26.231 INFO:tasks.workunit.client.0.smithi027.stdout:8/352: fdatasync d7/dc/d37/d46/f52 0 2024-06-06T08:43:26.232 INFO:tasks.workunit.client.0.smithi027.stdout:8/353: rename d7/db/d58/f64 to d7/d2f/f77 0 2024-06-06T08:43:26.234 INFO:tasks.workunit.client.0.smithi027.stdout:9/378: dwrite d3/f4 [4194304,4194304] 0 2024-06-06T08:43:26.235 INFO:tasks.workunit.client.0.smithi027.stdout:9/379: symlink d3/d4e/d75/dc/df/d34/d41/l87 0 2024-06-06T08:43:26.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/380: chown d3/d4e/d75/dc/l19 82922734 1 2024-06-06T08:43:26.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/381: dread - d3/d4e/d75/dc/df/d23/d36/d3a/f67 zero size 2024-06-06T08:43:26.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/382: mkdir d3/d4e/d75/dc/df/d34/d41/d88 0 2024-06-06T08:43:26.237 INFO:tasks.workunit.client.0.smithi027.stdout:9/383: readlink d3/d4e/d75/dc/df/l62 0 2024-06-06T08:43:26.238 INFO:tasks.workunit.client.0.smithi027.stdout:9/384: creat d3/f89 x:0 0 0 2024-06-06T08:43:26.239 INFO:tasks.workunit.client.0.smithi027.stdout:9/385: symlink d3/d4e/d75/dc/df/d34/l8a 0 2024-06-06T08:43:26.241 INFO:tasks.workunit.client.0.smithi027.stdout:8/354: write d7/db/f27 [2615873,16682] 0 2024-06-06T08:43:26.243 INFO:tasks.workunit.client.0.smithi027.stdout:8/355: creat d7/db/d60/f78 x:0 0 0 2024-06-06T08:43:26.245 INFO:tasks.workunit.client.0.smithi027.stdout:8/356: rename d7/f30 to d7/dc/d36/d3c/f79 0 2024-06-06T08:43:26.246 INFO:tasks.workunit.client.0.smithi027.stdout:8/357: chown d7/dc/f12 0 1 2024-06-06T08:43:26.247 INFO:tasks.workunit.client.0.smithi027.stdout:8/358: creat d7/db/d60/f7a x:0 0 0 2024-06-06T08:43:26.247 INFO:tasks.workunit.client.0.smithi027.stdout:8/359: dread - d7/d13/d15/f1c zero size 2024-06-06T08:43:26.249 INFO:tasks.workunit.client.0.smithi027.stdout:8/360: creat d7/dc/d36/f7b x:0 0 0 2024-06-06T08:43:26.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/386: dread d3/f15 [0,4194304] 0 2024-06-06T08:43:26.250 INFO:tasks.workunit.client.0.smithi027.stdout:9/387: read d3/d4e/d75/dc/df/f57 [36562,660] 0 2024-06-06T08:43:26.251 INFO:tasks.workunit.client.0.smithi027.stdout:8/361: rename d7/l44 to d7/db/d58/l7c 0 2024-06-06T08:43:26.254 INFO:tasks.workunit.client.0.smithi027.stdout:8/362: rename d7/c23 to d7/d13/d15/c7d 0 2024-06-06T08:43:26.255 INFO:tasks.workunit.client.0.smithi027.stdout:8/363: symlink d7/dc/d37/l7e 0 2024-06-06T08:43:26.257 INFO:tasks.workunit.client.0.smithi027.stdout:8/364: creat d7/dc/d37/d46/d50/f7f x:0 0 0 2024-06-06T08:43:26.257 INFO:tasks.workunit.client.0.smithi027.stdout:8/365: dread - d7/dc/d36/f7b zero size 2024-06-06T08:43:26.258 INFO:tasks.workunit.client.0.smithi027.stdout:8/366: rmdir d7/d2f 39 2024-06-06T08:43:26.258 INFO:tasks.workunit.client.0.smithi027.stdout:8/367: stat d7/dc/d37/d46/d50/c6c 0 2024-06-06T08:43:26.260 INFO:tasks.workunit.client.0.smithi027.stdout:8/368: rmdir d7/d13/d15 39 2024-06-06T08:43:26.261 INFO:tasks.workunit.client.0.smithi027.stdout:8/369: creat d7/dc/d37/f80 x:0 0 0 2024-06-06T08:43:26.263 INFO:tasks.workunit.client.0.smithi027.stdout:8/370: creat d7/db/d60/f81 x:0 0 0 2024-06-06T08:43:26.264 INFO:tasks.workunit.client.0.smithi027.stdout:8/371: mkdir d7/db/d58/d82 0 2024-06-06T08:43:26.266 INFO:tasks.workunit.client.0.smithi027.stdout:9/388: dwrite d3/d4e/d75/dc/df/d23/d36/f74 [0,4194304] 0 2024-06-06T08:43:26.274 INFO:tasks.workunit.client.0.smithi027.stdout:9/389: getdents d3/d4e 0 2024-06-06T08:43:26.275 INFO:tasks.workunit.client.0.smithi027.stdout:9/390: truncate d3/d4e/d75/dc/df/d34/d41/f79 1234364 0 2024-06-06T08:43:26.275 INFO:tasks.workunit.client.0.smithi027.stdout:9/391: fsync d3/d4e/d75/dc/df/d23/d36/f5a 0 2024-06-06T08:43:26.275 INFO:tasks.workunit.client.0.smithi027.stdout:9/392: chown d3/c26 729 1 2024-06-06T08:43:26.275 INFO:tasks.workunit.client.0.smithi027.stdout:9/393: write d3/d4e/d75/dc/df/f4f [1976286,83938] 0 2024-06-06T08:43:26.291 INFO:tasks.workunit.client.0.smithi027.stdout:8/372: dwrite d7/dc/d37/d46/d50/f62 [0,4194304] 0 2024-06-06T08:43:26.297 INFO:tasks.workunit.client.0.smithi027.stdout:8/373: rename d7/db to d7/dc/d36/d83 0 2024-06-06T08:43:26.298 INFO:tasks.workunit.client.0.smithi027.stdout:8/374: symlink d7/d13/d15/l84 0 2024-06-06T08:43:26.471 INFO:tasks.workunit.client.0.smithi027.stdout:1/384: sync 2024-06-06T08:43:26.471 INFO:tasks.workunit.client.0.smithi027.stdout:7/601: sync 2024-06-06T08:43:26.471 INFO:tasks.workunit.client.0.smithi027.stdout:0/379: sync 2024-06-06T08:43:26.471 INFO:tasks.workunit.client.0.smithi027.stdout:4/477: sync 2024-06-06T08:43:26.471 INFO:tasks.workunit.client.0.smithi027.stdout:2/469: sync 2024-06-06T08:43:26.471 INFO:tasks.workunit.client.0.smithi027.stdout:7/602: read - d17/d47/f5d zero size 2024-06-06T08:43:26.472 INFO:tasks.workunit.client.0.smithi027.stdout:4/478: chown d0/d1/d6/d4b/f56 1629385454 1 2024-06-06T08:43:26.472 INFO:tasks.workunit.client.0.smithi027.stdout:3/353: sync 2024-06-06T08:43:26.472 INFO:tasks.workunit.client.0.smithi027.stdout:3/354: stat d1/d1f/d4d 0 2024-06-06T08:43:26.472 INFO:tasks.workunit.client.0.smithi027.stdout:5/548: sync 2024-06-06T08:43:26.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/438: sync 2024-06-06T08:43:26.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/439: write d4/de/f6c [684738,105804] 0 2024-06-06T08:43:26.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/440: write d4/f69 [715721,53655] 0 2024-06-06T08:43:26.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/441: truncate d4/d1a/f2c 1347386 0 2024-06-06T08:43:26.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/442: readlink d4/de/l63 0 2024-06-06T08:43:26.473 INFO:tasks.workunit.client.0.smithi027.stdout:0/380: creat d15/d18/d23/d28/d2a/d30/d5c/f82 x:0 0 0 2024-06-06T08:43:26.475 INFO:tasks.workunit.client.0.smithi027.stdout:2/470: creat d6/fa7 x:0 0 0 2024-06-06T08:43:26.476 INFO:tasks.workunit.client.0.smithi027.stdout:0/381: mkdir d15/d18/d23/d28/d2a/d30/d5c/d83 0 2024-06-06T08:43:26.476 INFO:tasks.workunit.client.0.smithi027.stdout:0/382: write d15/d18/f1c [1017889,20205] 0 2024-06-06T08:43:26.477 INFO:tasks.workunit.client.0.smithi027.stdout:5/549: mknod d0/da/db/d46/d7f/d81/cab 0 2024-06-06T08:43:26.478 INFO:tasks.workunit.client.0.smithi027.stdout:2/471: creat d6/dc/d1d/d4d/d8e/fa8 x:0 0 0 2024-06-06T08:43:26.478 INFO:tasks.workunit.client.0.smithi027.stdout:6/443: mkdir d4/d9/dc/d8f 0 2024-06-06T08:43:26.480 INFO:tasks.workunit.client.0.smithi027.stdout:2/472: creat d6/dc/d3f/d77/d55/d69/d82/fa9 x:0 0 0 2024-06-06T08:43:26.481 INFO:tasks.workunit.client.0.smithi027.stdout:0/383: creat d15/d18/d23/d28/d31/f84 x:0 0 0 2024-06-06T08:43:26.481 INFO:tasks.workunit.client.0.smithi027.stdout:0/384: write d15/d18/d23/d3c/f67 [320543,39609] 0 2024-06-06T08:43:26.482 INFO:tasks.workunit.client.0.smithi027.stdout:6/444: mknod d4/de/c90 0 2024-06-06T08:43:26.483 INFO:tasks.workunit.client.0.smithi027.stdout:4/479: dread d0/fd [4194304,4194304] 0 2024-06-06T08:43:26.483 INFO:tasks.workunit.client.0.smithi027.stdout:4/480: chown d0/d1/f4f 463 1 2024-06-06T08:43:26.483 INFO:tasks.workunit.client.0.smithi027.stdout:2/473: mknod d6/dc/caa 0 2024-06-06T08:43:26.484 INFO:tasks.workunit.client.0.smithi027.stdout:9/394: sync 2024-06-06T08:43:26.484 INFO:tasks.workunit.client.0.smithi027.stdout:8/375: rename d7/dc/d36/d83/d58/d82 to d7/dc/d36/d41/d85 0 2024-06-06T08:43:26.484 INFO:tasks.workunit.client.0.smithi027.stdout:8/376: write d7/dc/f69 [986295,28048] 0 2024-06-06T08:43:26.484 INFO:tasks.workunit.client.0.smithi027.stdout:8/377: stat d7/dc/d36 0 2024-06-06T08:43:26.484 INFO:tasks.workunit.client.0.smithi027.stdout:4/481: creat d0/d1/f96 x:0 0 0 2024-06-06T08:43:26.485 INFO:tasks.workunit.client.0.smithi027.stdout:4/482: fsync d0/d1/d6/d8/d21/f4a 0 2024-06-06T08:43:26.486 INFO:tasks.workunit.client.0.smithi027.stdout:9/395: symlink d3/d4e/d75/dc/df/d23/d36/l8b 0 2024-06-06T08:43:26.486 INFO:tasks.workunit.client.0.smithi027.stdout:8/378: mknod d7/dc/d36/d83/c86 0 2024-06-06T08:43:26.488 INFO:tasks.workunit.client.0.smithi027.stdout:3/355: dread d1/f6 [0,4194304] 0 2024-06-06T08:43:26.488 INFO:tasks.workunit.client.0.smithi027.stdout:9/396: symlink d3/d4e/d75/dc/df/d34/d41/d88/l8c 0 2024-06-06T08:43:26.489 INFO:tasks.workunit.client.0.smithi027.stdout:7/603: rename d17/d47/f5d to d17/d32/d6c/dc2/fc5 0 2024-06-06T08:43:26.489 INFO:tasks.workunit.client.0.smithi027.stdout:7/604: stat d17/d32/d46/d4f/d5e/ca7 0 2024-06-06T08:43:26.491 INFO:tasks.workunit.client.0.smithi027.stdout:9/397: creat d3/d4e/d75/dc/f8d x:0 0 0 2024-06-06T08:43:26.491 INFO:tasks.workunit.client.0.smithi027.stdout:3/356: truncate d1/f13 637760 0 2024-06-06T08:43:26.492 INFO:tasks.workunit.client.0.smithi027.stdout:7/605: mknod d17/d32/d46/d51/d52/d70/d87/cc6 0 2024-06-06T08:43:26.493 INFO:tasks.workunit.client.0.smithi027.stdout:9/398: mknod d3/d76/c8e 0 2024-06-06T08:43:26.493 INFO:tasks.workunit.client.0.smithi027.stdout:9/399: truncate d3/d4e/d75/dc/df/f57 701000 0 2024-06-06T08:43:26.494 INFO:tasks.workunit.client.0.smithi027.stdout:7/606: mkdir d17/d32/d46/d83/d2a/d3b/dc7 0 2024-06-06T08:43:26.495 INFO:tasks.workunit.client.0.smithi027.stdout:6/445: dread d4/d1a/d2e/f5c [0,4194304] 0 2024-06-06T08:43:26.496 INFO:tasks.workunit.client.0.smithi027.stdout:3/357: symlink d1/d1f/l67 0 2024-06-06T08:43:26.496 INFO:tasks.workunit.client.0.smithi027.stdout:3/358: chown d1/d8/d22/d27/l3f 13433878 1 2024-06-06T08:43:26.496 INFO:tasks.workunit.client.0.smithi027.stdout:3/359: write d1/d8/f61 [823310,5950] 0 2024-06-06T08:43:26.496 INFO:tasks.workunit.client.0.smithi027.stdout:3/360: readlink d1/d8/d22/l5a 0 2024-06-06T08:43:26.497 INFO:tasks.workunit.client.0.smithi027.stdout:1/385: dwrite dc/de/d14/f87 [0,4194304] 0 2024-06-06T08:43:26.499 INFO:tasks.workunit.client.0.smithi027.stdout:7/607: mkdir d17/d32/d46/d4f/db6/dc8 0 2024-06-06T08:43:26.506 INFO:tasks.workunit.client.0.smithi027.stdout:5/550: dwrite d0/da/db/d1a/f7b [0,4194304] 0 2024-06-06T08:43:26.506 INFO:tasks.workunit.client.0.smithi027.stdout:1/386: rmdir dc/d67/d69/d32/d71 39 2024-06-06T08:43:26.512 INFO:tasks.workunit.client.0.smithi027.stdout:8/379: write d7/dc/d36/d83/f6e [3899816,40373] 0 2024-06-06T08:43:26.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/608: write d17/d32/f2f [1131713,96243] 0 2024-06-06T08:43:26.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/609: stat c6 0 2024-06-06T08:43:26.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/610: fdatasync d17/d32/d46/d4f/f55 0 2024-06-06T08:43:26.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/611: fsync d17/d47/f49 0 2024-06-06T08:43:26.515 INFO:tasks.workunit.client.0.smithi027.stdout:7/612: write d17/d32/f2f [1653746,2976] 0 2024-06-06T08:43:26.516 INFO:tasks.workunit.client.0.smithi027.stdout:0/385: dwrite d15/d18/d23/d38/d68/f7b [0,4194304] 0 2024-06-06T08:43:26.516 INFO:tasks.workunit.client.0.smithi027.stdout:0/386: chown d15/d18/d23/d28/d31 23 1 2024-06-06T08:43:26.517 INFO:tasks.workunit.client.0.smithi027.stdout:8/380: link d7/dc/d36/d83/f1d d7/d2f/f87 0 2024-06-06T08:43:26.518 INFO:tasks.workunit.client.0.smithi027.stdout:4/483: dwrite d0/fd [8388608,4194304] 0 2024-06-06T08:43:26.519 INFO:tasks.workunit.client.0.smithi027.stdout:5/551: symlink d0/da/d47/lac 0 2024-06-06T08:43:26.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/381: link d7/dc/d36/l71 d7/dc/d37/d46/d50/d73/l88 0 2024-06-06T08:43:26.523 INFO:tasks.workunit.client.0.smithi027.stdout:4/484: mkdir d0/d1d/d34/d7f/d97 0 2024-06-06T08:43:26.523 INFO:tasks.workunit.client.0.smithi027.stdout:9/400: write d3/f15 [312956,17991] 0 2024-06-06T08:43:26.524 INFO:tasks.workunit.client.0.smithi027.stdout:5/552: getdents d0/da/d37 0 2024-06-06T08:43:26.524 INFO:tasks.workunit.client.0.smithi027.stdout:2/474: dwrite d6/f14 [0,4194304] 0 2024-06-06T08:43:26.524 INFO:tasks.workunit.client.0.smithi027.stdout:2/475: chown d6/dc/d3f/d77/d55/f6d 471589 1 2024-06-06T08:43:26.524 INFO:tasks.workunit.client.0.smithi027.stdout:2/476: chown d6/dc/d3f/d77/d55/d69 708260439 1 2024-06-06T08:43:26.525 INFO:tasks.workunit.client.0.smithi027.stdout:3/361: read d1/d8/d22/d27/d50/f51 [3002464,13316] 0 2024-06-06T08:43:26.526 INFO:tasks.workunit.client.0.smithi027.stdout:7/613: symlink d17/d32/lc9 0 2024-06-06T08:43:26.526 INFO:tasks.workunit.client.0.smithi027.stdout:8/382: mknod d7/dc/d37/d46/d50/c89 0 2024-06-06T08:43:26.526 INFO:tasks.workunit.client.0.smithi027.stdout:8/383: readlink d7/dc/d36/d3c/l48 0 2024-06-06T08:43:26.526 INFO:tasks.workunit.client.0.smithi027.stdout:8/384: truncate d7/dc/d36/f75 355346 0 2024-06-06T08:43:26.527 INFO:tasks.workunit.client.0.smithi027.stdout:8/385: link d7/d13/d15/f1c d7/dc/d36/d83/d58/f8a 0 2024-06-06T08:43:26.527 INFO:tasks.workunit.client.0.smithi027.stdout:8/386: fdatasync d7/f22 0 2024-06-06T08:43:26.527 INFO:tasks.workunit.client.0.smithi027.stdout:0/387: write d15/d18/d23/d28/d2a/d2d/f3a [2475359,40612] 0 2024-06-06T08:43:26.528 INFO:tasks.workunit.client.0.smithi027.stdout:7/614: rmdir d17/d32/d46/d4f/d5e/d7b/dc1 0 2024-06-06T08:43:26.528 INFO:tasks.workunit.client.0.smithi027.stdout:0/388: mkdir d15/d18/d23/d38/d68/d85 0 2024-06-06T08:43:26.530 INFO:tasks.workunit.client.0.smithi027.stdout:8/387: symlink d7/dc/d36/l8b 0 2024-06-06T08:43:26.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/401: dread d3/f9 [0,4194304] 0 2024-06-06T08:43:26.532 INFO:tasks.workunit.client.0.smithi027.stdout:1/387: dread dc/d67/d69/f27 [0,4194304] 0 2024-06-06T08:43:26.533 INFO:tasks.workunit.client.0.smithi027.stdout:1/388: write dc/f84 [164745,71919] 0 2024-06-06T08:43:26.533 INFO:tasks.workunit.client.0.smithi027.stdout:1/389: write dc/fd [1984731,111009] 0 2024-06-06T08:43:26.533 INFO:tasks.workunit.client.0.smithi027.stdout:7/615: dread d17/d32/d46/d83/f45 [0,4194304] 0 2024-06-06T08:43:26.535 INFO:tasks.workunit.client.0.smithi027.stdout:1/390: link dc/d67/d69/d32/d71/l7b dc/de/d14/d4e/l88 0 2024-06-06T08:43:26.535 INFO:tasks.workunit.client.0.smithi027.stdout:1/391: chown dc/de/d14/d4e 2729079 1 2024-06-06T08:43:26.536 INFO:tasks.workunit.client.0.smithi027.stdout:7/616: unlink d17/d32/d46/d83/d2a/d3b/f42 0 2024-06-06T08:43:26.538 INFO:tasks.workunit.client.0.smithi027.stdout:2/477: dread d6/dc/d1d/f60 [0,4194304] 0 2024-06-06T08:43:26.541 INFO:tasks.workunit.client.0.smithi027.stdout:7/617: unlink f14 0 2024-06-06T08:43:26.541 INFO:tasks.workunit.client.0.smithi027.stdout:7/618: write d17/d32/f3a [6301529,47398] 0 2024-06-06T08:43:26.543 INFO:tasks.workunit.client.0.smithi027.stdout:2/478: symlink d6/dc/d3f/d77/lab 0 2024-06-06T08:43:26.547 INFO:tasks.workunit.client.0.smithi027.stdout:8/388: dread d7/dc/d36/d83/f6e [0,4194304] 0 2024-06-06T08:43:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:6/446: dwrite d4/f57 [0,4194304] 0 2024-06-06T08:43:26.551 INFO:tasks.workunit.client.0.smithi027.stdout:8/389: creat d7/d13/d15/f8c x:0 0 0 2024-06-06T08:43:26.558 INFO:tasks.workunit.client.0.smithi027.stdout:3/362: dwrite d1/d1f/d25/f3a [0,4194304] 0 2024-06-06T08:43:26.558 INFO:tasks.workunit.client.0.smithi027.stdout:3/363: write d1/d8/d22/d27/f32 [689903,87388] 0 2024-06-06T08:43:26.561 INFO:tasks.workunit.client.0.smithi027.stdout:5/553: dwrite d0/da/f35 [4194304,4194304] 0 2024-06-06T08:43:26.561 INFO:tasks.workunit.client.0.smithi027.stdout:4/485: dwrite d0/f5c [0,4194304] 0 2024-06-06T08:43:26.562 INFO:tasks.workunit.client.0.smithi027.stdout:3/364: symlink d1/da/d44/d54/l68 0 2024-06-06T08:43:26.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/402: dwrite d3/f20 [0,4194304] 0 2024-06-06T08:43:26.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/403: read - d3/d4e/d75/dc/df/d34/f65 zero size 2024-06-06T08:43:26.567 INFO:tasks.workunit.client.0.smithi027.stdout:0/389: dread f4 [8388608,4194304] 0 2024-06-06T08:43:26.567 INFO:tasks.workunit.client.0.smithi027.stdout:0/390: write d15/d18/d23/d28/d2a/d2d/f34 [4290154,82856] 0 2024-06-06T08:43:26.568 INFO:tasks.workunit.client.0.smithi027.stdout:0/391: chown d15/d18/d23/d38/d68/l7a 190262280 1 2024-06-06T08:43:26.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/479: dwrite d6/dc/d3f/d77/f4c [0,4194304] 0 2024-06-06T08:43:26.571 INFO:tasks.workunit.client.0.smithi027.stdout:9/404: symlink d3/d4e/d75/dc/df/d23/d78/l8f 0 2024-06-06T08:43:26.571 INFO:tasks.workunit.client.0.smithi027.stdout:9/405: dread - d3/d4e/d75/dc/df/d23/d36/d3a/f68 zero size 2024-06-06T08:43:26.574 INFO:tasks.workunit.client.0.smithi027.stdout:9/406: unlink d3/d4e/d75/dc/df/d34/d41/d88/l8c 0 2024-06-06T08:43:26.579 INFO:tasks.workunit.client.0.smithi027.stdout:6/447: getdents d4/de/d45/d6a 0 2024-06-06T08:43:26.583 INFO:tasks.workunit.client.0.smithi027.stdout:4/486: dwrite d0/d1/f14 [8388608,4194304] 0 2024-06-06T08:43:26.587 INFO:tasks.workunit.client.0.smithi027.stdout:2/480: dwrite d6/dc/d3f/f50 [0,4194304] 0 2024-06-06T08:43:26.592 INFO:tasks.workunit.client.0.smithi027.stdout:5/554: rename d0/da/db/d46/d7f/d81/cab to d0/cad 0 2024-06-06T08:43:26.594 INFO:tasks.workunit.client.0.smithi027.stdout:0/392: rename d15/d18/d23/d28/f4c to d15/d18/d23/d28/d2a/d30/d5c/d83/f86 0 2024-06-06T08:43:26.598 INFO:tasks.workunit.client.0.smithi027.stdout:9/407: rename d3/d4e/d75/dc/f30 to d3/d4e/d75/dc/df/f90 0 2024-06-06T08:43:26.598 INFO:tasks.workunit.client.0.smithi027.stdout:9/408: dread - d3/d4e/d75/dc/f80 zero size 2024-06-06T08:43:26.598 INFO:tasks.workunit.client.0.smithi027.stdout:9/409: dread - d3/d4e/d75/f50 zero size 2024-06-06T08:43:26.599 INFO:tasks.workunit.client.0.smithi027.stdout:9/410: creat d3/d4e/d75/dc/d71/f91 x:0 0 0 2024-06-06T08:43:26.601 INFO:tasks.workunit.client.0.smithi027.stdout:0/393: dread fc [0,4194304] 0 2024-06-06T08:43:26.602 INFO:tasks.workunit.client.0.smithi027.stdout:9/411: creat d3/d4e/d75/dc/d71/f92 x:0 0 0 2024-06-06T08:43:26.602 INFO:tasks.workunit.client.0.smithi027.stdout:9/412: fsync d3/f15 0 2024-06-06T08:43:26.602 INFO:tasks.workunit.client.0.smithi027.stdout:9/413: read - d3/d4e/d75/dc/df/d34/f65 zero size 2024-06-06T08:43:26.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/394: unlink d15/d18/d23/d38/f77 0 2024-06-06T08:43:26.606 INFO:tasks.workunit.client.0.smithi027.stdout:0/395: creat d15/d18/d64/f87 x:0 0 0 2024-06-06T08:43:26.608 INFO:tasks.workunit.client.0.smithi027.stdout:0/396: getdents d15/d18/d23 0 2024-06-06T08:43:26.609 INFO:tasks.workunit.client.0.smithi027.stdout:0/397: getdents d15/d18/d23/d38/d3e 0 2024-06-06T08:43:26.613 INFO:tasks.workunit.client.0.smithi027.stdout:4/487: dwrite d0/d1d/d2a/f57 [0,4194304] 0 2024-06-06T08:43:26.615 INFO:tasks.workunit.client.0.smithi027.stdout:4/488: mkdir d0/d1d/d2a/d98 0 2024-06-06T08:43:26.616 INFO:tasks.workunit.client.0.smithi027.stdout:5/555: dwrite d0/da/db/d1a/f7b [0,4194304] 0 2024-06-06T08:43:26.618 INFO:tasks.workunit.client.0.smithi027.stdout:9/414: dwrite d3/f4 [4194304,4194304] 0 2024-06-06T08:43:26.619 INFO:tasks.workunit.client.0.smithi027.stdout:9/415: write d3/d4e/d75/dc/df/f39 [1475897,117746] 0 2024-06-06T08:43:26.619 INFO:tasks.workunit.client.0.smithi027.stdout:5/556: link d0/da/d37/d2b/f67 d0/da/fae 0 2024-06-06T08:43:26.620 INFO:tasks.workunit.client.0.smithi027.stdout:9/416: symlink d3/d4e/d75/dc/df/d23/d36/d3a/l93 0 2024-06-06T08:43:26.620 INFO:tasks.workunit.client.0.smithi027.stdout:9/417: fsync d3/d4e/d75/dc/df/f57 0 2024-06-06T08:43:26.630 INFO:tasks.workunit.client.0.smithi027.stdout:6/448: rename d4/d1a/c1f to d4/d9/d49/c91 0 2024-06-06T08:43:26.631 INFO:tasks.workunit.client.0.smithi027.stdout:6/449: creat d4/d13/f92 x:0 0 0 2024-06-06T08:43:26.631 INFO:tasks.workunit.client.0.smithi027.stdout:6/450: mknod d4/d9/dc/d8f/c93 0 2024-06-06T08:43:26.631 INFO:tasks.workunit.client.0.smithi027.stdout:6/451: chown d4/d1a/d2e/d44/f5a 160993 1 2024-06-06T08:43:26.631 INFO:tasks.workunit.client.0.smithi027.stdout:6/452: dread - d4/d9/d6e/f80 zero size 2024-06-06T08:43:26.631 INFO:tasks.workunit.client.0.smithi027.stdout:6/453: write d4/d9/dc/d1c/f30 [764636,102177] 0 2024-06-06T08:43:26.632 INFO:tasks.workunit.client.0.smithi027.stdout:9/418: dread d3/d4e/d75/dc/df/d34/f60 [0,4194304] 0 2024-06-06T08:43:26.632 INFO:tasks.workunit.client.0.smithi027.stdout:6/454: creat d4/d9/f94 x:0 0 0 2024-06-06T08:43:26.632 INFO:tasks.workunit.client.0.smithi027.stdout:6/455: write d4/de/d45/d6a/f4b [783415,16761] 0 2024-06-06T08:43:26.633 INFO:tasks.workunit.client.0.smithi027.stdout:9/419: creat d3/d4e/d75/dc/df/d23/d36/f94 x:0 0 0 2024-06-06T08:43:26.633 INFO:tasks.workunit.client.0.smithi027.stdout:9/420: stat d3/d4e/d75/dc/df/d34/f48 0 2024-06-06T08:43:26.633 INFO:tasks.workunit.client.0.smithi027.stdout:9/421: chown d3/d4e/d75/dc/df/d34/l6a 0 1 2024-06-06T08:43:26.636 INFO:tasks.workunit.client.0.smithi027.stdout:4/489: dwrite d0/d1d/f59 [0,4194304] 0 2024-06-06T08:43:26.647 INFO:tasks.workunit.client.0.smithi027.stdout:6/456: rename d4/c40 to d4/d9/d49/c95 0 2024-06-06T08:43:26.648 INFO:tasks.workunit.client.0.smithi027.stdout:6/457: write d4/d9/dc/d1c/f30 [115769,87919] 0 2024-06-06T08:43:26.649 INFO:tasks.workunit.client.0.smithi027.stdout:6/458: truncate d4/d9/d49/f62 3410552 0 2024-06-06T08:43:26.650 INFO:tasks.workunit.client.0.smithi027.stdout:6/459: creat d4/d9/f96 x:0 0 0 2024-06-06T08:43:26.652 INFO:tasks.workunit.client.0.smithi027.stdout:6/460: symlink d4/d9/dc/d18/l97 0 2024-06-06T08:43:26.653 INFO:tasks.workunit.client.0.smithi027.stdout:6/461: symlink d4/d13/l98 0 2024-06-06T08:43:26.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/490: dwrite d0/d1d/d34/f6b [0,4194304] 0 2024-06-06T08:43:26.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/491: fdatasync d0/d1/d6/d8/d21/f5d 0 2024-06-06T08:43:26.657 INFO:tasks.workunit.client.0.smithi027.stdout:6/462: unlink d4/de/l63 0 2024-06-06T08:43:26.657 INFO:tasks.workunit.client.0.smithi027.stdout:6/463: readlink d4/d9/dc/d1c/l5b 0 2024-06-06T08:43:26.657 INFO:tasks.workunit.client.0.smithi027.stdout:4/492: dread d0/d1/f7c [0,4194304] 0 2024-06-06T08:43:26.658 INFO:tasks.workunit.client.0.smithi027.stdout:4/493: mknod d0/c99 0 2024-06-06T08:43:26.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/464: rmdir d4/d9/d8e 0 2024-06-06T08:43:26.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/465: write d4/d9/d49/f77 [193253,110534] 0 2024-06-06T08:43:26.661 INFO:tasks.workunit.client.0.smithi027.stdout:6/466: creat d4/d1a/d2e/d44/f99 x:0 0 0 2024-06-06T08:43:26.663 INFO:tasks.workunit.client.0.smithi027.stdout:6/467: write d4/de/d45/f48 [740367,124316] 0 2024-06-06T08:43:26.675 INFO:tasks.workunit.client.0.smithi027.stdout:6/468: write d4/d9/f19 [384678,25992] 0 2024-06-06T08:43:26.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/494: dwrite d0/d63/f6d [0,4194304] 0 2024-06-06T08:43:26.676 INFO:tasks.workunit.client.0.smithi027.stdout:6/469: creat d4/d9/d6e/f9a x:0 0 0 2024-06-06T08:43:26.676 INFO:tasks.workunit.client.0.smithi027.stdout:6/470: getdents d4/d9/dc 0 2024-06-06T08:43:26.678 INFO:tasks.workunit.client.0.smithi027.stdout:4/495: rmdir d0/d63/d7e/d3b 39 2024-06-06T08:43:26.678 INFO:tasks.workunit.client.0.smithi027.stdout:4/496: stat d0/l7b 0 2024-06-06T08:43:26.693 INFO:tasks.workunit.client.0.smithi027.stdout:4/497: dwrite d0/d1/d6/d8/d21/f74 [4194304,4194304] 0 2024-06-06T08:43:26.696 INFO:tasks.workunit.client.0.smithi027.stdout:6/471: dwrite d4/d9/d6e/f80 [0,4194304] 0 2024-06-06T08:43:26.698 INFO:tasks.workunit.client.0.smithi027.stdout:6/472: getdents d4/d1a/d2e 0 2024-06-06T08:43:26.700 INFO:tasks.workunit.client.0.smithi027.stdout:6/473: symlink d4/d1a/l9b 0 2024-06-06T08:43:26.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/474: rmdir d4/d9/dc/d8f 39 2024-06-06T08:43:26.709 INFO:tasks.workunit.client.0.smithi027.stdout:4/498: dread d0/d1/d6/f2f [0,4194304] 0 2024-06-06T08:43:26.710 INFO:tasks.workunit.client.0.smithi027.stdout:4/499: link d0/c7 d0/d1/c9a 0 2024-06-06T08:43:26.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/500: mknod d0/d1/d6/d8/d21/c9b 0 2024-06-06T08:43:26.722 INFO:tasks.workunit.client.0.smithi027.stdout:6/475: dwrite d4/d9/f96 [0,4194304] 0 2024-06-06T08:43:26.724 INFO:tasks.workunit.client.0.smithi027.stdout:6/476: symlink d4/l9c 0 2024-06-06T08:43:26.732 INFO:tasks.workunit.client.0.smithi027.stdout:4/501: dwrite d0/d63/d7e/d3b/f8c [0,4194304] 0 2024-06-06T08:43:26.734 INFO:tasks.workunit.client.0.smithi027.stdout:4/502: mkdir d0/d1d/d34/d7f/d97/d9c 0 2024-06-06T08:43:26.736 INFO:tasks.workunit.client.0.smithi027.stdout:6/477: rename d4/d9/dc/f87 to d4/d1a/d2e/f9d 0 2024-06-06T08:43:26.737 INFO:tasks.workunit.client.0.smithi027.stdout:4/503: write d0/d1/d6/d4b/f56 [64162,88246] 0 2024-06-06T08:43:26.737 INFO:tasks.workunit.client.0.smithi027.stdout:4/504: readlink d0/d1d/d77/l5f 0 2024-06-06T08:43:26.738 INFO:tasks.workunit.client.0.smithi027.stdout:6/478: creat d4/d9/dc/d1c/f9e x:0 0 0 2024-06-06T08:43:26.738 INFO:tasks.workunit.client.0.smithi027.stdout:6/479: truncate d4/de/d45/f89 853132 0 2024-06-06T08:43:26.748 INFO:tasks.workunit.client.0.smithi027.stdout:6/480: dread d4/d9/dc/d1c/f50 [0,4194304] 0 2024-06-06T08:43:26.754 INFO:tasks.workunit.client.0.smithi027.stdout:4/505: dwrite d0/d1/d6/d8/d21/f30 [0,4194304] 0 2024-06-06T08:43:26.754 INFO:tasks.workunit.client.0.smithi027.stdout:4/506: truncate d0/d63/d7e/d3b/f68 415052 0 2024-06-06T08:43:26.756 INFO:tasks.workunit.client.0.smithi027.stdout:6/481: write d4/d9/f96 [1751988,88410] 0 2024-06-06T08:43:26.757 INFO:tasks.workunit.client.0.smithi027.stdout:6/482: rename d4/d9/d6e/f9a to d4/d9/dc/f9f 0 2024-06-06T08:43:26.758 INFO:tasks.workunit.client.0.smithi027.stdout:4/507: symlink d0/d63/d7e/d3b/l9d 0 2024-06-06T08:43:26.758 INFO:tasks.workunit.client.0.smithi027.stdout:6/483: symlink d4/d9/dc/d8f/la0 0 2024-06-06T08:43:26.761 INFO:tasks.workunit.client.0.smithi027.stdout:6/484: mknod d4/d9/dc/d1c/ca1 0 2024-06-06T08:43:26.762 INFO:tasks.workunit.client.0.smithi027.stdout:6/485: mkdir d4/da2 0 2024-06-06T08:43:26.764 INFO:tasks.workunit.client.0.smithi027.stdout:4/508: dread d0/f1c [0,4194304] 0 2024-06-06T08:43:26.764 INFO:tasks.workunit.client.0.smithi027.stdout:4/509: chown d0/f1c 41 1 2024-06-06T08:43:26.765 INFO:tasks.workunit.client.0.smithi027.stdout:4/510: creat d0/d1/d6/f9e x:0 0 0 2024-06-06T08:43:26.765 INFO:tasks.workunit.client.0.smithi027.stdout:4/511: fdatasync d0/d1d/d34/f6b 0 2024-06-06T08:43:26.767 INFO:tasks.workunit.client.0.smithi027.stdout:4/512: creat d0/d1/d6/d8/d82/f9f x:0 0 0 2024-06-06T08:43:26.767 INFO:tasks.workunit.client.0.smithi027.stdout:4/513: truncate d0/d1/d6/f9e 431029 0 2024-06-06T08:43:26.767 INFO:tasks.workunit.client.0.smithi027.stdout:4/514: truncate d0/d1/d6/f33 3738524 0 2024-06-06T08:43:26.774 INFO:tasks.workunit.client.0.smithi027.stdout:4/515: dread d0/d1/d6/f2f [0,4194304] 0 2024-06-06T08:43:26.776 INFO:tasks.workunit.client.0.smithi027.stdout:4/516: mknod d0/d1/d6/d8/d82/d8b/ca0 0 2024-06-06T08:43:26.776 INFO:tasks.workunit.client.0.smithi027.stdout:4/517: write d0/d1d/d2a/f7d [169895,112465] 0 2024-06-06T08:43:26.776 INFO:tasks.workunit.client.0.smithi027.stdout:6/486: dwrite d4/d1a/f2c [0,4194304] 0 2024-06-06T08:43:26.778 INFO:tasks.workunit.client.0.smithi027.stdout:6/487: creat d4/d9/dc/d8f/fa3 x:0 0 0 2024-06-06T08:43:26.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/488: mknod d4/d9/dc/d18/ca4 0 2024-06-06T08:43:26.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/489: fsync d4/d1a/d2e/f5c 0 2024-06-06T08:43:26.788 INFO:tasks.workunit.client.0.smithi027.stdout:6/490: dread d4/d1a/f2c [0,4194304] 0 2024-06-06T08:43:26.794 INFO:tasks.workunit.client.0.smithi027.stdout:1/392: sync 2024-06-06T08:43:26.798 INFO:tasks.workunit.client.0.smithi027.stdout:6/491: link d4/de/d45/d6a/f4b d4/d9/dc/d1c/d52/fa5 0 2024-06-06T08:43:26.800 INFO:tasks.workunit.client.0.smithi027.stdout:6/492: dread d4/d1a/d2e/d44/f74 [0,4194304] 0 2024-06-06T08:43:26.802 INFO:tasks.workunit.client.0.smithi027.stdout:6/493: creat d4/da2/fa6 x:0 0 0 2024-06-06T08:43:26.802 INFO:tasks.workunit.client.0.smithi027.stdout:6/494: write d4/de/d45/f75 [645378,67420] 0 2024-06-06T08:43:26.817 INFO:tasks.workunit.client.0.smithi027.stdout:6/495: dwrite d4/d9/f96 [0,4194304] 0 2024-06-06T08:43:26.819 INFO:tasks.workunit.client.0.smithi027.stdout:6/496: symlink d4/d9/dc/d8f/la7 0 2024-06-06T08:43:26.819 INFO:tasks.workunit.client.0.smithi027.stdout:6/497: fsync d4/d9/dc/f43 0 2024-06-06T08:43:26.820 INFO:tasks.workunit.client.0.smithi027.stdout:6/498: rmdir d4/d1a/d2e/d44 39 2024-06-06T08:43:26.822 INFO:tasks.workunit.client.0.smithi027.stdout:6/499: rename d4/d9/f19 to d4/d9/dc/fa8 0 2024-06-06T08:43:26.822 INFO:tasks.workunit.client.0.smithi027.stdout:6/500: fsync d4/de/d45/d6a/f76 0 2024-06-06T08:43:26.822 INFO:tasks.workunit.client.0.smithi027.stdout:6/501: read - d4/d1a/d2e/f70 zero size 2024-06-06T08:43:26.832 INFO:tasks.workunit.client.0.smithi027.stdout:6/502: rename d4/d1a/d2e/f70 to d4/d9/dc/d1c/d52/fa9 0 2024-06-06T08:43:26.832 INFO:tasks.workunit.client.0.smithi027.stdout:6/503: chown d4/de/d45/d6a 1562055 1 2024-06-06T08:43:26.835 INFO:tasks.workunit.client.0.smithi027.stdout:6/504: unlink d4/d9/dc/d18/c3d 0 2024-06-06T08:43:26.836 INFO:tasks.workunit.client.0.smithi027.stdout:6/505: getdents d4/d9/d49 0 2024-06-06T08:43:26.837 INFO:tasks.workunit.client.0.smithi027.stdout:6/506: getdents d4 0 2024-06-06T08:43:26.837 INFO:tasks.workunit.client.0.smithi027.stdout:6/507: dread - d4/f79 zero size 2024-06-06T08:43:26.853 INFO:tasks.workunit.client.0.smithi027.stdout:6/508: dwrite d4/d9/f26 [0,4194304] 0 2024-06-06T08:43:26.863 INFO:tasks.workunit.client.0.smithi027.stdout:6/509: write d4/de/f11 [3366188,26459] 0 2024-06-06T08:43:26.863 INFO:tasks.workunit.client.0.smithi027.stdout:6/510: chown d4/d9/f2d 104696 1 2024-06-06T08:43:26.879 INFO:tasks.workunit.client.0.smithi027.stdout:6/511: rename d4/d13/c2b to d4/d9/dc/d8f/caa 0 2024-06-06T08:43:26.895 INFO:tasks.workunit.client.0.smithi027.stdout:6/512: dwrite d4/d13/f92 [0,4194304] 0 2024-06-06T08:43:26.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/513: dwrite d4/d9/dc/d8f/fa3 [0,4194304] 0 2024-06-06T08:43:26.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/514: chown d4/d9/dc/d1c/f2a 1114211727 1 2024-06-06T08:43:26.919 INFO:tasks.workunit.client.0.smithi027.stdout:6/515: dread d4/f29 [0,4194304] 0 2024-06-06T08:43:26.919 INFO:tasks.workunit.client.0.smithi027.stdout:6/516: chown d4/de/l31 1777024860 1 2024-06-06T08:43:26.919 INFO:tasks.workunit.client.0.smithi027.stdout:6/517: chown d4/d9/dc/d8f/c93 1 1 2024-06-06T08:43:26.923 INFO:tasks.workunit.client.0.smithi027.stdout:6/518: getdents d4/d9/dc/d1c 0 2024-06-06T08:43:26.923 INFO:tasks.workunit.client.0.smithi027.stdout:6/519: stat d4/d9/dc/l3f 0 2024-06-06T08:43:26.924 INFO:tasks.workunit.client.0.smithi027.stdout:6/520: symlink d4/d9/lab 0 2024-06-06T08:43:26.925 INFO:tasks.workunit.client.0.smithi027.stdout:6/521: write d4/de/d45/d6a/f4b [1591472,95645] 0 2024-06-06T08:43:26.926 INFO:tasks.workunit.client.0.smithi027.stdout:6/522: unlink d4/de/l31 0 2024-06-06T08:43:26.937 INFO:tasks.workunit.client.0.smithi027.stdout:6/523: write d4/d9/f26 [1333268,124917] 0 2024-06-06T08:43:26.938 INFO:tasks.workunit.client.0.smithi027.stdout:6/524: mknod d4/d9/dc/d1c/d52/cac 0 2024-06-06T08:43:26.948 INFO:tasks.workunit.client.0.smithi027.stdout:6/525: link d4/d13/f92 d4/d9/fad 0 2024-06-06T08:43:26.948 INFO:tasks.workunit.client.0.smithi027.stdout:6/526: truncate d4/d1a/d2e/f3c 1339531 0 2024-06-06T08:43:26.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/527: write d4/d9/d49/f62 [136709,11224] 0 2024-06-06T08:43:26.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:26 smithi027 ceph-mon[32335]: scrub summary: idle 2024-06-06T08:43:26.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:26 smithi027 ceph-mon[32335]: pgmap v98: 129 pgs: 129 active+clean; 700 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 97 MiB/s wr, 1.41k op/s 2024-06-06T08:43:26.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:26 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:26.956 INFO:tasks.workunit.client.0.smithi027.stdout:7/619: sync 2024-06-06T08:43:26.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/528: truncate d4/d9/d49/f71 530309 0 2024-06-06T08:43:26.958 INFO:tasks.workunit.client.0.smithi027.stdout:6/529: mknod d4/de/d45/d6a/cae 0 2024-06-06T08:43:26.961 INFO:tasks.workunit.client.0.smithi027.stdout:6/530: unlink d4/d9/dc/d8f/la0 0 2024-06-06T08:43:26.973 INFO:tasks.workunit.client.0.smithi027.stdout:6/531: rename d4/d1a/d2e/f65 to d4/d9/d6e/faf 0 2024-06-06T08:43:26.973 INFO:tasks.workunit.client.0.smithi027.stdout:6/532: symlink d4/d9/dc/lb0 0 2024-06-06T08:43:26.974 INFO:tasks.workunit.client.0.smithi027.stdout:6/533: chown d4/d9/dc 32303 1 2024-06-06T08:43:26.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:26 smithi203 ceph-mon[36579]: scrub summary: idle 2024-06-06T08:43:26.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:26 smithi203 ceph-mon[36579]: pgmap v98: 129 pgs: 129 active+clean; 700 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 97 MiB/s wr, 1.41k op/s 2024-06-06T08:43:26.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:26 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:26 smithi176 ceph-mon[36786]: scrub summary: idle 2024-06-06T08:43:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:26 smithi176 ceph-mon[36786]: pgmap v98: 129 pgs: 129 active+clean; 700 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 97 MiB/s wr, 1.41k op/s 2024-06-06T08:43:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:26 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:43:27.046 INFO:tasks.workunit.client.0.smithi027.stdout:8/390: sync 2024-06-06T08:43:27.046 INFO:tasks.workunit.client.0.smithi027.stdout:8/391: fdatasync d7/dc/d36/d83/d60/f81 0 2024-06-06T08:43:27.046 INFO:tasks.workunit.client.0.smithi027.stdout:8/392: chown d7/d13/d15 2479 1 2024-06-06T08:43:27.047 INFO:tasks.workunit.client.0.smithi027.stdout:8/393: creat d7/dc/d37/d46/f8d x:0 0 0 2024-06-06T08:43:27.060 INFO:tasks.workunit.client.0.smithi027.stdout:8/394: dwrite d7/d2f/f87 [4194304,4194304] 0 2024-06-06T08:43:27.060 INFO:tasks.workunit.client.0.smithi027.stdout:8/395: readlink d7/l55 0 2024-06-06T08:43:27.063 INFO:tasks.workunit.client.0.smithi027.stdout:8/396: rmdir d7/dc/d36/d83/d58 39 2024-06-06T08:43:27.063 INFO:tasks.workunit.client.0.smithi027.stdout:8/397: chown d7/f2c 28958 1 2024-06-06T08:43:27.063 INFO:tasks.workunit.client.0.smithi027.stdout:8/398: stat d7/dc/d36/d41/f25 0 2024-06-06T08:43:27.071 INFO:tasks.workunit.client.0.smithi027.stdout:8/399: dread d7/d13/f24 [0,4194304] 0 2024-06-06T08:43:27.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/400: dread d7/f2c [0,4194304] 0 2024-06-06T08:43:27.084 INFO:tasks.workunit.client.0.smithi027.stdout:8/401: mknod d7/dc/d36/d41/d85/c8e 0 2024-06-06T08:43:27.085 INFO:tasks.workunit.client.0.smithi027.stdout:8/402: stat d7/d2f/c3e 0 2024-06-06T08:43:27.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/403: write d7/d13/f24 [1651509,123279] 0 2024-06-06T08:43:27.098 INFO:tasks.workunit.client.0.smithi027.stdout:8/404: truncate d7/d13/f24 2815111 0 2024-06-06T08:43:27.098 INFO:tasks.workunit.client.0.smithi027.stdout:8/405: stat d7/dc/d36/d83/l14 0 2024-06-06T08:43:27.099 INFO:tasks.workunit.client.0.smithi027.stdout:8/406: mknod d7/dc/d37/d46/c8f 0 2024-06-06T08:43:27.101 INFO:tasks.workunit.client.0.smithi027.stdout:8/407: creat d7/f90 x:0 0 0 2024-06-06T08:43:27.101 INFO:tasks.workunit.client.0.smithi027.stdout:8/408: dread - d7/dc/d37/f80 zero size 2024-06-06T08:43:27.102 INFO:tasks.workunit.client.0.smithi027.stdout:8/409: read - d7/dc/d37/d46/d50/f7f zero size 2024-06-06T08:43:27.102 INFO:tasks.workunit.client.0.smithi027.stdout:8/410: write d7/dc/d36/d83/d60/f7a [88654,17645] 0 2024-06-06T08:43:27.102 INFO:tasks.workunit.client.0.smithi027.stdout:8/411: truncate d7/dc/f69 195636 0 2024-06-06T08:43:27.102 INFO:tasks.workunit.client.0.smithi027.stdout:8/412: write d7/d2f/f70 [297692,16011] 0 2024-06-06T08:43:27.103 INFO:tasks.workunit.client.0.smithi027.stdout:8/413: creat d7/dc/d36/d83/f91 x:0 0 0 2024-06-06T08:43:27.104 INFO:tasks.workunit.client.0.smithi027.stdout:8/414: dread - d7/dc/d36/d3c/f72 zero size 2024-06-06T08:43:27.105 INFO:tasks.workunit.client.0.smithi027.stdout:8/415: creat d7/d2f/f92 x:0 0 0 2024-06-06T08:43:27.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/416: mkdir d7/dc/d36/d83/d60/d93 0 2024-06-06T08:43:27.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/417: stat d7/dc/d37/l49 0 2024-06-06T08:43:27.188 INFO:tasks.workunit.client.0.smithi027.stdout:4/518: sync 2024-06-06T08:43:27.188 INFO:tasks.workunit.client.0.smithi027.stdout:9/422: sync 2024-06-06T08:43:27.188 INFO:tasks.workunit.client.0.smithi027.stdout:3/365: sync 2024-06-06T08:43:27.188 INFO:tasks.workunit.client.0.smithi027.stdout:2/481: sync 2024-06-06T08:43:27.188 INFO:tasks.workunit.client.0.smithi027.stdout:0/398: sync 2024-06-06T08:43:27.188 INFO:tasks.workunit.client.0.smithi027.stdout:1/393: sync 2024-06-06T08:43:27.189 INFO:tasks.workunit.client.0.smithi027.stdout:3/366: truncate d1/da/d44/f49 4915476 0 2024-06-06T08:43:27.189 INFO:tasks.workunit.client.0.smithi027.stdout:5/557: sync 2024-06-06T08:43:27.190 INFO:tasks.workunit.client.0.smithi027.stdout:3/367: creat d1/d1f/d45/f69 x:0 0 0 2024-06-06T08:43:27.194 INFO:tasks.workunit.client.0.smithi027.stdout:1/394: mknod dc/de/d14/c89 0 2024-06-06T08:43:27.194 INFO:tasks.workunit.client.0.smithi027.stdout:1/395: chown dc/de/d14/d4e/c7d 50404 1 2024-06-06T08:43:27.194 INFO:tasks.workunit.client.0.smithi027.stdout:1/396: dread - dc/de/d22/f82 zero size 2024-06-06T08:43:27.194 INFO:tasks.workunit.client.0.smithi027.stdout:1/397: truncate dc/de/d3d/d4f/f77 1339 0 2024-06-06T08:43:27.194 INFO:tasks.workunit.client.0.smithi027.stdout:1/398: write dc/de/d3d/f50 [1494647,92493] 0 2024-06-06T08:43:27.195 INFO:tasks.workunit.client.0.smithi027.stdout:3/368: creat d1/d1f/d4d/f6a x:0 0 0 2024-06-06T08:43:27.196 INFO:tasks.workunit.client.0.smithi027.stdout:3/369: creat d1/d1f/d4d/f6b x:0 0 0 2024-06-06T08:43:27.197 INFO:tasks.workunit.client.0.smithi027.stdout:3/370: fsync d1/d1f/d25/f4c 0 2024-06-06T08:43:27.197 INFO:tasks.workunit.client.0.smithi027.stdout:1/399: dread dc/de/d14/f63 [0,4194304] 0 2024-06-06T08:43:27.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/400: creat dc/de/d14/f8a x:0 0 0 2024-06-06T08:43:27.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/401: read dc/de/d14/f49 [3835951,107527] 0 2024-06-06T08:43:27.205 INFO:tasks.workunit.client.0.smithi027.stdout:4/519: dwrite d0/d63/d7e/f89 [4194304,4194304] 0 2024-06-06T08:43:27.209 INFO:tasks.workunit.client.0.smithi027.stdout:5/558: dwrite d0/da/d47/f68 [4194304,4194304] 0 2024-06-06T08:43:27.214 INFO:tasks.workunit.client.0.smithi027.stdout:5/559: creat d0/da/d20/faf x:0 0 0 2024-06-06T08:43:27.216 INFO:tasks.workunit.client.0.smithi027.stdout:5/560: write d0/da/d37/f41 [182532,122375] 0 2024-06-06T08:43:27.217 INFO:tasks.workunit.client.0.smithi027.stdout:5/561: mkdir d0/da/db/d46/d7f/d81/d97/db0 0 2024-06-06T08:43:27.217 INFO:tasks.workunit.client.0.smithi027.stdout:5/562: read - d0/da/db/d46/d4c/d70/f9e zero size 2024-06-06T08:43:27.219 INFO:tasks.workunit.client.0.smithi027.stdout:5/563: truncate d0/da/db/d1a/f78 69790 0 2024-06-06T08:43:27.219 INFO:tasks.workunit.client.0.smithi027.stdout:5/564: readlink d0/da/db/d1a/d5a/l96 0 2024-06-06T08:43:27.220 INFO:tasks.workunit.client.0.smithi027.stdout:4/520: dread d0/d1/d6/d8/d21/f2d [0,4194304] 0 2024-06-06T08:43:27.225 INFO:tasks.workunit.client.0.smithi027.stdout:0/399: dwrite d15/d18/d23/d38/f51 [4194304,4194304] 0 2024-06-06T08:43:27.225 INFO:tasks.workunit.client.0.smithi027.stdout:1/402: dwrite dc/de/d3d/f3f [0,4194304] 0 2024-06-06T08:43:27.225 INFO:tasks.workunit.client.0.smithi027.stdout:1/403: chown dc/de/d14/f73 21 1 2024-06-06T08:43:27.225 INFO:tasks.workunit.client.0.smithi027.stdout:1/404: fsync dc/de/d3d/f42 0 2024-06-06T08:43:27.229 INFO:tasks.workunit.client.0.smithi027.stdout:9/423: rename d3/d4e/d75/dc/df to d3/d95 0 2024-06-06T08:43:27.229 INFO:tasks.workunit.client.0.smithi027.stdout:0/400: creat d15/f88 x:0 0 0 2024-06-06T08:43:27.229 INFO:tasks.workunit.client.0.smithi027.stdout:9/424: write d3/d4e/d75/dc/d71/f92 [144749,4395] 0 2024-06-06T08:43:27.230 INFO:tasks.workunit.client.0.smithi027.stdout:0/401: getdents d15/d18/d23/d28/d2a/d30/d5c/d83 0 2024-06-06T08:43:27.231 INFO:tasks.workunit.client.0.smithi027.stdout:3/371: rename d1/d1f/l2b to d1/da/d44/l6c 0 2024-06-06T08:43:27.231 INFO:tasks.workunit.client.0.smithi027.stdout:3/372: fsync d1/f41 0 2024-06-06T08:43:27.231 INFO:tasks.workunit.client.0.smithi027.stdout:3/373: chown d1/da/d44/d54/c56 48 1 2024-06-06T08:43:27.232 INFO:tasks.workunit.client.0.smithi027.stdout:3/374: fdatasync d1/f41 0 2024-06-06T08:43:27.232 INFO:tasks.workunit.client.0.smithi027.stdout:9/425: creat d3/d95/d23/d36/d3a/f96 x:0 0 0 2024-06-06T08:43:27.236 INFO:tasks.workunit.client.0.smithi027.stdout:5/565: dwrite d0/da/db/d46/d77/f8c [4194304,4194304] 0 2024-06-06T08:43:27.237 INFO:tasks.workunit.client.0.smithi027.stdout:5/566: unlink d0/da/d20/c2e 0 2024-06-06T08:43:27.239 INFO:tasks.workunit.client.0.smithi027.stdout:5/567: creat d0/da/db/d46/d7f/d81/d97/fb1 x:0 0 0 2024-06-06T08:43:27.239 INFO:tasks.workunit.client.0.smithi027.stdout:3/375: getdents d1/da/d44 0 2024-06-06T08:43:27.239 INFO:tasks.workunit.client.0.smithi027.stdout:3/376: stat d1/d8/d22/d2d 0 2024-06-06T08:43:27.241 INFO:tasks.workunit.client.0.smithi027.stdout:3/377: mkdir d1/da/d6d 0 2024-06-06T08:43:27.241 INFO:tasks.workunit.client.0.smithi027.stdout:3/378: stat d1/d1f/l35 0 2024-06-06T08:43:27.242 INFO:tasks.workunit.client.0.smithi027.stdout:1/405: dwrite dc/de/d14/f73 [0,4194304] 0 2024-06-06T08:43:27.247 INFO:tasks.workunit.client.0.smithi027.stdout:0/402: dwrite f9 [0,4194304] 0 2024-06-06T08:43:27.251 INFO:tasks.workunit.client.0.smithi027.stdout:0/403: mknod d15/d18/d23/d28/d2a/d30/c89 0 2024-06-06T08:43:27.257 INFO:tasks.workunit.client.0.smithi027.stdout:4/521: rename d0/d1/d6/d8/d21/f5d to d0/d1/d75/fa1 0 2024-06-06T08:43:27.260 INFO:tasks.workunit.client.0.smithi027.stdout:4/522: dread d0/d1/d6/d4b/f56 [0,4194304] 0 2024-06-06T08:43:27.260 INFO:tasks.workunit.client.0.smithi027.stdout:9/426: rename d3/d95/c47 to d3/d95/d23/d36/d3a/d84/c97 0 2024-06-06T08:43:27.262 INFO:tasks.workunit.client.0.smithi027.stdout:1/406: dwrite dc/de/f13 [0,4194304] 0 2024-06-06T08:43:27.262 INFO:tasks.workunit.client.0.smithi027.stdout:1/407: truncate dc/de/d3d/d4f/f77 238327 0 2024-06-06T08:43:27.262 INFO:tasks.workunit.client.0.smithi027.stdout:1/408: chown dc/f84 81 1 2024-06-06T08:43:27.263 INFO:tasks.workunit.client.0.smithi027.stdout:4/523: creat d0/d1d/d34/fa2 x:0 0 0 2024-06-06T08:43:27.263 INFO:tasks.workunit.client.0.smithi027.stdout:4/524: truncate d0/d1d/f79 997883 0 2024-06-06T08:43:27.263 INFO:tasks.workunit.client.0.smithi027.stdout:3/379: rename d1/d1f/d45/d5f/l64 to d1/d8/d22/d27/l6e 0 2024-06-06T08:43:27.264 INFO:tasks.workunit.client.0.smithi027.stdout:1/409: creat dc/d67/d69/d32/f8b x:0 0 0 2024-06-06T08:43:27.266 INFO:tasks.workunit.client.0.smithi027.stdout:4/525: unlink d0/d63/d7e/d3b/f8c 0 2024-06-06T08:43:27.267 INFO:tasks.workunit.client.0.smithi027.stdout:3/380: creat d1/d1f/f6f x:0 0 0 2024-06-06T08:43:27.267 INFO:tasks.workunit.client.0.smithi027.stdout:3/381: readlink d1/l4 0 2024-06-06T08:43:27.268 INFO:tasks.workunit.client.0.smithi027.stdout:3/382: symlink d1/d1f/d5b/l70 0 2024-06-06T08:43:27.269 INFO:tasks.workunit.client.0.smithi027.stdout:0/404: read d15/d18/d23/d38/f51 [6017840,43904] 0 2024-06-06T08:43:27.269 INFO:tasks.workunit.client.0.smithi027.stdout:0/405: write d15/d18/d64/f87 [199317,128861] 0 2024-06-06T08:43:27.269 INFO:tasks.workunit.client.0.smithi027.stdout:0/406: write d15/d18/d23/d28/d31/f84 [286627,82240] 0 2024-06-06T08:43:27.271 INFO:tasks.workunit.client.0.smithi027.stdout:0/407: creat d15/d18/d64/f8a x:0 0 0 2024-06-06T08:43:27.274 INFO:tasks.workunit.client.0.smithi027.stdout:0/408: dread d15/d18/d23/d28/d2a/d2d/f54 [0,4194304] 0 2024-06-06T08:43:27.276 INFO:tasks.workunit.client.0.smithi027.stdout:1/410: write dc/de/d14/d4e/f5d [1039850,84705] 0 2024-06-06T08:43:27.280 INFO:tasks.workunit.client.0.smithi027.stdout:0/409: dread d15/d18/f1c [0,4194304] 0 2024-06-06T08:43:27.280 INFO:tasks.workunit.client.0.smithi027.stdout:0/410: truncate d15/d18/f73 410896 0 2024-06-06T08:43:27.283 INFO:tasks.workunit.client.0.smithi027.stdout:3/383: dread d1/da/d44/f49 [0,4194304] 0 2024-06-06T08:43:27.283 INFO:tasks.workunit.client.0.smithi027.stdout:3/384: fsync d1/d8/d22/d27/d50/f51 0 2024-06-06T08:43:27.283 INFO:tasks.workunit.client.0.smithi027.stdout:4/526: write d0/d63/d7e/f89 [6216870,29711] 0 2024-06-06T08:43:27.285 INFO:tasks.workunit.client.0.smithi027.stdout:3/385: unlink d1/c66 0 2024-06-06T08:43:27.286 INFO:tasks.workunit.client.0.smithi027.stdout:0/411: read d15/f4f [1243176,127446] 0 2024-06-06T08:43:27.287 INFO:tasks.workunit.client.0.smithi027.stdout:4/527: dread d0/d1d/d34/f4d [0,4194304] 0 2024-06-06T08:43:27.287 INFO:tasks.workunit.client.0.smithi027.stdout:3/386: rename d1/da/d44/l6c to d1/da/d44/l71 0 2024-06-06T08:43:27.288 INFO:tasks.workunit.client.0.smithi027.stdout:0/412: chown d15/d18/d64/l69 439367250 1 2024-06-06T08:43:27.289 INFO:tasks.workunit.client.0.smithi027.stdout:4/528: mkdir d0/d63/d7e/da3 0 2024-06-06T08:43:27.290 INFO:tasks.workunit.client.0.smithi027.stdout:3/387: creat d1/d8/d22/d27/d50/f72 x:0 0 0 2024-06-06T08:43:27.291 INFO:tasks.workunit.client.0.smithi027.stdout:4/529: dread d0/d1/f4f [0,4194304] 0 2024-06-06T08:43:27.292 INFO:tasks.workunit.client.0.smithi027.stdout:4/530: symlink d0/d1d/la4 0 2024-06-06T08:43:27.294 INFO:tasks.workunit.client.0.smithi027.stdout:4/531: mknod d0/d63/ca5 0 2024-06-06T08:43:27.295 INFO:tasks.workunit.client.0.smithi027.stdout:4/532: symlink d0/d1/d6/la6 0 2024-06-06T08:43:27.296 INFO:tasks.workunit.client.0.smithi027.stdout:4/533: mknod d0/d1/d6/d8/d82/d8b/ca7 0 2024-06-06T08:43:27.296 INFO:tasks.workunit.client.0.smithi027.stdout:4/534: stat d0/d1/d6/d8/d82/d8b 0 2024-06-06T08:43:27.299 INFO:tasks.workunit.client.0.smithi027.stdout:0/413: write f9 [2958492,87757] 0 2024-06-06T08:43:27.301 INFO:tasks.workunit.client.0.smithi027.stdout:0/414: mknod d15/d18/d23/d28/d2a/d30/d5c/d83/c8b 0 2024-06-06T08:43:27.303 INFO:tasks.workunit.client.0.smithi027.stdout:0/415: mkdir d15/d18/d40/d8c 0 2024-06-06T08:43:27.303 INFO:tasks.workunit.client.0.smithi027.stdout:0/416: fdatasync d15/f4f 0 2024-06-06T08:43:27.303 INFO:tasks.workunit.client.0.smithi027.stdout:0/417: read - d15/f88 zero size 2024-06-06T08:43:27.303 INFO:tasks.workunit.client.0.smithi027.stdout:3/388: write d1/d8/d22/f46 [1171452,12620] 0 2024-06-06T08:43:27.303 INFO:tasks.workunit.client.0.smithi027.stdout:1/411: dwrite dc/d67/d69/f27 [0,4194304] 0 2024-06-06T08:43:27.304 INFO:tasks.workunit.client.0.smithi027.stdout:1/412: chown dc/d67/d69/f52 0 1 2024-06-06T08:43:27.306 INFO:tasks.workunit.client.0.smithi027.stdout:3/389: dread d1/f1e [0,4194304] 0 2024-06-06T08:43:27.313 INFO:tasks.workunit.client.0.smithi027.stdout:1/413: write dc/de/d14/f87 [1318132,48030] 0 2024-06-06T08:43:27.316 INFO:tasks.workunit.client.0.smithi027.stdout:4/535: dwrite d0/d1d/d2a/f5e [0,4194304] 0 2024-06-06T08:43:27.316 INFO:tasks.workunit.client.0.smithi027.stdout:4/536: chown d0/f55 149818 1 2024-06-06T08:43:27.318 INFO:tasks.workunit.client.0.smithi027.stdout:1/414: truncate dc/de/d14/d4e/f62 2458699 0 2024-06-06T08:43:27.323 INFO:tasks.workunit.client.0.smithi027.stdout:4/537: creat d0/d1d/d2a/d98/fa8 x:0 0 0 2024-06-06T08:43:27.324 INFO:tasks.workunit.client.0.smithi027.stdout:4/538: creat d0/d63/d7e/d3b/fa9 x:0 0 0 2024-06-06T08:43:27.333 INFO:tasks.workunit.client.0.smithi027.stdout:1/415: dwrite dc/f30 [0,4194304] 0 2024-06-06T08:43:27.333 INFO:tasks.workunit.client.0.smithi027.stdout:1/416: stat dc/de/d14 0 2024-06-06T08:43:27.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/418: link d15/d18/d64/f87 d15/d18/d23/d38/d68/d85/f8d 0 2024-06-06T08:43:27.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/419: fdatasync d15/d18/d23/d38/d68/f6a 0 2024-06-06T08:43:27.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/420: write d15/d18/d23/f27 [520985,86215] 0 2024-06-06T08:43:27.340 INFO:tasks.workunit.client.0.smithi027.stdout:0/421: creat d15/d18/d23/d5b/f8e x:0 0 0 2024-06-06T08:43:27.343 INFO:tasks.workunit.client.0.smithi027.stdout:7/620: sync 2024-06-06T08:43:27.343 INFO:tasks.workunit.client.0.smithi027.stdout:0/422: creat d15/d18/d23/d38/d68/f8f x:0 0 0 2024-06-06T08:43:27.348 INFO:tasks.workunit.client.0.smithi027.stdout:4/539: dwrite d0/d1/f96 [0,4194304] 0 2024-06-06T08:43:27.351 INFO:tasks.workunit.client.0.smithi027.stdout:4/540: dread d0/d1/d6/d8/d21/f4a [0,4194304] 0 2024-06-06T08:43:27.357 INFO:tasks.workunit.client.0.smithi027.stdout:1/417: dwrite dc/de/d14/d4e/f53 [0,4194304] 0 2024-06-06T08:43:27.359 INFO:tasks.workunit.client.0.smithi027.stdout:0/423: rename d15/d18/d23/d28/d2a/d30/c39 to d15/d18/d23/d28/c90 0 2024-06-06T08:43:27.360 INFO:tasks.workunit.client.0.smithi027.stdout:0/424: symlink d15/d18/d23/d5b/l91 0 2024-06-06T08:43:27.360 INFO:tasks.workunit.client.0.smithi027.stdout:0/425: fsync d15/d18/d23/d38/d68/f7b 0 2024-06-06T08:43:27.361 INFO:tasks.workunit.client.0.smithi027.stdout:0/426: write d15/d18/f1c [1541938,1964] 0 2024-06-06T08:43:27.361 INFO:tasks.workunit.client.0.smithi027.stdout:1/418: dread dc/f84 [0,4194304] 0 2024-06-06T08:43:27.362 INFO:tasks.workunit.client.0.smithi027.stdout:0/427: symlink d15/l92 0 2024-06-06T08:43:27.362 INFO:tasks.workunit.client.0.smithi027.stdout:0/428: write d15/d18/f4b [4564970,100229] 0 2024-06-06T08:43:27.362 INFO:tasks.workunit.client.0.smithi027.stdout:0/429: chown f10 498 1 2024-06-06T08:43:27.363 INFO:tasks.workunit.client.0.smithi027.stdout:0/430: symlink d15/d18/d23/d38/d3e/l93 0 2024-06-06T08:43:27.379 INFO:tasks.workunit.client.0.smithi027.stdout:0/431: read d15/d18/d23/d38/d3e/f5a [615374,89428] 0 2024-06-06T08:43:27.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/621: dwrite d17/d32/d46/d83/d72/fc3 [0,4194304] 0 2024-06-06T08:43:27.380 INFO:tasks.workunit.client.0.smithi027.stdout:0/432: readlink d15/d18/d40/l66 0 2024-06-06T08:43:27.380 INFO:tasks.workunit.client.0.smithi027.stdout:0/433: creat d15/d18/d23/d6f/f94 x:0 0 0 2024-06-06T08:43:27.381 INFO:tasks.workunit.client.0.smithi027.stdout:0/434: creat d15/d18/d23/d3c/f95 x:0 0 0 2024-06-06T08:43:27.383 INFO:tasks.workunit.client.0.smithi027.stdout:1/419: read dc/de/d3d/f3f [3727320,71053] 0 2024-06-06T08:43:27.383 INFO:tasks.workunit.client.0.smithi027.stdout:1/420: chown dc/f30 346 1 2024-06-06T08:43:27.383 INFO:tasks.workunit.client.0.smithi027.stdout:1/421: fdatasync dc/de/d22/f35 0 2024-06-06T08:43:27.384 INFO:tasks.workunit.client.0.smithi027.stdout:1/422: creat dc/de/d14/d4e/f8c x:0 0 0 2024-06-06T08:43:27.385 INFO:tasks.workunit.client.0.smithi027.stdout:0/435: dread d15/d18/d23/d28/d2a/d2d/f58 [0,4194304] 0 2024-06-06T08:43:27.386 INFO:tasks.workunit.client.0.smithi027.stdout:1/423: symlink dc/de/d14/l8d 0 2024-06-06T08:43:27.386 INFO:tasks.workunit.client.0.smithi027.stdout:1/424: write dc/d67/d69/d32/d71/f74 [225312,28773] 0 2024-06-06T08:43:27.386 INFO:tasks.workunit.client.0.smithi027.stdout:1/425: stat dc/de 0 2024-06-06T08:43:27.386 INFO:tasks.workunit.client.0.smithi027.stdout:7/622: rename d17/d32/d61/f6b to d17/d32/fca 0 2024-06-06T08:43:27.387 INFO:tasks.workunit.client.0.smithi027.stdout:7/623: creat d17/d32/fcb x:0 0 0 2024-06-06T08:43:27.389 INFO:tasks.workunit.client.0.smithi027.stdout:7/624: getdents d17/d32/d6c 0 2024-06-06T08:43:27.389 INFO:tasks.workunit.client.0.smithi027.stdout:7/625: stat d17/d32/d46/d83/d2a/d3b/cb1 0 2024-06-06T08:43:27.391 INFO:tasks.workunit.client.0.smithi027.stdout:7/626: symlink d17/d32/d46/d83/d77/d8c/dae/lcc 0 2024-06-06T08:43:27.391 INFO:tasks.workunit.client.0.smithi027.stdout:7/627: chown d17/d47/f49 87484301 1 2024-06-06T08:43:27.404 INFO:tasks.workunit.client.0.smithi027.stdout:0/436: dwrite f10 [0,4194304] 0 2024-06-06T08:43:27.405 INFO:tasks.workunit.client.0.smithi027.stdout:1/426: dwrite dc/de/d3d/d4f/f77 [0,4194304] 0 2024-06-06T08:43:27.405 INFO:tasks.workunit.client.0.smithi027.stdout:1/427: write dc/de/d3d/f44 [384707,53421] 0 2024-06-06T08:43:27.406 INFO:tasks.workunit.client.0.smithi027.stdout:0/437: creat d15/d18/d23/d28/d31/f96 x:0 0 0 2024-06-06T08:43:27.407 INFO:tasks.workunit.client.0.smithi027.stdout:0/438: mknod d15/d18/d23/d28/c97 0 2024-06-06T08:43:27.409 INFO:tasks.workunit.client.0.smithi027.stdout:0/439: creat d15/d18/d23/d28/d2a/d30/d5d/f98 x:0 0 0 2024-06-06T08:43:27.413 INFO:tasks.workunit.client.0.smithi027.stdout:7/628: dwrite f15 [4194304,4194304] 0 2024-06-06T08:43:27.413 INFO:tasks.workunit.client.0.smithi027.stdout:7/629: write d17/d32/d46/d4f/f50 [3358448,129962] 0 2024-06-06T08:43:27.422 INFO:tasks.workunit.client.0.smithi027.stdout:0/440: read d15/f22 [1793630,72916] 0 2024-06-06T08:43:27.426 INFO:tasks.workunit.client.0.smithi027.stdout:1/428: dwrite dc/de/d14/d4e/f5d [0,4194304] 0 2024-06-06T08:43:27.427 INFO:tasks.workunit.client.0.smithi027.stdout:1/429: write dc/de/d3d/f3e [1017017,64064] 0 2024-06-06T08:43:27.427 INFO:tasks.workunit.client.0.smithi027.stdout:1/430: chown dc/d67/d69/d32/f4d 2058729 1 2024-06-06T08:43:27.427 INFO:tasks.workunit.client.0.smithi027.stdout:1/431: write f5 [2974395,81526] 0 2024-06-06T08:43:27.427 INFO:tasks.workunit.client.0.smithi027.stdout:1/432: readlink dc/l23 0 2024-06-06T08:43:27.432 INFO:tasks.workunit.client.0.smithi027.stdout:1/433: dread dc/de/d3d/f42 [0,4194304] 0 2024-06-06T08:43:27.432 INFO:tasks.workunit.client.0.smithi027.stdout:1/434: chown dc/de/c51 63495976 1 2024-06-06T08:43:27.434 INFO:tasks.workunit.client.0.smithi027.stdout:1/435: rmdir dc/de/d22 39 2024-06-06T08:43:27.436 INFO:tasks.workunit.client.0.smithi027.stdout:7/630: dwrite d17/d32/d46/d51/d52/d70/d87/fa3 [0,4194304] 0 2024-06-06T08:43:27.436 INFO:tasks.workunit.client.0.smithi027.stdout:7/631: chown d17/d32/d46/d51/f6f 7757209 1 2024-06-06T08:43:27.438 INFO:tasks.workunit.client.0.smithi027.stdout:0/441: dread d15/d18/f4b [0,4194304] 0 2024-06-06T08:43:27.440 INFO:tasks.workunit.client.0.smithi027.stdout:0/442: unlink d15/f88 0 2024-06-06T08:43:27.440 INFO:tasks.workunit.client.0.smithi027.stdout:0/443: chown d15/d18/d23/d38/d3e/f5a 32 1 2024-06-06T08:43:27.441 INFO:tasks.workunit.client.0.smithi027.stdout:0/444: read - d15/d18/d23/d28/d31/f5e zero size 2024-06-06T08:43:27.447 INFO:tasks.workunit.client.0.smithi027.stdout:0/445: getdents d15/d18/d23/d28/d2a/d30/d5c 0 2024-06-06T08:43:27.448 INFO:tasks.workunit.client.0.smithi027.stdout:0/446: fsync d15/d18/d23/d28/d2a/d30/d5c/f82 0 2024-06-06T08:43:27.448 INFO:tasks.workunit.client.0.smithi027.stdout:0/447: stat d15/d18/d23/d28/d2a/d2d/f58 0 2024-06-06T08:43:27.448 INFO:tasks.workunit.client.0.smithi027.stdout:0/448: write d15/d18/d23/d28/d2a/d2d/f58 [1969206,21281] 0 2024-06-06T08:43:27.448 INFO:tasks.workunit.client.0.smithi027.stdout:0/449: fsync d15/d18/d23/d28/d2a/d30/d5c/d83/f86 0 2024-06-06T08:43:27.449 INFO:tasks.workunit.client.0.smithi027.stdout:7/632: dread d17/d32/d46/d4f/d5e/f67 [0,4194304] 0 2024-06-06T08:43:27.449 INFO:tasks.workunit.client.0.smithi027.stdout:0/450: creat d15/d18/d23/d38/d3e/f99 x:0 0 0 2024-06-06T08:43:27.449 INFO:tasks.workunit.client.0.smithi027.stdout:0/451: truncate d15/d18/d23/d28/d2a/d2d/f56 819024 0 2024-06-06T08:43:27.450 INFO:tasks.workunit.client.0.smithi027.stdout:1/436: read dc/f30 [1386487,93759] 0 2024-06-06T08:43:27.450 INFO:tasks.workunit.client.0.smithi027.stdout:6/534: sync 2024-06-06T08:43:27.450 INFO:tasks.workunit.client.0.smithi027.stdout:7/633: link d17/d32/d46/d4f/d5e/f79 d17/d32/d46/d4f/db6/dc8/fcd 0 2024-06-06T08:43:27.451 INFO:tasks.workunit.client.0.smithi027.stdout:0/452: symlink d15/d18/d23/l9a 0 2024-06-06T08:43:27.451 INFO:tasks.workunit.client.0.smithi027.stdout:0/453: dread - d15/d18/d23/d38/d3e/f99 zero size 2024-06-06T08:43:27.451 INFO:tasks.workunit.client.0.smithi027.stdout:0/454: fdatasync d15/d18/d23/d28/d31/f5e 0 2024-06-06T08:43:27.451 INFO:tasks.workunit.client.0.smithi027.stdout:0/455: fsync f9 0 2024-06-06T08:43:27.451 INFO:tasks.workunit.client.0.smithi027.stdout:7/634: mkdir d17/d32/d46/d51/d52/d70/dce 0 2024-06-06T08:43:27.466 INFO:tasks.workunit.client.0.smithi027.stdout:1/437: write dc/f36 [7756351,53365] 0 2024-06-06T08:43:27.468 INFO:tasks.workunit.client.0.smithi027.stdout:1/438: symlink dc/d67/d69/d32/d71/l8e 0 2024-06-06T08:43:27.468 INFO:tasks.workunit.client.0.smithi027.stdout:1/439: fdatasync dc/de/d3d/d4f/f77 0 2024-06-06T08:43:27.470 INFO:tasks.workunit.client.0.smithi027.stdout:0/456: write d15/d18/d23/d5b/f6b [48847,69751] 0 2024-06-06T08:43:27.470 INFO:tasks.workunit.client.0.smithi027.stdout:0/457: chown d15/d18/d23/d38/f51 1528040 1 2024-06-06T08:43:27.470 INFO:tasks.workunit.client.0.smithi027.stdout:1/440: link dc/de/d14/d4e/c6f dc/de/d3d/d4f/c8f 0 2024-06-06T08:43:27.472 INFO:tasks.workunit.client.0.smithi027.stdout:0/458: creat d15/d18/d40/f9b x:0 0 0 2024-06-06T08:43:27.472 INFO:tasks.workunit.client.0.smithi027.stdout:7/635: dread d17/d32/fca [4194304,4194304] 0 2024-06-06T08:43:27.472 INFO:tasks.workunit.client.0.smithi027.stdout:0/459: mkdir d15/d18/d40/d8c/d9c 0 2024-06-06T08:43:27.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/636: rmdir d17/d32/d46/d83/d2a/d3b 39 2024-06-06T08:43:27.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/637: fdatasync d17/d47/f49 0 2024-06-06T08:43:27.484 INFO:tasks.workunit.client.0.smithi027.stdout:7/638: dread d17/d32/d46/d4f/f50 [0,4194304] 0 2024-06-06T08:43:27.484 INFO:tasks.workunit.client.0.smithi027.stdout:7/639: read d17/f2e [184464,120770] 0 2024-06-06T08:43:27.486 INFO:tasks.workunit.client.0.smithi027.stdout:7/640: creat d17/d32/d46/d83/d77/d8c/d90/fcf x:0 0 0 2024-06-06T08:43:27.486 INFO:tasks.workunit.client.0.smithi027.stdout:7/641: readlink d17/d32/d46/d83/d77/d8c/d90/lc4 0 2024-06-06T08:43:27.486 INFO:tasks.workunit.client.0.smithi027.stdout:7/642: write d17/d32/d46/d51/d52/f85 [1124339,24993] 0 2024-06-06T08:43:27.488 INFO:tasks.workunit.client.0.smithi027.stdout:0/460: dwrite d15/d18/d23/d38/d68/f8f [0,4194304] 0 2024-06-06T08:43:27.488 INFO:tasks.workunit.client.0.smithi027.stdout:0/461: chown d15/d18/d23/d38/d68/f7b 441610 1 2024-06-06T08:43:27.488 INFO:tasks.workunit.client.0.smithi027.stdout:0/462: dread - d15/d18/d23/d28/d31/f5e zero size 2024-06-06T08:43:27.488 INFO:tasks.workunit.client.0.smithi027.stdout:7/643: mknod d17/d32/d46/d51/d7f/cd0 0 2024-06-06T08:43:27.488 INFO:tasks.workunit.client.0.smithi027.stdout:7/644: write d17/d47/fa9 [241579,85826] 0 2024-06-06T08:43:27.489 INFO:tasks.workunit.client.0.smithi027.stdout:0/463: read d15/d18/d23/d28/d2a/d2d/f54 [523742,84571] 0 2024-06-06T08:43:27.490 INFO:tasks.workunit.client.0.smithi027.stdout:0/464: creat d15/d18/d23/d38/d68/f9d x:0 0 0 2024-06-06T08:43:27.492 INFO:tasks.workunit.client.0.smithi027.stdout:0/465: creat d15/d18/d40/f9e x:0 0 0 2024-06-06T08:43:27.493 INFO:tasks.workunit.client.0.smithi027.stdout:0/466: link d15/d18/c26 d15/d18/d23/d38/d3e/c9f 0 2024-06-06T08:43:27.495 INFO:tasks.workunit.client.0.smithi027.stdout:0/467: mkdir d15/d18/d23/d5b/da0 0 2024-06-06T08:43:27.505 INFO:tasks.workunit.client.0.smithi027.stdout:7/645: rename d17/d32/d46/d51/fa8 to d17/d32/d61/d75/fd1 0 2024-06-06T08:43:27.507 INFO:tasks.workunit.client.0.smithi027.stdout:7/646: symlink d17/d32/d6c/ld2 0 2024-06-06T08:43:27.507 INFO:tasks.workunit.client.0.smithi027.stdout:7/647: read - d17/d32/d46/d51/d52/d76/fb8 zero size 2024-06-06T08:43:27.509 INFO:tasks.workunit.client.0.smithi027.stdout:7/648: creat d17/d32/d46/d51/d52/fd3 x:0 0 0 2024-06-06T08:43:27.509 INFO:tasks.workunit.client.0.smithi027.stdout:0/468: dwrite d15/d18/d40/f9b [0,4194304] 0 2024-06-06T08:43:27.510 INFO:tasks.workunit.client.0.smithi027.stdout:7/649: symlink d17/d32/d46/d4f/db6/ld4 0 2024-06-06T08:43:27.513 INFO:tasks.workunit.client.0.smithi027.stdout:7/650: creat d17/d32/d46/d83/fd5 x:0 0 0 2024-06-06T08:43:27.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/651: mkdir d17/d32/d46/d51/d52/d70/dd6 0 2024-06-06T08:43:27.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/469: truncate d15/d18/d23/f27 3241768 0 2024-06-06T08:43:27.516 INFO:tasks.workunit.client.0.smithi027.stdout:7/652: symlink d17/d32/d46/d83/d77/d8c/d91/daa/ld7 0 2024-06-06T08:43:27.516 INFO:tasks.workunit.client.0.smithi027.stdout:0/470: creat d15/fa1 x:0 0 0 2024-06-06T08:43:27.516 INFO:tasks.workunit.client.0.smithi027.stdout:0/471: chown d15/d18/d40/l66 34 1 2024-06-06T08:43:27.517 INFO:tasks.workunit.client.0.smithi027.stdout:7/653: symlink d17/d32/d46/d83/d72/ld8 0 2024-06-06T08:43:27.517 INFO:tasks.workunit.client.0.smithi027.stdout:7/654: chown fd 1036853 1 2024-06-06T08:43:27.517 INFO:tasks.workunit.client.0.smithi027.stdout:7/655: write d17/d47/f4d [1125985,109509] 0 2024-06-06T08:43:27.517 INFO:tasks.workunit.client.0.smithi027.stdout:0/472: mknod d15/d18/d23/d28/d2a/d30/d5d/ca2 0 2024-06-06T08:43:27.518 INFO:tasks.workunit.client.0.smithi027.stdout:7/656: symlink d17/d32/d46/d51/d52/d70/d87/ld9 0 2024-06-06T08:43:27.520 INFO:tasks.workunit.client.0.smithi027.stdout:7/657: symlink d17/d32/d46/d4f/db6/lda 0 2024-06-06T08:43:27.531 INFO:tasks.workunit.client.0.smithi027.stdout:7/658: write d17/fa4 [539980,50901] 0 2024-06-06T08:43:27.532 INFO:tasks.workunit.client.0.smithi027.stdout:7/659: chown d17/d32/d46/d4f/f50 12957 1 2024-06-06T08:43:27.532 INFO:tasks.workunit.client.0.smithi027.stdout:7/660: readlink d17/d32/d46/d83/l43 0 2024-06-06T08:43:27.532 INFO:tasks.workunit.client.0.smithi027.stdout:7/661: read - d17/d32/d46/d51/d52/d76/fb8 zero size 2024-06-06T08:43:27.532 INFO:tasks.workunit.client.0.smithi027.stdout:7/662: write d17/f1b [2879829,41009] 0 2024-06-06T08:43:27.532 INFO:tasks.workunit.client.0.smithi027.stdout:7/663: dread - d17/d32/d6c/dc2/fc5 zero size 2024-06-06T08:43:27.532 INFO:tasks.workunit.client.0.smithi027.stdout:7/664: write d17/d32/d46/d83/fd5 [1004649,111624] 0 2024-06-06T08:43:27.533 INFO:tasks.workunit.client.0.smithi027.stdout:7/665: link d17/d32/d46/d83/d77/fb4 d17/d32/d46/d83/d77/d8c/d91/daa/fdb 0 2024-06-06T08:43:27.540 INFO:tasks.workunit.client.0.smithi027.stdout:7/666: dread d17/d32/d46/d51/f71 [0,4194304] 0 2024-06-06T08:43:27.549 INFO:tasks.workunit.client.0.smithi027.stdout:9/427: sync 2024-06-06T08:43:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/390: sync 2024-06-06T08:43:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:2/482: sync 2024-06-06T08:43:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/391: fdatasync d1/d1f/d25/f4c 0 2024-06-06T08:43:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:8/418: sync 2024-06-06T08:43:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:5/568: sync 2024-06-06T08:43:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:2/483: readlink d6/d1e/d2b/l9b 0 2024-06-06T08:43:27.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/392: getdents d1/da/d44 0 2024-06-06T08:43:27.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/393: chown d1/da/f18 1 1 2024-06-06T08:43:27.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/394: chown d1/d1f/d4d/f6a 211 1 2024-06-06T08:43:27.554 INFO:tasks.workunit.client.0.smithi027.stdout:2/484: truncate d6/d1e/f1f 1535207 0 2024-06-06T08:43:27.555 INFO:tasks.workunit.client.0.smithi027.stdout:9/428: getdents d3/d4e/d75 0 2024-06-06T08:43:27.555 INFO:tasks.workunit.client.0.smithi027.stdout:9/429: chown d3/d76/f86 2276002 1 2024-06-06T08:43:27.556 INFO:tasks.workunit.client.0.smithi027.stdout:8/419: creat d7/dc/d36/d3c/f94 x:0 0 0 2024-06-06T08:43:27.559 INFO:tasks.workunit.client.0.smithi027.stdout:5/569: mknod d0/da/d47/cb2 0 2024-06-06T08:43:27.559 INFO:tasks.workunit.client.0.smithi027.stdout:5/570: stat d0/da/db/d1a/d5a/l5f 0 2024-06-06T08:43:27.559 INFO:tasks.workunit.client.0.smithi027.stdout:5/571: readlink d0/da/l21 0 2024-06-06T08:43:27.559 INFO:tasks.workunit.client.0.smithi027.stdout:2/485: creat d6/dc/d3f/d48/fac x:0 0 0 2024-06-06T08:43:27.561 INFO:tasks.workunit.client.0.smithi027.stdout:9/430: unlink d3/d76/f86 0 2024-06-06T08:43:27.561 INFO:tasks.workunit.client.0.smithi027.stdout:9/431: truncate d3/f20 4255200 0 2024-06-06T08:43:27.563 INFO:tasks.workunit.client.0.smithi027.stdout:5/572: unlink d0/da/l30 0 2024-06-06T08:43:27.564 INFO:tasks.workunit.client.0.smithi027.stdout:2/486: symlink d6/dc/d1d/lad 0 2024-06-06T08:43:27.565 INFO:tasks.workunit.client.0.smithi027.stdout:2/487: readlink d6/dc/d3f/d77/d55/d69/l76 0 2024-06-06T08:43:27.565 INFO:tasks.workunit.client.0.smithi027.stdout:2/488: dread - d6/dc/d3f/d77/d55/f74 zero size 2024-06-06T08:43:27.565 INFO:tasks.workunit.client.0.smithi027.stdout:2/489: stat d6/d1e/f73 0 2024-06-06T08:43:27.565 INFO:tasks.workunit.client.0.smithi027.stdout:2/490: fdatasync d6/dd/f23 0 2024-06-06T08:43:27.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/432: unlink d3/d76/c7d 0 2024-06-06T08:43:27.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/433: dread - d3/d4e/d75/dc/f8d zero size 2024-06-06T08:43:27.567 INFO:tasks.workunit.client.0.smithi027.stdout:2/491: creat d6/dc/d3f/d48/d83/fae x:0 0 0 2024-06-06T08:43:27.569 INFO:tasks.workunit.client.0.smithi027.stdout:9/434: mknod d3/d95/d34/c98 0 2024-06-06T08:43:27.570 INFO:tasks.workunit.client.0.smithi027.stdout:9/435: mknod d3/d95/d23/c99 0 2024-06-06T08:43:27.572 INFO:tasks.workunit.client.0.smithi027.stdout:9/436: mknod d3/d95/d34/c9a 0 2024-06-06T08:43:27.572 INFO:tasks.workunit.client.0.smithi027.stdout:9/437: chown d3/d95/d34/d73/f85 22811691 1 2024-06-06T08:43:27.572 INFO:tasks.workunit.client.0.smithi027.stdout:9/438: write d3/f5f [5062768,8965] 0 2024-06-06T08:43:27.572 INFO:tasks.workunit.client.0.smithi027.stdout:8/420: dwrite d7/dc/d36/d3c/f72 [0,4194304] 0 2024-06-06T08:43:27.572 INFO:tasks.workunit.client.0.smithi027.stdout:8/421: dread - d7/f90 zero size 2024-06-06T08:43:27.572 INFO:tasks.workunit.client.0.smithi027.stdout:8/422: write d7/dc/f12 [12308,2449] 0 2024-06-06T08:43:27.573 INFO:tasks.workunit.client.0.smithi027.stdout:9/439: rename d3/d4e/d75/dc/d71 to d3/d4e/d75/dc/d9b 0 2024-06-06T08:43:27.574 INFO:tasks.workunit.client.0.smithi027.stdout:9/440: link d3/d95/d23/d36/f5d d3/d95/d34/d41/f9c 0 2024-06-06T08:43:27.582 INFO:tasks.workunit.client.0.smithi027.stdout:8/423: write d7/dc/d36/d3c/f61 [631571,68329] 0 2024-06-06T08:43:27.583 INFO:tasks.workunit.client.0.smithi027.stdout:8/424: dread d7/dc/d37/d46/d50/f76 [0,4194304] 0 2024-06-06T08:43:27.585 INFO:tasks.workunit.client.0.smithi027.stdout:2/492: dwrite d6/dc/d3f/f49 [0,4194304] 0 2024-06-06T08:43:27.585 INFO:tasks.workunit.client.0.smithi027.stdout:5/573: dwrite d0/da/db/d46/d77/f8f [0,4194304] 0 2024-06-06T08:43:27.586 INFO:tasks.workunit.client.0.smithi027.stdout:8/425: rename d7/dc/c5f to d7/d13/c95 0 2024-06-06T08:43:27.590 INFO:tasks.workunit.client.0.smithi027.stdout:2/493: creat d6/d1e/d2b/d5a/faf x:0 0 0 2024-06-06T08:43:27.593 INFO:tasks.workunit.client.0.smithi027.stdout:8/426: creat d7/dc/d36/d83/d58/f96 x:0 0 0 2024-06-06T08:43:27.593 INFO:tasks.workunit.client.0.smithi027.stdout:8/427: readlink d7/d13/l47 0 2024-06-06T08:43:27.595 INFO:tasks.workunit.client.0.smithi027.stdout:8/428: rename d7/d13/c6f to d7/dc/d37/d46/d50/d73/c97 0 2024-06-06T08:43:27.597 INFO:tasks.workunit.client.0.smithi027.stdout:9/441: dwrite d3/d95/d23/d36/f61 [0,4194304] 0 2024-06-06T08:43:27.597 INFO:tasks.workunit.client.0.smithi027.stdout:9/442: fsync d3/d4e/d75/dc/d9b/f92 0 2024-06-06T08:43:27.597 INFO:tasks.workunit.client.0.smithi027.stdout:2/494: mkdir d6/dc/d3f/db0 0 2024-06-06T08:43:27.597 INFO:tasks.workunit.client.0.smithi027.stdout:8/429: creat d7/d2f/f98 x:0 0 0 2024-06-06T08:43:27.600 INFO:tasks.workunit.client.0.smithi027.stdout:9/443: creat d3/d4e/f9d x:0 0 0 2024-06-06T08:43:27.602 INFO:tasks.workunit.client.0.smithi027.stdout:9/444: symlink d3/d95/d23/d78/l9e 0 2024-06-06T08:43:27.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/430: rename d7/d13/d15/c7d to d7/dc/c99 0 2024-06-06T08:43:27.606 INFO:tasks.workunit.client.0.smithi027.stdout:8/431: link d7/dc/d36/d83/l14 d7/l9a 0 2024-06-06T08:43:27.606 INFO:tasks.workunit.client.0.smithi027.stdout:8/432: fdatasync d7/d2f/f77 0 2024-06-06T08:43:27.617 INFO:tasks.workunit.client.0.smithi027.stdout:2/495: dwrite d6/dc/f29 [0,4194304] 0 2024-06-06T08:43:27.618 INFO:tasks.workunit.client.0.smithi027.stdout:2/496: symlink d6/dc/d1d/lb1 0 2024-06-06T08:43:27.621 INFO:tasks.workunit.client.0.smithi027.stdout:9/445: dwrite d3/d95/d34/f48 [0,4194304] 0 2024-06-06T08:43:27.623 INFO:tasks.workunit.client.0.smithi027.stdout:8/433: dwrite d7/dc/d36/d83/f27 [0,4194304] 0 2024-06-06T08:43:27.623 INFO:tasks.workunit.client.0.smithi027.stdout:9/446: symlink d3/d95/d23/d36/d3a/d84/l9f 0 2024-06-06T08:43:27.624 INFO:tasks.workunit.client.0.smithi027.stdout:8/434: getdents d7/dc/d37/d46/d50 0 2024-06-06T08:43:27.625 INFO:tasks.workunit.client.0.smithi027.stdout:9/447: link d3/d95/d23/d36/l8b d3/d95/d23/la0 0 2024-06-06T08:43:27.633 INFO:tasks.workunit.client.0.smithi027.stdout:2/497: write d6/dc/f25 [1983234,120719] 0 2024-06-06T08:43:27.634 INFO:tasks.workunit.client.0.smithi027.stdout:2/498: creat d6/dc/fb2 x:0 0 0 2024-06-06T08:43:27.640 INFO:tasks.workunit.client.0.smithi027.stdout:2/499: write d6/dc/d3f/d77/d55/d69/d82/f79 [102775,118431] 0 2024-06-06T08:43:27.641 INFO:tasks.workunit.client.0.smithi027.stdout:8/435: dwrite d7/dc/d36/d3c/f94 [0,4194304] 0 2024-06-06T08:43:27.641 INFO:tasks.workunit.client.0.smithi027.stdout:4/541: sync 2024-06-06T08:43:27.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/535: sync 2024-06-06T08:43:27.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/536: fdatasync d4/d9/d6e/faf 0 2024-06-06T08:43:27.641 INFO:tasks.workunit.client.0.smithi027.stdout:4/542: write d0/d1/d6/f33 [2340584,6931] 0 2024-06-06T08:43:27.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/537: chown d4/d9/dc/d1c/d52/cac 3446357 1 2024-06-06T08:43:27.643 INFO:tasks.workunit.client.0.smithi027.stdout:4/543: dread d0/d1d/d2a/f7d [0,4194304] 0 2024-06-06T08:43:27.648 INFO:tasks.workunit.client.0.smithi027.stdout:2/500: creat d6/dd/d6b/fb3 x:0 0 0 2024-06-06T08:43:27.648 INFO:tasks.workunit.client.0.smithi027.stdout:8/436: mknod d7/d2f/c9b 0 2024-06-06T08:43:27.648 INFO:tasks.workunit.client.0.smithi027.stdout:9/448: rename d3/d95/d23/c6f to d3/d76/ca1 0 2024-06-06T08:43:27.648 INFO:tasks.workunit.client.0.smithi027.stdout:8/437: truncate d7/d13/d15/f8c 819572 0 2024-06-06T08:43:27.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/449: mknod d3/d95/d34/d73/ca2 0 2024-06-06T08:43:27.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/450: stat d3/d4e/d75/dc/c1c 0 2024-06-06T08:43:27.655 INFO:tasks.workunit.client.0.smithi027.stdout:2/501: creat d6/dc/d3f/d77/d55/fb4 x:0 0 0 2024-06-06T08:43:27.655 INFO:tasks.workunit.client.0.smithi027.stdout:8/438: mkdir d7/d9c 0 2024-06-06T08:43:27.655 INFO:tasks.workunit.client.0.smithi027.stdout:8/439: rename d7/d13/d15/f8c to d7/d9c/f9d 0 2024-06-06T08:43:27.669 INFO:tasks.workunit.client.0.smithi027.stdout:4/544: read d0/d1/d6/d8/f49 [1272574,48201] 0 2024-06-06T08:43:27.669 INFO:tasks.workunit.client.0.smithi027.stdout:6/538: dwrite d4/f5 [4194304,4194304] 0 2024-06-06T08:43:27.671 INFO:tasks.workunit.client.0.smithi027.stdout:4/545: mkdir d0/d63/d7e/da3/daa 0 2024-06-06T08:43:27.671 INFO:tasks.workunit.client.0.smithi027.stdout:2/502: write d6/dc/d3f/d77/f4c [1383059,15422] 0 2024-06-06T08:43:27.673 INFO:tasks.workunit.client.0.smithi027.stdout:2/503: getdents d6/dc/d3f/d48 0 2024-06-06T08:43:27.676 INFO:tasks.workunit.client.0.smithi027.stdout:9/451: dwrite d3/d95/d23/d36/f61 [0,4194304] 0 2024-06-06T08:43:27.677 INFO:tasks.workunit.client.0.smithi027.stdout:8/440: dwrite d7/d2f/f87 [4194304,4194304] 0 2024-06-06T08:43:27.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/546: dread d0/d1/d6/d8/f6a [0,4194304] 0 2024-06-06T08:43:27.684 INFO:tasks.workunit.client.0.smithi027.stdout:2/504: dread d6/dc/d3f/d77/d55/d69/d82/f79 [0,4194304] 0 2024-06-06T08:43:27.684 INFO:tasks.workunit.client.0.smithi027.stdout:2/505: chown d6/d1e/d2b/d5a/d7b/l98 15513 1 2024-06-06T08:43:27.685 INFO:tasks.workunit.client.0.smithi027.stdout:4/547: creat d0/d1/d6/d8/d82/fab x:0 0 0 2024-06-06T08:43:27.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/506: truncate d6/dc/d3f/f50 732932 0 2024-06-06T08:43:27.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/507: chown d6/dd 1600152 1 2024-06-06T08:43:27.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/508: stat d6/dc/d3f/d77/d55/f6d 0 2024-06-06T08:43:27.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/509: write d6/d3b/d61/fa5 [297407,85788] 0 2024-06-06T08:43:27.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/510: write d6/dd/f23 [3662918,112531] 0 2024-06-06T08:43:27.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/511: write d6/f8b [370584,129677] 0 2024-06-06T08:43:27.687 INFO:tasks.workunit.client.0.smithi027.stdout:4/548: mknod d0/d1d/d2a/d98/cac 0 2024-06-06T08:43:27.687 INFO:tasks.workunit.client.0.smithi027.stdout:4/549: fdatasync d0/f1c 0 2024-06-06T08:43:27.689 INFO:tasks.workunit.client.0.smithi027.stdout:4/550: unlink d0/d1/d6/d8/d21/c76 0 2024-06-06T08:43:27.689 INFO:tasks.workunit.client.0.smithi027.stdout:6/539: rename d4/d9/dc/d18/l34 to d4/da2/lb1 0 2024-06-06T08:43:27.691 INFO:tasks.workunit.client.0.smithi027.stdout:4/551: mkdir d0/d1d/d34/d7f/d97/dad 0 2024-06-06T08:43:27.691 INFO:tasks.workunit.client.0.smithi027.stdout:4/552: chown d0/d1/d6/d8/f49 7 1 2024-06-06T08:43:27.691 INFO:tasks.workunit.client.0.smithi027.stdout:9/452: rename d3/d95/d23/d36/f61 to d3/d95/d34/fa3 0 2024-06-06T08:43:27.692 INFO:tasks.workunit.client.0.smithi027.stdout:4/553: chown d0/d1/d6/d8/d82/d8b/ca7 1383 1 2024-06-06T08:43:27.692 INFO:tasks.workunit.client.0.smithi027.stdout:9/453: symlink d3/d95/d23/d36/la4 0 2024-06-06T08:43:27.693 INFO:tasks.workunit.client.0.smithi027.stdout:6/540: symlink d4/lb2 0 2024-06-06T08:43:27.694 INFO:tasks.workunit.client.0.smithi027.stdout:8/441: dwrite d7/dc/d37/d46/f8d [0,4194304] 0 2024-06-06T08:43:27.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/512: dwrite d6/f8f [0,4194304] 0 2024-06-06T08:43:27.706 INFO:tasks.workunit.client.0.smithi027.stdout:6/541: getdents d4/d9 0 2024-06-06T08:43:27.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/454: dread d3/d95/d23/d36/d3a/f6d [0,4194304] 0 2024-06-06T08:43:27.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/455: dread - d3/d95/d23/d36/d3a/f68 zero size 2024-06-06T08:43:27.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/456: write d3/f5f [5791081,4240] 0 2024-06-06T08:43:27.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/457: dread - d3/d95/d23/d36/d3a/f67 zero size 2024-06-06T08:43:27.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/458: chown d3/l14 20608 1 2024-06-06T08:43:27.708 INFO:tasks.workunit.client.0.smithi027.stdout:6/542: mknod d4/d9/d49/cb3 0 2024-06-06T08:43:27.714 INFO:tasks.workunit.client.0.smithi027.stdout:6/543: dread d4/d9/f28 [0,4194304] 0 2024-06-06T08:43:27.714 INFO:tasks.workunit.client.0.smithi027.stdout:6/544: dread - d4/d1a/d2e/d44/f99 zero size 2024-06-06T08:43:27.715 INFO:tasks.workunit.client.0.smithi027.stdout:6/545: creat d4/de/fb4 x:0 0 0 2024-06-06T08:43:27.717 INFO:tasks.workunit.client.0.smithi027.stdout:4/554: rename d0/d1/d6/d8/f49 to d0/d1/d6/fae 0 2024-06-06T08:43:27.717 INFO:tasks.workunit.client.0.smithi027.stdout:4/555: chown d0/d1d/d34/f40 4802191 1 2024-06-06T08:43:27.721 INFO:tasks.workunit.client.0.smithi027.stdout:6/546: getdents d4/de/d45/d6a 0 2024-06-06T08:43:27.721 INFO:tasks.workunit.client.0.smithi027.stdout:6/547: getdents d4/de/d45/d6a 0 2024-06-06T08:43:27.721 INFO:tasks.workunit.client.0.smithi027.stdout:6/548: fsync d4/de/f11 0 2024-06-06T08:43:27.722 INFO:tasks.workunit.client.0.smithi027.stdout:6/549: stat d4/de/d45/d6a/f4b 0 2024-06-06T08:43:27.723 INFO:tasks.workunit.client.0.smithi027.stdout:6/550: rmdir d4/d13 39 2024-06-06T08:43:27.723 INFO:tasks.workunit.client.0.smithi027.stdout:6/551: readlink d4/d13/l98 0 2024-06-06T08:43:27.724 INFO:tasks.workunit.client.0.smithi027.stdout:2/513: dwrite d6/dc/d3f/d48/f62 [0,4194304] 0 2024-06-06T08:43:27.724 INFO:tasks.workunit.client.0.smithi027.stdout:8/442: dwrite d7/dc/d36/f7b [0,4194304] 0 2024-06-06T08:43:27.724 INFO:tasks.workunit.client.0.smithi027.stdout:9/459: dread d3/d95/d23/d36/f5a [0,4194304] 0 2024-06-06T08:43:27.724 INFO:tasks.workunit.client.0.smithi027.stdout:2/514: chown d6/d1e/f68 2 1 2024-06-06T08:43:27.724 INFO:tasks.workunit.client.0.smithi027.stdout:9/460: truncate d3/d95/d34/f60 5020843 0 2024-06-06T08:43:27.725 INFO:tasks.workunit.client.0.smithi027.stdout:9/461: write d3/d95/f5e [296012,91961] 0 2024-06-06T08:43:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:2/515: link d6/dc/d3f/l99 d6/dc/d3f/db0/lb5 0 2024-06-06T08:43:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:2/516: chown d6/dc/d3f/d77/lab 13409014 1 2024-06-06T08:43:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:2/517: chown d6/d1e/c35 4459 1 2024-06-06T08:43:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:2/518: dread - d6/d1e/d2b/d5a/f7d zero size 2024-06-06T08:43:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:2/519: fdatasync d6/dd/f23 0 2024-06-06T08:43:27.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/520: creat d6/dc/d3f/d77/d55/fb6 x:0 0 0 2024-06-06T08:43:27.733 INFO:tasks.workunit.client.0.smithi027.stdout:2/521: mkdir d6/d1e/d2b/d5a/db7 0 2024-06-06T08:43:27.733 INFO:tasks.workunit.client.0.smithi027.stdout:2/522: write d6/d3b/f52 [78968,61915] 0 2024-06-06T08:43:27.735 INFO:tasks.workunit.client.0.smithi027.stdout:1/441: sync 2024-06-06T08:43:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:3/395: sync 2024-06-06T08:43:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:1/442: chown dc/d67/d69/d32/d71/f74 9 1 2024-06-06T08:43:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:5/574: sync 2024-06-06T08:43:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:1/443: fdatasync dc/de/d22/f3b 0 2024-06-06T08:43:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/473: sync 2024-06-06T08:43:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:1/444: dread - dc/d67/d69/d32/d37/d56/d78/f83 zero size 2024-06-06T08:43:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/667: sync 2024-06-06T08:43:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:1/445: truncate dc/f84 997585 0 2024-06-06T08:43:27.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/474: write d15/d18/d23/d38/d3e/f5a [2716826,39949] 0 2024-06-06T08:43:27.737 INFO:tasks.workunit.client.0.smithi027.stdout:4/556: dwrite d0/d1d/d2a/f5e [0,4194304] 0 2024-06-06T08:43:27.742 INFO:tasks.workunit.client.0.smithi027.stdout:6/552: dwrite d4/de/d45/d6a/f4b [0,4194304] 0 2024-06-06T08:43:27.743 INFO:tasks.workunit.client.0.smithi027.stdout:3/396: unlink d1/d1f/d25/f4c 0 2024-06-06T08:43:27.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/668: dread d17/d32/d46/d51/f35 [0,4194304] 0 2024-06-06T08:43:27.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/669: fsync d17/d32/d46/d51/d52/d76/fb8 0 2024-06-06T08:43:27.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/670: fsync d17/d32/d46/d83/d2a/d3b/f5b 0 2024-06-06T08:43:27.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/671: getdents d17/d32/d46/d51/d52/d70/d87/dc0 0 2024-06-06T08:43:27.744 INFO:tasks.workunit.client.0.smithi027.stdout:5/575: link d0/da/d37/d2b/f5b d0/da/db/d1a/d5a/fb3 0 2024-06-06T08:43:27.746 INFO:tasks.workunit.client.0.smithi027.stdout:1/446: link dc/fd dc/de/d3d/d4a/d76/f90 0 2024-06-06T08:43:27.746 INFO:tasks.workunit.client.0.smithi027.stdout:1/447: chown dc/de/d3d/d4a/d76 0 1 2024-06-06T08:43:27.748 INFO:tasks.workunit.client.0.smithi027.stdout:4/557: symlink d0/d63/d7e/da3/laf 0 2024-06-06T08:43:27.748 INFO:tasks.workunit.client.0.smithi027.stdout:4/558: chown d0/d1/d6/d4b/l78 1 1 2024-06-06T08:43:27.750 INFO:tasks.workunit.client.0.smithi027.stdout:6/553: creat d4/da2/fb5 x:0 0 0 2024-06-06T08:43:27.751 INFO:tasks.workunit.client.0.smithi027.stdout:3/397: creat d1/d8/d22/f73 x:0 0 0 2024-06-06T08:43:27.753 INFO:tasks.workunit.client.0.smithi027.stdout:5/576: creat d0/da/fb4 x:0 0 0 2024-06-06T08:43:27.753 INFO:tasks.workunit.client.0.smithi027.stdout:2/523: dwrite d6/fa7 [0,4194304] 0 2024-06-06T08:43:27.755 INFO:tasks.workunit.client.0.smithi027.stdout:5/577: creat d0/da/db/d46/d7f/d81/fb5 x:0 0 0 2024-06-06T08:43:27.756 INFO:tasks.workunit.client.0.smithi027.stdout:4/559: creat d0/d1/d6/d4b/fb0 x:0 0 0 2024-06-06T08:43:27.756 INFO:tasks.workunit.client.0.smithi027.stdout:4/560: truncate d0/d1/d6/d8/d21/f74 6078621 0 2024-06-06T08:43:27.758 INFO:tasks.workunit.client.0.smithi027.stdout:3/398: creat d1/da/d44/f74 x:0 0 0 2024-06-06T08:43:27.758 INFO:tasks.workunit.client.0.smithi027.stdout:3/399: dread - d1/d8/f5c zero size 2024-06-06T08:43:27.758 INFO:tasks.workunit.client.0.smithi027.stdout:3/400: truncate d1/d8/d22/d27/d50/f72 228171 0 2024-06-06T08:43:27.758 INFO:tasks.workunit.client.0.smithi027.stdout:3/401: write d1/d1f/d4d/f6b [827312,54407] 0 2024-06-06T08:43:27.758 INFO:tasks.workunit.client.0.smithi027.stdout:4/561: dread d0/d1d/f79 [0,4194304] 0 2024-06-06T08:43:27.758 INFO:tasks.workunit.client.0.smithi027.stdout:1/448: rmdir dc/d67/d69/d32/d37 39 2024-06-06T08:43:27.762 INFO:tasks.workunit.client.0.smithi027.stdout:4/562: creat d0/d1d/d77/d6e/fb1 x:0 0 0 2024-06-06T08:43:27.771 INFO:tasks.workunit.client.0.smithi027.stdout:2/524: dread d6/dd/f18 [0,4194304] 0 2024-06-06T08:43:27.772 INFO:tasks.workunit.client.0.smithi027.stdout:0/475: dwrite d15/d18/d23/d28/d2a/d2d/f58 [0,4194304] 0 2024-06-06T08:43:27.772 INFO:tasks.workunit.client.0.smithi027.stdout:2/525: truncate f5 5403462 0 2024-06-06T08:43:27.772 INFO:tasks.workunit.client.0.smithi027.stdout:2/526: truncate d6/d1e/d2b/d5a/f7d 170415 0 2024-06-06T08:43:27.774 INFO:tasks.workunit.client.0.smithi027.stdout:6/554: dwrite d4/d1a/d2e/d44/f99 [0,4194304] 0 2024-06-06T08:43:27.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/555: symlink d4/d9/dc/d1c/d52/lb6 0 2024-06-06T08:43:27.782 INFO:tasks.workunit.client.0.smithi027.stdout:0/476: symlink d15/d18/d23/d28/d2a/d30/d5c/la3 0 2024-06-06T08:43:27.784 INFO:tasks.workunit.client.0.smithi027.stdout:2/527: read d6/dc/d1d/f5b [362800,123735] 0 2024-06-06T08:43:27.784 INFO:tasks.workunit.client.0.smithi027.stdout:2/528: write d6/dc/d3f/d77/f4c [1892645,100942] 0 2024-06-06T08:43:27.784 INFO:tasks.workunit.client.0.smithi027.stdout:2/529: fdatasync d6/dc/f54 0 2024-06-06T08:43:27.784 INFO:tasks.workunit.client.0.smithi027.stdout:2/530: dread - d6/dc/d3f/d48/fac zero size 2024-06-06T08:43:27.784 INFO:tasks.workunit.client.0.smithi027.stdout:2/531: truncate d6/dd/d6b/fb3 1021370 0 2024-06-06T08:43:27.784 INFO:tasks.workunit.client.0.smithi027.stdout:1/449: dwrite dc/de/d3d/f85 [0,4194304] 0 2024-06-06T08:43:27.787 INFO:tasks.workunit.client.0.smithi027.stdout:6/556: rmdir d4/de 39 2024-06-06T08:43:27.789 INFO:tasks.workunit.client.0.smithi027.stdout:4/563: read d0/d1d/d34/f6b [2060541,42399] 0 2024-06-06T08:43:27.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/402: dwrite d1/d8/d22/d27/f14 [4194304,4194304] 0 2024-06-06T08:43:27.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/403: stat d1/d1f/d25/l4a 0 2024-06-06T08:43:27.797 INFO:tasks.workunit.client.0.smithi027.stdout:0/477: dread d15/d18/d40/f9b [0,4194304] 0 2024-06-06T08:43:27.797 INFO:tasks.workunit.client.0.smithi027.stdout:0/478: fsync d15/d18/d23/d5b/f6b 0 2024-06-06T08:43:27.798 INFO:tasks.workunit.client.0.smithi027.stdout:0/479: mknod d15/d18/d23/d38/d68/d85/ca4 0 2024-06-06T08:43:27.798 INFO:tasks.workunit.client.0.smithi027.stdout:6/557: dread d4/d13/f21 [0,4194304] 0 2024-06-06T08:43:27.801 INFO:tasks.workunit.client.0.smithi027.stdout:0/480: unlink d15/d18/d23/d38/d68/f6a 0 2024-06-06T08:43:27.801 INFO:tasks.workunit.client.0.smithi027.stdout:0/481: readlink d15/l36 0 2024-06-06T08:43:27.802 INFO:tasks.workunit.client.0.smithi027.stdout:9/462: rename d3/l4c to d3/d95/d34/d73/la5 0 2024-06-06T08:43:27.802 INFO:tasks.workunit.client.0.smithi027.stdout:1/450: dread dc/d67/d69/f27 [0,4194304] 0 2024-06-06T08:43:27.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/482: mknod d15/d18/d23/d28/d2a/ca5 0 2024-06-06T08:43:27.804 INFO:tasks.workunit.client.0.smithi027.stdout:0/483: chown d15/d18/d23/d28/d2a/c2b 26 1 2024-06-06T08:43:27.804 INFO:tasks.workunit.client.0.smithi027.stdout:3/404: dread d1/f6 [0,4194304] 0 2024-06-06T08:43:27.804 INFO:tasks.workunit.client.0.smithi027.stdout:3/405: chown d1/f6 25 1 2024-06-06T08:43:27.805 INFO:tasks.workunit.client.0.smithi027.stdout:0/484: truncate f10 2088711 0 2024-06-06T08:43:27.806 INFO:tasks.workunit.client.0.smithi027.stdout:0/485: dread d15/d18/d23/d28/d2a/f33 [0,4194304] 0 2024-06-06T08:43:27.806 INFO:tasks.workunit.client.0.smithi027.stdout:0/486: readlink d15/l3f 0 2024-06-06T08:43:27.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/564: dwrite d0/f5c [0,4194304] 0 2024-06-06T08:43:27.808 INFO:tasks.workunit.client.0.smithi027.stdout:4/565: rmdir d0/d1/d75 39 2024-06-06T08:43:27.811 INFO:tasks.workunit.client.0.smithi027.stdout:4/566: rename d0/d1d/la4 to d0/d1/lb2 0 2024-06-06T08:43:27.811 INFO:tasks.workunit.client.0.smithi027.stdout:4/567: truncate d0/d1/d75/fa1 402117 0 2024-06-06T08:43:27.812 INFO:tasks.workunit.client.0.smithi027.stdout:4/568: write d0/d1/d6/f2f [5058899,8552] 0 2024-06-06T08:43:27.812 INFO:tasks.workunit.client.0.smithi027.stdout:1/451: dread dc/de/d3d/d4a/d76/f90 [0,4194304] 0 2024-06-06T08:43:27.812 INFO:tasks.workunit.client.0.smithi027.stdout:1/452: dread - dc/de/d3d/d4f/f64 zero size 2024-06-06T08:43:27.815 INFO:tasks.workunit.client.0.smithi027.stdout:1/453: link dc/d67/d69/d32/f4d dc/de/d14/f91 0 2024-06-06T08:43:27.817 INFO:tasks.workunit.client.0.smithi027.stdout:6/558: unlink d4/de/c3e 0 2024-06-06T08:43:27.818 INFO:tasks.workunit.client.0.smithi027.stdout:6/559: getdents d4/d13 0 2024-06-06T08:43:27.822 INFO:tasks.workunit.client.0.smithi027.stdout:3/406: dwrite d1/d8/f16 [4194304,4194304] 0 2024-06-06T08:43:27.822 INFO:tasks.workunit.client.0.smithi027.stdout:3/407: stat d1/d8/d22/d27/f14 0 2024-06-06T08:43:27.823 INFO:tasks.workunit.client.0.smithi027.stdout:3/408: stat d1/d8/d22/l5a 0 2024-06-06T08:43:27.825 INFO:tasks.workunit.client.0.smithi027.stdout:4/569: link d0/d1/d6/d8/d21/f74 d0/d63/d7e/fb3 0 2024-06-06T08:43:27.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/570: mknod d0/d1/d6/cb4 0 2024-06-06T08:43:27.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/571: write d0/d1d/d2a/f7d [359895,59894] 0 2024-06-06T08:43:27.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/572: truncate d0/d1/f4f 938277 0 2024-06-06T08:43:27.838 INFO:tasks.workunit.client.0.smithi027.stdout:6/560: dwrite d4/de/d45/d6a/f4b [0,4194304] 0 2024-06-06T08:43:27.844 INFO:tasks.workunit.client.0.smithi027.stdout:4/573: dwrite d0/d1d/d34/f40 [0,4194304] 0 2024-06-06T08:43:27.845 INFO:tasks.workunit.client.0.smithi027.stdout:4/574: readlink d0/d1d/l2c 0 2024-06-06T08:43:27.860 INFO:tasks.workunit.client.0.smithi027.stdout:4/575: write d0/d1/d6/d8/f58 [1535726,110856] 0 2024-06-06T08:43:27.860 INFO:tasks.workunit.client.0.smithi027.stdout:6/561: dwrite d4/d1a/d2e/d44/f74 [0,4194304] 0 2024-06-06T08:43:27.860 INFO:tasks.workunit.client.0.smithi027.stdout:4/576: link d0/c6c d0/d63/cb5 0 2024-06-06T08:43:27.861 INFO:tasks.workunit.client.0.smithi027.stdout:4/577: mkdir d0/d1d/d77/d6e/db6 0 2024-06-06T08:43:27.861 INFO:tasks.workunit.client.0.smithi027.stdout:4/578: chown d0/d1/c4c 2 1 2024-06-06T08:43:27.862 INFO:tasks.workunit.client.0.smithi027.stdout:4/579: creat d0/d1d/d34/d7f/fb7 x:0 0 0 2024-06-06T08:43:27.862 INFO:tasks.workunit.client.0.smithi027.stdout:4/580: fsync d0/d1d/d34/d7f/f94 0 2024-06-06T08:43:27.863 INFO:tasks.workunit.client.0.smithi027.stdout:4/581: truncate d0/d1d/d34/d7f/f94 3073258 0 2024-06-06T08:43:27.864 INFO:tasks.workunit.client.0.smithi027.stdout:4/582: symlink d0/d1d/d34/d7f/d97/lb8 0 2024-06-06T08:43:27.870 INFO:tasks.workunit.client.0.smithi027.stdout:4/583: dread d0/d1d/d2a/f7d [0,4194304] 0 2024-06-06T08:43:27.871 INFO:tasks.workunit.client.0.smithi027.stdout:4/584: mknod d0/d1d/d77/cb9 0 2024-06-06T08:43:27.871 INFO:tasks.workunit.client.0.smithi027.stdout:4/585: fdatasync d0/d1/d6/d4b/fb0 0 2024-06-06T08:43:27.873 INFO:tasks.workunit.client.0.smithi027.stdout:4/586: truncate d0/d1d/f59 3264532 0 2024-06-06T08:43:27.873 INFO:tasks.workunit.client.0.smithi027.stdout:4/587: truncate d0/d1d/f54 4562468 0 2024-06-06T08:43:27.874 INFO:tasks.workunit.client.0.smithi027.stdout:4/588: write d0/d1d/d2a/f7d [99564,6881] 0 2024-06-06T08:43:27.879 INFO:tasks.workunit.client.0.smithi027.stdout:6/562: dwrite d4/de/d45/d6a/f4b [0,4194304] 0 2024-06-06T08:43:27.893 INFO:tasks.workunit.client.0.smithi027.stdout:4/589: dwrite d0/d1/d6/fae [4194304,4194304] 0 2024-06-06T08:43:27.895 INFO:tasks.workunit.client.0.smithi027.stdout:4/590: symlink d0/d1d/d34/lba 0 2024-06-06T08:43:27.896 INFO:tasks.workunit.client.0.smithi027.stdout:6/563: dwrite d4/f29 [0,4194304] 0 2024-06-06T08:43:27.898 INFO:tasks.workunit.client.0.smithi027.stdout:4/591: rmdir d0/d1/d6/d8/d21 39 2024-06-06T08:43:27.906 INFO:tasks.workunit.client.0.smithi027.stdout:4/592: dread d0/f5c [0,4194304] 0 2024-06-06T08:43:27.907 INFO:tasks.workunit.client.0.smithi027.stdout:6/564: rename d4/l42 to d4/d9/dc/d18/lb7 0 2024-06-06T08:43:27.908 INFO:tasks.workunit.client.0.smithi027.stdout:6/565: symlink d4/da2/lb8 0 2024-06-06T08:43:27.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/593: dread d0/d1/d6/d8/f58 [4194304,4194304] 0 2024-06-06T08:43:27.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/594: creat d0/d1d/d34/d7f/d97/fbb x:0 0 0 2024-06-06T08:43:27.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/595: chown d0/d1d/d34/f6b 67416 1 2024-06-06T08:43:27.927 INFO:tasks.workunit.client.0.smithi027.stdout:6/566: dwrite d4/d9/dc/d1c/d52/fa9 [0,4194304] 0 2024-06-06T08:43:27.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/596: dread d0/d1/d6/fae [0,4194304] 0 2024-06-06T08:43:27.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/597: write d0/d1/d6/d8/f58 [4128978,10698] 0 2024-06-06T08:43:27.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/598: dread - d0/d1/d6/d4b/f80 zero size 2024-06-06T08:43:27.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/599: write d0/d1d/d34/d7f/d97/fbb [1027315,54971] 0 2024-06-06T08:43:27.935 INFO:tasks.workunit.client.0.smithi027.stdout:4/600: creat d0/d1/d6/fbc x:0 0 0 2024-06-06T08:43:27.935 INFO:tasks.workunit.client.0.smithi027.stdout:4/601: chown d0/d1d/d77/cb9 455 1 2024-06-06T08:43:28.104 INFO:tasks.workunit.client.0.smithi027.stdout:8/443: sync 2024-06-06T08:43:28.104 INFO:tasks.workunit.client.0.smithi027.stdout:8/444: read d7/dc/d36/d3c/f61 [1024139,109793] 0 2024-06-06T08:43:28.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/445: symlink d7/dc/d36/d41/d85/l9e 0 2024-06-06T08:43:28.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/446: chown d7/dc/d36/d83/f91 18891 1 2024-06-06T08:43:28.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/447: truncate d7/dc/d37/d46/d50/f62 632651 0 2024-06-06T08:43:28.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/448: chown d7/dc/d36/l8b 96480 1 2024-06-06T08:43:28.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/449: chown d7/d2f/f70 2430371 1 2024-06-06T08:43:28.107 INFO:tasks.workunit.client.0.smithi027.stdout:8/450: dread - d7/dc/d37/f80 zero size 2024-06-06T08:43:28.107 INFO:tasks.workunit.client.0.smithi027.stdout:8/451: creat d7/dc/f9f x:0 0 0 2024-06-06T08:43:28.108 INFO:tasks.workunit.client.0.smithi027.stdout:8/452: creat d7/d13/fa0 x:0 0 0 2024-06-06T08:43:28.110 INFO:tasks.workunit.client.0.smithi027.stdout:8/453: unlink d7/f90 0 2024-06-06T08:43:28.111 INFO:tasks.workunit.client.0.smithi027.stdout:8/454: rmdir d7/dc 39 2024-06-06T08:43:28.126 INFO:tasks.workunit.client.0.smithi027.stdout:8/455: dwrite d7/dc/d36/d83/d60/f78 [0,4194304] 0 2024-06-06T08:43:28.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/456: read d7/dc/d36/d3c/f94 [3040199,63062] 0 2024-06-06T08:43:28.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/457: dread - d7/dc/d36/d83/d58/f8a zero size 2024-06-06T08:43:28.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/458: chown d7/d2f 0 1 2024-06-06T08:43:28.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/459: chown d7/d13/l40 44694650 1 2024-06-06T08:43:28.137 INFO:tasks.workunit.client.0.smithi027.stdout:8/460: symlink d7/dc/d36/d41/d85/la1 0 2024-06-06T08:43:28.137 INFO:tasks.workunit.client.0.smithi027.stdout:8/461: readlink d7/dc/d36/d41/d85/la1 0 2024-06-06T08:43:28.137 INFO:tasks.workunit.client.0.smithi027.stdout:8/462: chown d7/dc/d37/l7e 748528231 1 2024-06-06T08:43:28.139 INFO:tasks.workunit.client.0.smithi027.stdout:8/463: rename d7/dc/d37/d46/d50/f62 to d7/dc/d36/d41/d85/fa2 0 2024-06-06T08:43:28.139 INFO:tasks.workunit.client.0.smithi027.stdout:8/464: chown d7/dc/d36/d83/f19 42742804 1 2024-06-06T08:43:28.150 INFO:tasks.workunit.client.0.smithi027.stdout:8/465: rename d7/dc/d36/d41/l56 to d7/la3 0 2024-06-06T08:43:28.163 INFO:tasks.workunit.client.0.smithi027.stdout:8/466: dwrite d7/dc/d36/d83/d60/f68 [0,4194304] 0 2024-06-06T08:43:28.174 INFO:tasks.workunit.client.0.smithi027.stdout:8/467: dread d7/dc/d36/f7b [0,4194304] 0 2024-06-06T08:43:28.176 INFO:tasks.workunit.client.0.smithi027.stdout:8/468: unlink d7/dc/d36/d41/f25 0 2024-06-06T08:43:28.177 INFO:tasks.workunit.client.0.smithi027.stdout:8/469: link d7/dc/d37/d46/f52 d7/dc/d36/d83/fa4 0 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:5/578: sync 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:9/463: sync 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:7/672: sync 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:3/409: sync 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:2/532: sync 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:6/567: sync 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/487: sync 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:9/464: dread - d3/d4e/d75/dc/f8d zero size 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:3/410: dread - d1/d1f/d25/f5d zero size 2024-06-06T08:43:28.241 INFO:tasks.workunit.client.0.smithi027.stdout:9/465: stat d3/d95/d34/fa3 0 2024-06-06T08:43:28.242 INFO:tasks.workunit.client.0.smithi027.stdout:5/579: symlink d0/da/db/d1a/lb6 0 2024-06-06T08:43:28.242 INFO:tasks.workunit.client.0.smithi027.stdout:7/673: mknod d17/d32/d46/d83/d77/d8c/cdc 0 2024-06-06T08:43:28.242 INFO:tasks.workunit.client.0.smithi027.stdout:2/533: dread d6/f6e [0,4194304] 0 2024-06-06T08:43:28.242 INFO:tasks.workunit.client.0.smithi027.stdout:9/466: mknod d3/d4e/ca6 0 2024-06-06T08:43:28.243 INFO:tasks.workunit.client.0.smithi027.stdout:7/674: creat d17/d32/d46/d83/d77/fdd x:0 0 0 2024-06-06T08:43:28.243 INFO:tasks.workunit.client.0.smithi027.stdout:7/675: fsync d17/d32/d46/d4f/d5e/f8b 0 2024-06-06T08:43:28.244 INFO:tasks.workunit.client.0.smithi027.stdout:7/676: symlink d17/d32/d46/d83/d72/lde 0 2024-06-06T08:43:28.244 INFO:tasks.workunit.client.0.smithi027.stdout:5/580: unlink d0/da/d37/f48 0 2024-06-06T08:43:28.245 INFO:tasks.workunit.client.0.smithi027.stdout:0/488: stat d15/d18/d40 0 2024-06-06T08:43:28.245 INFO:tasks.workunit.client.0.smithi027.stdout:2/534: creat d6/d1e/d2b/d5a/db7/fb8 x:0 0 0 2024-06-06T08:43:28.246 INFO:tasks.workunit.client.0.smithi027.stdout:5/581: symlink d0/da/db/d46/d7f/d81/d97/lb7 0 2024-06-06T08:43:28.247 INFO:tasks.workunit.client.0.smithi027.stdout:9/467: rmdir d3/d95/d34 39 2024-06-06T08:43:28.247 INFO:tasks.workunit.client.0.smithi027.stdout:9/468: dread - d3/d95/d23/d36/d3a/f67 zero size 2024-06-06T08:43:28.247 INFO:tasks.workunit.client.0.smithi027.stdout:2/535: rmdir d6/dc 39 2024-06-06T08:43:28.248 INFO:tasks.workunit.client.0.smithi027.stdout:2/536: write d6/dc/d3f/d77/d55/fb6 [378633,5094] 0 2024-06-06T08:43:28.251 INFO:tasks.workunit.client.0.smithi027.stdout:2/537: mknod d6/d3b/d80/cb9 0 2024-06-06T08:43:28.253 INFO:tasks.workunit.client.0.smithi027.stdout:2/538: rename d6/d1e/d2b/d5a/f7d to d6/d1e/d2b/d5a/fba 0 2024-06-06T08:43:28.255 INFO:tasks.workunit.client.0.smithi027.stdout:6/568: dread d4/d13/f92 [0,4194304] 0 2024-06-06T08:43:28.255 INFO:tasks.workunit.client.0.smithi027.stdout:3/411: dread d1/d8/d22/d27/f14 [4194304,4194304] 0 2024-06-06T08:43:28.255 INFO:tasks.workunit.client.0.smithi027.stdout:0/489: dread d15/d18/d23/d3c/f67 [0,4194304] 0 2024-06-06T08:43:28.257 INFO:tasks.workunit.client.0.smithi027.stdout:6/569: symlink d4/d9/d49/lb9 0 2024-06-06T08:43:28.257 INFO:tasks.workunit.client.0.smithi027.stdout:6/570: chown d4/de/d45/f48 54812023 1 2024-06-06T08:43:28.260 INFO:tasks.workunit.client.0.smithi027.stdout:6/571: dread d4/f25 [0,4194304] 0 2024-06-06T08:43:28.261 INFO:tasks.workunit.client.0.smithi027.stdout:5/582: read d0/da/d37/d2b/f4a [3301998,10852] 0 2024-06-06T08:43:28.262 INFO:tasks.workunit.client.0.smithi027.stdout:5/583: read - d0/da/db/d46/d4c/f82 zero size 2024-06-06T08:43:28.262 INFO:tasks.workunit.client.0.smithi027.stdout:5/584: chown d0/d6a/f8a 1195149 1 2024-06-06T08:43:28.266 INFO:tasks.workunit.client.0.smithi027.stdout:6/572: rmdir d4/d9/dc 39 2024-06-06T08:43:28.268 INFO:tasks.workunit.client.0.smithi027.stdout:6/573: mkdir d4/d13/dba 0 2024-06-06T08:43:28.269 INFO:tasks.workunit.client.0.smithi027.stdout:7/677: dwrite d17/d32/d46/d83/d77/d8c/d91/daa/fdb [0,4194304] 0 2024-06-06T08:43:28.271 INFO:tasks.workunit.client.0.smithi027.stdout:7/678: mknod d17/d32/d46/d4f/d5e/d7b/d7c/cdf 0 2024-06-06T08:43:28.281 INFO:tasks.workunit.client.0.smithi027.stdout:7/679: dread d17/d32/d61/f74 [0,4194304] 0 2024-06-06T08:43:28.281 INFO:tasks.workunit.client.0.smithi027.stdout:7/680: read - d17/d32/fcb zero size 2024-06-06T08:43:28.282 INFO:tasks.workunit.client.0.smithi027.stdout:0/490: dwrite fc [0,4194304] 0 2024-06-06T08:43:28.283 INFO:tasks.workunit.client.0.smithi027.stdout:7/681: mkdir d17/d32/d46/d4f/db6/dc8/de0 0 2024-06-06T08:43:28.284 INFO:tasks.workunit.client.0.smithi027.stdout:2/539: dwrite d6/d1e/f9a [0,4194304] 0 2024-06-06T08:43:28.284 INFO:tasks.workunit.client.0.smithi027.stdout:0/491: creat d15/d18/d40/d8c/d9c/fa6 x:0 0 0 2024-06-06T08:43:28.285 INFO:tasks.workunit.client.0.smithi027.stdout:6/574: dwrite d4/d1a/d2e/f5c [0,4194304] 0 2024-06-06T08:43:28.285 INFO:tasks.workunit.client.0.smithi027.stdout:6/575: stat d4/f5 0 2024-06-06T08:43:28.287 INFO:tasks.workunit.client.0.smithi027.stdout:5/585: dwrite d0/da/db/d46/d7f/f8e [0,4194304] 0 2024-06-06T08:43:28.287 INFO:tasks.workunit.client.0.smithi027.stdout:2/540: link d6/d1e/d2b/d5a/l75 d6/dc/d3f/db0/lbb 0 2024-06-06T08:43:28.288 INFO:tasks.workunit.client.0.smithi027.stdout:2/541: chown d6/dc/d3f/d77/d55/d69/d82/f79 1625 1 2024-06-06T08:43:28.288 INFO:tasks.workunit.client.0.smithi027.stdout:2/542: chown d6/dc/d3f/d77/d55/fb4 598917 1 2024-06-06T08:43:28.288 INFO:tasks.workunit.client.0.smithi027.stdout:3/412: dwrite d1/f20 [0,4194304] 0 2024-06-06T08:43:28.288 INFO:tasks.workunit.client.0.smithi027.stdout:0/492: mkdir d15/da7 0 2024-06-06T08:43:28.289 INFO:tasks.workunit.client.0.smithi027.stdout:1/454: sync 2024-06-06T08:43:28.289 INFO:tasks.workunit.client.0.smithi027.stdout:4/602: sync 2024-06-06T08:43:28.289 INFO:tasks.workunit.client.0.smithi027.stdout:1/455: write dc/de/d22/f3b [5052237,5515] 0 2024-06-06T08:43:28.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/543: mkdir d6/dc/d1d/dbc 0 2024-06-06T08:43:28.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/603: dread d0/d1/d6/d8/f6a [0,4194304] 0 2024-06-06T08:43:28.298 INFO:tasks.workunit.client.0.smithi027.stdout:0/493: mknod d15/da7/ca8 0 2024-06-06T08:43:28.298 INFO:tasks.workunit.client.0.smithi027.stdout:0/494: chown d15/d18/d23/d28/d2a/c46 65736792 1 2024-06-06T08:43:28.300 INFO:tasks.workunit.client.0.smithi027.stdout:6/576: dread d4/d9/dc/d18/f2f [0,4194304] 0 2024-06-06T08:43:28.300 INFO:tasks.workunit.client.0.smithi027.stdout:2/544: rename d6/d3b/f52 to d6/fbd 0 2024-06-06T08:43:28.300 INFO:tasks.workunit.client.0.smithi027.stdout:6/577: write d4/d9/d49/f77 [958440,32594] 0 2024-06-06T08:43:28.301 INFO:tasks.workunit.client.0.smithi027.stdout:5/586: read d0/da/db/d1a/d5a/fb3 [790333,12644] 0 2024-06-06T08:43:28.303 INFO:tasks.workunit.client.0.smithi027.stdout:1/456: write dc/f36 [316652,51782] 0 2024-06-06T08:43:28.314 INFO:tasks.workunit.client.0.smithi027.stdout:3/413: dread d1/f29 [0,4194304] 0 2024-06-06T08:43:28.314 INFO:tasks.workunit.client.0.smithi027.stdout:3/414: fdatasync d1/f29 0 2024-06-06T08:43:28.314 INFO:tasks.workunit.client.0.smithi027.stdout:3/415: chown d1/da/d6d 3 1 2024-06-06T08:43:28.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/604: dread d0/fd [4194304,4194304] 0 2024-06-06T08:43:28.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/605: fdatasync d0/d1d/d2a/f5e 0 2024-06-06T08:43:28.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/416: getdents d1/d1f/d25 0 2024-06-06T08:43:28.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/417: truncate d1/da/f5e 682657 0 2024-06-06T08:43:28.316 INFO:tasks.workunit.client.0.smithi027.stdout:3/418: mknod d1/da/d6d/c75 0 2024-06-06T08:43:28.317 INFO:tasks.workunit.client.0.smithi027.stdout:5/587: read d0/f1 [405135,26271] 0 2024-06-06T08:43:28.317 INFO:tasks.workunit.client.0.smithi027.stdout:5/588: write d0/da/d20/f36 [1985632,19608] 0 2024-06-06T08:43:28.320 INFO:tasks.workunit.client.0.smithi027.stdout:3/419: creat d1/d1f/d25/f76 x:0 0 0 2024-06-06T08:43:28.321 INFO:tasks.workunit.client.0.smithi027.stdout:5/589: rename d0/d6a/c6b to d0/da/db/d46/d77/cb8 0 2024-06-06T08:43:28.322 INFO:tasks.workunit.client.0.smithi027.stdout:3/420: creat d1/da/f77 x:0 0 0 2024-06-06T08:43:28.323 INFO:tasks.workunit.client.0.smithi027.stdout:2/545: dwrite d6/d1e/d2b/f57 [0,4194304] 0 2024-06-06T08:43:28.324 INFO:tasks.workunit.client.0.smithi027.stdout:6/578: dread d4/d9/dc/d1c/d52/fa9 [0,4194304] 0 2024-06-06T08:43:28.324 INFO:tasks.workunit.client.0.smithi027.stdout:5/590: unlink d0/f4e 0 2024-06-06T08:43:28.324 INFO:tasks.workunit.client.0.smithi027.stdout:6/579: chown d4/d9/d6e/faf 48 1 2024-06-06T08:43:28.325 INFO:tasks.workunit.client.0.smithi027.stdout:5/591: read d0/f4 [953843,102046] 0 2024-06-06T08:43:28.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/546: rename d6/c1b to d6/dc/d3f/d77/cbe 0 2024-06-06T08:43:28.326 INFO:tasks.workunit.client.0.smithi027.stdout:5/592: symlink d0/da/db/lb9 0 2024-06-06T08:43:28.327 INFO:tasks.workunit.client.0.smithi027.stdout:3/421: link d1/f1e d1/d8/d22/d27/f78 0 2024-06-06T08:43:28.328 INFO:tasks.workunit.client.0.smithi027.stdout:5/593: mknod d0/da/db/d1a/da9/cba 0 2024-06-06T08:43:28.328 INFO:tasks.workunit.client.0.smithi027.stdout:5/594: write d0/da/db/f51 [5140756,18886] 0 2024-06-06T08:43:28.329 INFO:tasks.workunit.client.0.smithi027.stdout:5/595: chown d0/da/d20/c39 1815621 1 2024-06-06T08:43:28.329 INFO:tasks.workunit.client.0.smithi027.stdout:5/596: symlink d0/da/db/d1a/lbb 0 2024-06-06T08:43:28.329 INFO:tasks.workunit.client.0.smithi027.stdout:5/597: readlink d0/da/db/d1a/lbb 0 2024-06-06T08:43:28.331 INFO:tasks.workunit.client.0.smithi027.stdout:0/495: dwrite d15/d18/f1c [0,4194304] 0 2024-06-06T08:43:28.331 INFO:tasks.workunit.client.0.smithi027.stdout:0/496: truncate d15/d18/f1c 4839805 0 2024-06-06T08:43:28.331 INFO:tasks.workunit.client.0.smithi027.stdout:5/598: mkdir d0/da/db/d46/d7f/d81/d97/db0/dbc 0 2024-06-06T08:43:28.333 INFO:tasks.workunit.client.0.smithi027.stdout:1/457: dwrite dc/f30 [4194304,4194304] 0 2024-06-06T08:43:28.333 INFO:tasks.workunit.client.0.smithi027.stdout:0/497: getdents d15/d18/d23/d28/d2a/d30/d5d 0 2024-06-06T08:43:28.337 INFO:tasks.workunit.client.0.smithi027.stdout:1/458: unlink dc/c18 0 2024-06-06T08:43:28.339 INFO:tasks.workunit.client.0.smithi027.stdout:4/606: dwrite d0/d1/ff [0,4194304] 0 2024-06-06T08:43:28.339 INFO:tasks.workunit.client.0.smithi027.stdout:3/422: mkdir d1/d8/d22/d27/d79 0 2024-06-06T08:43:28.345 INFO:tasks.workunit.client.0.smithi027.stdout:4/607: unlink d0/d1/c4 0 2024-06-06T08:43:28.345 INFO:tasks.workunit.client.0.smithi027.stdout:2/547: dread d6/d1e/d2b/f58 [0,4194304] 0 2024-06-06T08:43:28.345 INFO:tasks.workunit.client.0.smithi027.stdout:2/548: fdatasync d6/dc/d1d/f5b 0 2024-06-06T08:43:28.348 INFO:tasks.workunit.client.0.smithi027.stdout:3/423: symlink d1/d8/d22/d27/d50/l7a 0 2024-06-06T08:43:28.349 INFO:tasks.workunit.client.0.smithi027.stdout:1/459: dread dc/f30 [4194304,4194304] 0 2024-06-06T08:43:28.349 INFO:tasks.workunit.client.0.smithi027.stdout:6/580: dwrite d4/de/d45/d6a/f4b [0,4194304] 0 2024-06-06T08:43:28.350 INFO:tasks.workunit.client.0.smithi027.stdout:6/581: write d4/d9/f94 [344070,94327] 0 2024-06-06T08:43:28.350 INFO:tasks.workunit.client.0.smithi027.stdout:6/582: dread - d4/de/d45/d6a/f76 zero size 2024-06-06T08:43:28.350 INFO:tasks.workunit.client.0.smithi027.stdout:4/608: getdents d0/d63 0 2024-06-06T08:43:28.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/424: symlink d1/d8/d59/l7b 0 2024-06-06T08:43:28.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/425: stat d1/da/d6d/c75 0 2024-06-06T08:43:28.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/426: read d1/f2 [7939562,44256] 0 2024-06-06T08:43:28.356 INFO:tasks.workunit.client.0.smithi027.stdout:1/460: symlink dc/de/d3d/d4f/l92 0 2024-06-06T08:43:28.358 INFO:tasks.workunit.client.0.smithi027.stdout:0/498: dwrite d15/d18/f73 [0,4194304] 0 2024-06-06T08:43:28.359 INFO:tasks.workunit.client.0.smithi027.stdout:4/609: creat d0/d1d/d34/d7f/d97/d9c/fbd x:0 0 0 2024-06-06T08:43:28.362 INFO:tasks.workunit.client.0.smithi027.stdout:4/610: mkdir d0/d1d/d34/d7f/d97/d9c/dbe 0 2024-06-06T08:43:28.370 INFO:tasks.workunit.client.0.smithi027.stdout:6/583: dread d4/d9/dc/d1c/d52/fa5 [0,4194304] 0 2024-06-06T08:43:28.371 INFO:tasks.workunit.client.0.smithi027.stdout:6/584: symlink d4/d9/d6e/lbb 0 2024-06-06T08:43:28.371 INFO:tasks.workunit.client.0.smithi027.stdout:6/585: fdatasync d4/d9/dc/d1c/d52/fa5 0 2024-06-06T08:43:28.373 INFO:tasks.workunit.client.0.smithi027.stdout:2/549: dwrite d6/dc/d1d/f36 [0,4194304] 0 2024-06-06T08:43:28.373 INFO:tasks.workunit.client.0.smithi027.stdout:4/611: dread d0/d1/f14 [8388608,4194304] 0 2024-06-06T08:43:28.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/550: getdents d6/dc/d3f/d77 0 2024-06-06T08:43:28.378 INFO:tasks.workunit.client.0.smithi027.stdout:4/612: symlink d0/d1d/d77/d6e/db6/lbf 0 2024-06-06T08:43:28.378 INFO:tasks.workunit.client.0.smithi027.stdout:1/461: dwrite dc/de/d3d/f50 [0,4194304] 0 2024-06-06T08:43:28.379 INFO:tasks.workunit.client.0.smithi027.stdout:2/551: symlink d6/dc/d3f/d77/d55/d69/lbf 0 2024-06-06T08:43:28.387 INFO:tasks.workunit.client.0.smithi027.stdout:1/462: read dc/de/d14/f73 [4022567,124206] 0 2024-06-06T08:43:28.391 INFO:tasks.workunit.client.0.smithi027.stdout:6/586: write d4/de/d45/d6a/f4b [1941779,118025] 0 2024-06-06T08:43:28.396 INFO:tasks.workunit.client.0.smithi027.stdout:5/599: rename d0/da/db/d46/d7f/d81/d97/la5 to d0/da/db/d46/d7f/lbd 0 2024-06-06T08:43:28.398 INFO:tasks.workunit.client.0.smithi027.stdout:2/552: dwrite d6/d1e/f27 [0,4194304] 0 2024-06-06T08:43:28.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/553: readlink d6/dc/d1d/lb1 0 2024-06-06T08:43:28.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/554: chown d6/d1e/d2b/f57 152710 1 2024-06-06T08:43:28.400 INFO:tasks.workunit.client.0.smithi027.stdout:3/427: rename d1/f13 to d1/d1f/d45/d5f/f7c 0 2024-06-06T08:43:28.400 INFO:tasks.workunit.client.0.smithi027.stdout:1/463: dread dc/f30 [0,4194304] 0 2024-06-06T08:43:28.400 INFO:tasks.workunit.client.0.smithi027.stdout:4/613: dwrite d0/d1d/d34/fa2 [0,4194304] 0 2024-06-06T08:43:28.400 INFO:tasks.workunit.client.0.smithi027.stdout:1/464: readlink dc/l2d 0 2024-06-06T08:43:28.401 INFO:tasks.workunit.client.0.smithi027.stdout:3/428: dread d1/da/d44/f49 [4194304,4194304] 0 2024-06-06T08:43:28.404 INFO:tasks.workunit.client.0.smithi027.stdout:6/587: write d4/d9/fad [3116215,91036] 0 2024-06-06T08:43:28.412 INFO:tasks.workunit.client.0.smithi027.stdout:1/465: chown dc/de/d14/c17 446451065 1 2024-06-06T08:43:28.412 INFO:tasks.workunit.client.0.smithi027.stdout:3/429: read d1/d8/d22/d27/f14 [7902919,97020] 0 2024-06-06T08:43:28.413 INFO:tasks.workunit.client.0.smithi027.stdout:4/614: mknod d0/d1d/d34/d7f/d97/cc0 0 2024-06-06T08:43:28.413 INFO:tasks.workunit.client.0.smithi027.stdout:1/466: stat dc/de/d14/f73 0 2024-06-06T08:43:28.414 INFO:tasks.workunit.client.0.smithi027.stdout:6/588: mknod d4/de/d45/d6a/cbc 0 2024-06-06T08:43:28.416 INFO:tasks.workunit.client.0.smithi027.stdout:4/615: symlink d0/d1/d6/d8/d21/lc1 0 2024-06-06T08:43:28.416 INFO:tasks.workunit.client.0.smithi027.stdout:4/616: truncate d0/d1/d6/d8/d82/fab 843437 0 2024-06-06T08:43:28.416 INFO:tasks.workunit.client.0.smithi027.stdout:4/617: chown d0/d1d/d34 24596439 1 2024-06-06T08:43:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:4/618: creat d0/d1/d6/fc2 x:0 0 0 2024-06-06T08:43:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:4/619: fsync d0/d1d/d34/f6b 0 2024-06-06T08:43:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:5/600: dread d0/da/d37/d2b/f67 [4194304,4194304] 0 2024-06-06T08:43:28.418 INFO:tasks.workunit.client.0.smithi027.stdout:4/620: creat d0/d1d/d34/d7f/d97/d9c/dbe/fc3 x:0 0 0 2024-06-06T08:43:28.419 INFO:tasks.workunit.client.0.smithi027.stdout:4/621: mknod d0/d1d/d60/cc4 0 2024-06-06T08:43:28.423 INFO:tasks.workunit.client.0.smithi027.stdout:3/430: dread d1/d1f/d4d/f52 [0,4194304] 0 2024-06-06T08:43:28.425 INFO:tasks.workunit.client.0.smithi027.stdout:2/555: dwrite d6/dc/fb2 [0,4194304] 0 2024-06-06T08:43:28.426 INFO:tasks.workunit.client.0.smithi027.stdout:3/431: link d1/da/f18 d1/d1f/d25/f7d 0 2024-06-06T08:43:28.426 INFO:tasks.workunit.client.0.smithi027.stdout:3/432: dread - d1/d1f/f6f zero size 2024-06-06T08:43:28.427 INFO:tasks.workunit.client.0.smithi027.stdout:2/556: dread d6/dc/d1d/f60 [0,4194304] 0 2024-06-06T08:43:28.434 INFO:tasks.workunit.client.0.smithi027.stdout:5/601: write d0/da/db/d1a/f71 [5928194,127695] 0 2024-06-06T08:43:28.438 INFO:tasks.workunit.client.0.smithi027.stdout:5/602: creat d0/da/db/d1a/da9/fbe x:0 0 0 2024-06-06T08:43:28.445 INFO:tasks.workunit.client.0.smithi027.stdout:1/467: dwrite f5 [0,4194304] 0 2024-06-06T08:43:28.445 INFO:tasks.workunit.client.0.smithi027.stdout:4/622: dwrite d0/d1d/d34/f6b [0,4194304] 0 2024-06-06T08:43:28.446 INFO:tasks.workunit.client.0.smithi027.stdout:6/589: dwrite d4/d9/dc/f43 [0,4194304] 0 2024-06-06T08:43:28.449 INFO:tasks.workunit.client.0.smithi027.stdout:4/623: unlink d0/d63/c6f 0 2024-06-06T08:43:28.453 INFO:tasks.workunit.client.0.smithi027.stdout:2/557: dwrite d6/f14 [0,4194304] 0 2024-06-06T08:43:28.453 INFO:tasks.workunit.client.0.smithi027.stdout:2/558: chown d6/dc/d3f/d77/c65 8247863 1 2024-06-06T08:43:28.455 INFO:tasks.workunit.client.0.smithi027.stdout:2/559: creat d6/fc0 x:0 0 0 2024-06-06T08:43:28.456 INFO:tasks.workunit.client.0.smithi027.stdout:2/560: symlink d6/dc/d1d/d4d/lc1 0 2024-06-06T08:43:28.456 INFO:tasks.workunit.client.0.smithi027.stdout:2/561: chown d6/dc/f54 0 1 2024-06-06T08:43:28.458 INFO:tasks.workunit.client.0.smithi027.stdout:6/590: read d4/d1a/f2c [882863,61006] 0 2024-06-06T08:43:28.459 INFO:tasks.workunit.client.0.smithi027.stdout:3/433: dwrite d1/d8/f16 [4194304,4194304] 0 2024-06-06T08:43:28.460 INFO:tasks.workunit.client.0.smithi027.stdout:2/562: dread d6/dc/d3f/d77/d55/d69/d82/f79 [0,4194304] 0 2024-06-06T08:43:28.460 INFO:tasks.workunit.client.0.smithi027.stdout:2/563: stat d6/dc/f40 0 2024-06-06T08:43:28.461 INFO:tasks.workunit.client.0.smithi027.stdout:3/434: mknod d1/d1f/c7e 0 2024-06-06T08:43:28.462 INFO:tasks.workunit.client.0.smithi027.stdout:3/435: creat d1/d8/d22/d2d/f7f x:0 0 0 2024-06-06T08:43:28.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/468: dwrite dc/d67/d69/d32/f5b [0,4194304] 0 2024-06-06T08:43:28.468 INFO:tasks.workunit.client.0.smithi027.stdout:1/469: creat dc/d67/d69/d32/d37/d56/f93 x:0 0 0 2024-06-06T08:43:28.468 INFO:tasks.workunit.client.0.smithi027.stdout:8/470: sync 2024-06-06T08:43:28.468 INFO:tasks.workunit.client.0.smithi027.stdout:8/471: chown d7/dc/d36/d83/d58/c5a 847 1 2024-06-06T08:43:28.468 INFO:tasks.workunit.client.0.smithi027.stdout:8/472: fsync d7/dc/d36/f75 0 2024-06-06T08:43:28.469 INFO:tasks.workunit.client.0.smithi027.stdout:1/470: creat dc/d67/d69/d32/f94 x:0 0 0 2024-06-06T08:43:28.469 INFO:tasks.workunit.client.0.smithi027.stdout:1/471: truncate dc/d67/d69/d32/f8b 811181 0 2024-06-06T08:43:28.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/624: rename d0/d1/d6/d8/d21/f30 to d0/d1d/d60/fc5 0 2024-06-06T08:43:28.476 INFO:tasks.workunit.client.0.smithi027.stdout:8/473: dread d7/d13/f24 [0,4194304] 0 2024-06-06T08:43:28.476 INFO:tasks.workunit.client.0.smithi027.stdout:6/591: dwrite d4/d9/dc/d1c/f9e [0,4194304] 0 2024-06-06T08:43:28.476 INFO:tasks.workunit.client.0.smithi027.stdout:4/625: dread d0/d1/f12 [0,4194304] 0 2024-06-06T08:43:28.486 INFO:tasks.workunit.client.0.smithi027.stdout:8/474: dread d7/dc/d36/d83/f6e [0,4194304] 0 2024-06-06T08:43:28.488 INFO:tasks.workunit.client.0.smithi027.stdout:4/626: rename d0/d1d/d77 to d0/d1d/d60/dc6 0 2024-06-06T08:43:28.493 INFO:tasks.workunit.client.0.smithi027.stdout:6/592: dwrite d4/d9/f58 [0,4194304] 0 2024-06-06T08:43:28.495 INFO:tasks.workunit.client.0.smithi027.stdout:6/593: creat d4/de/d45/d6a/fbd x:0 0 0 2024-06-06T08:43:28.495 INFO:tasks.workunit.client.0.smithi027.stdout:6/594: fdatasync d4/d1a/d2e/f5c 0 2024-06-06T08:43:28.498 INFO:tasks.workunit.client.0.smithi027.stdout:8/475: dread d7/dc/d36/d83/f19 [0,4194304] 0 2024-06-06T08:43:28.503 INFO:tasks.workunit.client.0.smithi027.stdout:8/476: read d7/dc/d36/d83/fa4 [18599,77022] 0 2024-06-06T08:43:28.503 INFO:tasks.workunit.client.0.smithi027.stdout:8/477: readlink d7/d13/d15/l84 0 2024-06-06T08:43:28.503 INFO:tasks.workunit.client.0.smithi027.stdout:6/595: dread d4/d9/dc/fa8 [0,4194304] 0 2024-06-06T08:43:28.504 INFO:tasks.workunit.client.0.smithi027.stdout:8/478: unlink d7/d13/c95 0 2024-06-06T08:43:28.505 INFO:tasks.workunit.client.0.smithi027.stdout:8/479: rmdir d7/dc/d37 39 2024-06-06T08:43:28.506 INFO:tasks.workunit.client.0.smithi027.stdout:6/596: dread d4/d9/d49/f8d [0,4194304] 0 2024-06-06T08:43:28.507 INFO:tasks.workunit.client.0.smithi027.stdout:6/597: creat d4/d9/dc/d18/fbe x:0 0 0 2024-06-06T08:43:28.514 INFO:tasks.workunit.client.0.smithi027.stdout:8/480: write d7/dc/d36/d83/f19 [2269979,57358] 0 2024-06-06T08:43:28.516 INFO:tasks.workunit.client.0.smithi027.stdout:8/481: unlink d7/d9c/f9d 0 2024-06-06T08:43:28.517 INFO:tasks.workunit.client.0.smithi027.stdout:8/482: mkdir d7/dc/d36/d83/d60/da5 0 2024-06-06T08:43:28.523 INFO:tasks.workunit.client.0.smithi027.stdout:6/598: dwrite d4/d9/dc/d18/f24 [0,4194304] 0 2024-06-06T08:43:28.525 INFO:tasks.workunit.client.0.smithi027.stdout:6/599: rmdir d4/de 39 2024-06-06T08:43:28.526 INFO:tasks.workunit.client.0.smithi027.stdout:6/600: creat d4/d9/d6e/fbf x:0 0 0 2024-06-06T08:43:28.527 INFO:tasks.workunit.client.0.smithi027.stdout:6/601: stat d4/d9/dc/d1c/l5b 0 2024-06-06T08:43:28.533 INFO:tasks.workunit.client.0.smithi027.stdout:8/483: dwrite d7/dc/d36/d83/d58/f8a [0,4194304] 0 2024-06-06T08:43:28.536 INFO:tasks.workunit.client.0.smithi027.stdout:8/484: mknod d7/dc/d36/d41/ca6 0 2024-06-06T08:43:28.536 INFO:tasks.workunit.client.0.smithi027.stdout:8/485: fsync d7/dc/d36/d83/d60/f7a 0 2024-06-06T08:43:28.538 INFO:tasks.workunit.client.0.smithi027.stdout:8/486: mknod d7/dc/d36/d83/ca7 0 2024-06-06T08:43:28.538 INFO:tasks.workunit.client.0.smithi027.stdout:8/487: chown d7/dc/d36/d3c/c53 884483770 1 2024-06-06T08:43:28.540 INFO:tasks.workunit.client.0.smithi027.stdout:8/488: getdents d7/dc/d37/d46/d50 0 2024-06-06T08:43:28.541 INFO:tasks.workunit.client.0.smithi027.stdout:8/489: readlink d7/dc/d36/l71 0 2024-06-06T08:43:28.543 INFO:tasks.workunit.client.0.smithi027.stdout:8/490: mknod d7/d2f/ca8 0 2024-06-06T08:43:28.543 INFO:tasks.workunit.client.0.smithi027.stdout:6/602: dwrite d4/d9/dc/d1c/f50 [0,4194304] 0 2024-06-06T08:43:28.546 INFO:tasks.workunit.client.0.smithi027.stdout:8/491: rmdir d7/dc/d36/d83/d60/da5 0 2024-06-06T08:43:28.552 INFO:tasks.workunit.client.0.smithi027.stdout:6/603: dread d4/d1a/d2e/f5c [0,4194304] 0 2024-06-06T08:43:28.553 INFO:tasks.workunit.client.0.smithi027.stdout:8/492: write d7/dc/d36/d41/f34 [4630983,3470] 0 2024-06-06T08:43:28.553 INFO:tasks.workunit.client.0.smithi027.stdout:6/604: creat d4/d1a/d2e/d44/fc0 x:0 0 0 2024-06-06T08:43:28.554 INFO:tasks.workunit.client.0.smithi027.stdout:8/493: creat d7/d9c/fa9 x:0 0 0 2024-06-06T08:43:28.555 INFO:tasks.workunit.client.0.smithi027.stdout:8/494: creat d7/d13/faa x:0 0 0 2024-06-06T08:43:28.556 INFO:tasks.workunit.client.0.smithi027.stdout:6/605: read d4/d9/d49/f77 [741483,91772] 0 2024-06-06T08:43:28.557 INFO:tasks.workunit.client.0.smithi027.stdout:8/495: mknod d7/d13/d15/cab 0 2024-06-06T08:43:28.557 INFO:tasks.workunit.client.0.smithi027.stdout:6/606: creat d4/d1a/d2e/fc1 x:0 0 0 2024-06-06T08:43:28.558 INFO:tasks.workunit.client.0.smithi027.stdout:8/496: getdents d7/dc/d37/d46 0 2024-06-06T08:43:28.558 INFO:tasks.workunit.client.0.smithi027.stdout:8/497: write d7/d2f/f31 [1061610,107166] 0 2024-06-06T08:43:28.575 INFO:tasks.workunit.client.0.smithi027.stdout:6/607: dwrite d4/d9/dc/d1c/d52/fa9 [0,4194304] 0 2024-06-06T08:43:28.577 INFO:tasks.workunit.client.0.smithi027.stdout:6/608: creat d4/d9/dc/d1c/d52/fc2 x:0 0 0 2024-06-06T08:43:28.578 INFO:tasks.workunit.client.0.smithi027.stdout:6/609: truncate d4/d9/dc/f43 3902425 0 2024-06-06T08:43:28.580 INFO:tasks.workunit.client.0.smithi027.stdout:8/498: dwrite d7/f2c [0,4194304] 0 2024-06-06T08:43:28.583 INFO:tasks.workunit.client.0.smithi027.stdout:6/610: rename d4/d9/dc/d1c/f30 to d4/fc3 0 2024-06-06T08:43:28.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/499: unlink d7/la3 0 2024-06-06T08:43:28.589 INFO:tasks.workunit.client.0.smithi027.stdout:8/500: rename d7/dc/d37/d46/d50 to d7/dc/d36/d83/d60/d93/dac 0 2024-06-06T08:43:28.591 INFO:tasks.workunit.client.0.smithi027.stdout:8/501: rename d7/dc/d36/d41/d85/fa2 to d7/dc/d36/d83/d60/fad 0 2024-06-06T08:43:28.593 INFO:tasks.workunit.client.0.smithi027.stdout:8/502: rename d7/d2f/f70 to d7/d2f/fae 0 2024-06-06T08:43:28.594 INFO:tasks.workunit.client.0.smithi027.stdout:8/503: mkdir d7/d9c/daf 0 2024-06-06T08:43:28.602 INFO:tasks.workunit.client.0.smithi027.stdout:6/611: dwrite d4/de/f8c [0,4194304] 0 2024-06-06T08:43:28.603 INFO:tasks.workunit.client.0.smithi027.stdout:6/612: rename d4/d9/dc/c5e to d4/d9/d6e/cc4 0 2024-06-06T08:43:28.606 INFO:tasks.workunit.client.0.smithi027.stdout:6/613: mkdir d4/d9/dc5 0 2024-06-06T08:43:28.606 INFO:tasks.workunit.client.0.smithi027.stdout:6/614: write d4/d9/f58 [4509197,31380] 0 2024-06-06T08:43:28.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/615: creat d4/da2/fc6 x:0 0 0 2024-06-06T08:43:28.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/616: chown d4/d9/d49/f86 100 1 2024-06-06T08:43:28.627 INFO:tasks.workunit.client.0.smithi027.stdout:6/617: dwrite d4/d9/d49/f62 [0,4194304] 0 2024-06-06T08:43:28.636 INFO:tasks.workunit.client.0.smithi027.stdout:6/618: write d4/d9/dc/d18/f24 [3763915,116099] 0 2024-06-06T08:43:28.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/619: dread d4/d1a/d2e/d44/f74 [0,4194304] 0 2024-06-06T08:43:28.646 INFO:tasks.workunit.client.0.smithi027.stdout:6/620: symlink d4/d9/dc5/lc7 0 2024-06-06T08:43:28.656 INFO:tasks.workunit.client.0.smithi027.stdout:6/621: dread d4/d9/f26 [0,4194304] 0 2024-06-06T08:43:28.656 INFO:tasks.workunit.client.0.smithi027.stdout:6/622: write d4/de/d45/f75 [1573558,25600] 0 2024-06-06T08:43:28.656 INFO:tasks.workunit.client.0.smithi027.stdout:6/623: fdatasync d4/d9/dc/d18/f2f 0 2024-06-06T08:43:28.656 INFO:tasks.workunit.client.0.smithi027.stdout:6/624: write d4/da2/fa6 [405026,111426] 0 2024-06-06T08:43:28.664 INFO:tasks.workunit.client.0.smithi027.stdout:6/625: link d4/d1a/d2e/f5c d4/d9/dc/fc8 0 2024-06-06T08:43:28.664 INFO:tasks.workunit.client.0.smithi027.stdout:6/626: fsync d4/d9/f26 0 2024-06-06T08:43:28.664 INFO:tasks.workunit.client.0.smithi027.stdout:6/627: stat d4/d9/dc/d18/c14 0 2024-06-06T08:43:28.671 INFO:tasks.workunit.client.0.smithi027.stdout:6/628: link d4/d9/d49/lb9 d4/d9/dc/d1c/lc9 0 2024-06-06T08:43:28.672 INFO:tasks.workunit.client.0.smithi027.stdout:6/629: mkdir d4/d1a/d2e/d44/dca 0 2024-06-06T08:43:28.672 INFO:tasks.workunit.client.0.smithi027.stdout:6/630: chown d4/d1a/d2e/fc1 2129016067 1 2024-06-06T08:43:28.690 INFO:tasks.workunit.client.0.smithi027.stdout:6/631: dwrite d4/d9/fad [0,4194304] 0 2024-06-06T08:43:28.692 INFO:tasks.workunit.client.0.smithi027.stdout:7/682: sync 2024-06-06T08:43:28.692 INFO:tasks.workunit.client.0.smithi027.stdout:5/603: sync 2024-06-06T08:43:28.692 INFO:tasks.workunit.client.0.smithi027.stdout:0/499: sync 2024-06-06T08:43:28.692 INFO:tasks.workunit.client.0.smithi027.stdout:9/469: sync 2024-06-06T08:43:28.692 INFO:tasks.workunit.client.0.smithi027.stdout:5/604: chown d0/da/f43 6649805 1 2024-06-06T08:43:28.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/500: dread - d15/d18/d23/d28/f55 zero size 2024-06-06T08:43:28.693 INFO:tasks.workunit.client.0.smithi027.stdout:7/683: mknod d17/d32/d46/d83/d2a/d3b/ce1 0 2024-06-06T08:43:28.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/684: truncate fd 1251869 0 2024-06-06T08:43:28.698 INFO:tasks.workunit.client.0.smithi027.stdout:0/501: chown d15/d18/d23/d28/d31/l60 446 1 2024-06-06T08:43:28.698 INFO:tasks.workunit.client.0.smithi027.stdout:7/685: fsync d17/f26 0 2024-06-06T08:43:28.698 INFO:tasks.workunit.client.0.smithi027.stdout:0/502: mknod d15/d18/d23/ca9 0 2024-06-06T08:43:28.699 INFO:tasks.workunit.client.0.smithi027.stdout:0/503: getdents d15/d18/d23/d38/d68/d85 0 2024-06-06T08:43:28.701 INFO:tasks.workunit.client.0.smithi027.stdout:0/504: symlink d15/laa 0 2024-06-06T08:43:28.701 INFO:tasks.workunit.client.0.smithi027.stdout:0/505: truncate d15/d18/d23/d28/d2a/d30/d5c/d83/f86 1267016 0 2024-06-06T08:43:28.701 INFO:tasks.workunit.client.0.smithi027.stdout:0/506: readlink d15/l36 0 2024-06-06T08:43:28.704 INFO:tasks.workunit.client.0.smithi027.stdout:0/507: mknod d15/d18/d23/d38/cab 0 2024-06-06T08:43:28.704 INFO:tasks.workunit.client.0.smithi027.stdout:0/508: fsync d15/d18/d23/d38/d3e/f99 0 2024-06-06T08:43:28.715 INFO:tasks.workunit.client.0.smithi027.stdout:6/632: dwrite d4/d1a/d2e/d44/f99 [0,4194304] 0 2024-06-06T08:43:28.715 INFO:tasks.workunit.client.0.smithi027.stdout:9/470: dwrite d3/d4e/d75/dc/d9b/f91 [0,4194304] 0 2024-06-06T08:43:28.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/686: dwrite d17/d32/d46/d83/d72/f98 [0,4194304] 0 2024-06-06T08:43:28.722 INFO:tasks.workunit.client.0.smithi027.stdout:5/605: dwrite d0/da/d37/f41 [0,4194304] 0 2024-06-06T08:43:28.723 INFO:tasks.workunit.client.0.smithi027.stdout:9/471: creat d3/d95/d34/d73/fa7 x:0 0 0 2024-06-06T08:43:28.723 INFO:tasks.workunit.client.0.smithi027.stdout:9/472: write d3/d95/f5e [686623,66510] 0 2024-06-06T08:43:28.725 INFO:tasks.workunit.client.0.smithi027.stdout:5/606: creat d0/fbf x:0 0 0 2024-06-06T08:43:28.726 INFO:tasks.workunit.client.0.smithi027.stdout:9/473: creat d3/d4e/fa8 x:0 0 0 2024-06-06T08:43:28.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/509: dwrite d15/d18/d64/f8a [0,4194304] 0 2024-06-06T08:43:28.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/510: truncate d15/d18/d23/d6f/f94 223896 0 2024-06-06T08:43:28.728 INFO:tasks.workunit.client.0.smithi027.stdout:5/607: truncate d0/da/db/d46/d77/f8f 3232617 0 2024-06-06T08:43:28.728 INFO:tasks.workunit.client.0.smithi027.stdout:9/474: mkdir d3/d95/d23/da9 0 2024-06-06T08:43:28.731 INFO:tasks.workunit.client.0.smithi027.stdout:0/511: symlink d15/d18/d23/d38/lac 0 2024-06-06T08:43:28.732 INFO:tasks.workunit.client.0.smithi027.stdout:0/512: read d15/f22 [2495271,110942] 0 2024-06-06T08:43:28.732 INFO:tasks.workunit.client.0.smithi027.stdout:0/513: chown d15/l92 4230 1 2024-06-06T08:43:28.737 INFO:tasks.workunit.client.0.smithi027.stdout:9/475: rmdir d3/d95/d23 39 2024-06-06T08:43:28.742 INFO:tasks.workunit.client.0.smithi027.stdout:9/476: creat d3/d95/faa x:0 0 0 2024-06-06T08:43:28.743 INFO:tasks.workunit.client.0.smithi027.stdout:9/477: stat d3/d95/d23/d36/d3a 0 2024-06-06T08:43:28.743 INFO:tasks.workunit.client.0.smithi027.stdout:9/478: fsync d3/d95/d23/d36/f94 0 2024-06-06T08:43:28.743 INFO:tasks.workunit.client.0.smithi027.stdout:9/479: truncate d3/d95/d34/d41/f9c 435221 0 2024-06-06T08:43:28.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/687: dwrite d17/d32/d46/d51/d52/d76/faf [0,4194304] 0 2024-06-06T08:43:28.743 INFO:tasks.workunit.client.0.smithi027.stdout:9/480: rmdir d3/d95/d23/d78 39 2024-06-06T08:43:28.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/608: dwrite d0/da/f72 [0,4194304] 0 2024-06-06T08:43:28.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/609: fdatasync d0/da/db/d1a/da3/fa6 0 2024-06-06T08:43:28.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/610: read d0/da/f43 [2686920,84172] 0 2024-06-06T08:43:28.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/611: chown d0/da/db/d1a 37 1 2024-06-06T08:43:28.757 INFO:tasks.workunit.client.0.smithi027.stdout:0/514: dwrite d15/d18/d40/f76 [0,4194304] 0 2024-06-06T08:43:28.757 INFO:tasks.workunit.client.0.smithi027.stdout:0/515: fsync d15/d18/d23/d28/d2a/d30/d5d/f98 0 2024-06-06T08:43:28.757 INFO:tasks.workunit.client.0.smithi027.stdout:0/516: dread - d15/fa1 zero size 2024-06-06T08:43:28.761 INFO:tasks.workunit.client.0.smithi027.stdout:9/481: dwrite d3/d95/d34/f65 [0,4194304] 0 2024-06-06T08:43:28.762 INFO:tasks.workunit.client.0.smithi027.stdout:9/482: write d3/d4e/d75/dc/f2d [1713786,67377] 0 2024-06-06T08:43:28.767 INFO:tasks.workunit.client.0.smithi027.stdout:0/517: dread d15/d18/d23/d38/d68/f8f [0,4194304] 0 2024-06-06T08:43:28.784 INFO:tasks.workunit.client.0.smithi027.stdout:0/518: dwrite d15/f22 [0,4194304] 0 2024-06-06T08:43:28.814 INFO:tasks.workunit.client.0.smithi027.stdout:2/564: sync 2024-06-06T08:43:28.814 INFO:tasks.workunit.client.0.smithi027.stdout:3/436: sync 2024-06-06T08:43:28.814 INFO:tasks.workunit.client.0.smithi027.stdout:1/472: sync 2024-06-06T08:43:28.814 INFO:tasks.workunit.client.0.smithi027.stdout:4/627: sync 2024-06-06T08:43:28.815 INFO:tasks.workunit.client.0.smithi027.stdout:1/473: stat dc/d67/d69/d32/d71/f74 0 2024-06-06T08:43:28.817 INFO:tasks.workunit.client.0.smithi027.stdout:3/437: creat d1/d1f/d25/f80 x:0 0 0 2024-06-06T08:43:28.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/628: mknod d0/d1d/d60/dc6/d6e/db6/cc7 0 2024-06-06T08:43:28.819 INFO:tasks.workunit.client.0.smithi027.stdout:1/474: mknod dc/de/d14/c95 0 2024-06-06T08:43:28.819 INFO:tasks.workunit.client.0.smithi027.stdout:1/475: chown dc/de/d3d/c43 13962615 1 2024-06-06T08:43:28.819 INFO:tasks.workunit.client.0.smithi027.stdout:1/476: fdatasync dc/de/d22/f82 0 2024-06-06T08:43:28.820 INFO:tasks.workunit.client.0.smithi027.stdout:3/438: creat d1/da/d6d/f81 x:0 0 0 2024-06-06T08:43:28.822 INFO:tasks.workunit.client.0.smithi027.stdout:4/629: creat d0/d63/d7e/fc8 x:0 0 0 2024-06-06T08:43:28.822 INFO:tasks.workunit.client.0.smithi027.stdout:4/630: chown d0/d1d/d60/dc6/d6e/db6 632626381 1 2024-06-06T08:43:28.824 INFO:tasks.workunit.client.0.smithi027.stdout:3/439: creat d1/d8/d22/d27/d50/f82 x:0 0 0 2024-06-06T08:43:28.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/440: symlink d1/d1f/d5b/l83 0 2024-06-06T08:43:28.827 INFO:tasks.workunit.client.0.smithi027.stdout:3/441: creat d1/f84 x:0 0 0 2024-06-06T08:43:28.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/442: rmdir d1/d1f/d45/d5f 39 2024-06-06T08:43:28.830 INFO:tasks.workunit.client.0.smithi027.stdout:3/443: creat d1/f85 x:0 0 0 2024-06-06T08:43:28.830 INFO:tasks.workunit.client.0.smithi027.stdout:3/444: chown d1/f1e 4588 1 2024-06-06T08:43:28.830 INFO:tasks.workunit.client.0.smithi027.stdout:3/445: fsync d1/d1f/d25/f76 0 2024-06-06T08:43:28.831 INFO:tasks.workunit.client.0.smithi027.stdout:3/446: readlink d1/d1f/d5b/l70 0 2024-06-06T08:43:28.831 INFO:tasks.workunit.client.0.smithi027.stdout:3/447: truncate d1/da/f77 537588 0 2024-06-06T08:43:28.833 INFO:tasks.workunit.client.0.smithi027.stdout:3/448: creat d1/da/d6d/f86 x:0 0 0 2024-06-06T08:43:28.834 INFO:tasks.workunit.client.0.smithi027.stdout:4/631: dread d0/d1/d6/fae [4194304,4194304] 0 2024-06-06T08:43:28.838 INFO:tasks.workunit.client.0.smithi027.stdout:2/565: unlink d6/d1e/d2b/d5a/db7/fb8 0 2024-06-06T08:43:28.850 INFO:tasks.workunit.client.0.smithi027.stdout:3/449: dwrite d1/d1f/d25/f76 [0,4194304] 0 2024-06-06T08:43:28.850 INFO:tasks.workunit.client.0.smithi027.stdout:2/566: dread d6/d1e/f1f [0,4194304] 0 2024-06-06T08:43:28.850 INFO:tasks.workunit.client.0.smithi027.stdout:2/567: chown d6/dc/d3f/d77/l78 1247899 1 2024-06-06T08:43:28.854 INFO:tasks.workunit.client.0.smithi027.stdout:3/450: truncate d1/d8/d22/d27/f78 1807797 0 2024-06-06T08:43:28.854 INFO:tasks.workunit.client.0.smithi027.stdout:3/451: dread - d1/d8/f55 zero size 2024-06-06T08:43:28.855 INFO:tasks.workunit.client.0.smithi027.stdout:9/483: creat d3/d95/d23/da9/fab x:0 0 0 2024-06-06T08:43:28.861 INFO:tasks.workunit.client.0.smithi027.stdout:3/452: dread d1/d8/d22/d27/d50/f51 [0,4194304] 0 2024-06-06T08:43:28.862 INFO:tasks.workunit.client.0.smithi027.stdout:2/568: write d6/dc/f44 [1780831,94252] 0 2024-06-06T08:43:28.863 INFO:tasks.workunit.client.0.smithi027.stdout:2/569: truncate d6/dc/d3f/d77/d55/fb4 468347 0 2024-06-06T08:43:28.865 INFO:tasks.workunit.client.0.smithi027.stdout:2/570: dread f1 [0,4194304] 0 2024-06-06T08:43:28.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/453: symlink d1/d8/d22/l87 0 2024-06-06T08:43:28.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/454: readlink d1/d8/d22/d27/d50/l7a 0 2024-06-06T08:43:28.869 INFO:tasks.workunit.client.0.smithi027.stdout:5/612: getdents d0/da/db/d46/d7f/d81/d97 0 2024-06-06T08:43:28.870 INFO:tasks.workunit.client.0.smithi027.stdout:3/455: dread d1/d1f/d45/d5f/f7c [0,4194304] 0 2024-06-06T08:43:28.870 INFO:tasks.workunit.client.0.smithi027.stdout:3/456: readlink d1/d1f/d5b/l83 0 2024-06-06T08:43:28.874 INFO:tasks.workunit.client.0.smithi027.stdout:2/571: symlink d6/dc/d1d/dbc/lc2 0 2024-06-06T08:43:28.876 INFO:tasks.workunit.client.0.smithi027.stdout:5/613: creat d0/da/d20/fc0 x:0 0 0 2024-06-06T08:43:28.882 INFO:tasks.workunit.client.0.smithi027.stdout:3/457: unlink d1/d1f/d4d/f6a 0 2024-06-06T08:43:28.883 INFO:tasks.workunit.client.0.smithi027.stdout:5/614: creat d0/da/db/d46/fc1 x:0 0 0 2024-06-06T08:43:28.892 INFO:tasks.workunit.client.0.smithi027.stdout:2/572: dwrite d6/dc/f22 [0,4194304] 0 2024-06-06T08:43:28.912 INFO:tasks.workunit.client.0.smithi027.stdout:0/519: creat d15/d18/d40/d8c/d9c/fad x:0 0 0 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 -- 172.21.15.27:0/2391652670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 msgr2=0x7f6d7410f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2391652670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 0x7f6d7410f370 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00b0d0 tx=0x7f6d6c00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 -- 172.21.15.27:0/2391652670 shutdown_connections 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2391652670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 0x7f6d7410f370 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2391652670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d74102770 0x7f6d74102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2391652670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d74103120 0x7f6d74107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 -- 172.21.15.27:0/2391652670 >> 172.21.15.27:0/2391652670 conn(0x7f6d74079560 msgr2=0x7f6d74079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 -- 172.21.15.27:0/2391652670 shutdown_connections 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 -- 172.21.15.27:0/2391652670 wait complete. 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.685+0000 7f6d793e5700 1 Processor -- start 2024-06-06T08:43:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d793e5700 1 -- start start 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d793e5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d74102770 0x7f6d74074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d793e5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d74103120 0x7f6d74074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d793e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 0x7f6d74075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d793e5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d74112850 con 0x7f6d74107760 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d793e5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741126d0 con 0x7f6d74102770 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d793e5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741129d0 con 0x7f6d74103120 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 0x7f6d74075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 0x7f6d74075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33118/0 (socket says 172.21.15.27:33118) 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d737fe700 1 -- 172.21.15.27:0/2205778201 learned_addr learned my addr 172.21.15.27:0/2205778201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d72ffd700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d74102770 0x7f6d74074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d737fe700 1 -- 172.21.15.27:0/2205778201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d74103120 msgr2=0x7f6d74074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d74103120 0x7f6d74074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d737fe700 1 -- 172.21.15.27:0/2205778201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d74102770 msgr2=0x7f6d74074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d74102770 0x7f6d74074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d737fe700 1 -- 172.21.15.27:0/2205778201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d741a7100 con 0x7f6d74107760 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.686+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 0x7f6d74075210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00d010 tx=0x7f6d6c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.687+0000 7f6d5bfff700 1 -- 172.21.15.27:0/2205778201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c003c00 con 0x7f6d74107760 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.687+0000 7f6d5bfff700 1 -- 172.21.15.27:0/2205778201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c003da0 con 0x7f6d74107760 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.687+0000 7f6d793e5700 1 -- 172.21.15.27:0/2205778201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d741a7390 con 0x7f6d74107760 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.687+0000 7f6d5bfff700 1 -- 172.21.15.27:0/2205778201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c01b960 con 0x7f6d74107760 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.687+0000 7f6d793e5700 1 -- 172.21.15.27:0/2205778201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d741a7840 con 0x7f6d74107760 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.689+0000 7f6d5bfff700 1 -- 172.21.15.27:0/2205778201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6d6c0055e0 con 0x7f6d74107760 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.689+0000 7f6d5bfff700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d5c0a2eb0 0x7f6d5c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d72ffd700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d5c0a2eb0 0x7f6d5c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d5bfff700 1 -- 172.21.15.27:0/2205778201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6d6c00f020 con 0x7f6d74107760 2024-06-06T08:43:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d72ffd700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d5c0a2eb0 0x7f6d5c0a5370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009c60 tx=0x7f6d64009260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d83c68b80 1 Processor -- start 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d83c68b80 1 -- start start 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a67cd0 0x564aa3a680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a685f0 0x564aa3a61f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d83c68b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564aa39b39a0 con 0x564aa3a62460 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d83c68b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564aa395b8b0 con 0x564aa3a67cd0 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.690+0000 7f6d83c68b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564aa39af960 con 0x564aa3a685f0 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33122/0 (socket says 172.21.15.27:33122) 2024-06-06T08:43:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d737fe700 1 -- 172.21.15.27:0/4139436795 learned_addr learned my addr 172.21.15.27:0/4139436795 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d727fc700 1 --2- 172.21.15.27:0/4139436795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a685f0 0x564aa3a61f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d72ffd700 1 --2- 172.21.15.27:0/4139436795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a67cd0 0x564aa3a680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d737fe700 1 -- 172.21.15.27:0/4139436795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a685f0 msgr2=0x564aa3a61f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d737fe700 1 --2- 172.21.15.27:0/4139436795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a685f0 0x564aa3a61f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d737fe700 1 -- 172.21.15.27:0/4139436795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a67cd0 msgr2=0x564aa3a680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d737fe700 1 --2- 172.21.15.27:0/4139436795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a67cd0 0x564aa3a680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d737fe700 1 -- 172.21.15.27:0/4139436795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa39520e0 con 0x564aa3a62460 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d737fe700 1 --2- 172.21.15.27:0/4139436795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a628a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c099040 tx=0x7f6d6c00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6d793ef0b30952 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d597fa700 1 -- 172.21.15.27:0/4139436795 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c09cbe0 con 0x564aa3a62460 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d597fa700 1 -- 172.21.15.27:0/4139436795 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c0a0f90 con 0x564aa3a62460 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.691+0000 7f6d597fa700 1 -- 172.21.15.27:0/4139436795 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0ce170 con 0x564aa3a62460 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.692+0000 7f6d83c68b80 1 -- 172.21.15.27:0/4139436795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 msgr2=0x564aa3a628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.692+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/4139436795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a628a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c099040 tx=0x7f6d6c00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.692+0000 7f6d83c68b80 1 -- 172.21.15.27:0/4139436795 shutdown_connections 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.692+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/4139436795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a628a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.692+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/4139436795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a67cd0 0x564aa3a680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.692+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/4139436795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a685f0 0x564aa3a61f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.692+0000 7f6d83c68b80 1 -- 172.21.15.27:0/4139436795 >> 172.21.15.27:0/4139436795 conn(0x564aa398ce50 msgr2=0x564aa3a5d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.692+0000 7f6d83c68b80 1 -- 172.21.15.27:0/4139436795 shutdown_connections 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.692+0000 7f6d83c68b80 1 -- 172.21.15.27:0/4139436795 wait complete. 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d83c68b80 1 -- start start 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d83c68b80 1 Processor -- start 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 0x564aa3a6e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a6b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a6bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d83c68b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564aa3a6ed80 con 0x564aa3a685f0 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d83c68b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x564aa3953d80 con 0x564aa3a62460 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d83c68b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564aa3a6ef00 con 0x564aa3a67cd0 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d83c68b80 10 client.0 _ll_register_callbacks cb 0x564aa394dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a6bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a6bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33124/0 (socket says 172.21.15.27:33124) 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a6b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.693+0000 7f6d737fe700 1 -- 172.21.15.27:0/2415246601 learned_addr learned my addr 172.21.15.27:0/2415246601 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.694+0000 7f6d737fe700 1 -- 172.21.15.27:0/2415246601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 msgr2=0x564aa3a6b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.694+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a6b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.694+0000 7f6d737fe700 1 -- 172.21.15.27:0/2415246601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 msgr2=0x564aa3a6e320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.694+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 0x564aa3a6e320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.694+0000 7f6d737fe700 1 -- 172.21.15.27:0/2415246601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa3902a60 con 0x564aa3a685f0 2024-06-06T08:43:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.694+0000 7f6d727fc700 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a6b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.694+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a6bc20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009fd0 tx=0x7f6d6c005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3a5667b9169cd5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.694+0000 7f6d56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.695+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2415246601 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0ba060 con 0x564aa3a685f0 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.695+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2415246601 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6d6c099250 con 0x564aa3a685f0 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.695+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2415246601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c0a0f90 con 0x564aa3a685f0 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.695+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2415246601 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c09ce40 con 0x564aa3a685f0 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.695+0000 7f6d83c68b80 10 client.16209 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.695+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2415246601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564aa3a6ed80 con 0x564aa3a685f0 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.695+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2415246601 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c0a0f90 con 0x564aa3a685f0 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.696+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2415246601 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6d6c011070 con 0x564aa3a685f0 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.696+0000 7f6d83c68b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.696+0000 7f6d83c68b80 10 client.16209 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.696+0000 7f6d83c68b80 20 client.16209 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.696+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564aa3a841d0 0x564aa3a865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.696+0000 7f6d83c68b80 4 client.16209 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.696+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2415246601 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aa39c0ea0 con 0x564aa3a841d0 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.697+0000 7f6d72ffd700 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564aa3a841d0 0x564aa3a865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.697+0000 7f6d56ff5700 10 client.16209 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.697+0000 7f6d72ffd700 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564aa3a841d0 0x564aa3a865c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa63a7d0dfb24863 server_cookie=417d29467a4e4ba7 in_seq=0 out_seq=0 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.698+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2415246601 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564aa39c0ea0 con 0x564aa3a841d0 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.698+0000 7f6d56ff5700 10 client.16209 handle_command_reply: tid=0 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.698+0000 7f6d83c68b80 1 client.16209 shutdown 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.699+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2415246601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 msgr2=0x564aa3a6bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.699+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a6bc20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009fd0 tx=0x7f6d6c005980 comp rx=0 tx=0).stop 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.699+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2415246601 shutdown_connections 2024-06-06T08:43:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.699+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564aa3a841d0 0x564aa3a865c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.699+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a6bc20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.699+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 0x564aa3a6e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.699+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2415246601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a6b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.699+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2415246601 >> 172.21.15.27:0/2415246601 conn(0x564aa398ce50 msgr2=0x564aa3a5cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.699+0000 7f6d56ff5700 0 client.16209 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.700+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2415246601 shutdown_connections 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.700+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2415246601 wait complete. 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.701+0000 7f6d83c68b80 20 client.16209 trim_cache size 0 max 16384 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.734+0000 7f6d83c68b80 1 Processor -- start 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.734+0000 7f6d83c68b80 1 -- start start 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a61f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a6bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a685f0 0x564aa3a69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d83c68b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741aecb0 con 0x564aa3a62460 2024-06-06T08:43:28.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d83c68b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741ac480 con 0x564aa3a685f0 2024-06-06T08:43:28.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d83c68b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741aa2f0 con 0x564aa3a67cd0 2024-06-06T08:43:28.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a61f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a61f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33138/0 (socket says 172.21.15.27:33138) 2024-06-06T08:43:28.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d72ffd700 1 -- 172.21.15.27:0/3999146916 learned_addr learned my addr 172.21.15.27:0/3999146916 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:28.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d737fe700 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a685f0 0x564aa3a69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.735+0000 7f6d727fc700 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a6bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.941 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:28 smithi027 ceph-mon[32335]: pgmap v99: 129 pgs: 129 active+clean; 700 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 88 MiB/s wr, 752 op/s 2024-06-06T08:43:28.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d737fe700 1 -- 172.21.15.27:0/3999146916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 msgr2=0x564aa3a6bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:28.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d737fe700 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a6bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d737fe700 1 -- 172.21.15.27:0/3999146916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 msgr2=0x564aa3a61f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:28.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d737fe700 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a61f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d737fe700 1 -- 172.21.15.27:0/3999146916 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa3a6aa10 con 0x564aa3a685f0 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d72ffd700 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a61f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d737fe700 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a685f0 0x564aa3a69260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c099040 tx=0x7f6d6c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddad21eaac6a82ce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d58ff9700 1 -- 172.21.15.27:0/3999146916 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0ba070 con 0x564aa3a685f0 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d58ff9700 1 -- 172.21.15.27:0/3999146916 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c0a0f90 con 0x564aa3a685f0 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.736+0000 7f6d58ff9700 1 -- 172.21.15.27:0/3999146916 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c09c1a0 con 0x564aa3a685f0 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.737+0000 7f6d83c68b80 1 -- 172.21.15.27:0/3999146916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a685f0 msgr2=0x564aa3a69260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.737+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a685f0 0x564aa3a69260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c099040 tx=0x7f6d6c0099e0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.738+0000 7f6d83c68b80 1 -- 172.21.15.27:0/3999146916 shutdown_connections 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.738+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a685f0 0x564aa3a69260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.738+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a62460 0x564aa3a61f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.738+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/3999146916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a6bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.738+0000 7f6d83c68b80 1 -- 172.21.15.27:0/3999146916 >> 172.21.15.27:0/3999146916 conn(0x564aa398ce50 msgr2=0x564aa3a5cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:28.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.738+0000 7f6d83c68b80 1 -- 172.21.15.27:0/3999146916 shutdown_connections 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.738+0000 7f6d83c68b80 1 -- 172.21.15.27:0/3999146916 wait complete. 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.738+0000 7f6d83c68b80 1 -- start start 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.738+0000 7f6d83c68b80 1 Processor -- start 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.739+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 0x564aa3a8b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.739+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a8bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.739+0000 7f6d83c68b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a8c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.739+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 0x564aa3a8b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.739+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 0x564aa3a8b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:44726/0 (socket says 172.21.15.27:44726) 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.739+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a8bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.739+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a8bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:41124/0 (socket says 172.21.15.27:41124) 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.739+0000 7f6d72ffd700 1 -- 172.21.15.27:0/2353324096 learned_addr learned my addr 172.21.15.27:0/2353324096 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.740+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a8c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.740+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741aa940 con 0x564aa3a685f0 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.740+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741abdb0 con 0x564aa3a62460 2024-06-06T08:43:28.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.740+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741ab010 con 0x564aa3a67cd0 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.740+0000 7f6d83c68b80 10 client.0 _ll_register_callbacks cb 0x564aa394dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.741+0000 7f6d737fe700 1 -- 172.21.15.27:0/2353324096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 msgr2=0x564aa3a8bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.741+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a8bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.741+0000 7f6d737fe700 1 -- 172.21.15.27:0/2353324096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 msgr2=0x564aa3a8b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.741+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 0x564aa3a8b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.741+0000 7f6d737fe700 1 -- 172.21.15.27:0/2353324096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa3a80040 con 0x564aa3a685f0 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.741+0000 7f6d737fe700 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a8c2e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c09c6b0 tx=0x7f6d6c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a52a1eee4c078bd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.741+0000 7f6d56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.741+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2353324096 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0ba070 con 0x564aa3a685f0 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.742+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2353324096 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6d6c0a5330 con 0x564aa3a685f0 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.742+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c0a0f90 con 0x564aa3a685f0 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.742+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2353324096 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c0aa050 con 0x564aa3a685f0 2024-06-06T08:43:28.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.742+0000 7f6d83c68b80 10 client.16215 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.742+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564aa3a80040 con 0x564aa3a685f0 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.743+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2353324096 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c0a0f90 con 0x564aa3a685f0 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.743+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2353324096 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6d6c084020 con 0x564aa3a685f0 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.743+0000 7f6d83c68b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.743+0000 7f6d83c68b80 10 client.16215 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.743+0000 7f6d83c68b80 20 client.16215 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.743+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564aa3ae6c40 0x564aa3ae9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.743+0000 7f6d83c68b80 4 client.16215 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.743+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x564aa39bb420 con 0x564aa3ae6c40 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.744+0000 7f6d72ffd700 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564aa3ae6c40 0x564aa3ae9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.745+0000 7f6d56ff5700 10 client.16215 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:28.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:28.745+0000 7f6d72ffd700 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564aa3ae6c40 0x564aa3ae9030 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16a1fdef6fc54e99 server_cookie=7f240cc7b2f7d479 in_seq=0 out_seq=0 2024-06-06T08:43:28.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:28 smithi203 ceph-mon[36579]: pgmap v99: 129 pgs: 129 active+clean; 700 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 88 MiB/s wr, 752 op/s 2024-06-06T08:43:29.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:28 smithi176 ceph-mon[36786]: pgmap v99: 129 pgs: 129 active+clean; 700 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 88 MiB/s wr, 752 op/s 2024-06-06T08:43:29.169 INFO:tasks.workunit.client.0.smithi027.stdout:7/688: link d17/d32/d46/d51/d52/d70/d87/cc6 d17/d32/d46/d4f/db6/dc8/de0/ce2 0 2024-06-06T08:43:30.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:30 smithi027 ceph-mon[32335]: pgmap v100: 129 pgs: 129 active+clean; 1.2 GiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s rd, 176 MiB/s wr, 888 op/s 2024-06-06T08:43:30.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:30 smithi203 ceph-mon[36579]: pgmap v100: 129 pgs: 129 active+clean; 1.2 GiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s rd, 176 MiB/s wr, 888 op/s 2024-06-06T08:43:31.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:30 smithi176 ceph-mon[36786]: pgmap v100: 129 pgs: 129 active+clean; 1.2 GiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 146 MiB/s rd, 176 MiB/s wr, 888 op/s 2024-06-06T08:43:32.044 INFO:tasks.workunit.client.0.smithi027.stdout:5/615: readlink d0/da/d37/d2b/l9a 0 2024-06-06T08:43:32.052 INFO:tasks.workunit.client.0.smithi027.stdout:7/689: chown d17/d32/d46/d83 1246897 1 2024-06-06T08:43:32.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:32 smithi027 ceph-mon[32335]: pgmap v101: 129 pgs: 129 active+clean; 1.2 GiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 121 MiB/s rd, 140 MiB/s wr, 832 op/s 2024-06-06T08:43:32.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:32 smithi203 ceph-mon[36579]: pgmap v101: 129 pgs: 129 active+clean; 1.2 GiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 121 MiB/s rd, 140 MiB/s wr, 832 op/s 2024-06-06T08:43:33.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:32 smithi176 ceph-mon[36786]: pgmap v101: 129 pgs: 129 active+clean; 1.2 GiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 121 MiB/s rd, 140 MiB/s wr, 832 op/s 2024-06-06T08:43:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.522+0000 7f714d4a5640 1 -- 172.21.15.27:0/2004397438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f714810e360 msgr2=0x7f7148110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.522+0000 7f714d4a5640 1 --2- 172.21.15.27:0/2004397438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f714810e360 0x7f7148110750 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7140009900 tx=0x7f714002a410 comp rx=0 tx=0).stop 2024-06-06T08:43:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.522+0000 7f714d4a5640 1 -- 172.21.15.27:0/2004397438 shutdown_connections 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.522+0000 7f714d4a5640 1 --2- 172.21.15.27:0/2004397438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f714810e360 0x7f7148110750 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.522+0000 7f714d4a5640 1 --2- 172.21.15.27:0/2004397438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71481010d0 0x7f714810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.522+0000 7f714d4a5640 1 --2- 172.21.15.27:0/2004397438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71481007b0 0x7f7148100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.522+0000 7f714d4a5640 1 -- 172.21.15.27:0/2004397438 >> 172.21.15.27:0/2004397438 conn(0x7f7148077a20 msgr2=0x7f7148077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.522+0000 7f714d4a5640 1 -- 172.21.15.27:0/2004397438 shutdown_connections 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.522+0000 7f714d4a5640 1 -- 172.21.15.27:0/2004397438 wait complete. 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.523+0000 7f714d4a5640 1 Processor -- start 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.523+0000 7f714d4a5640 1 -- start start 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f714d4a5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71481007b0 0x7f714819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f714d4a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71481010d0 0x7f714819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f714d4a5640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f714810e360 0x7f714819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f714d4a5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7148113f90 con 0x7f71481010d0 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f714d4a5640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f7148113e10 con 0x7f714810e360 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f714d4a5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7148114110 con 0x7f71481007b0 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f71467fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71481010d0 0x7f714819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f7146ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71481007b0 0x7f714819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f71467fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71481010d0 0x7f714819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60488/0 (socket says 172.21.15.27:60488) 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f71467fc640 1 -- 172.21.15.27:0/1364147321 learned_addr learned my addr 172.21.15.27:0/1364147321 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f71477fe640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f714810e360 0x7f714819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f7146ffd640 1 -- 172.21.15.27:0/1364147321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f714810e360 msgr2=0x7f714819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f7146ffd640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f714810e360 0x7f714819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f7146ffd640 1 -- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71481010d0 msgr2=0x7f714819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f7146ffd640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71481010d0 0x7f714819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f7146ffd640 1 -- 172.21.15.27:0/1364147321 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714819f460 con 0x7f71481007b0 2024-06-06T08:43:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f71467fc640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71481010d0 0x7f714819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.524+0000 7f7146ffd640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71481007b0 0x7f714819a470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f713800eb00 tx=0x7f713800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.525+0000 7f713ffff640 1 -- 172.21.15.27:0/1364147321 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138010430 con 0x7f71481007b0 2024-06-06T08:43:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.525+0000 7f713ffff640 1 -- 172.21.15.27:0/1364147321 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7138010ab0 con 0x7f71481007b0 2024-06-06T08:43:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.525+0000 7f714d4a5640 1 -- 172.21.15.27:0/1364147321 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714819f750 con 0x7f71481007b0 2024-06-06T08:43:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.525+0000 7f713ffff640 1 -- 172.21.15.27:0/1364147321 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138017bb0 con 0x7f71481007b0 2024-06-06T08:43:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.525+0000 7f714d4a5640 1 -- 172.21.15.27:0/1364147321 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71481a6fe0 con 0x7f71481007b0 2024-06-06T08:43:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.527+0000 7f713ffff640 1 -- 172.21.15.27:0/1364147321 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7138010c60 con 0x7f71481007b0 2024-06-06T08:43:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.527+0000 7f71501b2740 1 Processor -- start 2024-06-06T08:43:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.527+0000 7f71501b2740 1 -- start start 2024-06-06T08:43:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71501b2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1ec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71501b2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71501b2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71501b2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b46a1504f0 con 0x55b46a1f7350 2024-06-06T08:43:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71501b2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b46a0ee870 con 0x55b46a1ec170 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71501b2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b46a14c4b0 con 0x55b46a1eca90 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f7146ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71467fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71477fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f7146ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:57672/0 (socket says 172.21.15.27:57672) 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71477fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1ec550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:53486/0 (socket says 172.21.15.27:53486) 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f7146ffd640 1 -- 172.21.15.27:0/3867382220 learned_addr learned my addr 172.21.15.27:0/3867382220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f71467fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60498/0 (socket says 172.21.15.27:60498) 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f713ffff640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f71340a2e20 0x7f71340a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f7146ffd640 1 -- 172.21.15.27:0/3867382220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 msgr2=0x55b46a1ec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.528+0000 7f7146ffd640 1 --2- 172.21.15.27:0/3867382220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1ec550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f7146ffd640 1 -- 172.21.15.27:0/3867382220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 msgr2=0x55b46a1f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f7146ffd640 1 --2- 172.21.15.27:0/3867382220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f7146ffd640 1 -- 172.21.15.27:0/3867382220 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46a10a9b0 con 0x55b46a1eca90 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f71477fe640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f71340a2e20 0x7f71340a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f713ffff640 1 -- 172.21.15.27:0/1364147321 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f71380d0150 con 0x7f71481007b0 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f7146ffd640 1 --2- 172.21.15.27:0/3867382220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f6e10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f71380cd3c0 tx=0x7f7138066e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44f90a5d6088eb9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f713d7fa640 1 -- 172.21.15.27:0/3867382220 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713809c020 con 0x55b46a1eca90 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f713d7fa640 1 -- 172.21.15.27:0/3867382220 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7138070ab0 con 0x55b46a1eca90 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f71477fe640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f71340a2e20 0x7f71340a52e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7140006010 tx=0x7f7140009260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.529+0000 7f713d7fa640 1 -- 172.21.15.27:0/3867382220 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71380d3cf0 con 0x55b46a1eca90 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 -- 172.21.15.27:0/3867382220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 msgr2=0x55b46a1f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 --2- 172.21.15.27:0/3867382220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f6e10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f71380cd3c0 tx=0x7f7138066e80 comp rx=0 tx=0).stop 2024-06-06T08:43:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 -- 172.21.15.27:0/3867382220 shutdown_connections 2024-06-06T08:43:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 --2- 172.21.15.27:0/3867382220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 --2- 172.21.15.27:0/3867382220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f6e10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 --2- 172.21.15.27:0/3867382220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 -- 172.21.15.27:0/3867382220 >> 172.21.15.27:0/3867382220 conn(0x55b46a1e8680 msgr2=0x55b46a1e8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 -- 172.21.15.27:0/3867382220 shutdown_connections 2024-06-06T08:43:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 -- 172.21.15.27:0/3867382220 wait complete. 2024-06-06T08:43:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.530+0000 7f71501b2740 1 -- start start 2024-06-06T08:43:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71501b2740 1 Processor -- start 2024-06-06T08:43:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71501b2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71501b2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71501b2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71501b2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b46a1f9f20 con 0x55b46a1f7350 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71501b2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55b46a0d9ae0 con 0x55b46a1ec170 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71501b2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b46a1fa0a0 con 0x55b46a1eca90 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71501b2740 10 client.0 _ll_register_callbacks cb 0x55b46a1129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71467fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f7146ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71467fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:57688/0 (socket says 172.21.15.27:57688) 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f7146ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:53498/0 (socket says 172.21.15.27:53498) 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71467fc640 1 -- 172.21.15.27:0/2946393931 learned_addr learned my addr 172.21.15.27:0/2946393931 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:33.536 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71477fe640 1 -- 172.21.15.27:0/2946393931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 msgr2=0x55b46a1f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71477fe640 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71477fe640 1 -- 172.21.15.27:0/2946393931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 msgr2=0x55b46a1f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.531+0000 7f71477fe640 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f71477fe640 1 -- 172.21.15.27:0/2946393931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b469e64210 con 0x55b46a1f7350 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f71477fe640 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f4cf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f714002a930 tx=0x7f714002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff0e7b00232acf0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f712e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f712e7fc640 1 -- 172.21.15.27:0/2946393931 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714004d020 con 0x55b46a1f7350 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f712e7fc640 1 -- 172.21.15.27:0/2946393931 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f7140004460 con 0x55b46a1f7350 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f71501b2740 1 -- 172.21.15.27:0/2946393931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b46a163f10 con 0x55b46a1f7350 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f712e7fc640 1 -- 172.21.15.27:0/2946393931 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7140058050 con 0x55b46a1f7350 2024-06-06T08:43:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f71501b2740 10 client.16227 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f71501b2740 1 -- 172.21.15.27:0/2946393931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b469f30620 con 0x55b46a1f7350 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.532+0000 7f712e7fc640 1 -- 172.21.15.27:0/2946393931 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b46a163f10 con 0x55b46a1f7350 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f712e7fc640 1 -- 172.21.15.27:0/2946393931 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f714003e8f0 con 0x55b46a1f7350 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f71501b2740 10 client.16227 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f71501b2740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f71501b2740 20 client.16227 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f71501b2740 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b46a207560 0x55b46a209970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f71501b2740 4 client.16227 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f71501b2740 1 -- 172.21.15.27:0/2946393931 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b46a15dc20 con 0x55b46a207560 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f7146ffd640 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b46a207560 0x55b46a209970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f7146ffd640 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b46a207560 0x55b46a209970 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fade8dea66f7d8c0 server_cookie=5ce928062ae24b48 in_seq=0 out_seq=0 2024-06-06T08:43:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.533+0000 7f712e7fc640 10 client.16227 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f712e7fc640 1 -- 172.21.15.27:0/2946393931 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b46a15dc20 con 0x55b46a207560 2024-06-06T08:43:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f712e7fc640 10 client.16227 handle_command_reply: tid=0 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f71501b2740 1 client.16227 shutdown 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f71501b2740 1 -- 172.21.15.27:0/2946393931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 msgr2=0x55b46a1f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f71501b2740 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f4cf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f714002a930 tx=0x7f714002a410 comp rx=0 tx=0).stop 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f71501b2740 1 -- 172.21.15.27:0/2946393931 shutdown_connections 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f71501b2740 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b46a207560 0x55b46a209970 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f71501b2740 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f4cf0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f71501b2740 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a1f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f712e7fc640 0 client.16227 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f71501b2740 1 --2- 172.21.15.27:0/2946393931 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.538+0000 7f71501b2740 1 -- 172.21.15.27:0/2946393931 >> 172.21.15.27:0/2946393931 conn(0x55b46a1e8680 msgr2=0x55b46a1ee400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.539+0000 7f71501b2740 1 -- 172.21.15.27:0/2946393931 shutdown_connections 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.539+0000 7f71501b2740 1 -- 172.21.15.27:0/2946393931 wait complete. 2024-06-06T08:43:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.539+0000 7f71501b2740 20 client.16227 trim_cache size 0 max 16384 2024-06-06T08:43:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.554+0000 7f71501b2740 1 Processor -- start 2024-06-06T08:43:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.554+0000 7f71501b2740 1 -- start start 2024-06-06T08:43:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71501b2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1eeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71501b2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a20f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71501b2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a20f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71501b2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b3970 con 0x55b46a1f7350 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71501b2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b1180 con 0x55b46a1ec170 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71501b2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71481aefd0 con 0x55b46a1eca90 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a20f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f7146ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1eeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71467fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a20f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f7146ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1eeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:53504/0 (socket says 172.21.15.27:53504) 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a20f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60522/0 (socket says 172.21.15.27:60522) 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f7146ffd640 1 -- 172.21.15.27:0/504845392 learned_addr learned my addr 172.21.15.27:0/504845392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71477fe640 1 -- 172.21.15.27:0/504845392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 msgr2=0x55b46a20f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71477fe640 1 --2- 172.21.15.27:0/504845392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a20f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71477fe640 1 -- 172.21.15.27:0/504845392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 msgr2=0x55b46a1eeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71477fe640 1 --2- 172.21.15.27:0/504845392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1eeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71477fe640 1 -- 172.21.15.27:0/504845392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46a1ed590 con 0x55b46a1f7350 2024-06-06T08:43:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f7146ffd640 1 --2- 172.21.15.27:0/504845392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1eeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.555+0000 7f71467fc640 1 --2- 172.21.15.27:0/504845392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a20f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.556+0000 7f71477fe640 1 --2- 172.21.15.27:0/504845392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a20f7c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f71400096c0 tx=0x7f7140031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa624d458e21f29 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.556+0000 7f713cff9640 1 -- 172.21.15.27:0/504845392 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140060020 con 0x55b46a1f7350 2024-06-06T08:43:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.556+0000 7f713cff9640 1 -- 172.21.15.27:0/504845392 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7140041070 con 0x55b46a1f7350 2024-06-06T08:43:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.556+0000 7f713cff9640 1 -- 172.21.15.27:0/504845392 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714004d070 con 0x55b46a1f7350 2024-06-06T08:43:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 -- 172.21.15.27:0/504845392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 msgr2=0x55b46a20f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 --2- 172.21.15.27:0/504845392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a20f7c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f71400096c0 tx=0x7f7140031460 comp rx=0 tx=0).stop 2024-06-06T08:43:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 -- 172.21.15.27:0/504845392 shutdown_connections 2024-06-06T08:43:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 --2- 172.21.15.27:0/504845392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a20f7c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 --2- 172.21.15.27:0/504845392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1eca90 0x55b46a20f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 --2- 172.21.15.27:0/504845392 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1ec170 0x55b46a1eeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 -- 172.21.15.27:0/504845392 >> 172.21.15.27:0/504845392 conn(0x55b46a1e8680 msgr2=0x55b46a1e8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 -- 172.21.15.27:0/504845392 shutdown_connections 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 -- 172.21.15.27:0/504845392 wait complete. 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 -- start start 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 Processor -- start 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1ec170 0x55b46a1ff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.557+0000 7f71501b2740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1eca90 0x55b46a1ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71501b2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71501b2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f71481af640 con 0x55b46a1f7350 2024-06-06T08:43:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71501b2740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f71481b0ab0 con 0x55b46a1eca90 2024-06-06T08:43:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71501b2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f71481afd10 con 0x55b46a1ec170 2024-06-06T08:43:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71501b2740 10 client.0 _ll_register_callbacks cb 0x7f7140009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60530/0 (socket says 172.21.15.27:60530) 2024-06-06T08:43:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f7146ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1ec170 0x55b46a1ff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71477fe640 1 -- 172.21.15.27:0/1432902427 learned_addr learned my addr 172.21.15.27:0/1432902427 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71467fc640 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1eca90 0x55b46a1ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71477fe640 1 -- 172.21.15.27:0/1432902427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1ec170 msgr2=0x55b46a1ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71477fe640 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1ec170 0x55b46a1ff6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71477fe640 1 -- 172.21.15.27:0/1432902427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1eca90 msgr2=0x55b46a1ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71477fe640 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1eca90 0x55b46a1ffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71477fe640 1 -- 172.21.15.27:0/1432902427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46a1f15f0 con 0x55b46a1f7350 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71477fe640 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f0dc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7140009850 tx=0x7f71400035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3f97f98127ed61 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f712e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f712e7fc640 1 -- 172.21.15.27:0/1432902427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140069060 con 0x55b46a1f7350 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f71501b2740 1 -- 172.21.15.27:0/1432902427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7140041070 con 0x55b46a1f7350 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.558+0000 7f712e7fc640 1 -- 172.21.15.27:0/1432902427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f7140003be0 con 0x55b46a1f7350 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f712e7fc640 1 -- 172.21.15.27:0/1432902427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7140070050 con 0x55b46a1f7350 2024-06-06T08:43:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f71501b2740 10 client.16233 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f71501b2740 1 -- 172.21.15.27:0/1432902427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b46a271dd0 con 0x55b46a1f7350 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f712e7fc640 1 -- 172.21.15.27:0/1432902427 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7140041070 con 0x55b46a1f7350 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f712e7fc640 1 -- 172.21.15.27:0/1432902427 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f714003c5d0 con 0x55b46a1f7350 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f71501b2740 10 client.16233 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f71501b2740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f71501b2740 20 client.16233 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f71501b2740 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b46a2723e0 0x55b46a274800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f71501b2740 4 client.16233 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f71501b2740 1 -- 172.21.15.27:0/1432902427 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b46a157f70 con 0x55b46a2723e0 2024-06-06T08:43:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.559+0000 7f7146ffd640 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b46a2723e0 0x55b46a274800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.560+0000 7f712e7fc640 10 client.16233 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:33.560+0000 7f7146ffd640 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b46a2723e0 0x55b46a274800 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b404226426e36d4 server_cookie=7cd05605b03fb802 in_seq=0 out_seq=0 2024-06-06T08:43:33.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:33 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:43:33.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:33 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:43:34.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:33 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:43:34.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:34 smithi027 ceph-mon[32335]: pgmap v102: 129 pgs: 129 active+clean; 1.2 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 121 MiB/s rd, 142 MiB/s wr, 834 op/s 2024-06-06T08:43:34.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:34 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:43:34.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:34 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:43:34.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:34 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:43:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:34 smithi203 ceph-mon[36579]: pgmap v102: 129 pgs: 129 active+clean; 1.2 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 121 MiB/s rd, 142 MiB/s wr, 834 op/s 2024-06-06T08:43:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:34 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:43:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:34 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:43:34.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:34 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:43:35.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:34 smithi176 ceph-mon[36786]: pgmap v102: 129 pgs: 129 active+clean; 1.2 GiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 121 MiB/s rd, 142 MiB/s wr, 834 op/s 2024-06-06T08:43:35.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:34 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:43:35.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:34 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:43:35.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:34 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:43:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f712e7fc640 1 -- 172.21.15.27:0/1432902427 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b46a157f70 con 0x55b46a2723e0 2024-06-06T08:43:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f712e7fc640 10 client.16233 handle_command_reply: tid=0 2024-06-06T08:43:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 client.16233 shutdown 2024-06-06T08:43:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 -- 172.21.15.27:0/1432902427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 msgr2=0x55b46a1f0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f0dc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7140009850 tx=0x7f71400035f0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 -- 172.21.15.27:0/1432902427 shutdown_connections 2024-06-06T08:43:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55b46a2723e0 0x55b46a274800 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b46a1f7350 0x55b46a1f0dc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b46a1eca90 0x55b46a1ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 --2- 172.21.15.27:0/1432902427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b46a1ec170 0x55b46a1ff6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 -- 172.21.15.27:0/1432902427 >> 172.21.15.27:0/1432902427 conn(0x55b46a1e8680 msgr2=0x55b46a1e6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f712e7fc640 0 client.16233 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 -- 172.21.15.27:0/1432902427 shutdown_connections 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 1 -- 172.21.15.27:0/1432902427 wait complete. 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.103+0000 7f71501b2740 20 client.16233 trim_cache size 0 max 16384 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:43:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:43:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:43:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 -- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f71340a2e20 msgr2=0x7f71340a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f71340a2e20 0x7f71340a52e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7140006010 tx=0x7f7140009260 comp rx=0 tx=0).stop 2024-06-06T08:43:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 -- 172.21.15.27:0/1364147321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71481007b0 msgr2=0x7f714819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71481007b0 0x7f714819a470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f713800eb00 tx=0x7f713800c2d0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 -- 172.21.15.27:0/1364147321 shutdown_connections 2024-06-06T08:43:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f71340a2e20 0x7f71340a52e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f714810e360 0x7f714819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71481010d0 0x7f714819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 --2- 172.21.15.27:0/1364147321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71481007b0 0x7f714819a470 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 -- 172.21.15.27:0/1364147321 >> 172.21.15.27:0/1364147321 conn(0x7f7148077a20 msgr2=0x7f714810f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 -- 172.21.15.27:0/1364147321 shutdown_connections 2024-06-06T08:43:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:35.104+0000 7f714d4a5640 1 -- 172.21.15.27:0/1364147321 wait complete. 2024-06-06T08:43:36.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:36 smithi027 ceph-mon[32335]: pgmap v103: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 MiB/s rd, 140 MiB/s wr, 959 op/s 2024-06-06T08:43:36.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:36 smithi203 ceph-mon[36579]: pgmap v103: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 MiB/s rd, 140 MiB/s wr, 959 op/s 2024-06-06T08:43:37.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:36 smithi176 ceph-mon[36786]: pgmap v103: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 MiB/s rd, 140 MiB/s wr, 959 op/s 2024-06-06T08:43:38.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:38 smithi027 ceph-mon[32335]: pgmap v104: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 94 MiB/s wr, 382 op/s 2024-06-06T08:43:38.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:38 smithi203 ceph-mon[36579]: pgmap v104: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 94 MiB/s wr, 382 op/s 2024-06-06T08:43:39.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:38 smithi176 ceph-mon[36786]: pgmap v104: 129 pgs: 129 active+clean; 1.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 94 MiB/s wr, 382 op/s 2024-06-06T08:43:39.993 INFO:tasks.workunit.client.0.smithi027.stdout:5/616: dread d0/da/db/d1a/f66 [0,4194304] 0 2024-06-06T08:43:40.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.033+0000 7f6d56ff5700 1 -- 172.21.15.27:0/2353324096 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564aa39bb420 con 0x564aa3ae6c40 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d56ff5700 10 client.16215 handle_command_reply: tid=0 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 client.16215 shutdown 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 msgr2=0x564aa3a8c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a8c2e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c09c6b0 tx=0x7f6d6c007ec0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 shutdown_connections 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x564aa3ae6c40 0x564aa3ae9030 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564aa3a685f0 0x564aa3a8c2e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564aa3a62460 0x564aa3a8b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 --2- 172.21.15.27:0/2353324096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564aa3a67cd0 0x564aa3a8bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d56ff5700 0 client.16215 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 >> 172.21.15.27:0/2353324096 conn(0x564aa398ce50 msgr2=0x564aa3a69260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:40.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 shutdown_connections 2024-06-06T08:43:40.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 1 -- 172.21.15.27:0/2353324096 wait complete. 2024-06-06T08:43:40.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.034+0000 7f6d83c68b80 20 client.16215 trim_cache size 0 max 16384 2024-06-06T08:43:40.035 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:43:40.035 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-06-06T08:43:40.035 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24482, 2024-06-06T08:43:40.035 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 6, 2024-06-06T08:43:40.035 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "3a2bb295": { 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 6, 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 145.2, 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-06-06T08:43:40.036 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-06-06T08:43:40.037 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:43:40.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-06-06T08:43:40.037 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:43:40.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:40.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:40.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:40.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:40.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:40.037 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:43:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 -- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d5c0a2eb0 msgr2=0x7f6d5c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d5c0a2eb0 0x7f6d5c0a5370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009c60 tx=0x7f6d64009260 comp rx=0 tx=0).stop 2024-06-06T08:43:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 -- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 msgr2=0x7f6d74075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 0x7f6d74075210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00d010 tx=0x7f6d6c0079a0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 -- 172.21.15.27:0/2205778201 shutdown_connections 2024-06-06T08:43:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d74107760 0x7f6d74075210 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6d5c0a2eb0 0x7f6d5c0a5370 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d74102770 0x7f6d74074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 --2- 172.21.15.27:0/2205778201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d74103120 0x7f6d74074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 -- 172.21.15.27:0/2205778201 >> 172.21.15.27:0/2205778201 conn(0x7f6d74079560 msgr2=0x7f6d741004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 -- 172.21.15.27:0/2205778201 shutdown_connections 2024-06-06T08:43:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:40.035+0000 7f6d793e5700 1 -- 172.21.15.27:0/2205778201 wait complete. 2024-06-06T08:43:40.789 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3a2bb295', quiesce took 30.3 seconds. Will release after: -5.100000000000001 2024-06-06T08:43:40.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:43:40.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:40 smithi027 ceph-mon[32335]: pgmap v105: 129 pgs: 129 active+clean; 1.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s rd, 95 MiB/s wr, 827 op/s 2024-06-06T08:43:40.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:40 smithi203 ceph-mon[36579]: pgmap v105: 129 pgs: 129 active+clean; 1.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s rd, 95 MiB/s wr, 827 op/s 2024-06-06T08:43:41.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:40 smithi176 ceph-mon[36786]: pgmap v105: 129 pgs: 129 active+clean; 1.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s rd, 95 MiB/s wr, 827 op/s 2024-06-06T08:43:41.051 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:43:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.732+0000 7f0e22e3d700 1 -- 172.21.15.27:0/1244659967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e1c107840 msgr2=0x7f0e1c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.732+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/1244659967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e1c107840 0x7f0e1c10f3c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400b0d0 tx=0x7f0e1400b3e0 comp rx=0 tx=0).stop 2024-06-06T08:43:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.732+0000 7f0e22e3d700 1 -- 172.21.15.27:0/1244659967 shutdown_connections 2024-06-06T08:43:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.732+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/1244659967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e1c107840 0x7f0e1c10f3c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:42.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.732+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/1244659967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c102740 0x7f0e1c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.732+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/1244659967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e1c1030f0 0x7f0e1c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.732+0000 7f0e22e3d700 1 -- 172.21.15.27:0/1244659967 >> 172.21.15.27:0/1244659967 conn(0x7f0e1c0795f0 msgr2=0x7f0e1c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.732+0000 7f0e22e3d700 1 -- 172.21.15.27:0/1244659967 shutdown_connections 2024-06-06T08:43:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.732+0000 7f0e22e3d700 1 -- 172.21.15.27:0/1244659967 wait complete. 2024-06-06T08:43:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e22e3d700 1 Processor -- start 2024-06-06T08:43:42.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e22e3d700 1 -- start start 2024-06-06T08:43:42.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e22e3d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c102740 0x7f0e1c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e22e3d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e1c1030f0 0x7f0e1c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e22e3d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e1c107840 0x7f0e1c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e22e3d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c112760 con 0x7f0e1c107840 2024-06-06T08:43:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e22e3d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c1125e0 con 0x7f0e1c102740 2024-06-06T08:43:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e22e3d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c1128e0 con 0x7f0e1c1030f0 2024-06-06T08:43:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e1bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e1c1030f0 0x7f0e1c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:42.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e213da700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e1c107840 0x7f0e1c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e1bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e1c1030f0 0x7f0e1c19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:38810/0 (socket says 172.21.15.27:38810) 2024-06-06T08:43:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e1bfff700 1 -- 172.21.15.27:0/3879268883 learned_addr learned my addr 172.21.15.27:0/3879268883 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.733+0000 7f0e20bd9700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c102740 0x7f0e1c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e213da700 1 -- 172.21.15.27:0/3879268883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e1c1030f0 msgr2=0x7f0e1c19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e213da700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e1c1030f0 0x7f0e1c19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:42.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e213da700 1 -- 172.21.15.27:0/3879268883 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c102740 msgr2=0x7f0e1c19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e213da700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c102740 0x7f0e1c19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e213da700 1 -- 172.21.15.27:0/3879268883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c1990e0 con 0x7f0e1c107840 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e1bfff700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e1c1030f0 0x7f0e1c19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e20bd9700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c102740 0x7f0e1c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e213da700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e1c107840 0x7f0e1c198980 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400b820 tx=0x7f0e14004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e19ffb700 1 -- 172.21.15.27:0/3879268883 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14021020 con 0x7f0e1c107840 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e19ffb700 1 -- 172.21.15.27:0/3879268883 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e14007b80 con 0x7f0e1c107840 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c199370 con 0x7f0e1c107840 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e19ffb700 1 -- 172.21.15.27:0/3879268883 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140052a0 con 0x7f0e1c107840 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.734+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c1a5640 con 0x7f0e1c107840 2024-06-06T08:43:42.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.735+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c069180 con 0x7f0e1c107840 2024-06-06T08:43:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.737+0000 7f0e19ffb700 1 -- 172.21.15.27:0/3879268883 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0e14024020 con 0x7f0e1c107840 2024-06-06T08:43:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.737+0000 7f0e19ffb700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0e0c0a2e10 0x7f0e0c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.737+0000 7f0e19ffb700 1 -- 172.21.15.27:0/3879268883 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f0e1400f020 con 0x7f0e1c107840 2024-06-06T08:43:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.737+0000 7f0e20bd9700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0e0c0a2e10 0x7f0e0c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:42.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.738+0000 7f0e20bd9700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0e0c0a2e10 0x7f0e0c0a52d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c107350 tx=0x7f0e1000b690 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:42.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.741+0000 7f0e19ffb700 1 -- 172.21.15.27:0/3879268883 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0e1409d6a0 con 0x7f0e1c107840 2024-06-06T08:43:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:42 smithi027 ceph-mon[32335]: pgmap v106: 129 pgs: 129 active+clean; 1.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 692 op/s 2024-06-06T08:43:42.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:42 smithi203 ceph-mon[36579]: pgmap v106: 129 pgs: 129 active+clean; 1.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 692 op/s 2024-06-06T08:43:42.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.996+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e1c199e60 con 0x7f0e1c107840 2024-06-06T08:43:42.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.996+0000 7f0e19ffb700 1 -- 172.21.15.27:0/3879268883 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11896 (secure 0 0 0) 0x7f0e1409cfa0 con 0x7f0e1c107840 2024-06-06T08:43:42.999 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:43:42.999 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:43:25.179611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:43:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0e0c0a2e10 msgr2=0x7f0e0c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0e0c0a2e10 0x7f0e0c0a52d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c107350 tx=0x7f0e1000b690 comp rx=0 tx=0).stop 2024-06-06T08:43:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e1c107840 msgr2=0x7f0e1c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e1c107840 0x7f0e1c198980 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400b820 tx=0x7f0e14004ce0 comp rx=0 tx=0).stop 2024-06-06T08:43:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 shutdown_connections 2024-06-06T08:43:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e1c107840 0x7f0e1c198980 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f0e0c0a2e10 0x7f0e0c0a52d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c102740 0x7f0e1c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 --2- 172.21.15.27:0/3879268883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e1c1030f0 0x7f0e1c19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 >> 172.21.15.27:0/3879268883 conn(0x7f0e1c0795f0 msgr2=0x7f0e1c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 shutdown_connections 2024-06-06T08:43:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:42.999+0000 7f0e22e3d700 1 -- 172.21.15.27:0/3879268883 wait complete. 2024-06-06T08:43:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-06-06T08:43:43.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:42 smithi176 ceph-mon[36786]: pgmap v106: 129 pgs: 129 active+clean; 1.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 692 op/s 2024-06-06T08:43:43.730 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 3a2bb295 --release --await' 2024-06-06T08:43:43.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 3a2bb295 --release --await 2024-06-06T08:43:43.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:43 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3879268883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:43:43.985 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:43:43.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:43 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3879268883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:43:44.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:43 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3879268883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:43:44.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:44 smithi027 ceph-mon[32335]: pgmap v107: 129 pgs: 129 active+clean; 1.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 766 op/s 2024-06-06T08:43:44.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:44 smithi203 ceph-mon[36579]: pgmap v107: 129 pgs: 129 active+clean; 1.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 766 op/s 2024-06-06T08:43:45.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:44 smithi176 ceph-mon[36786]: pgmap v107: 129 pgs: 129 active+clean; 1.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 766 op/s 2024-06-06T08:43:45.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 -- 172.21.15.27:0/2230530986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d34102f10 msgr2=0x7f4d3410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:45.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 --2- 172.21.15.27:0/2230530986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d34102f10 0x7f4d3410f2f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00b0d0 tx=0x7f4d2c00b3e0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 -- 172.21.15.27:0/2230530986 shutdown_connections 2024-06-06T08:43:45.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 --2- 172.21.15.27:0/2230530986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d34102f10 0x7f4d3410f2f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 --2- 172.21.15.27:0/2230530986 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d341073a0 0x7f4d34107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 --2- 172.21.15.27:0/2230530986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34102550 0x7f4d341029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 -- 172.21.15.27:0/2230530986 >> 172.21.15.27:0/2230530986 conn(0x7f4d34079650 msgr2=0x7f4d34079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:45.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 -- 172.21.15.27:0/2230530986 shutdown_connections 2024-06-06T08:43:45.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 -- 172.21.15.27:0/2230530986 wait complete. 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.772+0000 7f4d39839700 1 Processor -- start 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d39839700 1 -- start start 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d39839700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34102550 0x7f4d3419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d39839700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d34102f10 0x7f4d3419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d39839700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d341073a0 0x7f4d3419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d39839700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34112760 con 0x7f4d341073a0 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d39839700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341125e0 con 0x7f4d34102f10 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d39839700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341128e0 con 0x7f4d34102550 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d341073a0 0x7f4d3419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d341073a0 0x7f4d3419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54108/0 (socket says 172.21.15.27:54108) 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d337fe700 1 -- 172.21.15.27:0/1785255780 learned_addr learned my addr 172.21.15.27:0/1785255780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34102550 0x7f4d3419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d327fc700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d34102f10 0x7f4d3419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d337fe700 1 -- 172.21.15.27:0/1785255780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34102550 msgr2=0x7f4d3419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d337fe700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34102550 0x7f4d3419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d337fe700 1 -- 172.21.15.27:0/1785255780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d34102f10 msgr2=0x7f4d3419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d337fe700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d34102f10 0x7f4d3419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.773+0000 7f4d337fe700 1 -- 172.21.15.27:0/1785255780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3419fda0 con 0x7f4d341073a0 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.774+0000 7f4d337fe700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d341073a0 0x7f4d3419f660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006070 tx=0x7f4d2c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.774+0000 7f4d38837700 1 -- 172.21.15.27:0/1785255780 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c021070 con 0x7f4d341073a0 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.774+0000 7f4d39839700 1 -- 172.21.15.27:0/1785255780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d341a0030 con 0x7f4d341073a0 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.774+0000 7f4d38837700 1 -- 172.21.15.27:0/1785255780 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d2c007b80 con 0x7f4d341073a0 2024-06-06T08:43:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.774+0000 7f4d38837700 1 -- 172.21.15.27:0/1785255780 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c005330 con 0x7f4d341073a0 2024-06-06T08:43:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.774+0000 7f4d39839700 1 -- 172.21.15.27:0/1785255780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d341a7910 con 0x7f4d341073a0 2024-06-06T08:43:45.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.777+0000 7f4d38837700 1 -- 172.21.15.27:0/1785255780 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4d2c024070 con 0x7f4d341073a0 2024-06-06T08:43:45.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.778+0000 7f4d38837700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d200a2d90 0x7f4d200a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.778+0000 7f4d38837700 1 -- 172.21.15.27:0/1785255780 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f4d2c00f020 con 0x7f4d341073a0 2024-06-06T08:43:45.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.778+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d200a2d90 0x7f4d200a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.778+0000 7f4d440bcb80 1 Processor -- start 2024-06-06T08:43:45.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.778+0000 7f4d440bcb80 1 -- start start 2024-06-06T08:43:45.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.778+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d200a2d90 0x7f4d200a5250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005950 tx=0x7f4d1c00b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe30a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 0x55ebe30a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ebe2fec910 con 0x55ebe30a9d70 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ebe2f94820 con 0x55ebe30a3be0 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ebe2fe88d0 con 0x55ebe30a9450 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d327fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe30a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d32ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d327fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe30a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54124/0 (socket says 172.21.15.27:54124) 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d327fc700 1 -- 172.21.15.27:0/3326030847 learned_addr learned my addr 172.21.15.27:0/3326030847 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d337fe700 1 --2- 172.21.15.27:0/3326030847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 0x55ebe30a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d327fc700 1 -- 172.21.15.27:0/3326030847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 msgr2=0x55ebe30a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d327fc700 1 --2- 172.21.15.27:0/3326030847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d327fc700 1 -- 172.21.15.27:0/3326030847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 msgr2=0x55ebe30a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d327fc700 1 --2- 172.21.15.27:0/3326030847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 0x55ebe30a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d327fc700 1 -- 172.21.15.27:0/3326030847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebe2f8b050 con 0x55ebe30a9d70 2024-06-06T08:43:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d327fc700 1 --2- 172.21.15.27:0/3326030847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe30a36a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400bd10 tx=0x7f4d2400f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab154b165996f4af server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d29ffb700 1 -- 172.21.15.27:0/3326030847 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24015070 con 0x55ebe30a9d70 2024-06-06T08:43:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d29ffb700 1 -- 172.21.15.27:0/3326030847 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d2400fda0 con 0x55ebe30a9d70 2024-06-06T08:43:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.779+0000 7f4d29ffb700 1 -- 172.21.15.27:0/3326030847 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d240117c0 con 0x55ebe30a9d70 2024-06-06T08:43:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.780+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3326030847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 msgr2=0x55ebe30a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.780+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3326030847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe30a36a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400bd10 tx=0x7f4d2400f640 comp rx=0 tx=0).stop 2024-06-06T08:43:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.780+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3326030847 shutdown_connections 2024-06-06T08:43:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.780+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3326030847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 0x55ebe30a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.780+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3326030847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.780+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3326030847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe30a36a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.780+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3326030847 >> 172.21.15.27:0/3326030847 conn(0x55ebe2fc5dc0 msgr2=0x55ebe3096120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.780+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3326030847 shutdown_connections 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.780+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3326030847 wait complete. 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d440bcb80 1 -- start start 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d440bcb80 1 Processor -- start 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9450 0x55ebe30acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9d70 0x55ebe30ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ebe30b07e0 con 0x55ebe30a3be0 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55ebe2f8ccf0 con 0x55ebe30a9450 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ebe30b0960 con 0x55ebe30a9d70 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d440bcb80 10 client.0 _ll_register_callbacks cb 0x55ebe2f86a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:45.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d32ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d32ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54130/0 (socket says 172.21.15.27:54130) 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d32ffd700 1 -- 172.21.15.27:0/3100911487 learned_addr learned my addr 172.21.15.27:0/3100911487 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d327fc700 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9450 0x55ebe30acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.781+0000 7f4d337fe700 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9d70 0x55ebe30ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d32ffd700 1 -- 172.21.15.27:0/3100911487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9d70 msgr2=0x55ebe30ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9d70 0x55ebe30ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d32ffd700 1 -- 172.21.15.27:0/3100911487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9450 msgr2=0x55ebe30acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9450 0x55ebe30acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d32ffd700 1 -- 172.21.15.27:0/3100911487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebe2f3b9d0 con 0x55ebe30a3be0 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30afa40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0116f0 tx=0x7f4d1c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d2838a2a2d0ca1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:45.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d12ffd700 1 -- 172.21.15.27:0/3100911487 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c004a30 con 0x55ebe30a3be0 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d12ffd700 1 -- 172.21.15.27:0/3100911487 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f4d1c004bd0 con 0x55ebe30a3be0 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3100911487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d2400fda0 con 0x55ebe30a3be0 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d12ffd700 1 -- 172.21.15.27:0/3100911487 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4d1c004e10 con 0x55ebe30a3be0 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d440bcb80 10 client.16251 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3100911487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ebe30bb160 con 0x55ebe30a3be0 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.782+0000 7f4d12ffd700 1 -- 172.21.15.27:0/3100911487 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d2400fda0 con 0x55ebe30a3be0 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d12ffd700 1 -- 172.21.15.27:0/3100911487 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4d1c01f8a0 con 0x55ebe30a3be0 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d440bcb80 10 client.16251 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d440bcb80 10 client.16251 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d440bcb80 20 client.16251 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ebe30bdbd0 0x55ebe30c0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d440bcb80 4 client.16251 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3100911487 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebe2ff9e10 con 0x55ebe30bdbd0 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d327fc700 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ebe30bdbd0 0x55ebe30c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d327fc700 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ebe30bdbd0 0x55ebe30c0010 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b96f3c2189a881 server_cookie=32f9e0781a190246 in_seq=0 out_seq=0 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.783+0000 7f4d12ffd700 10 client.16251 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.784+0000 7f4d12ffd700 1 -- 172.21.15.27:0/3100911487 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ebe2ff9e10 con 0x55ebe30bdbd0 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.784+0000 7f4d12ffd700 10 client.16251 handle_command_reply: tid=0 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.784+0000 7f4d440bcb80 1 client.16251 shutdown 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.784+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3100911487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 msgr2=0x55ebe30afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.784+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30afa40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0116f0 tx=0x7f4d1c0047c0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3100911487 shutdown_connections 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9d70 0x55ebe30ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ebe30bdbd0 0x55ebe30c0010 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30afa40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/3100911487 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9450 0x55ebe30acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3100911487 >> 172.21.15.27:0/3100911487 conn(0x55ebe2fc5dc0 msgr2=0x55ebe30a2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d12ffd700 0 client.16251 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3100911487 shutdown_connections 2024-06-06T08:43:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d440bcb80 1 -- 172.21.15.27:0/3100911487 wait complete. 2024-06-06T08:43:45.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.785+0000 7f4d440bcb80 20 client.16251 trim_cache size 0 max 16384 2024-06-06T08:43:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d440bcb80 1 Processor -- start 2024-06-06T08:43:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d440bcb80 1 -- start start 2024-06-06T08:43:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9d70 0x55ebe30b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341aebd0 con 0x55ebe30a3be0 2024-06-06T08:43:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341ac3a0 con 0x55ebe30a9d70 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341aa1f0 con 0x55ebe30a9450 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d32ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d32ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54132/0 (socket says 172.21.15.27:54132) 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.805+0000 7f4d32ffd700 1 -- 172.21.15.27:0/1453548718 learned_addr learned my addr 172.21.15.27:0/1453548718 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d327fc700 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d337fe700 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9d70 0x55ebe30b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d32ffd700 1 -- 172.21.15.27:0/1453548718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 msgr2=0x55ebe30c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d32ffd700 1 -- 172.21.15.27:0/1453548718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9d70 msgr2=0x55ebe30b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9d70 0x55ebe30b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d32ffd700 1 -- 172.21.15.27:0/1453548718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebe30baf70 con 0x55ebe30a3be0 2024-06-06T08:43:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d327fc700 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:43:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d337fe700 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9d70 0x55ebe30b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:43:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30c5920 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c01fcc0 tx=0x7f4d1c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a001e4b0b8951dd6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d297fa700 1 -- 172.21.15.27:0/1453548718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c004830 con 0x55ebe30a3be0 2024-06-06T08:43:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d297fa700 1 -- 172.21.15.27:0/1453548718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d1c023070 con 0x55ebe30a3be0 2024-06-06T08:43:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.806+0000 7f4d297fa700 1 -- 172.21.15.27:0/1453548718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0053b0 con 0x55ebe30a3be0 2024-06-06T08:43:45.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 -- 172.21.15.27:0/1453548718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 msgr2=0x55ebe30c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:45.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30c5920 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c01fcc0 tx=0x7f4d1c00b960 comp rx=0 tx=0).stop 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 -- 172.21.15.27:0/1453548718 shutdown_connections 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a9d70 0x55ebe30b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a3be0 0x55ebe30c5920 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/1453548718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 -- 172.21.15.27:0/1453548718 >> 172.21.15.27:0/1453548718 conn(0x55ebe2fc5dc0 msgr2=0x55ebe30a6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 -- 172.21.15.27:0/1453548718 shutdown_connections 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 -- 172.21.15.27:0/1453548718 wait complete. 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 -- start start 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.807+0000 7f4d440bcb80 1 Processor -- start 2024-06-06T08:43:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 0x55ebe30b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d440bcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe31208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341aa860 con 0x55ebe30a9d70 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341abcd0 con 0x55ebe30a3be0 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d440bcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4d341aaf30 con 0x55ebe30a9450 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d440bcb80 10 client.0 _ll_register_callbacks cb 0x55ebe2f86a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe31208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe31208b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54144/0 (socket says 172.21.15.27:54144) 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d337fe700 1 -- 172.21.15.27:0/4035888103 learned_addr learned my addr 172.21.15.27:0/4035888103 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 0x55ebe30b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d327fc700 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d337fe700 1 -- 172.21.15.27:0/4035888103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 msgr2=0x55ebe30b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d337fe700 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d337fe700 1 -- 172.21.15.27:0/4035888103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 msgr2=0x55ebe30b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d337fe700 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 0x55ebe30b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d337fe700 1 -- 172.21.15.27:0/4035888103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebe3120fb0 con 0x55ebe30a9d70 2024-06-06T08:43:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d337fe700 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe31208b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0994f0 tx=0x7f4d2c09ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08fb482d2ec9d4a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d12ffd700 1 -- 172.21.15.27:0/4035888103 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c01d9c0 con 0x55ebe30a9d70 2024-06-06T08:43:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d12ffd700 1 -- 172.21.15.27:0/4035888103 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f4d2c0a5690 con 0x55ebe30a9d70 2024-06-06T08:43:45.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.808+0000 7f4d440bcb80 1 -- 172.21.15.27:0/4035888103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d1c023070 con 0x55ebe30a9d70 2024-06-06T08:43:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d12ffd700 1 -- 172.21.15.27:0/4035888103 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4d2c01db60 con 0x55ebe30a9d70 2024-06-06T08:43:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d440bcb80 10 client.16257 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d440bcb80 1 -- 172.21.15.27:0/4035888103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ebe3129490 con 0x55ebe30a9d70 2024-06-06T08:43:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d12ffd700 1 -- 172.21.15.27:0/4035888103 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d1c023070 con 0x55ebe30a9d70 2024-06-06T08:43:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d12ffd700 1 -- 172.21.15.27:0/4035888103 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4d2c011070 con 0x55ebe30a9d70 2024-06-06T08:43:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d440bcb80 10 client.16257 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d440bcb80 10 client.16257 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:43:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d440bcb80 20 client.16257 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ebe3129bf0 0x55ebe312c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d440bcb80 4 client.16257 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3a2bb295", "release": true, "await": true}] 2024-06-06T08:43:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d440bcb80 1 -- 172.21.15.27:0/4035888103 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3a2bb295", "release": true, "await": true}) -- 0x55ebe2ff4390 con 0x55ebe3129bf0 2024-06-06T08:43:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.809+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ebe3129bf0 0x55ebe312c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.810+0000 7f4d32ffd700 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ebe3129bf0 0x55ebe312c010 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db805d98a303cadf server_cookie=55a420fe3e552383 in_seq=0 out_seq=0 2024-06-06T08:43:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:45.810+0000 7f4d12ffd700 10 client.16257 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:45.912 INFO:tasks.workunit.client.0.smithi027.stdout:9/484: creat d3/d95/d23/d36/d3a/fac x:0 0 0 2024-06-06T08:43:46.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.437+0000 7f4d12ffd700 1 -- 172.21.15.27:0/4035888103 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55ebe2ff4390 con 0x55ebe3129bf0 2024-06-06T08:43:46.436 INFO:tasks.workunit.client.0.smithi027.stdout:9/485: creat d3/d4e/d75/dc/fad x:0 0 0 2024-06-06T08:43:46.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.437+0000 7f4d12ffd700 10 client.16257 handle_command_reply: tid=0 2024-06-06T08:43:46.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.437+0000 7f4d440bcb80 1 client.16257 shutdown 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.437+0000 7f4d440bcb80 1 -- 172.21.15.27:0/4035888103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 msgr2=0x55ebe31208b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.437+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe31208b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0994f0 tx=0x7f4d2c09ca70 comp rx=0 tx=0).stop 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.438+0000 7f4d440bcb80 1 -- 172.21.15.27:0/4035888103 shutdown_connections 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.438+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55ebe3129bf0 0x55ebe312c010 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.438+0000 7f4d12ffd700 0 client.16257 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.438+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebe30a9d70 0x55ebe31208b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.438+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ebe30a3be0 0x55ebe30b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.438+0000 7f4d440bcb80 1 --2- 172.21.15.27:0/4035888103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ebe30a9450 0x55ebe30b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.438+0000 7f4d440bcb80 1 -- 172.21.15.27:0/4035888103 >> 172.21.15.27:0/4035888103 conn(0x55ebe2fc5dc0 msgr2=0x55ebe3095830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:46.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.438+0000 7f4d440bcb80 1 -- 172.21.15.27:0/4035888103 shutdown_connections 2024-06-06T08:43:46.439 INFO:tasks.workunit.client.0.smithi027.stdout:9/486: readlink d3/l56 0 2024-06-06T08:43:46.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.438+0000 7f4d440bcb80 1 -- 172.21.15.27:0/4035888103 wait complete. 2024-06-06T08:43:46.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.439+0000 7f4d440bcb80 20 client.16257 trim_cache size 0 max 16384 2024-06-06T08:43:46.439 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:43:46.439 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24482, 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 8, 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "3a2bb295": { 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 8, 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 145.2, 2024-06-06T08:43:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:43:46.441 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:43:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.439+0000 7f4d39839700 1 -- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d200a2d90 msgr2=0x7f4d200a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.439+0000 7f4d39839700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d200a2d90 0x7f4d200a5250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005950 tx=0x7f4d1c00b410 comp rx=0 tx=0).stop 2024-06-06T08:43:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.439+0000 7f4d39839700 1 -- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d341073a0 msgr2=0x7f4d3419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.439+0000 7f4d39839700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d341073a0 0x7f4d3419f660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006070 tx=0x7f4d2c004d00 comp rx=0 tx=0).stop 2024-06-06T08:43:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.440+0000 7f4d39839700 1 -- 172.21.15.27:0/1785255780 shutdown_connections 2024-06-06T08:43:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.440+0000 7f4d39839700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d341073a0 0x7f4d3419f660 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.440+0000 7f4d39839700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d34102550 0x7f4d3419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:46.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.440+0000 7f4d39839700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d200a2d90 0x7f4d200a5250 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:46.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.440+0000 7f4d39839700 1 --2- 172.21.15.27:0/1785255780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4d34102f10 0x7f4d3419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:46.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.440+0000 7f4d39839700 1 -- 172.21.15.27:0/1785255780 >> 172.21.15.27:0/1785255780 conn(0x7f4d34079650 msgr2=0x7f4d341003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:46.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.440+0000 7f4d39839700 1 -- 172.21.15.27:0/1785255780 shutdown_connections 2024-06-06T08:43:46.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:46.440+0000 7f4d39839700 1 -- 172.21.15.27:0/1785255780 wait complete. 2024-06-06T08:43:46.443 INFO:tasks.workunit.client.0.smithi027.stdout:3/458: dwrite d1/f85 [0,4194304] 0 2024-06-06T08:43:46.445 INFO:tasks.workunit.client.0.smithi027.stdout:3/459: symlink d1/d1f/d5b/l88 0 2024-06-06T08:43:46.446 INFO:tasks.workunit.client.0.smithi027.stdout:3/460: write d1/d8/f15 [1435970,27523] 0 2024-06-06T08:43:46.455 INFO:tasks.workunit.client.0.smithi027.stdout:0/520: mkdir d15/dae 0 2024-06-06T08:43:46.462 INFO:tasks.workunit.client.0.smithi027.stdout:3/461: link d1/d1f/d5b/l83 d1/d1f/d25/l89 0 2024-06-06T08:43:46.465 INFO:tasks.workunit.client.0.smithi027.stdout:1/477: link dc/d67/d69/c2e dc/de/d14/d4e/c96 0 2024-06-06T08:43:46.474 INFO:tasks.workunit.client.0.smithi027.stdout:3/462: unlink d1/d8/d22/d27/l3b 0 2024-06-06T08:43:46.477 INFO:tasks.workunit.client.0.smithi027.stdout:3/463: creat d1/d8/f8a x:0 0 0 2024-06-06T08:43:46.478 INFO:tasks.workunit.client.0.smithi027.stdout:1/478: dread - dc/de/d3d/d4f/f64 zero size 2024-06-06T08:43:46.483 INFO:tasks.workunit.client.0.smithi027.stdout:3/464: symlink d1/d8/d59/l8b 0 2024-06-06T08:43:46.486 INFO:tasks.workunit.client.0.smithi027.stdout:1/479: unlink dc/de/d14/d4e/c57 0 2024-06-06T08:43:46.490 INFO:tasks.workunit.client.0.smithi027.stdout:7/690: truncate d17/d32/d61/f6a 1625127 0 2024-06-06T08:43:46.490 INFO:tasks.workunit.client.0.smithi027.stdout:7/691: chown d17/d32/fcb 15 1 2024-06-06T08:43:46.491 INFO:tasks.workunit.client.0.smithi027.stdout:7/692: fdatasync d17/d32/d46/d83/d77/fb4 0 2024-06-06T08:43:46.494 INFO:tasks.workunit.client.0.smithi027.stdout:0/521: creat d15/d18/d40/faf x:0 0 0 2024-06-06T08:43:46.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/617: rmdir d0/da/db 39 2024-06-06T08:43:46.497 INFO:tasks.workunit.client.0.smithi027.stdout:2/573: rmdir d6/d1e/d2b/d5a 39 2024-06-06T08:43:46.498 INFO:tasks.workunit.client.0.smithi027.stdout:0/522: stat d15/d18/d23/d38/d68/d85/ca4 0 2024-06-06T08:43:46.498 INFO:tasks.workunit.client.0.smithi027.stdout:1/480: symlink dc/d67/d69/d32/d37/l97 0 2024-06-06T08:43:46.499 INFO:tasks.workunit.client.0.smithi027.stdout:0/523: chown d15/d18/d40/d8c/d9c 18437754 1 2024-06-06T08:43:46.501 INFO:tasks.workunit.client.0.smithi027.stdout:3/465: dwrite d1/f41 [0,4194304] 0 2024-06-06T08:43:46.505 INFO:tasks.workunit.client.0.smithi027.stdout:7/693: truncate d17/d32/d46/d4f/d5e/fb0 558150 0 2024-06-06T08:43:46.505 INFO:tasks.workunit.client.0.smithi027.stdout:1/481: dread dc/d67/d69/d32/d37/f41 [0,4194304] 0 2024-06-06T08:43:46.505 INFO:tasks.workunit.client.0.smithi027.stdout:3/466: stat d1/d8/d22/f73 0 2024-06-06T08:43:46.510 INFO:tasks.workunit.client.0.smithi027.stdout:0/524: dread fb [0,4194304] 0 2024-06-06T08:43:46.512 INFO:tasks.workunit.client.0.smithi027.stdout:2/574: creat d6/dc/d1d/d4d/d8e/fc3 x:0 0 0 2024-06-06T08:43:46.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/525: dread d15/d18/d23/d28/d2a/d30/d5c/d83/f86 [0,4194304] 0 2024-06-06T08:43:46.515 INFO:tasks.workunit.client.0.smithi027.stdout:5/618: write d0/da/d37/d2b/f3d [2114199,21573] 0 2024-06-06T08:43:46.520 INFO:tasks.workunit.client.0.smithi027.stdout:1/482: read dc/de/f75 [1850766,115757] 0 2024-06-06T08:43:46.521 INFO:tasks.workunit.client.0.smithi027.stdout:7/694: dread d17/d32/d46/d51/d7f/f7a [0,4194304] 0 2024-06-06T08:43:46.523 INFO:tasks.workunit.client.0.smithi027.stdout:0/526: rmdir d15/d18/d23/d28/d2a 39 2024-06-06T08:43:46.524 INFO:tasks.workunit.client.0.smithi027.stdout:1/483: mkdir dc/d67/d69/d32/d37/d98 0 2024-06-06T08:43:46.525 INFO:tasks.workunit.client.0.smithi027.stdout:7/695: truncate d17/d47/f49 862025 0 2024-06-06T08:43:46.526 INFO:tasks.workunit.client.0.smithi027.stdout:0/527: creat d15/d18/d23/d28/d2a/d74/fb0 x:0 0 0 2024-06-06T08:43:46.527 INFO:tasks.workunit.client.0.smithi027.stdout:4/632: rename d0/d63/d7e to d0/d1/d6/d8/d82/d8b/dc9 0 2024-06-06T08:43:46.527 INFO:tasks.workunit.client.0.smithi027.stdout:0/528: fdatasync d15/d18/d23/d28/f4e 0 2024-06-06T08:43:46.530 INFO:tasks.workunit.client.0.smithi027.stdout:2/575: dread d6/dc/d3f/d77/f4c [0,4194304] 0 2024-06-06T08:43:46.532 INFO:tasks.workunit.client.0.smithi027.stdout:5/619: dread d0/f29 [0,4194304] 0 2024-06-06T08:43:46.534 INFO:tasks.workunit.client.0.smithi027.stdout:2/576: dread - d6/dc/d3f/d48/fac zero size 2024-06-06T08:43:46.534 INFO:tasks.workunit.client.0.smithi027.stdout:7/696: creat d17/d32/d46/d83/d77/d8c/d91/daa/fe3 x:0 0 0 2024-06-06T08:43:46.534 INFO:tasks.workunit.client.0.smithi027.stdout:1/484: write dc/d67/d69/d32/d37/d56/d78/f83 [28288,70548] 0 2024-06-06T08:43:46.535 INFO:tasks.workunit.client.0.smithi027.stdout:0/529: link d15/d18/c20 d15/d18/d23/d38/d68/cb1 0 2024-06-06T08:43:46.536 INFO:tasks.workunit.client.0.smithi027.stdout:2/577: chown d6/fc0 16035728 1 2024-06-06T08:43:46.536 INFO:tasks.workunit.client.0.smithi027.stdout:7/697: getdents d17/d32/d46/d83/d77/d8c 0 2024-06-06T08:43:46.539 INFO:tasks.workunit.client.0.smithi027.stdout:1/485: unlink dc/de/c11 0 2024-06-06T08:43:46.540 INFO:tasks.workunit.client.0.smithi027.stdout:5/620: write d0/da/db/d1a/da9/fbe [346448,80678] 0 2024-06-06T08:43:46.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/578: fdatasync d6/dc/d3f/d77/d55/fb4 0 2024-06-06T08:43:46.543 INFO:tasks.workunit.client.0.smithi027.stdout:2/579: write d6/dc/d3f/d48/f62 [4985899,9783] 0 2024-06-06T08:43:46.544 INFO:tasks.workunit.client.0.smithi027.stdout:1/486: mknod dc/d6d/c99 0 2024-06-06T08:43:46.548 INFO:tasks.workunit.client.0.smithi027.stdout:5/621: symlink d0/da/db/d46/d4c/lc2 0 2024-06-06T08:43:46.549 INFO:tasks.workunit.client.0.smithi027.stdout:5/622: truncate d0/da/db/fa0 327609 0 2024-06-06T08:43:46.555 INFO:tasks.workunit.client.0.smithi027.stdout:4/633: dread d0/d1d/d34/fa2 [0,4194304] 0 2024-06-06T08:43:46.556 INFO:tasks.workunit.client.0.smithi027.stdout:1/487: write dc/de/d22/f72 [1921,104425] 0 2024-06-06T08:43:46.558 INFO:tasks.workunit.client.0.smithi027.stdout:5/623: mknod d0/da/db/d46/d7f/d81/cc3 0 2024-06-06T08:43:46.559 INFO:tasks.workunit.client.0.smithi027.stdout:4/634: stat d0/d1d/d2a/d98 0 2024-06-06T08:43:46.560 INFO:tasks.workunit.client.0.smithi027.stdout:1/488: mknod dc/d67/d69/d32/d37/d56/d78/c9a 0 2024-06-06T08:43:46.561 INFO:tasks.workunit.client.0.smithi027.stdout:5/624: fsync d0/da/db/f1b 0 2024-06-06T08:43:46.563 INFO:tasks.workunit.client.0.smithi027.stdout:1/489: creat dc/de/d14/d4e/f9b x:0 0 0 2024-06-06T08:43:46.563 INFO:tasks.workunit.client.0.smithi027.stdout:0/530: dwrite d15/d18/d23/d28/d2a/d2d/f34 [0,4194304] 0 2024-06-06T08:43:46.563 INFO:tasks.workunit.client.0.smithi027.stdout:5/625: dread - d0/da/fb4 zero size 2024-06-06T08:43:46.566 INFO:tasks.workunit.client.0.smithi027.stdout:5/626: readlink d0/da/d37/l74 0 2024-06-06T08:43:46.566 INFO:tasks.workunit.client.0.smithi027.stdout:7/698: dwrite d17/d32/d46/d51/d52/fd3 [0,4194304] 0 2024-06-06T08:43:46.566 INFO:tasks.workunit.client.0.smithi027.stdout:7/699: fsync d17/d32/d46/d51/d7f/f7a 0 2024-06-06T08:43:46.568 INFO:tasks.workunit.client.0.smithi027.stdout:5/627: truncate d0/da/db/d1a/da9/faa 284019 0 2024-06-06T08:43:46.570 INFO:tasks.workunit.client.0.smithi027.stdout:7/700: getdents d17/d32/d46/d51/d52/d76 0 2024-06-06T08:43:46.572 INFO:tasks.workunit.client.0.smithi027.stdout:4/635: write d0/d1/f4f [1764343,88095] 0 2024-06-06T08:43:46.573 INFO:tasks.workunit.client.0.smithi027.stdout:7/701: mknod d17/d32/d61/d75/ce4 0 2024-06-06T08:43:46.575 INFO:tasks.workunit.client.0.smithi027.stdout:4/636: mknod d0/d1d/d2a/cca 0 2024-06-06T08:43:46.576 INFO:tasks.workunit.client.0.smithi027.stdout:5/628: mkdir d0/da/db/d46/d7f/d81/d97/db0/dbc/dc4 0 2024-06-06T08:43:46.577 INFO:tasks.workunit.client.0.smithi027.stdout:2/580: dwrite d6/dd/f24 [0,4194304] 0 2024-06-06T08:43:46.577 INFO:tasks.workunit.client.0.smithi027.stdout:5/629: write d0/da/f43 [1631840,10288] 0 2024-06-06T08:43:46.580 INFO:tasks.workunit.client.0.smithi027.stdout:0/531: dread d15/d18/d23/d38/d3e/f5a [0,4194304] 0 2024-06-06T08:43:46.584 INFO:tasks.workunit.client.0.smithi027.stdout:4/637: creat d0/d1d/d34/d7f/d97/d9c/dbe/fcb x:0 0 0 2024-06-06T08:43:46.586 INFO:tasks.workunit.client.0.smithi027.stdout:9/487: rename d3/d95/d34/f64 to d3/fae 0 2024-06-06T08:43:46.587 INFO:tasks.workunit.client.0.smithi027.stdout:9/488: fsync d3/d4e/d75/dc/fad 0 2024-06-06T08:43:46.588 INFO:tasks.workunit.client.0.smithi027.stdout:9/489: readlink d3/d95/d23/d78/l9e 0 2024-06-06T08:43:46.589 INFO:tasks.workunit.client.0.smithi027.stdout:9/490: readlink d3/d95/d23/d36/d3a/l93 0 2024-06-06T08:43:46.589 INFO:tasks.workunit.client.0.smithi027.stdout:0/532: creat d15/da7/fb2 x:0 0 0 2024-06-06T08:43:46.589 INFO:tasks.workunit.client.0.smithi027.stdout:1/490: dwrite dc/de/d14/d4e/f62 [0,4194304] 0 2024-06-06T08:43:46.592 INFO:tasks.workunit.client.0.smithi027.stdout:2/581: write d6/dc/d1d/d4d/f5f [946060,111484] 0 2024-06-06T08:43:46.594 INFO:tasks.workunit.client.0.smithi027.stdout:4/638: mknod d0/ccc 0 2024-06-06T08:43:46.595 INFO:tasks.workunit.client.0.smithi027.stdout:4/639: fsync d0/d1d/d34/d7f/f94 0 2024-06-06T08:43:46.597 INFO:tasks.workunit.client.0.smithi027.stdout:3/467: rename d1/d8/d22/d27/l3f to d1/d1f/d45/d5f/l8c 0 2024-06-06T08:43:46.597 INFO:tasks.workunit.client.0.smithi027.stdout:6/633: sync 2024-06-06T08:43:46.597 INFO:tasks.workunit.client.0.smithi027.stdout:8/504: sync 2024-06-06T08:43:46.598 INFO:tasks.workunit.client.0.smithi027.stdout:2/582: mknod d6/d3b/cc4 0 2024-06-06T08:43:46.599 INFO:tasks.workunit.client.0.smithi027.stdout:9/491: creat d3/d4e/d75/faf x:0 0 0 2024-06-06T08:43:46.599 INFO:tasks.workunit.client.0.smithi027.stdout:3/468: fdatasync d1/d1f/d25/f5d 0 2024-06-06T08:43:46.599 INFO:tasks.workunit.client.0.smithi027.stdout:5/630: dwrite d0/fe [0,4194304] 0 2024-06-06T08:43:46.600 INFO:tasks.workunit.client.0.smithi027.stdout:2/583: stat d6/d1e/d2b/d5a 0 2024-06-06T08:43:46.600 INFO:tasks.workunit.client.0.smithi027.stdout:2/584: chown d6/fc0 68 1 2024-06-06T08:43:46.601 INFO:tasks.workunit.client.0.smithi027.stdout:9/492: unlink d3/d95/d34/c43 0 2024-06-06T08:43:46.601 INFO:tasks.workunit.client.0.smithi027.stdout:3/469: fsync d1/f29 0 2024-06-06T08:43:46.602 INFO:tasks.workunit.client.0.smithi027.stdout:5/631: stat d0/da/c73 0 2024-06-06T08:43:46.603 INFO:tasks.workunit.client.0.smithi027.stdout:5/632: mknod d0/da/db/d46/d7f/d81/d97/db0/dbc/cc5 0 2024-06-06T08:43:46.605 INFO:tasks.workunit.client.0.smithi027.stdout:4/640: link d0/d1/f2 d0/d1d/d34/d7f/fcd 0 2024-06-06T08:43:46.605 INFO:tasks.workunit.client.0.smithi027.stdout:2/585: rmdir d6/dd 39 2024-06-06T08:43:46.607 INFO:tasks.workunit.client.0.smithi027.stdout:3/470: creat d1/d1f/d25/f8d x:0 0 0 2024-06-06T08:43:46.608 INFO:tasks.workunit.client.0.smithi027.stdout:9/493: mkdir d3/d4e/d75/db0 0 2024-06-06T08:43:46.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/634: mknod d4/d9/d6e/ccb 0 2024-06-06T08:43:46.609 INFO:tasks.workunit.client.0.smithi027.stdout:9/494: fsync d3/d95/f57 0 2024-06-06T08:43:46.609 INFO:tasks.workunit.client.0.smithi027.stdout:1/491: dwrite dc/f84 [0,4194304] 0 2024-06-06T08:43:46.609 INFO:tasks.workunit.client.0.smithi027.stdout:7/702: rename d17/d32/f3a to d17/d32/d61/d75/fe5 0 2024-06-06T08:43:46.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/641: fdatasync d0/f66 0 2024-06-06T08:43:46.611 INFO:tasks.workunit.client.0.smithi027.stdout:2/586: stat d6/d3b/l90 0 2024-06-06T08:43:46.611 INFO:tasks.workunit.client.0.smithi027.stdout:6/635: creat d4/d9/dc/d18/fcc x:0 0 0 2024-06-06T08:43:46.613 INFO:tasks.workunit.client.0.smithi027.stdout:7/703: mkdir d17/d32/d46/d51/d52/d70/dce/de6 0 2024-06-06T08:43:46.613 INFO:tasks.workunit.client.0.smithi027.stdout:8/505: symlink d7/d9c/daf/lb0 0 2024-06-06T08:43:46.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/492: chown dc/de/d3d/d4f/l7f 30541 1 2024-06-06T08:43:46.614 INFO:tasks.workunit.client.0.smithi027.stdout:7/704: mknod d17/d32/d46/d51/d52/d70/ce7 0 2024-06-06T08:43:46.615 INFO:tasks.workunit.client.0.smithi027.stdout:7/705: dread - d17/d32/d46/d83/d77/d8c/d91/daa/fe3 zero size 2024-06-06T08:43:46.615 INFO:tasks.workunit.client.0.smithi027.stdout:4/642: write d0/f1c [4170178,28785] 0 2024-06-06T08:43:46.619 INFO:tasks.workunit.client.0.smithi027.stdout:3/471: symlink d1/d8/d22/d27/l8e 0 2024-06-06T08:43:46.625 INFO:tasks.workunit.client.0.smithi027.stdout:9/495: dwrite d3/d95/faa [0,4194304] 0 2024-06-06T08:43:46.626 INFO:tasks.workunit.client.0.smithi027.stdout:6/636: creat d4/d13/fcd x:0 0 0 2024-06-06T08:43:46.627 INFO:tasks.workunit.client.0.smithi027.stdout:9/496: dread - d3/d95/d23/d36/d3a/f67 zero size 2024-06-06T08:43:46.629 INFO:tasks.workunit.client.0.smithi027.stdout:9/497: mkdir d3/d4e/d75/dc/d9b/db1 0 2024-06-06T08:43:46.629 INFO:tasks.workunit.client.0.smithi027.stdout:1/493: symlink dc/d67/d69/l9c 0 2024-06-06T08:43:46.630 INFO:tasks.workunit.client.0.smithi027.stdout:7/706: truncate d17/f3f 3525909 0 2024-06-06T08:43:46.632 INFO:tasks.workunit.client.0.smithi027.stdout:1/494: write dc/de/d14/f87 [3514034,54304] 0 2024-06-06T08:43:46.633 INFO:tasks.workunit.client.0.smithi027.stdout:2/587: write d6/dc/d1d/f36 [3474478,11705] 0 2024-06-06T08:43:46.633 INFO:tasks.workunit.client.0.smithi027.stdout:2/588: read - d6/fc0 zero size 2024-06-06T08:43:46.638 INFO:tasks.workunit.client.0.smithi027.stdout:6/637: dread d4/de/f6c [0,4194304] 0 2024-06-06T08:43:46.638 INFO:tasks.workunit.client.0.smithi027.stdout:3/472: fsync d1/da/d6d/f86 0 2024-06-06T08:43:46.638 INFO:tasks.workunit.client.0.smithi027.stdout:2/589: chown d6/d3b/d80 1022489 1 2024-06-06T08:43:46.638 INFO:tasks.workunit.client.0.smithi027.stdout:6/638: read - d4/d9/d6e/faf zero size 2024-06-06T08:43:46.638 INFO:tasks.workunit.client.0.smithi027.stdout:9/498: dread d3/d95/d34/d41/f9c [0,4194304] 0 2024-06-06T08:43:46.641 INFO:tasks.workunit.client.0.smithi027.stdout:4/643: dwrite d0/d1d/f79 [0,4194304] 0 2024-06-06T08:43:46.641 INFO:tasks.workunit.client.0.smithi027.stdout:8/506: unlink d7/dc/d36/d3c/f94 0 2024-06-06T08:43:46.642 INFO:tasks.workunit.client.0.smithi027.stdout:1/495: mkdir dc/d9d 0 2024-06-06T08:43:46.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/639: mkdir d4/d1a/d2e/d44/dce 0 2024-06-06T08:43:46.645 INFO:tasks.workunit.client.0.smithi027.stdout:7/707: mkdir d17/d32/d46/d4f/db6/dc8/de8 0 2024-06-06T08:43:46.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/499: unlink d3/f9 0 2024-06-06T08:43:46.650 INFO:tasks.workunit.client.0.smithi027.stdout:9/500: dread - d3/d95/d23/d36/d3a/fac zero size 2024-06-06T08:43:46.651 INFO:tasks.workunit.client.0.smithi027.stdout:9/501: truncate d3/d95/d34/d41/f79 1724359 0 2024-06-06T08:43:46.651 INFO:tasks.workunit.client.0.smithi027.stdout:3/473: readlink d1/da/d44/d54/l68 0 2024-06-06T08:43:46.652 INFO:tasks.workunit.client.0.smithi027.stdout:9/502: write d3/d95/d23/da9/fab [824541,58282] 0 2024-06-06T08:43:46.653 INFO:tasks.workunit.client.0.smithi027.stdout:5/633: dwrite d0/f44 [4194304,4194304] 0 2024-06-06T08:43:46.653 INFO:tasks.workunit.client.0.smithi027.stdout:9/503: read d3/d4e/d75/dc/f2a [24868,80514] 0 2024-06-06T08:43:46.655 INFO:tasks.workunit.client.0.smithi027.stdout:9/504: mkdir d3/d95/d23/d36/d3a/d84/db2 0 2024-06-06T08:43:46.656 INFO:tasks.workunit.client.0.smithi027.stdout:9/505: rmdir d3/d4e/d75/dc/d9b/db1 0 2024-06-06T08:43:46.658 INFO:tasks.workunit.client.0.smithi027.stdout:7/708: dread d17/d32/d46/d83/d77/d8c/d90/f9a [0,4194304] 0 2024-06-06T08:43:46.660 INFO:tasks.workunit.client.0.smithi027.stdout:2/590: dread d6/f14 [0,4194304] 0 2024-06-06T08:43:46.661 INFO:tasks.workunit.client.0.smithi027.stdout:4/644: creat d0/d1/d6/d4b/fce x:0 0 0 2024-06-06T08:43:46.661 INFO:tasks.workunit.client.0.smithi027.stdout:1/496: write dc/d67/d69/d32/f5b [3055823,14163] 0 2024-06-06T08:43:46.668 INFO:tasks.workunit.client.0.smithi027.stdout:6/640: unlink d4/d9/dc/d1c/d52/c7d 0 2024-06-06T08:43:46.668 INFO:tasks.workunit.client.0.smithi027.stdout:0/533: rename d15/c78 to d15/d18/d23/d28/d2a/cb3 0 2024-06-06T08:43:46.671 INFO:tasks.workunit.client.0.smithi027.stdout:6/641: dread - d4/d9/dc/f9f zero size 2024-06-06T08:43:46.674 INFO:tasks.workunit.client.0.smithi027.stdout:0/534: symlink d15/d18/d23/d28/lb4 0 2024-06-06T08:43:46.674 INFO:tasks.workunit.client.0.smithi027.stdout:5/634: dread - d0/da/db/d46/d54/f90 zero size 2024-06-06T08:43:46.676 INFO:tasks.workunit.client.0.smithi027.stdout:3/474: symlink d1/d8/d22/d27/d50/l8f 0 2024-06-06T08:43:46.678 INFO:tasks.workunit.client.0.smithi027.stdout:7/709: dread fd [0,4194304] 0 2024-06-06T08:43:46.682 INFO:tasks.workunit.client.0.smithi027.stdout:9/506: dwrite d3/d95/d23/d36/f6c [0,4194304] 0 2024-06-06T08:43:46.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/645: truncate d0/fd 8363821 0 2024-06-06T08:43:46.684 INFO:tasks.workunit.client.0.smithi027.stdout:9/507: creat d3/d95/d23/d78/fb3 x:0 0 0 2024-06-06T08:43:46.685 INFO:tasks.workunit.client.0.smithi027.stdout:3/475: write d1/f1e [1831057,101047] 0 2024-06-06T08:43:46.687 INFO:tasks.workunit.client.0.smithi027.stdout:9/508: fsync d3/d95/d34/d73/fa7 0 2024-06-06T08:43:46.690 INFO:tasks.workunit.client.0.smithi027.stdout:6/642: readlink d4/de/d45/l7b 0 2024-06-06T08:43:46.692 INFO:tasks.workunit.client.0.smithi027.stdout:0/535: write d15/d18/d23/d28/d2a/d30/d5d/f4a [3490208,97964] 0 2024-06-06T08:43:46.692 INFO:tasks.workunit.client.0.smithi027.stdout:8/507: unlink d7/dc/d36/d83/c5c 0 2024-06-06T08:43:46.692 INFO:tasks.workunit.client.0.smithi027.stdout:2/591: dwrite d6/fbd [0,4194304] 0 2024-06-06T08:43:46.693 INFO:tasks.workunit.client.0.smithi027.stdout:9/509: dread d3/d95/f28 [0,4194304] 0 2024-06-06T08:43:46.695 INFO:tasks.workunit.client.0.smithi027.stdout:0/536: mknod d15/d18/d23/d28/d2a/d30/cb5 0 2024-06-06T08:43:46.697 INFO:tasks.workunit.client.0.smithi027.stdout:9/510: mkdir d3/d4e/d75/db4 0 2024-06-06T08:43:46.699 INFO:tasks.workunit.client.0.smithi027.stdout:9/511: unlink d3/d4e/d75/f2c 0 2024-06-06T08:43:46.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/635: mkdir d0/da/db/d1a/dc6 0 2024-06-06T08:43:46.701 INFO:tasks.workunit.client.0.smithi027.stdout:7/710: dwrite d17/d32/d46/d4f/f55 [0,4194304] 0 2024-06-06T08:43:46.702 INFO:tasks.workunit.client.0.smithi027.stdout:2/592: dread d6/dc/d3f/f50 [0,4194304] 0 2024-06-06T08:43:46.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/711: write d17/d32/d46/d83/d2a/fbf [839333,70029] 0 2024-06-06T08:43:46.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/636: chown d0/da/d37/d2b/l53 120777952 1 2024-06-06T08:43:46.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/643: mkdir d4/de/d45/dcf 0 2024-06-06T08:43:46.705 INFO:tasks.workunit.client.0.smithi027.stdout:8/508: chown d7/dc/d36/d83/d58/c5a 218 1 2024-06-06T08:43:46.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:46 smithi027 ceph-mon[32335]: pgmap v108: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 767 op/s 2024-06-06T08:43:46.709 INFO:tasks.workunit.client.0.smithi027.stdout:2/593: dread d6/fe [0,4194304] 0 2024-06-06T08:43:46.710 INFO:tasks.workunit.client.0.smithi027.stdout:8/509: fsync d7/f42 0 2024-06-06T08:43:46.710 INFO:tasks.workunit.client.0.smithi027.stdout:9/512: write d3/d95/d23/d36/d3a/f6d [92504,29115] 0 2024-06-06T08:43:46.710 INFO:tasks.workunit.client.0.smithi027.stdout:9/513: write d3/d95/faa [4528700,90173] 0 2024-06-06T08:43:46.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/646: dwrite d0/d1/d6/f33 [4194304,4194304] 0 2024-06-06T08:43:46.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/514: mknod d3/d95/d23/d36/d3a/d84/db2/cb5 0 2024-06-06T08:43:46.713 INFO:tasks.workunit.client.0.smithi027.stdout:5/637: write d0/da/d20/fa1 [169638,79167] 0 2024-06-06T08:43:46.715 INFO:tasks.workunit.client.0.smithi027.stdout:8/510: write d7/dc/d36/d3c/f61 [3147063,37278] 0 2024-06-06T08:43:46.717 INFO:tasks.workunit.client.0.smithi027.stdout:5/638: creat d0/da/db/d1a/dc6/fc7 x:0 0 0 2024-06-06T08:43:46.717 INFO:tasks.workunit.client.0.smithi027.stdout:2/594: stat d6/dc/d3f/d77/d55/d69/l95 0 2024-06-06T08:43:46.719 INFO:tasks.workunit.client.0.smithi027.stdout:5/639: chown d0/da/f92 1416775491 1 2024-06-06T08:43:46.719 INFO:tasks.workunit.client.0.smithi027.stdout:2/595: chown d6/d1e/c35 46174808 1 2024-06-06T08:43:46.719 INFO:tasks.workunit.client.0.smithi027.stdout:7/712: write d17/d32/d46/d51/f71 [2568731,46542] 0 2024-06-06T08:43:46.723 INFO:tasks.workunit.client.0.smithi027.stdout:5/640: rmdir d0/da/db/d46/d4c/d70 39 2024-06-06T08:43:46.724 INFO:tasks.workunit.client.0.smithi027.stdout:3/476: rename d1/da to d1/d90 0 2024-06-06T08:43:46.725 INFO:tasks.workunit.client.0.smithi027.stdout:8/511: symlink d7/dc/d37/d46/lb1 0 2024-06-06T08:43:46.727 INFO:tasks.workunit.client.0.smithi027.stdout:3/477: chown d1/d8/d22/d27/l6e 8075 1 2024-06-06T08:43:46.728 INFO:tasks.workunit.client.0.smithi027.stdout:9/515: mkdir d3/d95/db6 0 2024-06-06T08:43:46.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/596: write d6/dc/f30 [814123,84689] 0 2024-06-06T08:43:46.731 INFO:tasks.workunit.client.0.smithi027.stdout:8/512: getdents d7/d2f 0 2024-06-06T08:43:46.734 INFO:tasks.workunit.client.0.smithi027.stdout:5/641: write d0/da/d37/d2b/f67 [7041281,64961] 0 2024-06-06T08:43:46.736 INFO:tasks.workunit.client.0.smithi027.stdout:5/642: creat d0/da/fc8 x:0 0 0 2024-06-06T08:43:46.739 INFO:tasks.workunit.client.0.smithi027.stdout:8/513: write d7/dc/d36/d83/f19 [4166412,122072] 0 2024-06-06T08:43:46.739 INFO:tasks.workunit.client.0.smithi027.stdout:8/514: fsync d7/d9c/fa9 0 2024-06-06T08:43:46.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/643: truncate d0/f44 653750 0 2024-06-06T08:43:46.748 INFO:tasks.workunit.client.0.smithi027.stdout:5/644: creat d0/da/db/d46/d4c/d70/fc9 x:0 0 0 2024-06-06T08:43:46.749 INFO:tasks.workunit.client.0.smithi027.stdout:5/645: getdents d0/da/db/d46/d54 0 2024-06-06T08:43:46.751 INFO:tasks.workunit.client.0.smithi027.stdout:3/478: dwrite d1/f29 [0,4194304] 0 2024-06-06T08:43:46.754 INFO:tasks.workunit.client.0.smithi027.stdout:7/713: dwrite d17/d32/fca [4194304,4194304] 0 2024-06-06T08:43:46.756 INFO:tasks.workunit.client.0.smithi027.stdout:0/537: rename f4 to d15/d18/d23/d28/d2a/d30/d5c/d83/fb6 0 2024-06-06T08:43:46.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/538: truncate d15/d18/d23/d28/d2a/d2d/f54 387086 0 2024-06-06T08:43:46.765 INFO:tasks.workunit.client.0.smithi027.stdout:7/714: truncate d17/f21 127960 0 2024-06-06T08:43:46.765 INFO:tasks.workunit.client.0.smithi027.stdout:0/539: mknod d15/d18/d23/d28/d2a/cb7 0 2024-06-06T08:43:46.772 INFO:tasks.workunit.client.0.smithi027.stdout:6/644: rename d4/d9/f28 to d4/d9/dc/d18/fd0 0 2024-06-06T08:43:46.773 INFO:tasks.workunit.client.0.smithi027.stdout:8/515: dwrite d7/d2f/f77 [0,4194304] 0 2024-06-06T08:43:46.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/645: dread - d4/de/fb4 zero size 2024-06-06T08:43:46.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/646: chown d4/d9 2769 1 2024-06-06T08:43:46.779 INFO:tasks.workunit.client.0.smithi027.stdout:9/516: dwrite d3/d4e/d75/dc/f2d [0,4194304] 0 2024-06-06T08:43:46.779 INFO:tasks.workunit.client.0.smithi027.stdout:2/597: dwrite d6/dc/d1d/f86 [0,4194304] 0 2024-06-06T08:43:46.784 INFO:tasks.workunit.client.0.smithi027.stdout:2/598: chown d6/d3b/d61 0 1 2024-06-06T08:43:46.786 INFO:tasks.workunit.client.0.smithi027.stdout:7/715: dwrite d17/d32/d46/d51/d52/d76/fb8 [0,4194304] 0 2024-06-06T08:43:46.786 INFO:tasks.workunit.client.0.smithi027.stdout:7/716: truncate d17/d32/fcb 717293 0 2024-06-06T08:43:46.787 INFO:tasks.workunit.client.0.smithi027.stdout:0/540: dwrite d15/d18/d23/d28/d2a/d30/d5c/f82 [0,4194304] 0 2024-06-06T08:43:46.788 INFO:tasks.workunit.client.0.smithi027.stdout:4/647: rename d0/c27 to d0/d1d/d34/d7f/d97/d9c/ccf 0 2024-06-06T08:43:46.788 INFO:tasks.workunit.client.0.smithi027.stdout:4/648: chown d0/d1d/d34/d7f/d97/d9c/dbe 29094 1 2024-06-06T08:43:46.788 INFO:tasks.workunit.client.0.smithi027.stdout:4/649: chown d0/d1d 473945 1 2024-06-06T08:43:46.790 INFO:tasks.workunit.client.0.smithi027.stdout:6/647: mknod d4/de/d45/cd1 0 2024-06-06T08:43:46.790 INFO:tasks.workunit.client.0.smithi027.stdout:8/516: read d7/dc/d36/f7b [184863,91622] 0 2024-06-06T08:43:46.791 INFO:tasks.workunit.client.0.smithi027.stdout:9/517: symlink d3/d4e/lb7 0 2024-06-06T08:43:46.792 INFO:tasks.workunit.client.0.smithi027.stdout:3/479: dwrite d1/d8/d22/d27/f36 [0,4194304] 0 2024-06-06T08:43:46.792 INFO:tasks.workunit.client.0.smithi027.stdout:2/599: truncate d6/dc/f22 258388 0 2024-06-06T08:43:46.793 INFO:tasks.workunit.client.0.smithi027.stdout:8/517: chown d7/dc/d37/d46/f8d 421429279 1 2024-06-06T08:43:46.795 INFO:tasks.workunit.client.0.smithi027.stdout:3/480: write d1/d8/d22/f46 [1903059,94507] 0 2024-06-06T08:43:46.796 INFO:tasks.workunit.client.0.smithi027.stdout:9/518: rmdir d3/d95/d34/d41/d88 0 2024-06-06T08:43:46.797 INFO:tasks.workunit.client.0.smithi027.stdout:4/650: mkdir d0/d1d/dd0 0 2024-06-06T08:43:46.798 INFO:tasks.workunit.client.0.smithi027.stdout:0/541: write d15/d18/f73 [3339501,21998] 0 2024-06-06T08:43:46.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/600: rmdir d6/d1e/d2b/d5a 39 2024-06-06T08:43:46.803 INFO:tasks.workunit.client.0.smithi027.stdout:4/651: creat d0/d1d/d34/d7f/d97/d9c/fd1 x:0 0 0 2024-06-06T08:43:46.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/542: creat d15/d18/d23/d28/d2a/d30/d5c/fb8 x:0 0 0 2024-06-06T08:43:46.804 INFO:tasks.workunit.client.0.smithi027.stdout:0/543: getdents d15/d18/d40/d8c/d9c 0 2024-06-06T08:43:46.804 INFO:tasks.workunit.client.0.smithi027.stdout:0/544: chown d15/d18/d23/d28/d2a/d30/f71 363 1 2024-06-06T08:43:46.805 INFO:tasks.workunit.client.0.smithi027.stdout:0/545: truncate d15/d18/d23/d5b/f6b 4496881 0 2024-06-06T08:43:46.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/652: readlink d0/d1/d6/l65 0 2024-06-06T08:43:46.807 INFO:tasks.workunit.client.0.smithi027.stdout:0/546: mknod d15/d18/d23/d28/d2a/cb9 0 2024-06-06T08:43:46.807 INFO:tasks.workunit.client.0.smithi027.stdout:9/519: write d3/d4e/d75/dc/f2d [3788708,112660] 0 2024-06-06T08:43:46.808 INFO:tasks.workunit.client.0.smithi027.stdout:2/601: mkdir d6/d1e/d2b/dc5 0 2024-06-06T08:43:46.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/653: stat d0/d1/d6/d8/f6a 0 2024-06-06T08:43:46.809 INFO:tasks.workunit.client.0.smithi027.stdout:6/648: dwrite d4/d1a/d2e/d44/f99 [0,4194304] 0 2024-06-06T08:43:46.810 INFO:tasks.workunit.client.0.smithi027.stdout:6/649: fsync d4/d1a/d2e/d44/f5a 0 2024-06-06T08:43:46.811 INFO:tasks.workunit.client.0.smithi027.stdout:8/518: creat d7/dc/d36/d83/d60/d93/dac/fb2 x:0 0 0 2024-06-06T08:43:46.813 INFO:tasks.workunit.client.0.smithi027.stdout:8/519: chown d7/dc/f12 3932605 1 2024-06-06T08:43:46.814 INFO:tasks.workunit.client.0.smithi027.stdout:4/654: truncate d0/d1d/d34/d7f/d97/d9c/dbe/fc3 214768 0 2024-06-06T08:43:46.816 INFO:tasks.workunit.client.0.smithi027.stdout:4/655: read - d0/d1/d6/d8/d82/d8b/dc9/fc8 zero size 2024-06-06T08:43:46.816 INFO:tasks.workunit.client.0.smithi027.stdout:8/520: chown d7/d13/f1e 0 1 2024-06-06T08:43:46.817 INFO:tasks.workunit.client.0.smithi027.stdout:8/521: chown d7/dc/d36 6172 1 2024-06-06T08:43:46.817 INFO:tasks.workunit.client.0.smithi027.stdout:9/520: link d3/d95/d23/l3c d3/d95/d23/d36/lb8 0 2024-06-06T08:43:46.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/602: mknod d6/dc/d3f/d77/d55/cc6 0 2024-06-06T08:43:46.818 INFO:tasks.workunit.client.0.smithi027.stdout:9/521: fdatasync d3/d4e/d75/dc/d9b/f91 0 2024-06-06T08:43:46.821 INFO:tasks.workunit.client.0.smithi027.stdout:3/481: dwrite d1/d8/d22/d27/d50/f51 [0,4194304] 0 2024-06-06T08:43:46.821 INFO:tasks.workunit.client.0.smithi027.stdout:9/522: rename d3/d95/d23/d36/f6c to d3/d95/d23/da9/fb9 0 2024-06-06T08:43:46.821 INFO:tasks.workunit.client.0.smithi027.stdout:9/523: dread - d3/d95/d23/d36/d3a/fac zero size 2024-06-06T08:43:46.822 INFO:tasks.workunit.client.0.smithi027.stdout:8/522: symlink d7/dc/d36/d83/d60/d93/lb3 0 2024-06-06T08:43:46.824 INFO:tasks.workunit.client.0.smithi027.stdout:9/524: creat d3/d95/fba x:0 0 0 2024-06-06T08:43:46.825 INFO:tasks.workunit.client.0.smithi027.stdout:8/523: creat d7/fb4 x:0 0 0 2024-06-06T08:43:46.827 INFO:tasks.workunit.client.0.smithi027.stdout:2/603: mknod d6/d1e/d2b/d5a/d7b/cc7 0 2024-06-06T08:43:46.827 INFO:tasks.workunit.client.0.smithi027.stdout:6/650: dwrite d4/de/f8c [0,4194304] 0 2024-06-06T08:43:46.827 INFO:tasks.workunit.client.0.smithi027.stdout:6/651: write d4/d1a/f2c [72779,96978] 0 2024-06-06T08:43:46.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/482: symlink d1/d8/d22/l91 0 2024-06-06T08:43:46.829 INFO:tasks.workunit.client.0.smithi027.stdout:9/525: fdatasync d3/f4 0 2024-06-06T08:43:46.830 INFO:tasks.workunit.client.0.smithi027.stdout:9/526: creat d3/d4e/d75/db0/fbb x:0 0 0 2024-06-06T08:43:46.830 INFO:tasks.workunit.client.0.smithi027.stdout:9/527: chown d3/d4e/lb7 15 1 2024-06-06T08:43:46.830 INFO:tasks.workunit.client.0.smithi027.stdout:2/604: getdents d6/dc/d3f/d77 0 2024-06-06T08:43:46.831 INFO:tasks.workunit.client.0.smithi027.stdout:0/547: dwrite d15/d18/d40/d8c/d9c/fad [0,4194304] 0 2024-06-06T08:43:46.831 INFO:tasks.workunit.client.0.smithi027.stdout:4/656: getdents d0/d1d/d60/dc6/d6e/db6 0 2024-06-06T08:43:46.831 INFO:tasks.workunit.client.0.smithi027.stdout:4/657: truncate d0/f66 5007004 0 2024-06-06T08:43:46.832 INFO:tasks.workunit.client.0.smithi027.stdout:9/528: symlink d3/d95/d34/d73/lbc 0 2024-06-06T08:43:46.834 INFO:tasks.workunit.client.0.smithi027.stdout:3/483: creat d1/d8/f92 x:0 0 0 2024-06-06T08:43:46.834 INFO:tasks.workunit.client.0.smithi027.stdout:3/484: chown d1/d1f/d5b/l83 120505 1 2024-06-06T08:43:46.836 INFO:tasks.workunit.client.0.smithi027.stdout:3/485: chown d1/d8/d22/d27/c1b 10176258 1 2024-06-06T08:43:46.839 INFO:tasks.workunit.client.0.smithi027.stdout:8/524: rename d7/l9a to d7/dc/d36/d83/d60/lb5 0 2024-06-06T08:43:46.841 INFO:tasks.workunit.client.0.smithi027.stdout:3/486: readlink d1/d8/l28 0 2024-06-06T08:43:46.845 INFO:tasks.workunit.client.0.smithi027.stdout:0/548: dread f2 [0,4194304] 0 2024-06-06T08:43:46.846 INFO:tasks.workunit.client.0.smithi027.stdout:6/652: mkdir d4/d9/dc/d1c/dd2 0 2024-06-06T08:43:46.846 INFO:tasks.workunit.client.0.smithi027.stdout:2/605: dread d6/dc/f44 [0,4194304] 0 2024-06-06T08:43:46.846 INFO:tasks.workunit.client.0.smithi027.stdout:8/525: dread d7/dc/d36/d83/fa4 [0,4194304] 0 2024-06-06T08:43:46.848 INFO:tasks.workunit.client.0.smithi027.stdout:3/487: write d1/d8/f61 [585121,52711] 0 2024-06-06T08:43:46.848 INFO:tasks.workunit.client.0.smithi027.stdout:6/653: rmdir d4/d9/dc/d8f 39 2024-06-06T08:43:46.849 INFO:tasks.workunit.client.0.smithi027.stdout:3/488: truncate d1/f85 2507313 0 2024-06-06T08:43:46.853 INFO:tasks.workunit.client.0.smithi027.stdout:3/489: mknod d1/d90/d44/c93 0 2024-06-06T08:43:46.854 INFO:tasks.workunit.client.0.smithi027.stdout:9/529: write d3/d95/f2f [734346,85213] 0 2024-06-06T08:43:46.854 INFO:tasks.workunit.client.0.smithi027.stdout:4/658: rename d0/d1/ce to d0/d1/d6/cd2 0 2024-06-06T08:43:46.854 INFO:tasks.workunit.client.0.smithi027.stdout:4/659: fsync d0/d1d/d34/d7f/f94 0 2024-06-06T08:43:46.855 INFO:tasks.workunit.client.0.smithi027.stdout:9/530: creat d3/d95/d23/d36/fbd x:0 0 0 2024-06-06T08:43:46.855 INFO:tasks.workunit.client.0.smithi027.stdout:3/490: fsync d1/d90/d44/f74 0 2024-06-06T08:43:46.857 INFO:tasks.workunit.client.0.smithi027.stdout:9/531: symlink d3/d95/d34/d41/lbe 0 2024-06-06T08:43:46.858 INFO:tasks.workunit.client.0.smithi027.stdout:3/491: truncate d1/d8/d22/d27/f36 1514689 0 2024-06-06T08:43:46.858 INFO:tasks.workunit.client.0.smithi027.stdout:3/492: chown d1/d1f/d5b/l88 82 1 2024-06-06T08:43:46.858 INFO:tasks.workunit.client.0.smithi027.stdout:4/660: symlink d0/ld3 0 2024-06-06T08:43:46.863 INFO:tasks.workunit.client.0.smithi027.stdout:4/661: readlink d0/d1d/d34/l3f 0 2024-06-06T08:43:46.864 INFO:tasks.workunit.client.0.smithi027.stdout:9/532: creat d3/d76/fbf x:0 0 0 2024-06-06T08:43:46.864 INFO:tasks.workunit.client.0.smithi027.stdout:9/533: chown d3/d4e/d75/l45 567688799 1 2024-06-06T08:43:46.865 INFO:tasks.workunit.client.0.smithi027.stdout:9/534: fdatasync d3/d95/d23/d36/f5a 0 2024-06-06T08:43:46.866 INFO:tasks.workunit.client.0.smithi027.stdout:2/606: dwrite d6/fbd [0,4194304] 0 2024-06-06T08:43:46.871 INFO:tasks.workunit.client.0.smithi027.stdout:8/526: write d7/dc/d36/d83/d60/fad [994087,36255] 0 2024-06-06T08:43:46.872 INFO:tasks.workunit.client.0.smithi027.stdout:2/607: truncate d6/dc/f12 1123714 0 2024-06-06T08:43:46.873 INFO:tasks.workunit.client.0.smithi027.stdout:4/662: dread d0/d1d/d34/d7f/fcd [0,4194304] 0 2024-06-06T08:43:46.876 INFO:tasks.workunit.client.0.smithi027.stdout:8/527: symlink d7/dc/d36/d83/d58/lb6 0 2024-06-06T08:43:46.880 INFO:tasks.workunit.client.0.smithi027.stdout:4/663: unlink d0/d63/cb5 0 2024-06-06T08:43:46.880 INFO:tasks.workunit.client.0.smithi027.stdout:4/664: write d0/d1/d6/d8/f6a [4388685,80838] 0 2024-06-06T08:43:46.891 INFO:tasks.workunit.client.0.smithi027.stdout:6/654: rename d4/d13/f92 to d4/d9/dc/fd3 0 2024-06-06T08:43:46.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:46 smithi203 ceph-mon[36579]: pgmap v108: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 767 op/s 2024-06-06T08:43:47.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:46 smithi176 ceph-mon[36786]: pgmap v108: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 767 op/s 2024-06-06T08:43:47.111 INFO:tasks.workunit.client.0.smithi027.stdout:8/528: dwrite d7/dc/d36/d83/fa4 [0,4194304] 0 2024-06-06T08:43:47.112 INFO:tasks.workunit.client.0.smithi027.stdout:6/655: read d4/da2/fa6 [147918,33207] 0 2024-06-06T08:43:47.113 INFO:tasks.workunit.client.0.smithi027.stdout:2/608: dwrite d6/d1e/d2b/f91 [0,4194304] 0 2024-06-06T08:43:47.115 INFO:tasks.workunit.client.0.smithi027.stdout:6/656: mkdir d4/d9/d6e/dd4 0 2024-06-06T08:43:47.116 INFO:tasks.workunit.client.0.smithi027.stdout:2/609: creat d6/dd/d6b/fc8 x:0 0 0 2024-06-06T08:43:47.116 INFO:tasks.workunit.client.0.smithi027.stdout:2/610: write d6/dd/d6b/fc8 [993433,48033] 0 2024-06-06T08:43:47.117 INFO:tasks.workunit.client.0.smithi027.stdout:6/657: getdents d4/de/d45 0 2024-06-06T08:43:47.118 INFO:tasks.workunit.client.0.smithi027.stdout:3/493: read d1/f85 [2481369,34762] 0 2024-06-06T08:43:47.119 INFO:tasks.workunit.client.0.smithi027.stdout:6/658: creat d4/d1a/d2e/d44/dce/fd5 x:0 0 0 2024-06-06T08:43:47.119 INFO:tasks.workunit.client.0.smithi027.stdout:6/659: write d4/d1a/d2e/d44/dce/fd5 [930404,111264] 0 2024-06-06T08:43:47.120 INFO:tasks.workunit.client.0.smithi027.stdout:6/660: mknod d4/de/d45/dcf/cd6 0 2024-06-06T08:43:47.121 INFO:tasks.workunit.client.0.smithi027.stdout:2/611: write d6/dc/d3f/d48/f62 [4332925,85439] 0 2024-06-06T08:43:47.121 INFO:tasks.workunit.client.0.smithi027.stdout:6/661: mknod d4/d9/dc5/cd7 0 2024-06-06T08:43:47.122 INFO:tasks.workunit.client.0.smithi027.stdout:2/612: truncate d6/f14 3925419 0 2024-06-06T08:43:47.123 INFO:tasks.workunit.client.0.smithi027.stdout:6/662: creat d4/d1a/d2e/d44/fd8 x:0 0 0 2024-06-06T08:43:47.123 INFO:tasks.workunit.client.0.smithi027.stdout:6/663: truncate d4/d9/dc/f9f 828432 0 2024-06-06T08:43:47.123 INFO:tasks.workunit.client.0.smithi027.stdout:2/613: readlink d6/l31 0 2024-06-06T08:43:47.124 INFO:tasks.workunit.client.0.smithi027.stdout:2/614: write d6/d3b/f43 [1798672,98325] 0 2024-06-06T08:43:47.128 INFO:tasks.workunit.client.0.smithi027.stdout:2/615: mknod d6/dc/d1d/dbc/cc9 0 2024-06-06T08:43:47.128 INFO:tasks.workunit.client.0.smithi027.stdout:6/664: write d4/d1a/d2e/f3c [1774777,29364] 0 2024-06-06T08:43:47.130 INFO:tasks.workunit.client.0.smithi027.stdout:3/494: write d1/d1f/d25/f76 [1282721,46095] 0 2024-06-06T08:43:47.131 INFO:tasks.workunit.client.0.smithi027.stdout:6/665: unlink d4/d9/d49/f62 0 2024-06-06T08:43:47.131 INFO:tasks.workunit.client.0.smithi027.stdout:6/666: chown d4/d9/d49/f71 443247 1 2024-06-06T08:43:47.131 INFO:tasks.workunit.client.0.smithi027.stdout:6/667: truncate d4/da2/fa6 1016928 0 2024-06-06T08:43:47.132 INFO:tasks.workunit.client.0.smithi027.stdout:2/616: unlink d6/dc/d3f/d48/f93 0 2024-06-06T08:43:47.132 INFO:tasks.workunit.client.0.smithi027.stdout:3/495: chown d1/d90/l47 1831335 1 2024-06-06T08:43:47.132 INFO:tasks.workunit.client.0.smithi027.stdout:3/496: write d1/d8/d22/d27/f36 [2387306,36978] 0 2024-06-06T08:43:47.134 INFO:tasks.workunit.client.0.smithi027.stdout:3/497: creat d1/d8/d59/f94 x:0 0 0 2024-06-06T08:43:47.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/529: dwrite d7/f22 [0,4194304] 0 2024-06-06T08:43:47.138 INFO:tasks.workunit.client.0.smithi027.stdout:3/498: rmdir d1/d8/d22/d27/d50 39 2024-06-06T08:43:47.140 INFO:tasks.workunit.client.0.smithi027.stdout:3/499: stat d1/d1f/f6f 0 2024-06-06T08:43:47.141 INFO:tasks.workunit.client.0.smithi027.stdout:3/500: getdents d1/d90/d44/d54 0 2024-06-06T08:43:47.142 INFO:tasks.workunit.client.0.smithi027.stdout:3/501: creat d1/d8/d22/f95 x:0 0 0 2024-06-06T08:43:47.144 INFO:tasks.workunit.client.0.smithi027.stdout:8/530: mkdir d7/dc/d36/d41/d85/db7 0 2024-06-06T08:43:47.144 INFO:tasks.workunit.client.0.smithi027.stdout:8/531: fsync d7/d2f/f87 0 2024-06-06T08:43:47.144 INFO:tasks.workunit.client.0.smithi027.stdout:3/502: symlink d1/d90/d44/l96 0 2024-06-06T08:43:47.146 INFO:tasks.workunit.client.0.smithi027.stdout:3/503: mknod d1/d90/d44/c97 0 2024-06-06T08:43:47.157 INFO:tasks.workunit.client.0.smithi027.stdout:8/532: dwrite d7/d2f/f77 [0,4194304] 0 2024-06-06T08:43:47.160 INFO:tasks.workunit.client.0.smithi027.stdout:8/533: creat d7/d2f/fb8 x:0 0 0 2024-06-06T08:43:47.160 INFO:tasks.workunit.client.0.smithi027.stdout:8/534: write d7/d2f/f98 [990197,37271] 0 2024-06-06T08:43:47.161 INFO:tasks.workunit.client.0.smithi027.stdout:3/504: dwrite d1/d8/f55 [0,4194304] 0 2024-06-06T08:43:47.164 INFO:tasks.workunit.client.0.smithi027.stdout:3/505: creat d1/d90/d44/d54/f98 x:0 0 0 2024-06-06T08:43:47.164 INFO:tasks.workunit.client.0.smithi027.stdout:3/506: write d1/d90/d6d/f86 [806411,22564] 0 2024-06-06T08:43:47.176 INFO:tasks.workunit.client.0.smithi027.stdout:8/535: dwrite d7/dc/d36/d83/d60/f78 [0,4194304] 0 2024-06-06T08:43:47.178 INFO:tasks.workunit.client.0.smithi027.stdout:8/536: symlink d7/dc/d36/d83/d60/d93/lb9 0 2024-06-06T08:43:47.180 INFO:tasks.workunit.client.0.smithi027.stdout:3/507: dwrite d1/d1f/f6f [0,4194304] 0 2024-06-06T08:43:47.180 INFO:tasks.workunit.client.0.smithi027.stdout:3/508: chown d1/d8/d22 562732 1 2024-06-06T08:43:47.182 INFO:tasks.workunit.client.0.smithi027.stdout:3/509: creat d1/d8/d22/d2d/f99 x:0 0 0 2024-06-06T08:43:47.184 INFO:tasks.workunit.client.0.smithi027.stdout:8/537: write d7/dc/d36/d83/d60/f81 [550206,85235] 0 2024-06-06T08:43:47.186 INFO:tasks.workunit.client.0.smithi027.stdout:8/538: symlink d7/d9c/lba 0 2024-06-06T08:43:47.188 INFO:tasks.workunit.client.0.smithi027.stdout:8/539: symlink d7/dc/d36/d3c/lbb 0 2024-06-06T08:43:47.189 INFO:tasks.workunit.client.0.smithi027.stdout:8/540: chown d7/dc/d36/d83/f11 3 1 2024-06-06T08:43:47.190 INFO:tasks.workunit.client.0.smithi027.stdout:8/541: rmdir d7/d2f 39 2024-06-06T08:43:47.191 INFO:tasks.workunit.client.0.smithi027.stdout:8/542: chown d7/dc/d36/d83/d60/lb5 5575728 1 2024-06-06T08:43:47.191 INFO:tasks.workunit.client.0.smithi027.stdout:8/543: chown d7/dc/d36/d3c/c53 1 1 2024-06-06T08:43:47.194 INFO:tasks.workunit.client.0.smithi027.stdout:8/544: unlink d7/d2f/ca8 0 2024-06-06T08:43:47.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/545: dread d7/dc/d36/d83/d60/f78 [0,4194304] 0 2024-06-06T08:43:47.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/546: write d7/dc/d37/d46/f8d [4863223,12012] 0 2024-06-06T08:43:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.208+0000 7fe35d3ab640 1 -- 172.21.15.27:0/2467352616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe358106890 msgr2=0x7fe358106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.208+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/2467352616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe358106890 0x7fe358106c70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe348009930 tx=0x7fe34802a420 comp rx=0 tx=0).stop 2024-06-06T08:43:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.208+0000 7fe35d3ab640 1 -- 172.21.15.27:0/2467352616 shutdown_connections 2024-06-06T08:43:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.208+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/2467352616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358101290 0x7fe35810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.208+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/2467352616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe3581008d0 0x7fe358100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.208+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/2467352616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe358106890 0x7fe358106c70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.208+0000 7fe35d3ab640 1 -- 172.21.15.27:0/2467352616 >> 172.21.15.27:0/2467352616 conn(0x7fe358077a20 msgr2=0x7fe358077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.208+0000 7fe35d3ab640 1 -- 172.21.15.27:0/2467352616 shutdown_connections 2024-06-06T08:43:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.208+0000 7fe35d3ab640 1 -- 172.21.15.27:0/2467352616 wait complete. 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe35d3ab640 1 Processor -- start 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe35d3ab640 1 -- start start 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe35d3ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3581008d0 0x7fe358198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe35d3ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358101290 0x7fe358198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe35d3ab640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe358106890 0x7fe35819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe35d3ab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe358110c60 con 0x7fe3581008d0 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe35d3ab640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe358110ae0 con 0x7fe358106890 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe35d3ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe358110de0 con 0x7fe358101290 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3581008d0 0x7fe358198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3581008d0 0x7fe358198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54164/0 (socket says 172.21.15.27:54164) 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.209+0000 7fe356ffd640 1 -- 172.21.15.27:0/1724351170 learned_addr learned my addr 172.21.15.27:0/1724351170 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358101290 0x7fe358198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe3577fe640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe358106890 0x7fe35819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe356ffd640 1 -- 172.21.15.27:0/1724351170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358101290 msgr2=0x7fe358198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358101290 0x7fe358198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe356ffd640 1 -- 172.21.15.27:0/1724351170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe358106890 msgr2=0x7fe35819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe358106890 0x7fe35819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe356ffd640 1 -- 172.21.15.27:0/1724351170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35819d1f0 con 0x7fe3581008d0 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358101290 0x7fe358198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3581008d0 0x7fe358198200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe348009230 tx=0x7fe348004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe354ff9640 1 -- 172.21.15.27:0/1724351170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34803f030 con 0x7fe3581008d0 2024-06-06T08:43:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe35d3ab640 1 -- 172.21.15.27:0/1724351170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35819d480 con 0x7fe3581008d0 2024-06-06T08:43:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe354ff9640 1 -- 172.21.15.27:0/1724351170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe348003ad0 con 0x7fe3581008d0 2024-06-06T08:43:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe354ff9640 1 -- 172.21.15.27:0/1724351170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348035070 con 0x7fe3581008d0 2024-06-06T08:43:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.210+0000 7fe35d3ab640 1 -- 172.21.15.27:0/1724351170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3581a4d70 con 0x7fe3581008d0 2024-06-06T08:43:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.213+0000 7fe354ff9640 1 -- 172.21.15.27:0/1724351170 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe348042030 con 0x7fe3581008d0 2024-06-06T08:43:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.213+0000 7fe354ff9640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3440a2da0 0x7fe3440a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.214+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3440a2da0 0x7fe3440a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.214+0000 7fe354ff9640 1 -- 172.21.15.27:0/1724351170 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fe3480eff00 con 0x7fe3581008d0 2024-06-06T08:43:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.214+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3440a2da0 0x7fe3440a5260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe3400058c0 tx=0x7fe340005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.214+0000 7fe36018b740 1 Processor -- start 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.214+0000 7fe36018b740 1 -- start start 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe36018b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcb3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe36018b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe36018b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcbe350 0x557a1bcc0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe36018b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557a1bc174f0 con 0x557a1bcb3170 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe36018b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557a1bbb5870 con 0x557a1bcb3a90 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe36018b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557a1bc134b0 con 0x557a1bcbe350 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcb3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcb3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54168/0 (socket says 172.21.15.27:54168) 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe356ffd640 1 -- 172.21.15.27:0/1551553044 learned_addr learned my addr 172.21.15.27:0/1551553044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe3577fe640 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcbe350 0x557a1bcc0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe356ffd640 1 -- 172.21.15.27:0/1551553044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcbe350 msgr2=0x557a1bcc0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcbe350 0x557a1bcc0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe356ffd640 1 -- 172.21.15.27:0/1551553044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 msgr2=0x557a1bcbde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe356ffd640 1 -- 172.21.15.27:0/1551553044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1bbd19b0 con 0x557a1bcb3170 2024-06-06T08:43:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:43:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.215+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcb3550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe3480b8a30 tx=0x7fe3480bc820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3417cd833d0c6b3f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.216+0000 7fe34d7fa640 1 -- 172.21.15.27:0/1551553044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348049040 con 0x557a1bcb3170 2024-06-06T08:43:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.216+0000 7fe34d7fa640 1 -- 172.21.15.27:0/1551553044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3480c4cd0 con 0x557a1bcb3170 2024-06-06T08:43:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.216+0000 7fe34d7fa640 1 -- 172.21.15.27:0/1551553044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3480c01d0 con 0x557a1bcb3170 2024-06-06T08:43:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.216+0000 7fe36018b740 1 -- 172.21.15.27:0/1551553044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 msgr2=0x557a1bcb3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.216+0000 7fe36018b740 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcb3550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe3480b8a30 tx=0x7fe3480bc820 comp rx=0 tx=0).stop 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 -- 172.21.15.27:0/1551553044 shutdown_connections 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcbe350 0x557a1bcc0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 --2- 172.21.15.27:0/1551553044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcb3550 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 -- 172.21.15.27:0/1551553044 >> 172.21.15.27:0/1551553044 conn(0x557a1bcaf680 msgr2=0x557a1bcafa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 -- 172.21.15.27:0/1551553044 shutdown_connections 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 -- 172.21.15.27:0/1551553044 wait complete. 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 -- start start 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 Processor -- start 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcbb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3a90 0x557a1bcbb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 0x557a1bcbbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557a1bcc0d40 con 0x557a1bcb3170 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557a1bba0ae0 con 0x557a1bcbe350 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557a1bcc0ec0 con 0x557a1bcb3a90 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe36018b740 10 client.0 _ll_register_callbacks cb 0x557a1bbd99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.217+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcbb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcbb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54178/0 (socket says 172.21.15.27:54178) 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe356ffd640 1 -- 172.21.15.27:0/3012159403 learned_addr learned my addr 172.21.15.27:0/3012159403 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe34ffff640 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3a90 0x557a1bcbb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe3577fe640 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 0x557a1bcbbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe356ffd640 1 -- 172.21.15.27:0/3012159403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3a90 msgr2=0x557a1bcbb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe356ffd640 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3a90 0x557a1bcbb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe356ffd640 1 -- 172.21.15.27:0/3012159403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 msgr2=0x557a1bcbbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe356ffd640 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 0x557a1bcbbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe356ffd640 1 -- 172.21.15.27:0/3012159403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1b92b210 con 0x557a1bcb3170 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe356ffd640 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcbb310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe3480f0220 tx=0x7fe348005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921e639bc9c40085 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe33e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe33e7fc640 1 -- 172.21.15.27:0/3012159403 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3480d4070 con 0x557a1bcb3170 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe33e7fc640 1 -- 172.21.15.27:0/3012159403 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fe34803bd60 con 0x557a1bcb3170 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe36018b740 1 -- 172.21.15.27:0/3012159403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3480c4cd0 con 0x557a1bcb3170 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe33e7fc640 1 -- 172.21.15.27:0/3012159403 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe348049030 con 0x557a1bcb3170 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe36018b740 10 client.16269 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.218+0000 7fe36018b740 1 -- 172.21.15.27:0/3012159403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a1b9f7620 con 0x557a1bcb3170 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.219+0000 7fe33e7fc640 1 -- 172.21.15.27:0/3012159403 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3480c4cd0 con 0x557a1bcb3170 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.219+0000 7fe33e7fc640 1 -- 172.21.15.27:0/3012159403 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe3480b86d0 con 0x557a1bcb3170 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.219+0000 7fe36018b740 10 client.16269 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.219+0000 7fe36018b740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.219+0000 7fe36018b740 20 client.16269 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.219+0000 7fe36018b740 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a1bcce0c0 0x557a1bcd04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.219+0000 7fe36018b740 4 client.16269 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.219+0000 7fe36018b740 1 -- 172.21.15.27:0/3012159403 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a1bc2af10 con 0x557a1bcce0c0 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.219+0000 7fe34ffff640 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a1bcce0c0 0x557a1bcd04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.220+0000 7fe34ffff640 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a1bcce0c0 0x557a1bcd04b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e110a7ec97dd40 server_cookie=71a912e25f8b19ce in_seq=0 out_seq=0 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.220+0000 7fe33e7fc640 10 client.16269 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.220+0000 7fe33e7fc640 1 -- 172.21.15.27:0/3012159403 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557a1bc2af10 con 0x557a1bcce0c0 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe33e7fc640 10 client.16269 handle_command_reply: tid=0 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 client.16269 shutdown 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 -- 172.21.15.27:0/3012159403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 msgr2=0x557a1bcbb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcbb310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe3480f0220 tx=0x7fe348005b70 comp rx=0 tx=0).stop 2024-06-06T08:43:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 -- 172.21.15.27:0/3012159403 shutdown_connections 2024-06-06T08:43:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a1bcce0c0 0x557a1bcd04b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 0x557a1bcbbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe33e7fc640 0 client.16269 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3a90 0x557a1bcbb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 --2- 172.21.15.27:0/3012159403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3170 0x557a1bcbb310 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 -- 172.21.15.27:0/3012159403 >> 172.21.15.27:0/3012159403 conn(0x557a1bcaf680 msgr2=0x557a1bcb5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 -- 172.21.15.27:0/3012159403 shutdown_connections 2024-06-06T08:43:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 1 -- 172.21.15.27:0/3012159403 wait complete. 2024-06-06T08:43:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.221+0000 7fe36018b740 20 client.16269 trim_cache size 0 max 16384 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.237+0000 7fe36018b740 1 Processor -- start 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.237+0000 7fe36018b740 1 -- start start 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.237+0000 7fe36018b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 0x557a1bcb5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.237+0000 7fe36018b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.237+0000 7fe36018b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcbe350 0x557a1bcd5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.237+0000 7fe36018b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3581b17c0 con 0x557a1bcbe350 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.237+0000 7fe36018b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe3581aefd0 con 0x557a1bcb3a90 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.237+0000 7fe36018b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3581ace20 con 0x557a1bcb3170 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe3577fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcbe350 0x557a1bcd5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe3577fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcbe350 0x557a1bcd5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54188/0 (socket says 172.21.15.27:54188) 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe3577fe640 1 -- 172.21.15.27:0/1397068507 learned_addr learned my addr 172.21.15.27:0/1397068507 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 0x557a1bcb5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe356ffd640 1 -- 172.21.15.27:0/1397068507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 msgr2=0x557a1bcbd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe356ffd640 1 -- 172.21.15.27:0/1397068507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcbe350 msgr2=0x557a1bcd5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcbe350 0x557a1bcd5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe356ffd640 1 -- 172.21.15.27:0/1397068507 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1bcb38c0 con 0x557a1bcb3170 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 0x557a1bcb5b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3480ed890 tx=0x7fe34802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f04f208c748b16bf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe34cff9640 1 -- 172.21.15.27:0/1397068507 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3480d4070 con 0x557a1bcb3170 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe34cff9640 1 -- 172.21.15.27:0/1397068507 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3480c4a80 con 0x557a1bcb3170 2024-06-06T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.238+0000 7fe34cff9640 1 -- 172.21.15.27:0/1397068507 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348003c80 con 0x557a1bcb3170 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 -- 172.21.15.27:0/1397068507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 msgr2=0x557a1bcb5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 0x557a1bcb5b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3480ed890 tx=0x7fe34802a420 comp rx=0 tx=0).stop 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 -- 172.21.15.27:0/1397068507 shutdown_connections 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcbe350 0x557a1bcd5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcb3a90 0x557a1bcbd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 --2- 172.21.15.27:0/1397068507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 0x557a1bcb5b30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 -- 172.21.15.27:0/1397068507 >> 172.21.15.27:0/1397068507 conn(0x557a1bcaf680 msgr2=0x557a1bcbcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 -- 172.21.15.27:0/1397068507 shutdown_connections 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 -- 172.21.15.27:0/1397068507 wait complete. 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.239+0000 7fe36018b740 1 -- start start 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe36018b740 1 Processor -- start 2024-06-06T08:43:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe36018b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 0x557a1bcc1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe36018b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3a90 0x557a1bcc1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe36018b740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 0x557a1bd36cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe36018b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3581ad490 con 0x557a1bcb3a90 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe36018b740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe3581ae900 con 0x557a1bcbe350 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe36018b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3581adb60 con 0x557a1bcb3170 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe36018b740 10 client.0 _ll_register_callbacks cb 0x7fe350009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe34ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3a90 0x557a1bcc1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe34ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3a90 0x557a1bcc1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54204/0 (socket says 172.21.15.27:54204) 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe34ffff640 1 -- 172.21.15.27:0/1678890955 learned_addr learned my addr 172.21.15.27:0/1678890955 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 0x557a1bcc1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe3577fe640 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 0x557a1bd36cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe34ffff640 1 -- 172.21.15.27:0/1678890955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 msgr2=0x557a1bcc1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 0x557a1bcc1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe34ffff640 1 -- 172.21.15.27:0/1678890955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 msgr2=0x557a1bd36cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 0x557a1bd36cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe34ffff640 1 -- 172.21.15.27:0/1678890955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1bd37200 con 0x557a1bcb3a90 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe34ffff640 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3a90 0x557a1bcc1990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe340014040 tx=0x7fe340004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3f1a39e0aaa5ef server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.240+0000 7fe33e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe33e7fc640 1 -- 172.21.15.27:0/1678890955 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34002f070 con 0x557a1bcb3a90 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe33e7fc640 1 -- 172.21.15.27:0/1678890955 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fe340004220 con 0x557a1bcb3a90 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe36018b740 1 -- 172.21.15.27:0/1678890955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3480c4a80 con 0x557a1bcb3a90 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe33e7fc640 1 -- 172.21.15.27:0/1678890955 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe34003a050 con 0x557a1bcb3a90 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe36018b740 10 client.16275 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe36018b740 1 -- 172.21.15.27:0/1678890955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a1bd38440 con 0x557a1bcb3a90 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe33e7fc640 1 -- 172.21.15.27:0/1678890955 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3480c4a80 con 0x557a1bcb3a90 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe33e7fc640 1 -- 172.21.15.27:0/1678890955 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe340003950 con 0x557a1bcb3a90 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe36018b740 10 client.16275 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe36018b740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe36018b740 20 client.16275 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe36018b740 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a1bcce0c0 0x557a1bd3ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe36018b740 4 client.16275 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:43:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.241+0000 7fe36018b740 1 -- 172.21.15.27:0/1678890955 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a1bc24c20 con 0x557a1bcce0c0 2024-06-06T08:43:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.242+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a1bcce0c0 0x557a1bd3ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.242+0000 7fe33e7fc640 10 client.16275 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.242+0000 7fe356ffd640 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a1bcce0c0 0x557a1bd3ac90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f97f39f43996971 server_cookie=2c043002ddca279 in_seq=0 out_seq=0 2024-06-06T08:43:48.592 INFO:tasks.workunit.client.0.smithi027.stdout:6/668: rename d4/d9/dc/d18/f2f to d4/da2/fd9 0 2024-06-06T08:43:48.594 INFO:tasks.workunit.client.0.smithi027.stdout:6/669: truncate d4/d1a/d2e/d44/f74 3686521 0 2024-06-06T08:43:48.596 INFO:tasks.workunit.client.0.smithi027.stdout:2/617: rename d6/d1e/d2b/d5a/l81 to d6/d1e/d2b/d5a/db7/lca 0 2024-06-06T08:43:48.603 INFO:tasks.workunit.client.0.smithi027.stdout:6/670: symlink d4/lda 0 2024-06-06T08:43:48.605 INFO:tasks.workunit.client.0.smithi027.stdout:6/671: mknod d4/d1a/d2e/d44/cdb 0 2024-06-06T08:43:48.606 INFO:tasks.workunit.client.0.smithi027.stdout:2/618: dread d6/dd/f24 [0,4194304] 0 2024-06-06T08:43:48.609 INFO:tasks.workunit.client.0.smithi027.stdout:2/619: link d6/dc/d3f/d48/f62 d6/dc/d3f/db0/fcb 0 2024-06-06T08:43:48.611 INFO:tasks.workunit.client.0.smithi027.stdout:2/620: creat d6/dc/d3f/db0/fcc x:0 0 0 2024-06-06T08:43:48.615 INFO:tasks.workunit.client.0.smithi027.stdout:6/672: dread d4/d9/dc/d1c/f9e [0,4194304] 0 2024-06-06T08:43:48.616 INFO:tasks.workunit.client.0.smithi027.stdout:8/547: rename d7/dc/d36/d83/d60/f81 to d7/dc/d36/d83/fbc 0 2024-06-06T08:43:48.619 INFO:tasks.workunit.client.0.smithi027.stdout:6/673: mkdir d4/de/d45/dcf/ddc 0 2024-06-06T08:43:48.622 INFO:tasks.workunit.client.0.smithi027.stdout:6/674: creat d4/d13/dba/fdd x:0 0 0 2024-06-06T08:43:48.625 INFO:tasks.workunit.client.0.smithi027.stdout:8/548: dread d7/dc/d36/d83/f19 [0,4194304] 0 2024-06-06T08:43:48.626 INFO:tasks.workunit.client.0.smithi027.stdout:8/549: write d7/dc/d36/d83/fbc [1062549,12270] 0 2024-06-06T08:43:48.631 INFO:tasks.workunit.client.0.smithi027.stdout:2/621: write d6/dc/d3f/d77/d55/d69/d82/fa3 [346643,95751] 0 2024-06-06T08:43:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.633+0000 7fe33e7fc640 1 -- 172.21.15.27:0/1678890955 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557a1bc24c20 con 0x557a1bcce0c0 2024-06-06T08:43:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.633+0000 7fe33e7fc640 10 client.16275 handle_command_reply: tid=0 2024-06-06T08:43:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.633+0000 7fe36018b740 1 client.16275 shutdown 2024-06-06T08:43:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.633+0000 7fe36018b740 1 -- 172.21.15.27:0/1678890955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3a90 msgr2=0x557a1bcc1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.633+0000 7fe36018b740 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3a90 0x557a1bcc1990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe340014040 tx=0x7fe340004730 comp rx=0 tx=0).stop 2024-06-06T08:43:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.633+0000 7fe36018b740 1 -- 172.21.15.27:0/1678890955 shutdown_connections 2024-06-06T08:43:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.633+0000 7fe36018b740 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a1bcce0c0 0x557a1bd3ac90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.634+0000 7fe33e7fc640 0 client.16275 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.634+0000 7fe36018b740 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a1bcbe350 0x557a1bd36cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.634+0000 7fe36018b740 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a1bcb3a90 0x557a1bcc1990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.634+0000 7fe36018b740 1 --2- 172.21.15.27:0/1678890955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a1bcb3170 0x557a1bcc1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.634+0000 7fe36018b740 1 -- 172.21.15.27:0/1678890955 >> 172.21.15.27:0/1678890955 conn(0x557a1bcaf680 msgr2=0x557a1bcbd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.634+0000 7fe36018b740 1 -- 172.21.15.27:0/1678890955 shutdown_connections 2024-06-06T08:43:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.634+0000 7fe36018b740 1 -- 172.21.15.27:0/1678890955 wait complete. 2024-06-06T08:43:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.634+0000 7fe36018b740 20 client.16275 trim_cache size 0 max 16384 2024-06-06T08:43:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:43:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:43:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:43:48.634 INFO:tasks.workunit.client.0.smithi027.stdout:5/646: sync 2024-06-06T08:43:48.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/497: sync 2024-06-06T08:43:48.634 INFO:tasks.workunit.client.0.smithi027.stdout:0/549: sync 2024-06-06T08:43:48.635 INFO:tasks.workunit.client.0.smithi027.stdout:9/535: sync 2024-06-06T08:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 -- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3440a2da0 msgr2=0x7fe3440a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3440a2da0 0x7fe3440a5260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe3400058c0 tx=0x7fe340005850 comp rx=0 tx=0).stop 2024-06-06T08:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 -- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3581008d0 msgr2=0x7fe358198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3581008d0 0x7fe358198200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe348009230 tx=0x7fe348004270 comp rx=0 tx=0).stop 2024-06-06T08:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 -- 172.21.15.27:0/1724351170 shutdown_connections 2024-06-06T08:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe3440a2da0 0x7fe3440a5260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe358106890 0x7fe35819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358101290 0x7fe358198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 --2- 172.21.15.27:0/1724351170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3581008d0 0x7fe358198200 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 -- 172.21.15.27:0/1724351170 >> 172.21.15.27:0/1724351170 conn(0x7fe358077a20 msgr2=0x7fe3580693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 -- 172.21.15.27:0/1724351170 shutdown_connections 2024-06-06T08:43:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:43:48.635+0000 7fe35d3ab640 1 -- 172.21.15.27:0/1724351170 wait complete. 2024-06-06T08:43:48.636 INFO:tasks.workunit.client.0.smithi027.stdout:3/510: sync 2024-06-06T08:43:48.636 INFO:tasks.workunit.client.0.smithi027.stdout:4/665: sync 2024-06-06T08:43:48.637 INFO:tasks.workunit.client.0.smithi027.stdout:7/717: sync 2024-06-06T08:43:48.637 INFO:tasks.workunit.client.0.smithi027.stdout:6/675: write d4/d9/dc/d1c/f50 [2495241,119751] 0 2024-06-06T08:43:48.637 INFO:tasks.workunit.client.0.smithi027.stdout:2/622: rename d6/d1e/l56 to d6/d1e/d2b/d5a/db7/lcd 0 2024-06-06T08:43:48.637 INFO:tasks.workunit.client.0.smithi027.stdout:5/647: mknod d0/da/db/d46/d7f/d81/d97/cca 0 2024-06-06T08:43:48.637 INFO:tasks.workunit.client.0.smithi027.stdout:9/536: chown d3/d95/d34/l8a 1173 1 2024-06-06T08:43:48.641 INFO:tasks.workunit.client.0.smithi027.stdout:0/550: unlink d15/d18/d23/d5b/f6b 0 2024-06-06T08:43:48.643 INFO:tasks.workunit.client.0.smithi027.stdout:1/498: readlink l7 0 2024-06-06T08:43:48.643 INFO:tasks.workunit.client.0.smithi027.stdout:0/551: creat d15/d18/d23/fba x:0 0 0 2024-06-06T08:43:48.644 INFO:tasks.workunit.client.0.smithi027.stdout:2/623: dread d6/dc/d1d/f60 [0,4194304] 0 2024-06-06T08:43:48.644 INFO:tasks.workunit.client.0.smithi027.stdout:5/648: dread d0/da/d37/d2b/f91 [0,4194304] 0 2024-06-06T08:43:48.646 INFO:tasks.workunit.client.0.smithi027.stdout:8/550: dwrite d7/d13/faa [0,4194304] 0 2024-06-06T08:43:48.646 INFO:tasks.workunit.client.0.smithi027.stdout:1/499: creat dc/de/d3d/d4f/f9e x:0 0 0 2024-06-06T08:43:48.649 INFO:tasks.workunit.client.0.smithi027.stdout:4/666: dread d0/d1/d6/d8/d21/f2d [0,4194304] 0 2024-06-06T08:43:48.649 INFO:tasks.workunit.client.0.smithi027.stdout:3/511: rename d1/d1f/d25/f80 to d1/d90/d6d/f9a 0 2024-06-06T08:43:48.651 INFO:tasks.workunit.client.0.smithi027.stdout:8/551: dread d7/d2f/f98 [0,4194304] 0 2024-06-06T08:43:48.652 INFO:tasks.workunit.client.0.smithi027.stdout:8/552: chown d7/dc/d36/d83/d60/d93/dac 679 1 2024-06-06T08:43:48.657 INFO:tasks.workunit.client.0.smithi027.stdout:7/718: write d17/f58 [3629714,128826] 0 2024-06-06T08:43:48.659 INFO:tasks.workunit.client.0.smithi027.stdout:7/719: fdatasync d17/d32/d46/d83/d77/fdd 0 2024-06-06T08:43:48.660 INFO:tasks.workunit.client.0.smithi027.stdout:3/512: rmdir d1/d90/d44 39 2024-06-06T08:43:48.661 INFO:tasks.workunit.client.0.smithi027.stdout:6/676: dwrite d4/d9/d49/f86 [0,4194304] 0 2024-06-06T08:43:48.662 INFO:tasks.workunit.client.0.smithi027.stdout:2/624: truncate d6/dc/d1d/f4b 28147 0 2024-06-06T08:43:48.662 INFO:tasks.workunit.client.0.smithi027.stdout:2/625: chown d6/dc/d3f/d77/l78 414988740 1 2024-06-06T08:43:48.662 INFO:tasks.workunit.client.0.smithi027.stdout:2/626: chown d6/dc/d1d/d4d 235 1 2024-06-06T08:43:48.663 INFO:tasks.workunit.client.0.smithi027.stdout:1/500: symlink dc/l9f 0 2024-06-06T08:43:48.664 INFO:tasks.workunit.client.0.smithi027.stdout:6/677: creat d4/d9/d6e/dd4/fde x:0 0 0 2024-06-06T08:43:48.664 INFO:tasks.workunit.client.0.smithi027.stdout:4/667: mkdir d0/d1/d6/dd4 0 2024-06-06T08:43:48.666 INFO:tasks.workunit.client.0.smithi027.stdout:0/552: mkdir d15/d18/d23/d28/d31/dbb 0 2024-06-06T08:43:48.667 INFO:tasks.workunit.client.0.smithi027.stdout:3/513: fsync d1/d1f/d25/f7d 0 2024-06-06T08:43:48.667 INFO:tasks.workunit.client.0.smithi027.stdout:3/514: stat d1/d8/d22/l91 0 2024-06-06T08:43:48.669 INFO:tasks.workunit.client.0.smithi027.stdout:4/668: readlink d0/d1d/d60/dc6/l5f 0 2024-06-06T08:43:48.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/553: symlink d15/d18/d23/d6f/lbc 0 2024-06-06T08:43:48.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/554: truncate d15/d18/d23/d28/d2a/d74/fb0 438366 0 2024-06-06T08:43:48.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/555: fdatasync d15/d18/d23/d28/d31/f5e 0 2024-06-06T08:43:48.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/556: truncate d15/d18/d40/f9e 660652 0 2024-06-06T08:43:48.669 INFO:tasks.workunit.client.0.smithi027.stdout:9/537: dwrite d3/d95/d34/d41/f9c [0,4194304] 0 2024-06-06T08:43:48.669 INFO:tasks.workunit.client.0.smithi027.stdout:8/553: dread d7/d13/f17 [0,4194304] 0 2024-06-06T08:43:48.671 INFO:tasks.workunit.client.0.smithi027.stdout:7/720: creat d17/d32/d46/d4f/db6/dc8/de0/fe9 x:0 0 0 2024-06-06T08:43:48.672 INFO:tasks.workunit.client.0.smithi027.stdout:7/721: getdents d17/d32/d46/d83/d77/d8c/d91/daa 0 2024-06-06T08:43:48.674 INFO:tasks.workunit.client.0.smithi027.stdout:6/678: symlink d4/da2/ldf 0 2024-06-06T08:43:48.674 INFO:tasks.workunit.client.0.smithi027.stdout:9/538: chown d3/d4e/f9d 4221 1 2024-06-06T08:43:48.675 INFO:tasks.workunit.client.0.smithi027.stdout:8/554: mknod d7/d2f/cbd 0 2024-06-06T08:43:48.677 INFO:tasks.workunit.client.0.smithi027.stdout:9/539: write d3/d95/f39 [2307333,43319] 0 2024-06-06T08:43:48.678 INFO:tasks.workunit.client.0.smithi027.stdout:0/557: getdents d15/d18/d23/d28/d2a/d30 0 2024-06-06T08:43:48.679 INFO:tasks.workunit.client.0.smithi027.stdout:9/540: chown d3/d4e/d75/dc/l82 254 1 2024-06-06T08:43:48.679 INFO:tasks.workunit.client.0.smithi027.stdout:6/679: mknod d4/ce0 0 2024-06-06T08:43:48.680 INFO:tasks.workunit.client.0.smithi027.stdout:0/558: fdatasync d15/d18/d23/f2e 0 2024-06-06T08:43:48.686 INFO:tasks.workunit.client.0.smithi027.stdout:9/541: symlink d3/d76/lc0 0 2024-06-06T08:43:48.687 INFO:tasks.workunit.client.0.smithi027.stdout:9/542: truncate d3/d95/d23/d36/f5d 4664867 0 2024-06-06T08:43:48.688 INFO:tasks.workunit.client.0.smithi027.stdout:8/555: mkdir d7/d13/d15/dbe 0 2024-06-06T08:43:48.688 INFO:tasks.workunit.client.0.smithi027.stdout:3/515: dread d1/d8/f16 [0,4194304] 0 2024-06-06T08:43:48.688 INFO:tasks.workunit.client.0.smithi027.stdout:9/543: fdatasync d3/f15 0 2024-06-06T08:43:48.692 INFO:tasks.workunit.client.0.smithi027.stdout:3/516: link d1/d8/f55 d1/d90/d6d/f9b 0 2024-06-06T08:43:48.692 INFO:tasks.workunit.client.0.smithi027.stdout:1/501: dwrite dc/f81 [0,4194304] 0 2024-06-06T08:43:48.693 INFO:tasks.workunit.client.0.smithi027.stdout:9/544: getdents d3/d76 0 2024-06-06T08:43:48.696 INFO:tasks.workunit.client.0.smithi027.stdout:9/545: symlink d3/d95/d23/d36/lc1 0 2024-06-06T08:43:48.696 INFO:tasks.workunit.client.0.smithi027.stdout:9/546: fdatasync d3/d95/d23/d36/d3a/f6d 0 2024-06-06T08:43:48.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/722: dread d17/d32/d46/d4f/d5e/d7b/f88 [0,4194304] 0 2024-06-06T08:43:48.698 INFO:tasks.workunit.client.0.smithi027.stdout:9/547: link d3/d4e/d75/dc/l82 d3/d95/db6/lc2 0 2024-06-06T08:43:48.698 INFO:tasks.workunit.client.0.smithi027.stdout:6/680: dread d4/d9/f2d [0,4194304] 0 2024-06-06T08:43:48.699 INFO:tasks.workunit.client.0.smithi027.stdout:6/681: fsync d4/d1a/d2e/f7e 0 2024-06-06T08:43:48.699 INFO:tasks.workunit.client.0.smithi027.stdout:7/723: creat d17/d32/d46/d51/d52/d70/dce/de6/fea x:0 0 0 2024-06-06T08:43:48.699 INFO:tasks.workunit.client.0.smithi027.stdout:3/517: mkdir d1/d1f/d45/d5f/d9c 0 2024-06-06T08:43:48.700 INFO:tasks.workunit.client.0.smithi027.stdout:9/548: truncate d3/d95/f2f 2448746 0 2024-06-06T08:43:48.700 INFO:tasks.workunit.client.0.smithi027.stdout:6/682: creat d4/d9/dc5/fe1 x:0 0 0 2024-06-06T08:43:48.701 INFO:tasks.workunit.client.0.smithi027.stdout:8/556: rmdir d7/dc/d36/d41 39 2024-06-06T08:43:48.704 INFO:tasks.workunit.client.0.smithi027.stdout:4/669: dwrite d0/d1d/d2a/f5e [0,4194304] 0 2024-06-06T08:43:48.704 INFO:tasks.workunit.client.0.smithi027.stdout:4/670: dread - d0/d1/d6/d8/d82/d8b/dc9/fc8 zero size 2024-06-06T08:43:48.705 INFO:tasks.workunit.client.0.smithi027.stdout:2/627: dwrite f5 [0,4194304] 0 2024-06-06T08:43:48.706 INFO:tasks.workunit.client.0.smithi027.stdout:1/502: unlink dc/de/d3d/c7e 0 2024-06-06T08:43:48.706 INFO:tasks.workunit.client.0.smithi027.stdout:1/503: fdatasync dc/de/d3d/d4f/f9e 0 2024-06-06T08:43:48.706 INFO:tasks.workunit.client.0.smithi027.stdout:8/557: readlink d7/dc/d37/l49 0 2024-06-06T08:43:48.708 INFO:tasks.workunit.client.0.smithi027.stdout:7/724: mknod d17/d32/d46/d51/d7f/db2/ceb 0 2024-06-06T08:43:48.708 INFO:tasks.workunit.client.0.smithi027.stdout:7/725: write d17/d32/d46/d4f/db6/dc8/de0/fe9 [168058,9203] 0 2024-06-06T08:43:48.708 INFO:tasks.workunit.client.0.smithi027.stdout:7/726: readlink d17/d32/d6c/ld2 0 2024-06-06T08:43:48.709 INFO:tasks.workunit.client.0.smithi027.stdout:3/518: symlink d1/d8/d22/d27/l9d 0 2024-06-06T08:43:48.709 INFO:tasks.workunit.client.0.smithi027.stdout:3/519: chown d1 41269644 1 2024-06-06T08:43:48.710 INFO:tasks.workunit.client.0.smithi027.stdout:0/559: dwrite d15/d18/d23/f27 [0,4194304] 0 2024-06-06T08:43:48.711 INFO:tasks.workunit.client.0.smithi027.stdout:6/683: truncate d4/f29 3244856 0 2024-06-06T08:43:48.712 INFO:tasks.workunit.client.0.smithi027.stdout:2/628: stat d6/d3b/l53 0 2024-06-06T08:43:48.716 INFO:tasks.workunit.client.0.smithi027.stdout:1/504: getdents dc/de/d14/d4e 0 2024-06-06T08:43:48.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/505: mkdir dc/da0 0 2024-06-06T08:43:48.724 INFO:tasks.workunit.client.0.smithi027.stdout:8/558: dread d7/dc/d36/d3c/f72 [0,4194304] 0 2024-06-06T08:43:48.726 INFO:tasks.workunit.client.0.smithi027.stdout:1/506: mkdir dc/d67/d69/d32/d37/d56/d78/da1 0 2024-06-06T08:43:48.727 INFO:tasks.workunit.client.0.smithi027.stdout:4/671: write d0/f13 [5545275,120044] 0 2024-06-06T08:43:48.729 INFO:tasks.workunit.client.0.smithi027.stdout:2/629: dread d6/dd/f18 [0,4194304] 0 2024-06-06T08:43:48.729 INFO:tasks.workunit.client.0.smithi027.stdout:9/549: dwrite d3/d95/d23/d36/d3a/f68 [0,4194304] 0 2024-06-06T08:43:48.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/630: readlink d6/d1e/d2b/d5a/l75 0 2024-06-06T08:43:48.734 INFO:tasks.workunit.client.0.smithi027.stdout:0/560: dread d15/d18/d64/f6d [0,4194304] 0 2024-06-06T08:43:48.734 INFO:tasks.workunit.client.0.smithi027.stdout:2/631: creat d6/dc/d3f/d48/d83/fce x:0 0 0 2024-06-06T08:43:48.734 INFO:tasks.workunit.client.0.smithi027.stdout:4/672: mkdir d0/d1/d6/d8/dd5 0 2024-06-06T08:43:48.735 INFO:tasks.workunit.client.0.smithi027.stdout:1/507: dread dc/de/d3d/f3e [0,4194304] 0 2024-06-06T08:43:48.736 INFO:tasks.workunit.client.0.smithi027.stdout:4/673: write d0/d1d/d2a/f7d [524510,64033] 0 2024-06-06T08:43:48.737 INFO:tasks.workunit.client.0.smithi027.stdout:8/559: dread d7/dc/d36/d3c/f79 [0,4194304] 0 2024-06-06T08:43:48.737 INFO:tasks.workunit.client.0.smithi027.stdout:0/561: creat d15/d18/d64/fbd x:0 0 0 2024-06-06T08:43:48.738 INFO:tasks.workunit.client.0.smithi027.stdout:1/508: chown dc/de/d14/c1e 158327 1 2024-06-06T08:43:48.739 INFO:tasks.workunit.client.0.smithi027.stdout:4/674: creat d0/d1d/dd0/fd6 x:0 0 0 2024-06-06T08:43:48.740 INFO:tasks.workunit.client.0.smithi027.stdout:8/560: fsync d7/dc/d36/d83/d60/f68 0 2024-06-06T08:43:48.741 INFO:tasks.workunit.client.0.smithi027.stdout:0/562: getdents d15/da7 0 2024-06-06T08:43:48.741 INFO:tasks.workunit.client.0.smithi027.stdout:1/509: dread - dc/de/d22/f79 zero size 2024-06-06T08:43:48.745 INFO:tasks.workunit.client.0.smithi027.stdout:2/632: link d6/dd/d6b/fb3 d6/dc/d1d/d4d/fcf 0 2024-06-06T08:43:48.746 INFO:tasks.workunit.client.0.smithi027.stdout:0/563: chown d15/d18/d23/d28/d2a/d74/l7f 67 1 2024-06-06T08:43:48.747 INFO:tasks.workunit.client.0.smithi027.stdout:2/633: chown d6/dc/d1d/d4d/lc1 18242 1 2024-06-06T08:43:48.751 INFO:tasks.workunit.client.0.smithi027.stdout:2/634: unlink f1 0 2024-06-06T08:43:48.752 INFO:tasks.workunit.client.0.smithi027.stdout:9/550: dwrite d3/d95/f2b [4194304,4194304] 0 2024-06-06T08:43:48.752 INFO:tasks.workunit.client.0.smithi027.stdout:9/551: write d3/d95/f2f [2142536,71464] 0 2024-06-06T08:43:48.752 INFO:tasks.workunit.client.0.smithi027.stdout:9/552: chown d3/d95/f39 3775 1 2024-06-06T08:43:48.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/561: dread d7/dc/d36/d83/d60/f68 [0,4194304] 0 2024-06-06T08:43:48.753 INFO:tasks.workunit.client.0.smithi027.stdout:9/553: getdents d3/d95/d23/d36/d3a/d84/db2 0 2024-06-06T08:43:48.763 INFO:tasks.workunit.client.0.smithi027.stdout:1/510: dread dc/de/f13 [0,4194304] 0 2024-06-06T08:43:48.765 INFO:tasks.workunit.client.0.smithi027.stdout:1/511: creat dc/d9d/fa2 x:0 0 0 2024-06-06T08:43:48.766 INFO:tasks.workunit.client.0.smithi027.stdout:4/675: dwrite d0/d1d/d2a/d98/fa8 [0,4194304] 0 2024-06-06T08:43:48.767 INFO:tasks.workunit.client.0.smithi027.stdout:2/635: dread d6/d3b/f43 [0,4194304] 0 2024-06-06T08:43:48.767 INFO:tasks.workunit.client.0.smithi027.stdout:4/676: readlink d0/d1/d6/d8/d21/l3e 0 2024-06-06T08:43:48.770 INFO:tasks.workunit.client.0.smithi027.stdout:8/562: dwrite d7/dc/d36/f7b [4194304,4194304] 0 2024-06-06T08:43:48.771 INFO:tasks.workunit.client.0.smithi027.stdout:9/554: dread d3/d95/d34/f60 [0,4194304] 0 2024-06-06T08:43:48.772 INFO:tasks.workunit.client.0.smithi027.stdout:9/555: symlink d3/d95/db6/lc3 0 2024-06-06T08:43:48.774 INFO:tasks.workunit.client.0.smithi027.stdout:7/727: rename d17/d32/d46/d83/fd5 to d17/d32/d61/d75/fec 0 2024-06-06T08:43:48.779 INFO:tasks.workunit.client.0.smithi027.stdout:8/563: chown d7/dc/d36/d83/l32 307345 1 2024-06-06T08:43:48.780 INFO:tasks.workunit.client.0.smithi027.stdout:2/636: truncate d6/d1e/f9a 1366043 0 2024-06-06T08:43:48.782 INFO:tasks.workunit.client.0.smithi027.stdout:4/677: truncate d0/d63/f6d 2800910 0 2024-06-06T08:43:48.782 INFO:tasks.workunit.client.0.smithi027.stdout:1/512: write dc/de/d14/d4e/f7a [759579,11268] 0 2024-06-06T08:43:48.786 INFO:tasks.workunit.client.0.smithi027.stdout:8/564: getdents d7/dc 0 2024-06-06T08:43:48.786 INFO:tasks.workunit.client.0.smithi027.stdout:8/565: dread - d7/dc/d36/d83/d60/d93/dac/fb2 zero size 2024-06-06T08:43:48.786 INFO:tasks.workunit.client.0.smithi027.stdout:8/566: stat d7/dc/c4b 0 2024-06-06T08:43:48.788 INFO:tasks.workunit.client.0.smithi027.stdout:4/678: chown d0/c6c 300 1 2024-06-06T08:43:48.791 INFO:tasks.workunit.client.0.smithi027.stdout:3/520: rename d1/d90/d44/c93 to d1/d1f/d5b/c9e 0 2024-06-06T08:43:48.793 INFO:tasks.workunit.client.0.smithi027.stdout:3/521: getdents d1/d1f 0 2024-06-06T08:43:48.793 INFO:tasks.workunit.client.0.smithi027.stdout:3/522: write d1/d8/d22/d27/d50/f51 [4383453,380] 0 2024-06-06T08:43:48.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/684: rename d4/d9/dc/d18/fbe to d4/d9/d49/fe2 0 2024-06-06T08:43:48.796 INFO:tasks.workunit.client.0.smithi027.stdout:6/685: stat d4/f59 0 2024-06-06T08:43:48.798 INFO:tasks.workunit.client.0.smithi027.stdout:6/686: dread d4/d9/dc/f9f [0,4194304] 0 2024-06-06T08:43:48.801 INFO:tasks.workunit.client.0.smithi027.stdout:8/567: write d7/dc/d36/d83/d58/f8a [3962195,128140] 0 2024-06-06T08:43:48.801 INFO:tasks.workunit.client.0.smithi027.stdout:8/568: dread - d7/dc/f9f zero size 2024-06-06T08:43:48.801 INFO:tasks.workunit.client.0.smithi027.stdout:8/569: getdents d7/d13/d15/dbe 0 2024-06-06T08:43:48.801 INFO:tasks.workunit.client.0.smithi027.stdout:4/679: truncate d0/d1/d6/d8/d82/d8b/dc9/fb3 1520364 0 2024-06-06T08:43:48.804 INFO:tasks.workunit.client.0.smithi027.stdout:3/523: write d1/d90/f42 [452116,50606] 0 2024-06-06T08:43:48.806 INFO:tasks.workunit.client.0.smithi027.stdout:3/524: creat d1/d1f/d5b/f9f x:0 0 0 2024-06-06T08:43:48.807 INFO:tasks.workunit.client.0.smithi027.stdout:6/687: dread d4/d9/dc/d18/f24 [0,4194304] 0 2024-06-06T08:43:48.808 INFO:tasks.workunit.client.0.smithi027.stdout:3/525: truncate d1/d8/d22/d27/d50/f82 545973 0 2024-06-06T08:43:48.808 INFO:tasks.workunit.client.0.smithi027.stdout:3/526: chown d1/d1f/d5b/c9e 1703536 1 2024-06-06T08:43:48.808 INFO:tasks.workunit.client.0.smithi027.stdout:6/688: truncate d4/d1a/f2c 3806645 0 2024-06-06T08:43:48.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/680: dread d0/d1/f4f [0,4194304] 0 2024-06-06T08:43:48.809 INFO:tasks.workunit.client.0.smithi027.stdout:8/570: mkdir d7/dc/d36/d41/dbf 0 2024-06-06T08:43:48.812 INFO:tasks.workunit.client.0.smithi027.stdout:4/681: chown d0/d1/ff 11 1 2024-06-06T08:43:48.817 INFO:tasks.workunit.client.0.smithi027.stdout:4/682: creat d0/d1/d6/d8/d82/d8b/dc9/fd7 x:0 0 0 2024-06-06T08:43:48.820 INFO:tasks.workunit.client.0.smithi027.stdout:6/689: write d4/d9/dc/d18/fd0 [781448,52778] 0 2024-06-06T08:43:48.820 INFO:tasks.workunit.client.0.smithi027.stdout:6/690: chown d4/d1a 311590709 1 2024-06-06T08:43:48.823 INFO:tasks.workunit.client.0.smithi027.stdout:4/683: mkdir d0/d1d/d34/dd8 0 2024-06-06T08:43:48.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/564: rename d15/d18/d23/d28/d2a/d30 to d15/d18/dbe 0 2024-06-06T08:43:48.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/565: chown d15/d18/d23/d28/d2a 14272 1 2024-06-06T08:43:48.825 INFO:tasks.workunit.client.0.smithi027.stdout:4/684: creat d0/d1d/d34/d7f/d97/fd9 x:0 0 0 2024-06-06T08:43:48.825 INFO:tasks.workunit.client.0.smithi027.stdout:4/685: write d0/d1/d6/d8/d82/d8b/dc9/fd7 [559360,44008] 0 2024-06-06T08:43:48.825 INFO:tasks.workunit.client.0.smithi027.stdout:0/566: creat d15/dae/fbf x:0 0 0 2024-06-06T08:43:48.826 INFO:tasks.workunit.client.0.smithi027.stdout:4/686: fsync d0/d1/fb 0 2024-06-06T08:43:48.828 INFO:tasks.workunit.client.0.smithi027.stdout:9/556: rename d3/d95/d23/d36/lc1 to d3/d95/db6/lc4 0 2024-06-06T08:43:48.828 INFO:tasks.workunit.client.0.smithi027.stdout:0/567: fdatasync d15/d18/d23/d38/d68/f7b 0 2024-06-06T08:43:48.829 INFO:tasks.workunit.client.0.smithi027.stdout:1/513: dwrite dc/de/d3d/d4a/f58 [0,4194304] 0 2024-06-06T08:43:48.829 INFO:tasks.workunit.client.0.smithi027.stdout:9/557: truncate d3/d95/d23/da9/fab 1237209 0 2024-06-06T08:43:48.830 INFO:tasks.workunit.client.0.smithi027.stdout:4/687: creat d0/d1/d6/d8/d82/fda x:0 0 0 2024-06-06T08:43:48.831 INFO:tasks.workunit.client.0.smithi027.stdout:2/637: dwrite d6/f2c [0,4194304] 0 2024-06-06T08:43:48.832 INFO:tasks.workunit.client.0.smithi027.stdout:1/514: truncate dc/de/d14/f15 183809 0 2024-06-06T08:43:48.832 INFO:tasks.workunit.client.0.smithi027.stdout:8/571: write d7/f2c [2399009,123370] 0 2024-06-06T08:43:48.832 INFO:tasks.workunit.client.0.smithi027.stdout:3/527: dwrite d1/d8/d22/d2d/f7f [0,4194304] 0 2024-06-06T08:43:48.836 INFO:tasks.workunit.client.0.smithi027.stdout:8/572: symlink d7/dc/lc0 0 2024-06-06T08:43:48.837 INFO:tasks.workunit.client.0.smithi027.stdout:1/515: mknod dc/de/d14/d80/ca3 0 2024-06-06T08:43:48.838 INFO:tasks.workunit.client.0.smithi027.stdout:3/528: rmdir d1/d1f/d25 39 2024-06-06T08:43:48.841 INFO:tasks.workunit.client.0.smithi027.stdout:4/688: rename d0/d1d/d60/l85 to d0/d1d/d34/ldb 0 2024-06-06T08:43:48.843 INFO:tasks.workunit.client.0.smithi027.stdout:8/573: rmdir d7/dc/d36/d83/d60 39 2024-06-06T08:43:48.844 INFO:tasks.workunit.client.0.smithi027.stdout:6/691: dwrite d4/d9/dc/d18/f24 [4194304,4194304] 0 2024-06-06T08:43:48.845 INFO:tasks.workunit.client.0.smithi027.stdout:4/689: dread d0/d1/d6/d8/d21/f4a [0,4194304] 0 2024-06-06T08:43:48.847 INFO:tasks.workunit.client.0.smithi027.stdout:6/692: creat d4/d1a/d2e/d44/dce/fe3 x:0 0 0 2024-06-06T08:43:48.849 INFO:tasks.workunit.client.0.smithi027.stdout:1/516: fdatasync dc/de/d3d/d4a/d76/f90 0 2024-06-06T08:43:48.849 INFO:tasks.workunit.client.0.smithi027.stdout:6/693: write d4/d9/dc/fa8 [2617056,61276] 0 2024-06-06T08:43:48.851 INFO:tasks.workunit.client.0.smithi027.stdout:4/690: creat d0/d1d/d2a/fdc x:0 0 0 2024-06-06T08:43:48.851 INFO:tasks.workunit.client.0.smithi027.stdout:8/574: mknod d7/dc/d36/d83/cc1 0 2024-06-06T08:43:48.852 INFO:tasks.workunit.client.0.smithi027.stdout:1/517: read - dc/d67/d69/d32/f4d zero size 2024-06-06T08:43:48.852 INFO:tasks.workunit.client.0.smithi027.stdout:1/518: fsync dc/de/d3d/f3e 0 2024-06-06T08:43:48.853 INFO:tasks.workunit.client.0.smithi027.stdout:6/694: creat d4/fe4 x:0 0 0 2024-06-06T08:43:48.854 INFO:tasks.workunit.client.0.smithi027.stdout:9/558: read d3/d4e/d75/dc/d9b/f91 [2170965,13306] 0 2024-06-06T08:43:48.854 INFO:tasks.workunit.client.0.smithi027.stdout:9/559: chown d3/d95/d23/d36/d3a/l93 1962428 1 2024-06-06T08:43:48.854 INFO:tasks.workunit.client.0.smithi027.stdout:4/691: creat d0/d1d/d2a/fdd x:0 0 0 2024-06-06T08:43:48.856 INFO:tasks.workunit.client.0.smithi027.stdout:9/560: creat d3/d95/d23/da9/fc5 x:0 0 0 2024-06-06T08:43:48.856 INFO:tasks.workunit.client.0.smithi027.stdout:6/695: creat d4/d1a/d2e/d44/dca/fe5 x:0 0 0 2024-06-06T08:43:48.856 INFO:tasks.workunit.client.0.smithi027.stdout:0/568: dread d15/d18/d40/f76 [0,4194304] 0 2024-06-06T08:43:48.857 INFO:tasks.workunit.client.0.smithi027.stdout:4/692: mkdir d0/d1/d6/dd4/dde 0 2024-06-06T08:43:48.857 INFO:tasks.workunit.client.0.smithi027.stdout:4/693: readlink d0/d1d/d34/l3f 0 2024-06-06T08:43:48.857 INFO:tasks.workunit.client.0.smithi027.stdout:4/694: rename d0/d1 to d0/d1/d6/d8/ddf 22 2024-06-06T08:43:48.858 INFO:tasks.workunit.client.0.smithi027.stdout:9/561: creat d3/fc6 x:0 0 0 2024-06-06T08:43:48.859 INFO:tasks.workunit.client.0.smithi027.stdout:6/696: creat d4/de/fe6 x:0 0 0 2024-06-06T08:43:48.859 INFO:tasks.workunit.client.0.smithi027.stdout:1/519: dread dc/de/f47 [0,4194304] 0 2024-06-06T08:43:48.860 INFO:tasks.workunit.client.0.smithi027.stdout:9/562: symlink d3/d95/d34/lc7 0 2024-06-06T08:43:48.860 INFO:tasks.workunit.client.0.smithi027.stdout:9/563: write d3/d95/f2f [492444,98647] 0 2024-06-06T08:43:48.860 INFO:tasks.workunit.client.0.smithi027.stdout:2/638: dwrite d6/dd/f23 [0,4194304] 0 2024-06-06T08:43:48.861 INFO:tasks.workunit.client.0.smithi027.stdout:2/639: stat d6/dc/d3f/d48/d83/fce 0 2024-06-06T08:43:48.861 INFO:tasks.workunit.client.0.smithi027.stdout:0/569: creat d15/fc0 x:0 0 0 2024-06-06T08:43:48.862 INFO:tasks.workunit.client.0.smithi027.stdout:9/564: mknod d3/d4e/d75/db4/cc8 0 2024-06-06T08:43:48.865 INFO:tasks.workunit.client.0.smithi027.stdout:0/570: read - d15/d18/dbe/d5d/f98 zero size 2024-06-06T08:43:48.867 INFO:tasks.workunit.client.0.smithi027.stdout:1/520: dread dc/de/d14/f63 [0,4194304] 0 2024-06-06T08:43:48.868 INFO:tasks.workunit.client.0.smithi027.stdout:0/571: mknod d15/d18/d23/d5b/cc1 0 2024-06-06T08:43:48.870 INFO:tasks.workunit.client.0.smithi027.stdout:0/572: creat d15/d18/d40/fc2 x:0 0 0 2024-06-06T08:43:48.871 INFO:tasks.workunit.client.0.smithi027.stdout:0/573: stat d15/d18/c61 0 2024-06-06T08:43:48.871 INFO:tasks.workunit.client.0.smithi027.stdout:3/529: dwrite d1/d90/d44/f49 [0,4194304] 0 2024-06-06T08:43:48.871 INFO:tasks.workunit.client.0.smithi027.stdout:8/575: dread d7/dc/f12 [0,4194304] 0 2024-06-06T08:43:48.875 INFO:tasks.workunit.client.0.smithi027.stdout:8/576: creat d7/dc/d36/d83/d60/d93/dac/d73/fc2 x:0 0 0 2024-06-06T08:43:48.876 INFO:tasks.workunit.client.0.smithi027.stdout:3/530: symlink d1/d1f/d4d/la0 0 2024-06-06T08:43:48.877 INFO:tasks.workunit.client.0.smithi027.stdout:6/697: dwrite d4/d1a/d2e/d44/dca/fe5 [0,4194304] 0 2024-06-06T08:43:48.878 INFO:tasks.workunit.client.0.smithi027.stdout:0/574: chown d15/d18/d23/d38/d68/d85/f8d 25249 1 2024-06-06T08:43:48.878 INFO:tasks.workunit.client.0.smithi027.stdout:0/575: chown d15/d18/dbe/d5c/d83 1 1 2024-06-06T08:43:48.878 INFO:tasks.workunit.client.0.smithi027.stdout:8/577: rmdir d7/dc 39 2024-06-06T08:43:48.879 INFO:tasks.workunit.client.0.smithi027.stdout:3/531: symlink d1/d90/d44/la1 0 2024-06-06T08:43:48.879 INFO:tasks.workunit.client.0.smithi027.stdout:3/532: write d1/d8/d22/f46 [182241,66137] 0 2024-06-06T08:43:48.880 INFO:tasks.workunit.client.0.smithi027.stdout:3/533: creat d1/d90/fa2 x:0 0 0 2024-06-06T08:43:48.880 INFO:tasks.workunit.client.0.smithi027.stdout:1/521: write dc/de/d14/d4e/f62 [2928699,3571] 0 2024-06-06T08:43:48.881 INFO:tasks.workunit.client.0.smithi027.stdout:1/522: stat dc/d67/d69/d32/f4d 0 2024-06-06T08:43:48.882 INFO:tasks.workunit.client.0.smithi027.stdout:0/576: mknod d15/d18/d40/cc3 0 2024-06-06T08:43:48.883 INFO:tasks.workunit.client.0.smithi027.stdout:0/577: read d15/d18/d23/d28/f4e [1360815,49203] 0 2024-06-06T08:43:48.884 INFO:tasks.workunit.client.0.smithi027.stdout:6/698: rename d4/d9/f96 to d4/d9/dc/d18/fe7 0 2024-06-06T08:43:48.884 INFO:tasks.workunit.client.0.smithi027.stdout:3/534: symlink d1/d8/d22/d27/la3 0 2024-06-06T08:43:48.885 INFO:tasks.workunit.client.0.smithi027.stdout:0/578: getdents d15/d18/d23/d28 0 2024-06-06T08:43:48.885 INFO:tasks.workunit.client.0.smithi027.stdout:0/579: dread - d15/d18/f81 zero size 2024-06-06T08:43:48.886 INFO:tasks.workunit.client.0.smithi027.stdout:6/699: readlink d4/d9/dc5/lc7 0 2024-06-06T08:43:48.886 INFO:tasks.workunit.client.0.smithi027.stdout:6/700: chown d4/de/fe6 546 1 2024-06-06T08:43:48.887 INFO:tasks.workunit.client.0.smithi027.stdout:3/535: rmdir d1/d1f/d45/d5f/d9c 0 2024-06-06T08:43:48.888 INFO:tasks.workunit.client.0.smithi027.stdout:1/523: unlink dc/d67/d69/c2e 0 2024-06-06T08:43:48.888 INFO:tasks.workunit.client.0.smithi027.stdout:0/580: mknod d15/d18/d23/d5b/da0/cc4 0 2024-06-06T08:43:48.890 INFO:tasks.workunit.client.0.smithi027.stdout:3/536: truncate d1/d1f/d25/f3a 1546762 0 2024-06-06T08:43:48.891 INFO:tasks.workunit.client.0.smithi027.stdout:3/537: dread d1/d8/d22/d27/d50/f82 [0,4194304] 0 2024-06-06T08:43:48.892 INFO:tasks.workunit.client.0.smithi027.stdout:1/524: rename dc/de/d14/d4e/f62 to dc/de/d14/d80/fa4 0 2024-06-06T08:43:48.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/525: getdents dc/d67 0 2024-06-06T08:43:48.896 INFO:tasks.workunit.client.0.smithi027.stdout:3/538: mknod d1/d90/ca4 0 2024-06-06T08:43:48.904 INFO:tasks.workunit.client.0.smithi027.stdout:6/701: dwrite d4/d9/f94 [0,4194304] 0 2024-06-06T08:43:48.907 INFO:tasks.workunit.client.0.smithi027.stdout:1/526: dread dc/de/d14/f19 [0,4194304] 0 2024-06-06T08:43:48.908 INFO:tasks.workunit.client.0.smithi027.stdout:1/527: fdatasync f5 0 2024-06-06T08:43:48.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/702: mkdir d4/de/d45/d6a/de8 0 2024-06-06T08:43:48.910 INFO:tasks.workunit.client.0.smithi027.stdout:1/528: link lb dc/da0/la5 0 2024-06-06T08:43:48.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/539: read d1/f29 [1601743,44474] 0 2024-06-06T08:43:48.913 INFO:tasks.workunit.client.0.smithi027.stdout:3/540: rename d1/f2 to d1/d90/fa5 0 2024-06-06T08:43:48.919 INFO:tasks.workunit.client.0.smithi027.stdout:6/703: rename d4/d1a/d2e/f9d to d4/d1a/d2e/fe9 0 2024-06-06T08:43:48.920 INFO:tasks.workunit.client.0.smithi027.stdout:6/704: chown d4/de/c36 881476 1 2024-06-06T08:43:48.927 INFO:tasks.workunit.client.0.smithi027.stdout:6/705: write d4/d9/f2d [966170,100924] 0 2024-06-06T08:43:48.928 INFO:tasks.workunit.client.0.smithi027.stdout:6/706: chown d4/f83 1104 1 2024-06-06T08:43:48.930 INFO:tasks.workunit.client.0.smithi027.stdout:6/707: rmdir d4/de/d45/dcf/ddc 0 2024-06-06T08:43:48.945 INFO:tasks.workunit.client.0.smithi027.stdout:6/708: dwrite d4/d9/d49/f8d [0,4194304] 0 2024-06-06T08:43:48.946 INFO:tasks.workunit.client.0.smithi027.stdout:6/709: mkdir d4/de/d45/dcf/dea 0 2024-06-06T08:43:48.946 INFO:tasks.workunit.client.0.smithi027.stdout:6/710: symlink d4/de/d45/dcf/dea/leb 0 2024-06-06T08:43:48.947 INFO:tasks.workunit.client.0.smithi027.stdout:6/711: creat d4/d1a/d2e/d44/dca/fec x:0 0 0 2024-06-06T08:43:48.948 INFO:tasks.workunit.client.0.smithi027.stdout:6/712: dread - d4/d9/dc/d18/fcc zero size 2024-06-06T08:43:48.955 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:48 smithi027 ceph-mon[32335]: pgmap v109: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.1 MiB/s wr, 523 op/s 2024-06-06T08:43:48.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/713: dread d4/d9/dc/f43 [0,4194304] 0 2024-06-06T08:43:48.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/714: getdents d4/d9/dc/d8f 0 2024-06-06T08:43:48.958 INFO:tasks.workunit.client.0.smithi027.stdout:6/715: creat d4/de/d45/dcf/dea/fed x:0 0 0 2024-06-06T08:43:48.958 INFO:tasks.workunit.client.0.smithi027.stdout:6/716: stat d4/d1a/d2e/f5c 0 2024-06-06T08:43:48.960 INFO:tasks.workunit.client.0.smithi027.stdout:6/717: rmdir d4/d9/dc/d1c 39 2024-06-06T08:43:48.961 INFO:tasks.workunit.client.0.smithi027.stdout:6/718: write d4/de/fb4 [623775,72354] 0 2024-06-06T08:43:48.961 INFO:tasks.workunit.client.0.smithi027.stdout:6/719: readlink d4/d1a/l61 0 2024-06-06T08:43:48.961 INFO:tasks.workunit.client.0.smithi027.stdout:6/720: fsync d4/f59 0 2024-06-06T08:43:48.967 INFO:tasks.workunit.client.0.smithi027.stdout:3/541: dwrite d1/d90/f77 [0,4194304] 0 2024-06-06T08:43:48.967 INFO:tasks.workunit.client.0.smithi027.stdout:3/542: stat d1/d8/d22/d27/l6e 0 2024-06-06T08:43:48.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/721: read d4/d9/dc/d1c/f9e [1031656,29120] 0 2024-06-06T08:43:48.969 INFO:tasks.workunit.client.0.smithi027.stdout:1/529: dwrite dc/de/d14/f19 [0,4194304] 0 2024-06-06T08:43:48.971 INFO:tasks.workunit.client.0.smithi027.stdout:3/543: creat d1/d90/d44/d54/fa6 x:0 0 0 2024-06-06T08:43:48.971 INFO:tasks.workunit.client.0.smithi027.stdout:3/544: readlink d1/d90/d44/d54/l68 0 2024-06-06T08:43:48.972 INFO:tasks.workunit.client.0.smithi027.stdout:1/530: mkdir dc/de/d14/d80/da6 0 2024-06-06T08:43:48.974 INFO:tasks.workunit.client.0.smithi027.stdout:1/531: chown dc/de/d22/f3b 31225 1 2024-06-06T08:43:48.974 INFO:tasks.workunit.client.0.smithi027.stdout:1/532: write dc/de/d3d/d4f/f9e [905565,27828] 0 2024-06-06T08:43:48.975 INFO:tasks.workunit.client.0.smithi027.stdout:1/533: mknod dc/d67/d69/d32/d37/d98/ca7 0 2024-06-06T08:43:48.981 INFO:tasks.workunit.client.0.smithi027.stdout:6/722: dread d4/d9/d49/f8d [0,4194304] 0 2024-06-06T08:43:48.981 INFO:tasks.workunit.client.0.smithi027.stdout:6/723: dread - d4/fe4 zero size 2024-06-06T08:43:48.985 INFO:tasks.workunit.client.0.smithi027.stdout:1/534: write dc/de/d14/d4e/f5d [2623858,79201] 0 2024-06-06T08:43:48.985 INFO:tasks.workunit.client.0.smithi027.stdout:1/535: stat dc/de/d14/d4e/f7a 0 2024-06-06T08:43:48.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:48 smithi203 ceph-mon[36579]: pgmap v109: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.1 MiB/s wr, 523 op/s 2024-06-06T08:43:48.993 INFO:tasks.workunit.client.0.smithi027.stdout:1/536: symlink dc/d67/d69/d32/d71/la8 0 2024-06-06T08:43:48.999 INFO:tasks.workunit.client.0.smithi027.stdout:6/724: write d4/f25 [1481783,88809] 0 2024-06-06T08:43:48.999 INFO:tasks.workunit.client.0.smithi027.stdout:6/725: write d4/d9/d49/f71 [397979,34243] 0 2024-06-06T08:43:49.000 INFO:tasks.workunit.client.0.smithi027.stdout:1/537: write dc/de/d3d/d4f/f64 [882966,13457] 0 2024-06-06T08:43:49.002 INFO:tasks.workunit.client.0.smithi027.stdout:1/538: mknod dc/d67/d69/d32/d37/d56/ca9 0 2024-06-06T08:43:49.003 INFO:tasks.workunit.client.0.smithi027.stdout:1/539: write dc/de/d3d/f85 [4473088,128145] 0 2024-06-06T08:43:49.003 INFO:tasks.workunit.client.0.smithi027.stdout:6/726: mknod d4/d9/dc/d1c/cee 0 2024-06-06T08:43:49.004 INFO:tasks.workunit.client.0.smithi027.stdout:1/540: symlink dc/de/d14/d80/da6/laa 0 2024-06-06T08:43:49.005 INFO:tasks.workunit.client.0.smithi027.stdout:1/541: creat dc/de/fab x:0 0 0 2024-06-06T08:43:49.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:48 smithi176 ceph-mon[36786]: pgmap v109: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.1 MiB/s wr, 523 op/s 2024-06-06T08:43:49.013 INFO:tasks.workunit.client.0.smithi027.stdout:6/727: dread d4/d1a/d2e/d44/dca/fe5 [0,4194304] 0 2024-06-06T08:43:49.014 INFO:tasks.workunit.client.0.smithi027.stdout:6/728: dread d4/de/d45/f89 [0,4194304] 0 2024-06-06T08:43:49.015 INFO:tasks.workunit.client.0.smithi027.stdout:6/729: write d4/d9/d6e/faf [515871,92920] 0 2024-06-06T08:43:49.020 INFO:tasks.workunit.client.0.smithi027.stdout:6/730: truncate d4/f25 1414614 0 2024-06-06T08:43:49.021 INFO:tasks.workunit.client.0.smithi027.stdout:6/731: creat d4/fef x:0 0 0 2024-06-06T08:43:49.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/542: dwrite dc/de/d3d/d4f/f64 [0,4194304] 0 2024-06-06T08:43:49.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/543: write dc/de/d22/f72 [236371,94644] 0 2024-06-06T08:43:49.031 INFO:tasks.workunit.client.0.smithi027.stdout:1/544: link dc/de/d14/c86 dc/de/d14/d80/da6/cac 0 2024-06-06T08:43:49.031 INFO:tasks.workunit.client.0.smithi027.stdout:1/545: stat dc/de/d3d/f3e 0 2024-06-06T08:43:49.031 INFO:tasks.workunit.client.0.smithi027.stdout:1/546: fdatasync dc/de/d14/d80/fa4 0 2024-06-06T08:43:49.033 INFO:tasks.workunit.client.0.smithi027.stdout:1/547: rmdir dc/d67/d69/d32/d37/d56/d78/da1 0 2024-06-06T08:43:49.035 INFO:tasks.workunit.client.0.smithi027.stdout:1/548: mknod dc/d67/d69/d32/d71/cad 0 2024-06-06T08:43:49.035 INFO:tasks.workunit.client.0.smithi027.stdout:1/549: stat dc/de/d14/d80/da6 0 2024-06-06T08:43:49.035 INFO:tasks.workunit.client.0.smithi027.stdout:1/550: chown dc/de/d14/d4e/l88 54155603 1 2024-06-06T08:43:49.036 INFO:tasks.workunit.client.0.smithi027.stdout:1/551: mknod dc/de/d14/d4e/cae 0 2024-06-06T08:43:49.036 INFO:tasks.workunit.client.0.smithi027.stdout:1/552: write dc/de/d14/d4e/f5d [116710,4299] 0 2024-06-06T08:43:49.050 INFO:tasks.workunit.client.0.smithi027.stdout:1/553: dwrite dc/f84 [0,4194304] 0 2024-06-06T08:43:49.053 INFO:tasks.workunit.client.0.smithi027.stdout:1/554: unlink dc/de/d22/f35 0 2024-06-06T08:43:49.072 INFO:tasks.workunit.client.0.smithi027.stdout:1/555: dwrite dc/de/d14/f91 [0,4194304] 0 2024-06-06T08:43:49.074 INFO:tasks.workunit.client.0.smithi027.stdout:1/556: truncate dc/de/f47 426157 0 2024-06-06T08:43:49.097 INFO:tasks.workunit.client.0.smithi027.stdout:1/557: dwrite dc/d67/d69/d32/d37/f41 [0,4194304] 0 2024-06-06T08:43:49.280 INFO:tasks.workunit.client.0.smithi027.stdout:5/649: sync 2024-06-06T08:43:49.283 INFO:tasks.workunit.client.0.smithi027.stdout:5/650: rename d0/da/db/d46/l55 to d0/da/db/d1a/da3/lcb 0 2024-06-06T08:43:49.284 INFO:tasks.workunit.client.0.smithi027.stdout:5/651: chown d0/da/db/f64 62 1 2024-06-06T08:43:49.299 INFO:tasks.workunit.client.0.smithi027.stdout:5/652: dwrite d0/da/db/d46/fc1 [0,4194304] 0 2024-06-06T08:43:49.301 INFO:tasks.workunit.client.0.smithi027.stdout:5/653: chown d0/da/db/d1a/d5a 27730 1 2024-06-06T08:43:49.305 INFO:tasks.workunit.client.0.smithi027.stdout:5/654: unlink d0/da/d20/f4f 0 2024-06-06T08:43:49.306 INFO:tasks.workunit.client.0.smithi027.stdout:5/655: dread - d0/da/db/fa4 zero size 2024-06-06T08:43:49.306 INFO:tasks.workunit.client.0.smithi027.stdout:5/656: chown d0/da/db 5 1 2024-06-06T08:43:49.306 INFO:tasks.workunit.client.0.smithi027.stdout:5/657: read d0/f44 [237467,116602] 0 2024-06-06T08:43:49.307 INFO:tasks.workunit.client.0.smithi027.stdout:5/658: getdents d0/da/db/d1a 0 2024-06-06T08:43:49.307 INFO:tasks.workunit.client.0.smithi027.stdout:5/659: stat d0/da/d37/d2b/l53 0 2024-06-06T08:43:49.329 INFO:tasks.workunit.client.0.smithi027.stdout:5/660: rename d0/da/d20/fc0 to d0/da/db/d46/d7f/d81/fcc 0 2024-06-06T08:43:49.330 INFO:tasks.workunit.client.0.smithi027.stdout:5/661: mkdir d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd 0 2024-06-06T08:43:49.331 INFO:tasks.workunit.client.0.smithi027.stdout:5/662: rmdir d0/da/db/d46/d54 39 2024-06-06T08:43:49.332 INFO:tasks.workunit.client.0.smithi027.stdout:5/663: dread - d0/da/db/d46/d4c/d70/f9e zero size 2024-06-06T08:43:49.333 INFO:tasks.workunit.client.0.smithi027.stdout:5/664: mknod d0/da/db/d1a/d5a/cce 0 2024-06-06T08:43:49.335 INFO:tasks.workunit.client.0.smithi027.stdout:5/665: rmdir d0/da/d20 39 2024-06-06T08:43:49.335 INFO:tasks.workunit.client.0.smithi027.stdout:5/666: fdatasync d0/da/d20/faf 0 2024-06-06T08:43:49.336 INFO:tasks.workunit.client.0.smithi027.stdout:5/667: write d0/f4 [2403540,92868] 0 2024-06-06T08:43:49.337 INFO:tasks.workunit.client.0.smithi027.stdout:5/668: mkdir d0/da/db/d46/d7f/d81/d97/db0/dbc/dcf 0 2024-06-06T08:43:49.337 INFO:tasks.workunit.client.0.smithi027.stdout:4/695: sync 2024-06-06T08:43:49.337 INFO:tasks.workunit.client.0.smithi027.stdout:8/578: sync 2024-06-06T08:43:49.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/581: sync 2024-06-06T08:43:49.338 INFO:tasks.workunit.client.0.smithi027.stdout:7/728: sync 2024-06-06T08:43:49.338 INFO:tasks.workunit.client.0.smithi027.stdout:2/640: sync 2024-06-06T08:43:49.338 INFO:tasks.workunit.client.0.smithi027.stdout:4/696: write d0/d1d/dd0/fd6 [387551,73617] 0 2024-06-06T08:43:49.338 INFO:tasks.workunit.client.0.smithi027.stdout:6/732: sync 2024-06-06T08:43:49.338 INFO:tasks.workunit.client.0.smithi027.stdout:6/733: fdatasync d4/d9/dc/d1c/f9e 0 2024-06-06T08:43:49.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/582: write d15/d18/f73 [4324766,129231] 0 2024-06-06T08:43:49.338 INFO:tasks.workunit.client.0.smithi027.stdout:1/558: sync 2024-06-06T08:43:49.339 INFO:tasks.workunit.client.0.smithi027.stdout:9/565: sync 2024-06-06T08:43:49.339 INFO:tasks.workunit.client.0.smithi027.stdout:3/545: sync 2024-06-06T08:43:49.340 INFO:tasks.workunit.client.0.smithi027.stdout:9/566: creat d3/d4e/d75/dc/fc9 x:0 0 0 2024-06-06T08:43:49.340 INFO:tasks.workunit.client.0.smithi027.stdout:6/734: mknod d4/de/d45/dcf/dea/cf0 0 2024-06-06T08:43:49.341 INFO:tasks.workunit.client.0.smithi027.stdout:2/641: creat d6/dc/d3f/fd0 x:0 0 0 2024-06-06T08:43:49.342 INFO:tasks.workunit.client.0.smithi027.stdout:1/559: write dc/f30 [196605,108356] 0 2024-06-06T08:43:49.343 INFO:tasks.workunit.client.0.smithi027.stdout:6/735: creat d4/d9/dc/d18/ff1 x:0 0 0 2024-06-06T08:43:49.344 INFO:tasks.workunit.client.0.smithi027.stdout:1/560: dread - dc/d67/d69/d32/f94 zero size 2024-06-06T08:43:49.346 INFO:tasks.workunit.client.0.smithi027.stdout:4/697: rename d0/d1d/d60/dc6/d6e/fb1 to d0/d1d/d34/d7f/d97/fe0 0 2024-06-06T08:43:49.347 INFO:tasks.workunit.client.0.smithi027.stdout:9/567: link d3/d95/d23/d36/la4 d3/d95/d23/d36/d3a/lca 0 2024-06-06T08:43:49.347 INFO:tasks.workunit.client.0.smithi027.stdout:9/568: write d3/d95/faa [4574570,122174] 0 2024-06-06T08:43:49.347 INFO:tasks.workunit.client.0.smithi027.stdout:9/569: chown d3/d4e/f9d 53143190 1 2024-06-06T08:43:49.350 INFO:tasks.workunit.client.0.smithi027.stdout:6/736: mkdir d4/d9/df2 0 2024-06-06T08:43:49.352 INFO:tasks.workunit.client.0.smithi027.stdout:1/561: rename dc/de/l59 to dc/d9d/laf 0 2024-06-06T08:43:49.355 INFO:tasks.workunit.client.0.smithi027.stdout:6/737: rmdir d4/de/d45 39 2024-06-06T08:43:49.355 INFO:tasks.workunit.client.0.smithi027.stdout:6/738: truncate d4/de/fe6 979883 0 2024-06-06T08:43:49.355 INFO:tasks.workunit.client.0.smithi027.stdout:6/739: fsync d4/fe4 0 2024-06-06T08:43:49.356 INFO:tasks.workunit.client.0.smithi027.stdout:4/698: getdents d0/d1d/d34/d7f/d97 0 2024-06-06T08:43:49.361 INFO:tasks.workunit.client.0.smithi027.stdout:8/579: dread d7/f4e [0,4194304] 0 2024-06-06T08:43:49.361 INFO:tasks.workunit.client.0.smithi027.stdout:0/583: dread d15/d18/d40/d8c/d9c/fad [0,4194304] 0 2024-06-06T08:43:49.361 INFO:tasks.workunit.client.0.smithi027.stdout:0/584: dread - d15/d18/d64/fbd zero size 2024-06-06T08:43:49.362 INFO:tasks.workunit.client.0.smithi027.stdout:0/585: readlink d15/d18/d23/l9a 0 2024-06-06T08:43:49.362 INFO:tasks.workunit.client.0.smithi027.stdout:8/580: chown d7/d13/c1b 48588 1 2024-06-06T08:43:49.363 INFO:tasks.workunit.client.0.smithi027.stdout:2/642: write d6/dd/f18 [2000911,1529] 0 2024-06-06T08:43:49.363 INFO:tasks.workunit.client.0.smithi027.stdout:0/586: symlink d15/d18/d23/d5b/da0/lc5 0 2024-06-06T08:43:49.365 INFO:tasks.workunit.client.0.smithi027.stdout:2/643: mkdir d6/dc/d3f/dd1 0 2024-06-06T08:43:49.366 INFO:tasks.workunit.client.0.smithi027.stdout:5/669: dwrite d0/fbf [0,4194304] 0 2024-06-06T08:43:49.366 INFO:tasks.workunit.client.0.smithi027.stdout:5/670: chown d0/da/db/fa0 1401853 1 2024-06-06T08:43:49.367 INFO:tasks.workunit.client.0.smithi027.stdout:9/570: dread d3/d95/d34/f48 [0,4194304] 0 2024-06-06T08:43:49.367 INFO:tasks.workunit.client.0.smithi027.stdout:4/699: rename d0/d63/c8a to d0/d1d/d34/d7f/d97/d9c/dbe/ce1 0 2024-06-06T08:43:49.367 INFO:tasks.workunit.client.0.smithi027.stdout:9/571: write d3/d95/d23/d36/f94 [202443,91291] 0 2024-06-06T08:43:49.369 INFO:tasks.workunit.client.0.smithi027.stdout:9/572: symlink d3/d4e/d75/db4/lcb 0 2024-06-06T08:43:49.369 INFO:tasks.workunit.client.0.smithi027.stdout:9/573: chown d3/d4e/d75/c70 1359 1 2024-06-06T08:43:49.372 INFO:tasks.workunit.client.0.smithi027.stdout:3/546: dread d1/d8/f55 [0,4194304] 0 2024-06-06T08:43:49.373 INFO:tasks.workunit.client.0.smithi027.stdout:3/547: getdents d1 0 2024-06-06T08:43:49.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/548: fsync d1/d90/f18 0 2024-06-06T08:43:49.374 INFO:tasks.workunit.client.0.smithi027.stdout:0/587: truncate d15/d18/d23/f27 628838 0 2024-06-06T08:43:49.375 INFO:tasks.workunit.client.0.smithi027.stdout:0/588: truncate d15/d18/d23/d28/d2a/d74/fb0 1249497 0 2024-06-06T08:43:49.375 INFO:tasks.workunit.client.0.smithi027.stdout:3/549: chown d1/d1f/d4d/f52 26 1 2024-06-06T08:43:49.375 INFO:tasks.workunit.client.0.smithi027.stdout:3/550: creat d1/d8/fa7 x:0 0 0 2024-06-06T08:43:49.376 INFO:tasks.workunit.client.0.smithi027.stdout:0/589: write d15/d18/d23/d6f/f94 [145676,50820] 0 2024-06-06T08:43:49.377 INFO:tasks.workunit.client.0.smithi027.stdout:0/590: read - d15/da7/fb2 zero size 2024-06-06T08:43:49.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/740: dwrite d4/d9/d49/fe2 [0,4194304] 0 2024-06-06T08:43:49.381 INFO:tasks.workunit.client.0.smithi027.stdout:0/591: symlink d15/d18/dbe/d5c/lc6 0 2024-06-06T08:43:49.383 INFO:tasks.workunit.client.0.smithi027.stdout:6/741: mkdir d4/d1a/d2e/df3 0 2024-06-06T08:43:49.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/742: symlink d4/d9/dc/d1c/dd2/lf4 0 2024-06-06T08:43:49.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/743: dread - d4/d1a/d2e/d44/dca/fec zero size 2024-06-06T08:43:49.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/744: fsync d4/d9/dc/d1c/f50 0 2024-06-06T08:43:49.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/745: chown d4/d9/dc5/fe1 7522 1 2024-06-06T08:43:49.385 INFO:tasks.workunit.client.0.smithi027.stdout:6/746: mkdir d4/d13/dba/df5 0 2024-06-06T08:43:49.385 INFO:tasks.workunit.client.0.smithi027.stdout:6/747: chown d4/d9/dc/d1c/dd2 245068322 1 2024-06-06T08:43:49.385 INFO:tasks.workunit.client.0.smithi027.stdout:0/592: mknod d15/d18/d23/d28/d2a/d2d/cc7 0 2024-06-06T08:43:49.385 INFO:tasks.workunit.client.0.smithi027.stdout:9/574: dread d3/d95/d23/d36/f5d [0,4194304] 0 2024-06-06T08:43:49.386 INFO:tasks.workunit.client.0.smithi027.stdout:6/748: truncate d4/d1a/d2e/d44/f5a 220350 0 2024-06-06T08:43:49.386 INFO:tasks.workunit.client.0.smithi027.stdout:6/749: write d4/f59 [104543,10240] 0 2024-06-06T08:43:49.386 INFO:tasks.workunit.client.0.smithi027.stdout:9/575: creat d3/d95/d34/d73/fcc x:0 0 0 2024-06-06T08:43:49.388 INFO:tasks.workunit.client.0.smithi027.stdout:6/750: symlink d4/d9/dc/d8f/lf6 0 2024-06-06T08:43:49.388 INFO:tasks.workunit.client.0.smithi027.stdout:9/576: symlink d3/d4e/d75/db0/lcd 0 2024-06-06T08:43:49.406 INFO:tasks.workunit.client.0.smithi027.stdout:1/562: dwrite dc/d67/d69/d32/f5b [4194304,4194304] 0 2024-06-06T08:43:49.407 INFO:tasks.workunit.client.0.smithi027.stdout:8/581: dwrite d7/dc/d37/f80 [0,4194304] 0 2024-06-06T08:43:49.408 INFO:tasks.workunit.client.0.smithi027.stdout:8/582: mkdir d7/dc/d36/d41/dbf/dc3 0 2024-06-06T08:43:49.411 INFO:tasks.workunit.client.0.smithi027.stdout:6/751: dwrite d4/d9/d49/f86 [0,4194304] 0 2024-06-06T08:43:49.412 INFO:tasks.workunit.client.0.smithi027.stdout:6/752: write d4/d1a/d2e/d44/dca/fec [618437,41622] 0 2024-06-06T08:43:49.412 INFO:tasks.workunit.client.0.smithi027.stdout:6/753: fdatasync d4/d9/f58 0 2024-06-06T08:43:49.413 INFO:tasks.workunit.client.0.smithi027.stdout:6/754: truncate d4/d1a/d2e/d44/dca/fe5 1217289 0 2024-06-06T08:43:49.414 INFO:tasks.workunit.client.0.smithi027.stdout:3/551: dwrite d1/d8/d22/d2d/f7f [0,4194304] 0 2024-06-06T08:43:49.421 INFO:tasks.workunit.client.0.smithi027.stdout:4/700: write d0/d1d/f69 [211782,116281] 0 2024-06-06T08:43:49.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/583: rmdir d7/dc/d36/d83/d58 39 2024-06-06T08:43:49.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/671: link d0/da/d20/c39 d0/da/db/d46/d7f/d81/cd0 0 2024-06-06T08:43:49.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/672: stat d0/da/db/d46/fc1 0 2024-06-06T08:43:49.422 INFO:tasks.workunit.client.0.smithi027.stdout:0/593: dwrite fc [4194304,4194304] 0 2024-06-06T08:43:49.422 INFO:tasks.workunit.client.0.smithi027.stdout:0/594: chown d15/d18/d40/d8c/d9c 1006091 1 2024-06-06T08:43:49.429 INFO:tasks.workunit.client.0.smithi027.stdout:4/701: readlink d0/d1/d6/la6 0 2024-06-06T08:43:49.432 INFO:tasks.workunit.client.0.smithi027.stdout:5/673: symlink d0/da/d20/ld1 0 2024-06-06T08:43:49.432 INFO:tasks.workunit.client.0.smithi027.stdout:5/674: getdents d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd 0 2024-06-06T08:43:49.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/755: dwrite d4/de/f6c [0,4194304] 0 2024-06-06T08:43:49.436 INFO:tasks.workunit.client.0.smithi027.stdout:3/552: dwrite d1/d8/d22/f95 [0,4194304] 0 2024-06-06T08:43:49.440 INFO:tasks.workunit.client.0.smithi027.stdout:8/584: dread d7/dc/d36/d83/fa4 [0,4194304] 0 2024-06-06T08:43:49.440 INFO:tasks.workunit.client.0.smithi027.stdout:6/756: dread d4/d1a/d2e/d44/f5a [0,4194304] 0 2024-06-06T08:43:49.440 INFO:tasks.workunit.client.0.smithi027.stdout:8/585: fdatasync d7/d2f/fb8 0 2024-06-06T08:43:49.440 INFO:tasks.workunit.client.0.smithi027.stdout:8/586: write d7/f2c [894428,129678] 0 2024-06-06T08:43:49.441 INFO:tasks.workunit.client.0.smithi027.stdout:6/757: truncate d4/d1a/d2e/d44/fc0 212485 0 2024-06-06T08:43:49.441 INFO:tasks.workunit.client.0.smithi027.stdout:3/553: rename d1/d1f/c7e to d1/d90/ca8 0 2024-06-06T08:43:49.442 INFO:tasks.workunit.client.0.smithi027.stdout:8/587: symlink d7/dc/lc4 0 2024-06-06T08:43:49.443 INFO:tasks.workunit.client.0.smithi027.stdout:8/588: truncate d7/dc/d36/d83/d60/f68 199325 0 2024-06-06T08:43:49.444 INFO:tasks.workunit.client.0.smithi027.stdout:2/644: dwrite d6/dc/d3f/d77/d55/f6d [0,4194304] 0 2024-06-06T08:43:49.448 INFO:tasks.workunit.client.0.smithi027.stdout:2/645: symlink d6/dc/d3f/d77/d55/d69/ld2 0 2024-06-06T08:43:49.450 INFO:tasks.workunit.client.0.smithi027.stdout:2/646: creat d6/fd3 x:0 0 0 2024-06-06T08:43:49.450 INFO:tasks.workunit.client.0.smithi027.stdout:2/647: chown d6/dc/d3f/d77/f4c 195039825 1 2024-06-06T08:43:49.454 INFO:tasks.workunit.client.0.smithi027.stdout:2/648: creat d6/dc/fd4 x:0 0 0 2024-06-06T08:43:49.456 INFO:tasks.workunit.client.0.smithi027.stdout:5/675: dread d0/da/f43 [0,4194304] 0 2024-06-06T08:43:49.458 INFO:tasks.workunit.client.0.smithi027.stdout:3/554: dread d1/d90/d44/f49 [0,4194304] 0 2024-06-06T08:43:49.458 INFO:tasks.workunit.client.0.smithi027.stdout:2/649: mknod d6/dc/d1d/d4d/d8e/cd5 0 2024-06-06T08:43:49.458 INFO:tasks.workunit.client.0.smithi027.stdout:6/758: read d4/de/d45/f75 [1497182,21170] 0 2024-06-06T08:43:49.460 INFO:tasks.workunit.client.0.smithi027.stdout:0/595: dwrite d15/fa1 [0,4194304] 0 2024-06-06T08:43:49.460 INFO:tasks.workunit.client.0.smithi027.stdout:0/596: fdatasync d15/d18/dbe/d5d/f98 0 2024-06-06T08:43:49.461 INFO:tasks.workunit.client.0.smithi027.stdout:0/597: creat d15/d18/d23/d5b/da0/fc8 x:0 0 0 2024-06-06T08:43:49.462 INFO:tasks.workunit.client.0.smithi027.stdout:0/598: mknod d15/d18/d23/d28/cc9 0 2024-06-06T08:43:49.462 INFO:tasks.workunit.client.0.smithi027.stdout:2/650: creat d6/d1e/d2b/d5a/fd6 x:0 0 0 2024-06-06T08:43:49.464 INFO:tasks.workunit.client.0.smithi027.stdout:0/599: creat d15/d18/d40/d8c/fca x:0 0 0 2024-06-06T08:43:49.464 INFO:tasks.workunit.client.0.smithi027.stdout:6/759: mkdir d4/de/d45/df7 0 2024-06-06T08:43:49.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/676: stat d0/da/db/d46/d77/f9d 0 2024-06-06T08:43:49.464 INFO:tasks.workunit.client.0.smithi027.stdout:0/600: chown d15/d18/d23/d38/d68/c72 578371 1 2024-06-06T08:43:49.465 INFO:tasks.workunit.client.0.smithi027.stdout:0/601: write d15/d18/d40/fc2 [262076,57664] 0 2024-06-06T08:43:49.466 INFO:tasks.workunit.client.0.smithi027.stdout:0/602: mkdir d15/d18/d23/d6f/dcb 0 2024-06-06T08:43:49.467 INFO:tasks.workunit.client.0.smithi027.stdout:3/555: mkdir d1/d8/da9 0 2024-06-06T08:43:49.468 INFO:tasks.workunit.client.0.smithi027.stdout:0/603: rmdir d15/d18/d23/d6f 39 2024-06-06T08:43:49.477 INFO:tasks.workunit.client.0.smithi027.stdout:6/760: creat d4/de/d45/df7/ff8 x:0 0 0 2024-06-06T08:43:49.477 INFO:tasks.workunit.client.0.smithi027.stdout:2/651: rename d6/l31 to d6/d1e/d2b/d5a/d7b/ld7 0 2024-06-06T08:43:49.479 INFO:tasks.workunit.client.0.smithi027.stdout:0/604: dread d15/d18/d23/d38/d68/d85/f8d [0,4194304] 0 2024-06-06T08:43:49.480 INFO:tasks.workunit.client.0.smithi027.stdout:0/605: rmdir d15/d18/d23/d28/d2a 39 2024-06-06T08:43:49.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/677: write d0/da/db/d46/d77/f9d [160358,646] 0 2024-06-06T08:43:49.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/678: readlink d0/da/d37/l74 0 2024-06-06T08:43:49.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/679: chown d0/da/db 17249603 1 2024-06-06T08:43:49.493 INFO:tasks.workunit.client.0.smithi027.stdout:2/652: dread d6/f2c [0,4194304] 0 2024-06-06T08:43:49.493 INFO:tasks.workunit.client.0.smithi027.stdout:5/680: read d0/f24 [5002949,113423] 0 2024-06-06T08:43:49.494 INFO:tasks.workunit.client.0.smithi027.stdout:2/653: truncate d6/fe 891076 0 2024-06-06T08:43:49.496 INFO:tasks.workunit.client.0.smithi027.stdout:3/556: dwrite d1/d1f/d25/f5d [0,4194304] 0 2024-06-06T08:43:49.502 INFO:tasks.workunit.client.0.smithi027.stdout:2/654: write d6/dc/d3f/f50 [546495,40470] 0 2024-06-06T08:43:49.502 INFO:tasks.workunit.client.0.smithi027.stdout:6/761: dwrite d4/de/d45/f89 [0,4194304] 0 2024-06-06T08:43:49.504 INFO:tasks.workunit.client.0.smithi027.stdout:0/606: dwrite d15/dae/fbf [0,4194304] 0 2024-06-06T08:43:49.504 INFO:tasks.workunit.client.0.smithi027.stdout:0/607: readlink d15/d18/l1d 0 2024-06-06T08:43:49.505 INFO:tasks.workunit.client.0.smithi027.stdout:0/608: dread - d15/d18/d23/d38/d68/f9d zero size 2024-06-06T08:43:49.505 INFO:tasks.workunit.client.0.smithi027.stdout:6/762: creat d4/de/d45/dcf/ff9 x:0 0 0 2024-06-06T08:43:49.509 INFO:tasks.workunit.client.0.smithi027.stdout:6/763: mkdir d4/d9/dc/d1c/dfa 0 2024-06-06T08:43:49.512 INFO:tasks.workunit.client.0.smithi027.stdout:2/655: write d6/fe [773161,70429] 0 2024-06-06T08:43:49.514 INFO:tasks.workunit.client.0.smithi027.stdout:0/609: getdents d15/d18/d64 0 2024-06-06T08:43:49.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/610: creat d15/d18/d23/d28/d31/dbb/fcc x:0 0 0 2024-06-06T08:43:49.516 INFO:tasks.workunit.client.0.smithi027.stdout:3/557: dwrite d1/d8/d22/d27/d50/f51 [0,4194304] 0 2024-06-06T08:43:49.517 INFO:tasks.workunit.client.0.smithi027.stdout:6/764: rename d4/f7f to d4/ffb 0 2024-06-06T08:43:49.522 INFO:tasks.workunit.client.0.smithi027.stdout:6/765: symlink d4/d1a/lfc 0 2024-06-06T08:43:49.522 INFO:tasks.workunit.client.0.smithi027.stdout:5/681: dwrite d0/da/db/f64 [0,4194304] 0 2024-06-06T08:43:49.524 INFO:tasks.workunit.client.0.smithi027.stdout:5/682: fsync d0/da/f35 0 2024-06-06T08:43:49.525 INFO:tasks.workunit.client.0.smithi027.stdout:6/766: mknod d4/d9/cfd 0 2024-06-06T08:43:49.525 INFO:tasks.workunit.client.0.smithi027.stdout:6/767: truncate d4/d1a/d2e/d44/dca/fec 1224648 0 2024-06-06T08:43:49.529 INFO:tasks.workunit.client.0.smithi027.stdout:5/683: unlink d0/da/fa2 0 2024-06-06T08:43:49.531 INFO:tasks.workunit.client.0.smithi027.stdout:6/768: rename d4/da2/fb5 to d4/d13/ffe 0 2024-06-06T08:43:49.532 INFO:tasks.workunit.client.0.smithi027.stdout:6/769: dread d4/de/fe6 [0,4194304] 0 2024-06-06T08:43:49.533 INFO:tasks.workunit.client.0.smithi027.stdout:5/684: creat d0/da/db/d46/d7f/d81/fd2 x:0 0 0 2024-06-06T08:43:49.533 INFO:tasks.workunit.client.0.smithi027.stdout:5/685: write d0/da/d20/faf [624349,19081] 0 2024-06-06T08:43:49.533 INFO:tasks.workunit.client.0.smithi027.stdout:3/558: dwrite d1/d1f/d25/f76 [0,4194304] 0 2024-06-06T08:43:49.534 INFO:tasks.workunit.client.0.smithi027.stdout:6/770: creat d4/d13/fff x:0 0 0 2024-06-06T08:43:49.534 INFO:tasks.workunit.client.0.smithi027.stdout:0/611: dread d15/d18/d23/f27 [0,4194304] 0 2024-06-06T08:43:49.536 INFO:tasks.workunit.client.0.smithi027.stdout:3/559: readlink d1/d8/l65 0 2024-06-06T08:43:49.536 INFO:tasks.workunit.client.0.smithi027.stdout:0/612: creat d15/d18/d23/d6f/fcd x:0 0 0 2024-06-06T08:43:49.536 INFO:tasks.workunit.client.0.smithi027.stdout:0/613: stat d15/d18/f81 0 2024-06-06T08:43:49.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/560: creat d1/d8/d22/d2d/faa x:0 0 0 2024-06-06T08:43:49.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/561: truncate d1/d1f/d5b/f9f 746833 0 2024-06-06T08:43:49.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/562: truncate d1/d1f/d25/f8d 663188 0 2024-06-06T08:43:49.538 INFO:tasks.workunit.client.0.smithi027.stdout:0/614: mknod d15/d18/dbe/d5d/cce 0 2024-06-06T08:43:49.539 INFO:tasks.workunit.client.0.smithi027.stdout:3/563: creat d1/d8/fab x:0 0 0 2024-06-06T08:43:49.539 INFO:tasks.workunit.client.0.smithi027.stdout:3/564: readlink d1/d8/d59/l8b 0 2024-06-06T08:43:49.544 INFO:tasks.workunit.client.0.smithi027.stdout:0/615: rmdir d15/d18/d40/d8c 39 2024-06-06T08:43:49.554 INFO:tasks.workunit.client.0.smithi027.stdout:3/565: dwrite d1/f85 [0,4194304] 0 2024-06-06T08:43:49.554 INFO:tasks.workunit.client.0.smithi027.stdout:0/616: dread f9 [0,4194304] 0 2024-06-06T08:43:49.555 INFO:tasks.workunit.client.0.smithi027.stdout:0/617: chown d15/d18/d40/l66 5297 1 2024-06-06T08:43:49.555 INFO:tasks.workunit.client.0.smithi027.stdout:0/618: write f2 [1075023,46929] 0 2024-06-06T08:43:49.556 INFO:tasks.workunit.client.0.smithi027.stdout:5/686: dwrite d0/da/fae [8388608,4194304] 0 2024-06-06T08:43:49.556 INFO:tasks.workunit.client.0.smithi027.stdout:0/619: write d15/d18/d40/d8c/d9c/fad [4703079,124775] 0 2024-06-06T08:43:49.556 INFO:tasks.workunit.client.0.smithi027.stdout:0/620: getdents d15/d18/d40/d8c/d9c 0 2024-06-06T08:43:49.556 INFO:tasks.workunit.client.0.smithi027.stdout:0/621: chown d15/fa1 201393 1 2024-06-06T08:43:49.558 INFO:tasks.workunit.client.0.smithi027.stdout:3/566: mkdir d1/d90/d6d/dac 0 2024-06-06T08:43:49.560 INFO:tasks.workunit.client.0.smithi027.stdout:6/771: link d4/d9/d49/l8a d4/l100 0 2024-06-06T08:43:49.560 INFO:tasks.workunit.client.0.smithi027.stdout:6/772: fdatasync d4/d9/dc/d1c/f9e 0 2024-06-06T08:43:49.560 INFO:tasks.workunit.client.0.smithi027.stdout:5/687: rmdir d0/da/db/d46/d4c 39 2024-06-06T08:43:49.561 INFO:tasks.workunit.client.0.smithi027.stdout:3/567: creat d1/d1f/d45/fad x:0 0 0 2024-06-06T08:43:49.564 INFO:tasks.workunit.client.0.smithi027.stdout:0/622: creat d15/d18/d64/fcf x:0 0 0 2024-06-06T08:43:49.564 INFO:tasks.workunit.client.0.smithi027.stdout:3/568: read d1/d8/d22/d27/d50/f72 [11954,89998] 0 2024-06-06T08:43:49.564 INFO:tasks.workunit.client.0.smithi027.stdout:3/569: fdatasync d1/d8/f8a 0 2024-06-06T08:43:49.565 INFO:tasks.workunit.client.0.smithi027.stdout:3/570: dread d1/d1f/d25/f8d [0,4194304] 0 2024-06-06T08:43:49.566 INFO:tasks.workunit.client.0.smithi027.stdout:3/571: creat d1/d8/d22/d27/fae x:0 0 0 2024-06-06T08:43:49.567 INFO:tasks.workunit.client.0.smithi027.stdout:0/623: link d15/d18/d23/c45 d15/d18/d23/d3c/cd0 0 2024-06-06T08:43:49.568 INFO:tasks.workunit.client.0.smithi027.stdout:0/624: dread - d15/d18/d23/d28/d31/dbb/fcc zero size 2024-06-06T08:43:49.576 INFO:tasks.workunit.client.0.smithi027.stdout:3/572: write d1/d8/d22/d2d/f7f [3236172,45551] 0 2024-06-06T08:43:49.576 INFO:tasks.workunit.client.0.smithi027.stdout:3/573: chown d1/d8/d59/f94 1026660 1 2024-06-06T08:43:49.576 INFO:tasks.workunit.client.0.smithi027.stdout:6/773: dwrite d4/de/d45/d6a/f76 [0,4194304] 0 2024-06-06T08:43:49.578 INFO:tasks.workunit.client.0.smithi027.stdout:3/574: symlink d1/d1f/laf 0 2024-06-06T08:43:49.580 INFO:tasks.workunit.client.0.smithi027.stdout:3/575: creat d1/d90/d44/fb0 x:0 0 0 2024-06-06T08:43:49.587 INFO:tasks.workunit.client.0.smithi027.stdout:3/576: write d1/f1e [225603,91003] 0 2024-06-06T08:43:49.588 INFO:tasks.workunit.client.0.smithi027.stdout:3/577: truncate d1/d8/d59/f94 942153 0 2024-06-06T08:43:49.588 INFO:tasks.workunit.client.0.smithi027.stdout:3/578: dread - d1/d90/d44/d54/f63 zero size 2024-06-06T08:43:49.589 INFO:tasks.workunit.client.0.smithi027.stdout:5/688: dwrite d0/da/db/f17 [0,4194304] 0 2024-06-06T08:43:49.589 INFO:tasks.workunit.client.0.smithi027.stdout:3/579: creat d1/d90/d44/d54/fb1 x:0 0 0 2024-06-06T08:43:49.589 INFO:tasks.workunit.client.0.smithi027.stdout:5/689: dread d0/da/db/d1a/f6f [0,4194304] 0 2024-06-06T08:43:49.590 INFO:tasks.workunit.client.0.smithi027.stdout:5/690: truncate d0/da/db/f1b 4363369 0 2024-06-06T08:43:49.591 INFO:tasks.workunit.client.0.smithi027.stdout:5/691: chown d0/da/db/d1a/d5a/f83 32427 1 2024-06-06T08:43:49.592 INFO:tasks.workunit.client.0.smithi027.stdout:6/774: dwrite d4/d9/d49/f8d [0,4194304] 0 2024-06-06T08:43:49.592 INFO:tasks.workunit.client.0.smithi027.stdout:6/775: stat d4/d9/dc/d18/fd0 0 2024-06-06T08:43:49.593 INFO:tasks.workunit.client.0.smithi027.stdout:6/776: truncate d4/d1a/d2e/d44/f99 2520049 0 2024-06-06T08:43:49.593 INFO:tasks.workunit.client.0.smithi027.stdout:6/777: dread - d4/de/d45/dcf/ff9 zero size 2024-06-06T08:43:49.595 INFO:tasks.workunit.client.0.smithi027.stdout:5/692: getdents d0/da/d47 0 2024-06-06T08:43:49.597 INFO:tasks.workunit.client.0.smithi027.stdout:5/693: chown d0/da/d37/d2b/f4a 2710 1 2024-06-06T08:43:49.611 INFO:tasks.workunit.client.0.smithi027.stdout:6/778: write d4/de/d45/d6a/f4b [708472,31523] 0 2024-06-06T08:43:49.611 INFO:tasks.workunit.client.0.smithi027.stdout:6/779: fdatasync d4/d13/f21 0 2024-06-06T08:43:49.620 INFO:tasks.workunit.client.0.smithi027.stdout:5/694: dread d0/da/f72 [0,4194304] 0 2024-06-06T08:43:49.627 INFO:tasks.workunit.client.0.smithi027.stdout:6/780: dwrite d4/de/d45/d6a/f76 [0,4194304] 0 2024-06-06T08:43:49.627 INFO:tasks.workunit.client.0.smithi027.stdout:6/781: fdatasync d4/ffb 0 2024-06-06T08:43:49.634 INFO:tasks.workunit.client.0.smithi027.stdout:5/695: rename d0/da/db/d46/d7f/d81/cc3 to d0/da/db/d1a/da3/cd3 0 2024-06-06T08:43:49.637 INFO:tasks.workunit.client.0.smithi027.stdout:5/696: link d0/da/db/d1a/c5e d0/da/d37/d2b/cd4 0 2024-06-06T08:43:49.640 INFO:tasks.workunit.client.0.smithi027.stdout:5/697: mknod d0/da/db/cd5 0 2024-06-06T08:43:49.641 INFO:tasks.workunit.client.0.smithi027.stdout:5/698: dread - d0/da/db/d46/d77/f79 zero size 2024-06-06T08:43:49.641 INFO:tasks.workunit.client.0.smithi027.stdout:5/699: fsync d0/da/d47/f94 0 2024-06-06T08:43:49.642 INFO:tasks.workunit.client.0.smithi027.stdout:5/700: creat d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd/fd6 x:0 0 0 2024-06-06T08:43:49.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/782: dwrite d4/d9/dc5/fe1 [0,4194304] 0 2024-06-06T08:43:49.657 INFO:tasks.workunit.client.0.smithi027.stdout:6/783: unlink d4/d9/dc/fc8 0 2024-06-06T08:43:49.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/701: dwrite d0/da/db/d46/d7f/f8e [0,4194304] 0 2024-06-06T08:43:49.660 INFO:tasks.workunit.client.0.smithi027.stdout:5/702: rmdir d0/da/db/d46/d7f/d81/d97/db0/dbc 39 2024-06-06T08:43:49.661 INFO:tasks.workunit.client.0.smithi027.stdout:7/729: sync 2024-06-06T08:43:49.664 INFO:tasks.workunit.client.0.smithi027.stdout:7/730: mkdir d17/d32/d6c/ded 0 2024-06-06T08:43:49.664 INFO:tasks.workunit.client.0.smithi027.stdout:7/731: write d17/d32/d46/d4f/f50 [2167850,39628] 0 2024-06-06T08:43:49.667 INFO:tasks.workunit.client.0.smithi027.stdout:6/784: link d4/d9/dc/d1c/cee d4/de/d45/d6a/de8/c101 0 2024-06-06T08:43:49.668 INFO:tasks.workunit.client.0.smithi027.stdout:6/785: creat d4/f102 x:0 0 0 2024-06-06T08:43:49.669 INFO:tasks.workunit.client.0.smithi027.stdout:7/732: write d17/d32/d46/d4f/d5e/f79 [291406,54777] 0 2024-06-06T08:43:49.670 INFO:tasks.workunit.client.0.smithi027.stdout:7/733: read d17/d47/f49 [73832,13122] 0 2024-06-06T08:43:49.672 INFO:tasks.workunit.client.0.smithi027.stdout:7/734: mknod d17/d32/d46/d51/d7f/db2/cee 0 2024-06-06T08:43:49.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/735: mknod d17/d32/d46/d4f/d5e/d7b/cef 0 2024-06-06T08:43:49.678 INFO:tasks.workunit.client.0.smithi027.stdout:5/703: dwrite d0/da/db/d46/d77/f9d [0,4194304] 0 2024-06-06T08:43:49.678 INFO:tasks.workunit.client.0.smithi027.stdout:7/736: symlink d17/d32/d46/d4f/d5e/lf0 0 2024-06-06T08:43:49.681 INFO:tasks.workunit.client.0.smithi027.stdout:6/786: dread d4/d9/dc/d18/fd0 [0,4194304] 0 2024-06-06T08:43:49.683 INFO:tasks.workunit.client.0.smithi027.stdout:7/737: creat d17/d47/ff1 x:0 0 0 2024-06-06T08:43:49.684 INFO:tasks.workunit.client.0.smithi027.stdout:6/787: readlink d4/l33 0 2024-06-06T08:43:49.686 INFO:tasks.workunit.client.0.smithi027.stdout:5/704: truncate d0/da/d37/d2b/f5b 2906864 0 2024-06-06T08:43:49.687 INFO:tasks.workunit.client.0.smithi027.stdout:7/738: write d17/fa4 [2035711,32385] 0 2024-06-06T08:43:49.690 INFO:tasks.workunit.client.0.smithi027.stdout:5/705: symlink d0/da/db/d1a/dc6/ld7 0 2024-06-06T08:43:49.690 INFO:tasks.workunit.client.0.smithi027.stdout:5/706: fdatasync d0/da/db/d46/d7f/d81/fb5 0 2024-06-06T08:43:49.691 INFO:tasks.workunit.client.0.smithi027.stdout:5/707: write d0/da/d20/f36 [5297584,67383] 0 2024-06-06T08:43:49.691 INFO:tasks.workunit.client.0.smithi027.stdout:5/708: chown d0/f24 0 1 2024-06-06T08:43:49.694 INFO:tasks.workunit.client.0.smithi027.stdout:5/709: creat d0/da/db/d1a/dc6/fd8 x:0 0 0 2024-06-06T08:43:49.698 INFO:tasks.workunit.client.0.smithi027.stdout:5/710: unlink d0/da/db/d1a/f7b 0 2024-06-06T08:43:49.699 INFO:tasks.workunit.client.0.smithi027.stdout:6/788: dwrite d4/d9/dc/f43 [4194304,4194304] 0 2024-06-06T08:43:49.700 INFO:tasks.workunit.client.0.smithi027.stdout:7/739: truncate d17/d32/fca 5143262 0 2024-06-06T08:43:49.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/711: getdents d0/da/db 0 2024-06-06T08:43:49.703 INFO:tasks.workunit.client.0.smithi027.stdout:5/712: mkdir d0/da/db/d46/d7f/d81/d97/db0/dbc/dd9 0 2024-06-06T08:43:49.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/789: creat d4/d9/dc5/f103 x:0 0 0 2024-06-06T08:43:49.705 INFO:tasks.workunit.client.0.smithi027.stdout:7/740: write d17/f1e [4380668,1412] 0 2024-06-06T08:43:49.710 INFO:tasks.workunit.client.0.smithi027.stdout:6/790: stat d4/d1a/d2e/d44/fc0 0 2024-06-06T08:43:49.721 INFO:tasks.workunit.client.0.smithi027.stdout:5/713: dread d0/da/d37/d2b/f5b [0,4194304] 0 2024-06-06T08:43:49.724 INFO:tasks.workunit.client.0.smithi027.stdout:5/714: creat d0/da/db/d1a/fda x:0 0 0 2024-06-06T08:43:49.724 INFO:tasks.workunit.client.0.smithi027.stdout:5/715: chown d0/da/f92 21292494 1 2024-06-06T08:43:49.726 INFO:tasks.workunit.client.0.smithi027.stdout:5/716: symlink d0/da/db/d1a/d7e/ldb 0 2024-06-06T08:43:49.726 INFO:tasks.workunit.client.0.smithi027.stdout:6/791: write d4/de/d45/d6a/f76 [1133246,80557] 0 2024-06-06T08:43:49.727 INFO:tasks.workunit.client.0.smithi027.stdout:6/792: creat d4/d9/d6e/dd4/f104 x:0 0 0 2024-06-06T08:43:49.728 INFO:tasks.workunit.client.0.smithi027.stdout:5/717: mkdir d0/da/db/d46/d7f/d81/ddc 0 2024-06-06T08:43:49.729 INFO:tasks.workunit.client.0.smithi027.stdout:6/793: symlink d4/d13/l105 0 2024-06-06T08:43:49.730 INFO:tasks.workunit.client.0.smithi027.stdout:5/718: write d0/da/db/d1a/da3/fa6 [814660,43313] 0 2024-06-06T08:43:49.730 INFO:tasks.workunit.client.0.smithi027.stdout:5/719: dread - d0/da/db/d1a/fda zero size 2024-06-06T08:43:49.731 INFO:tasks.workunit.client.0.smithi027.stdout:5/720: getdents d0/da/db/d46/d7f/d81/d97 0 2024-06-06T08:43:49.742 INFO:tasks.workunit.client.0.smithi027.stdout:6/794: write d4/d9/d49/fe2 [3965958,7513] 0 2024-06-06T08:43:49.743 INFO:tasks.workunit.client.0.smithi027.stdout:6/795: getdents d4/d1a/d2e/df3 0 2024-06-06T08:43:49.746 INFO:tasks.workunit.client.0.smithi027.stdout:6/796: symlink d4/d9/l106 0 2024-06-06T08:43:49.747 INFO:tasks.workunit.client.0.smithi027.stdout:6/797: creat d4/d9/dc/d1c/dfa/f107 x:0 0 0 2024-06-06T08:43:49.749 INFO:tasks.workunit.client.0.smithi027.stdout:6/798: creat d4/d9/dc/d1c/d52/f108 x:0 0 0 2024-06-06T08:43:49.750 INFO:tasks.workunit.client.0.smithi027.stdout:5/721: dwrite d0/da/db/d46/d7f/d81/fd2 [0,4194304] 0 2024-06-06T08:43:49.750 INFO:tasks.workunit.client.0.smithi027.stdout:5/722: getdents d0/da/db/d46/d7f/d81/d97/db0/dbc/dcf 0 2024-06-06T08:43:49.751 INFO:tasks.workunit.client.0.smithi027.stdout:6/799: rename d4/f57 to d4/de/d45/dcf/f109 0 2024-06-06T08:43:49.752 INFO:tasks.workunit.client.0.smithi027.stdout:5/723: creat d0/da/db/d46/d7f/d81/d97/db0/fdd x:0 0 0 2024-06-06T08:43:49.756 INFO:tasks.workunit.client.0.smithi027.stdout:6/800: mkdir d4/d9/dc/d18/d10a 0 2024-06-06T08:43:49.756 INFO:tasks.workunit.client.0.smithi027.stdout:1/563: sync 2024-06-06T08:43:49.756 INFO:tasks.workunit.client.0.smithi027.stdout:9/577: sync 2024-06-06T08:43:49.757 INFO:tasks.workunit.client.0.smithi027.stdout:9/578: write d3/d4e/fa8 [898876,78018] 0 2024-06-06T08:43:49.760 INFO:tasks.workunit.client.0.smithi027.stdout:9/579: dread d3/d4e/d75/f55 [0,4194304] 0 2024-06-06T08:43:49.762 INFO:tasks.workunit.client.0.smithi027.stdout:6/801: dread d4/d9/d49/f71 [0,4194304] 0 2024-06-06T08:43:49.763 INFO:tasks.workunit.client.0.smithi027.stdout:6/802: chown d4/l47 15 1 2024-06-06T08:43:49.771 INFO:tasks.workunit.client.0.smithi027.stdout:5/724: dwrite d0/da/f72 [4194304,4194304] 0 2024-06-06T08:43:49.772 INFO:tasks.workunit.client.0.smithi027.stdout:1/564: dwrite dc/d67/d69/d32/d37/d56/d78/f83 [0,4194304] 0 2024-06-06T08:43:49.775 INFO:tasks.workunit.client.0.smithi027.stdout:1/565: mkdir dc/de/d14/d80/da6/db0 0 2024-06-06T08:43:49.775 INFO:tasks.workunit.client.0.smithi027.stdout:1/566: chown dc/de/d14/d4e/f5d 1 1 2024-06-06T08:43:49.782 INFO:tasks.workunit.client.0.smithi027.stdout:9/580: dwrite d3/d4e/d75/dc/f80 [0,4194304] 0 2024-06-06T08:43:49.796 INFO:tasks.workunit.client.0.smithi027.stdout:6/803: dwrite d4/d13/fff [0,4194304] 0 2024-06-06T08:43:49.797 INFO:tasks.workunit.client.0.smithi027.stdout:5/725: dwrite d0/da/f72 [0,4194304] 0 2024-06-06T08:43:49.801 INFO:tasks.workunit.client.0.smithi027.stdout:1/567: dwrite dc/de/d14/f73 [0,4194304] 0 2024-06-06T08:43:49.802 INFO:tasks.workunit.client.0.smithi027.stdout:1/568: write dc/f36 [901894,89216] 0 2024-06-06T08:43:49.804 INFO:tasks.workunit.client.0.smithi027.stdout:1/569: symlink dc/d67/d69/d32/d71/lb1 0 2024-06-06T08:43:49.811 INFO:tasks.workunit.client.0.smithi027.stdout:6/804: dread d4/de/d45/d6a/f4b [0,4194304] 0 2024-06-06T08:43:49.812 INFO:tasks.workunit.client.0.smithi027.stdout:6/805: read d4/d9/d49/f77 [863108,128719] 0 2024-06-06T08:43:49.820 INFO:tasks.workunit.client.0.smithi027.stdout:5/726: dwrite d0/da/db/d1a/f66 [0,4194304] 0 2024-06-06T08:43:49.822 INFO:tasks.workunit.client.0.smithi027.stdout:5/727: stat d0/d6a/l89 0 2024-06-06T08:43:49.825 INFO:tasks.workunit.client.0.smithi027.stdout:5/728: unlink d0/da/db/f17 0 2024-06-06T08:43:49.826 INFO:tasks.workunit.client.0.smithi027.stdout:5/729: creat d0/da/db/d1a/fde x:0 0 0 2024-06-06T08:43:49.826 INFO:tasks.workunit.client.0.smithi027.stdout:5/730: dread - d0/da/d47/fa8 zero size 2024-06-06T08:43:49.826 INFO:tasks.workunit.client.0.smithi027.stdout:5/731: write d0/da/d37/d2b/f3d [2304783,91740] 0 2024-06-06T08:43:49.828 INFO:tasks.workunit.client.0.smithi027.stdout:5/732: unlink d0/da/db/d46/d4c/fa7 0 2024-06-06T08:43:49.829 INFO:tasks.workunit.client.0.smithi027.stdout:5/733: mknod d0/da/d47/cdf 0 2024-06-06T08:43:49.843 INFO:tasks.workunit.client.0.smithi027.stdout:9/581: dwrite d3/f20 [0,4194304] 0 2024-06-06T08:43:49.844 INFO:tasks.workunit.client.0.smithi027.stdout:9/582: unlink d3/d95/d23/d36/d3a/f67 0 2024-06-06T08:43:49.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T08:43:49.916 INFO:tasks.workunit.client.0.smithi027.stdout:5/734: dwrite d0/da/f72 [8388608,4194304] 0 2024-06-06T08:43:49.918 INFO:tasks.workunit.client.0.smithi027.stdout:2/656: sync 2024-06-06T08:43:49.919 INFO:tasks.workunit.client.0.smithi027.stdout:3/580: sync 2024-06-06T08:43:49.919 INFO:tasks.workunit.client.0.smithi027.stdout:4/702: sync 2024-06-06T08:43:49.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/589: sync 2024-06-06T08:43:49.919 INFO:tasks.workunit.client.0.smithi027.stdout:0/625: sync 2024-06-06T08:43:49.920 INFO:tasks.workunit.client.0.smithi027.stdout:4/703: creat d0/d1/d6/d8/fe2 x:0 0 0 2024-06-06T08:43:49.922 INFO:tasks.workunit.client.0.smithi027.stdout:0/626: unlink d15/d18/d23/d5b/l91 0 2024-06-06T08:43:49.922 INFO:tasks.workunit.client.0.smithi027.stdout:2/657: creat d6/dc/d3f/fd8 x:0 0 0 2024-06-06T08:43:49.922 INFO:tasks.workunit.client.0.smithi027.stdout:7/741: sync 2024-06-06T08:43:49.922 INFO:tasks.workunit.client.0.smithi027.stdout:3/581: mknod d1/d1f/d45/d5f/cb2 0 2024-06-06T08:43:49.923 INFO:tasks.workunit.client.0.smithi027.stdout:3/582: dread - d1/d8/d22/d27/fae zero size 2024-06-06T08:43:49.923 INFO:tasks.workunit.client.0.smithi027.stdout:4/704: chown d0/d1/d6/d4b/f80 115 1 2024-06-06T08:43:49.924 INFO:tasks.workunit.client.0.smithi027.stdout:7/742: mknod d17/d32/d6c/ded/cf2 0 2024-06-06T08:43:49.925 INFO:tasks.workunit.client.0.smithi027.stdout:7/743: creat d17/d32/d46/d51/d52/ff3 x:0 0 0 2024-06-06T08:43:49.926 INFO:tasks.workunit.client.0.smithi027.stdout:3/583: getdents d1/d8/d22/d27/d50 0 2024-06-06T08:43:49.928 INFO:tasks.workunit.client.0.smithi027.stdout:3/584: creat d1/d1f/d5b/fb3 x:0 0 0 2024-06-06T08:43:49.931 INFO:tasks.workunit.client.0.smithi027.stdout:8/590: truncate d7/dc/d36/d83/f11 329924 0 2024-06-06T08:43:49.931 INFO:tasks.workunit.client.0.smithi027.stdout:8/591: readlink d7/dc/d36/d41/d85/la1 0 2024-06-06T08:43:49.931 INFO:tasks.workunit.client.0.smithi027.stdout:3/585: mkdir d1/d90/d44/d54/db4 0 2024-06-06T08:43:49.935 INFO:tasks.workunit.client.0.smithi027.stdout:7/744: write d17/d32/d46/d4f/f55 [597126,20329] 0 2024-06-06T08:43:49.935 INFO:tasks.workunit.client.0.smithi027.stdout:5/735: dwrite d0/da/db/d46/d77/f9d [0,4194304] 0 2024-06-06T08:43:49.938 INFO:tasks.workunit.client.0.smithi027.stdout:2/658: rename d6/dc to d6/dd/dd9 0 2024-06-06T08:43:49.938 INFO:tasks.workunit.client.0.smithi027.stdout:8/592: mknod d7/dc/d36/d41/d85/cc5 0 2024-06-06T08:43:49.940 INFO:tasks.workunit.client.0.smithi027.stdout:0/627: write d15/d18/d64/f87 [855385,45476] 0 2024-06-06T08:43:49.942 INFO:tasks.workunit.client.0.smithi027.stdout:5/736: mkdir d0/da/db/d1a/d5a/de0 0 2024-06-06T08:43:49.942 INFO:tasks.workunit.client.0.smithi027.stdout:7/745: write d17/d32/fca [4962535,75410] 0 2024-06-06T08:43:49.943 INFO:tasks.workunit.client.0.smithi027.stdout:0/628: unlink d15/d18/dbe/d5d/ca2 0 2024-06-06T08:43:49.949 INFO:tasks.workunit.client.0.smithi027.stdout:5/737: rename d0/f44 to d0/da/db/d1a/da9/fe1 0 2024-06-06T08:43:49.949 INFO:tasks.workunit.client.0.smithi027.stdout:5/738: chown d0/da/db/fa0 30 1 2024-06-06T08:43:49.951 INFO:tasks.workunit.client.0.smithi027.stdout:5/739: rmdir d0/da/db/d46/d77 39 2024-06-06T08:43:49.951 INFO:tasks.workunit.client.0.smithi027.stdout:5/740: chown d0/da/db/d46/d7f/d81/d97/f9c 47 1 2024-06-06T08:43:49.952 INFO:tasks.workunit.client.0.smithi027.stdout:5/741: symlink d0/da/d20/le2 0 2024-06-06T08:43:49.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/586: dwrite d1/d8/d22/d27/f32 [0,4194304] 0 2024-06-06T08:43:49.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/587: chown d1/d1f/d25/f8d 872926942 1 2024-06-06T08:43:49.955 INFO:tasks.workunit.client.0.smithi027.stdout:5/742: rename d0/da/db/d1a/f78 to d0/da/db/d46/d4c/d70/fe3 0 2024-06-06T08:43:49.959 INFO:tasks.workunit.client.0.smithi027.stdout:5/743: symlink d0/da/d37/d2b/le4 0 2024-06-06T08:43:49.961 INFO:tasks.workunit.client.0.smithi027.stdout:7/746: dread d17/d32/d46/d4f/d5e/f67 [0,4194304] 0 2024-06-06T08:43:49.961 INFO:tasks.workunit.client.0.smithi027.stdout:7/747: fsync d17/d32/d46/d4f/f8f 0 2024-06-06T08:43:49.964 INFO:tasks.workunit.client.0.smithi027.stdout:8/593: dwrite d7/d2f/fae [0,4194304] 0 2024-06-06T08:43:49.966 INFO:tasks.workunit.client.0.smithi027.stdout:7/748: symlink d17/d32/d46/d51/d7f/lf4 0 2024-06-06T08:43:49.968 INFO:tasks.workunit.client.0.smithi027.stdout:7/749: getdents d17/d32/d46/d4f 0 2024-06-06T08:43:49.970 INFO:tasks.workunit.client.0.smithi027.stdout:8/594: mknod d7/dc/d36/d83/cc6 0 2024-06-06T08:43:49.972 INFO:tasks.workunit.client.0.smithi027.stdout:5/744: dread d0/da/f35 [0,4194304] 0 2024-06-06T08:43:49.972 INFO:tasks.workunit.client.0.smithi027.stdout:3/588: dwrite d1/d1f/d45/f69 [0,4194304] 0 2024-06-06T08:43:49.972 INFO:tasks.workunit.client.0.smithi027.stdout:3/589: chown d1/d8/d22/f46 662992611 1 2024-06-06T08:43:49.980 INFO:tasks.workunit.client.0.smithi027.stdout:5/745: mkdir d0/da/db/d1a/da3/de5 0 2024-06-06T08:43:49.980 INFO:tasks.workunit.client.0.smithi027.stdout:5/746: stat d0/da/db/d46/d7f/d81/d97/cca 0 2024-06-06T08:43:49.981 INFO:tasks.workunit.client.0.smithi027.stdout:3/590: symlink d1/d1f/d25/lb5 0 2024-06-06T08:43:49.983 INFO:tasks.workunit.client.0.smithi027.stdout:3/591: chown d1/d8/d59/l7b 119772445 1 2024-06-06T08:43:49.984 INFO:tasks.workunit.client.0.smithi027.stdout:3/592: read d1/d8/d22/d27/d50/f82 [371188,83242] 0 2024-06-06T08:43:49.985 INFO:tasks.workunit.client.0.smithi027.stdout:5/747: symlink d0/da/db/d1a/da3/le6 0 2024-06-06T08:43:49.986 INFO:tasks.workunit.client.0.smithi027.stdout:5/748: symlink d0/da/db/d46/d7f/d81/d97/db0/dbc/le7 0 2024-06-06T08:43:49.989 INFO:tasks.workunit.client.0.smithi027.stdout:3/593: rename d1/d90/d44/d54/f98 to d1/d1f/d25/fb6 0 2024-06-06T08:43:49.991 INFO:tasks.workunit.client.0.smithi027.stdout:3/594: getdents d1/d90 0 2024-06-06T08:43:49.991 INFO:tasks.workunit.client.0.smithi027.stdout:7/750: dwrite d17/d32/d46/d4f/d5e/f66 [4194304,4194304] 0 2024-06-06T08:43:49.995 INFO:tasks.workunit.client.0.smithi027.stdout:7/751: mknod d17/d32/d61/cf5 0 2024-06-06T08:43:50.002 INFO:tasks.workunit.client.0.smithi027.stdout:5/749: dwrite d0/da/db/fa4 [0,4194304] 0 2024-06-06T08:43:50.005 INFO:tasks.workunit.client.0.smithi027.stdout:5/750: truncate d0/da/db/fa0 501888 0 2024-06-06T08:43:50.005 INFO:tasks.workunit.client.0.smithi027.stdout:5/751: stat d0/da/d37/c2a 0 2024-06-06T08:43:50.063 INFO:tasks.workunit.client.0.smithi027.stdout:5/752: dwrite d0/da/d37/d2b/f3d [0,4194304] 0 2024-06-06T08:43:50.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/753: creat d0/da/db/d1a/d5a/de0/fe8 x:0 0 0 2024-06-06T08:43:50.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/754: dread - d0/da/db/d46/d7f/d81/fcc zero size 2024-06-06T08:43:50.134 INFO:tasks.workunit.client.0.smithi027.stdout:5/755: dwrite d0/d6a/f8a [0,4194304] 0 2024-06-06T08:43:50.137 INFO:tasks.workunit.client.0.smithi027.stdout:5/756: rename d0/da/db/d46/d7f/d81/ddc to d0/da/db/d46/d7f/d81/d97/de9 0 2024-06-06T08:43:50.138 INFO:tasks.workunit.client.0.smithi027.stdout:5/757: creat d0/da/d37/fea x:0 0 0 2024-06-06T08:43:50.139 INFO:tasks.workunit.client.0.smithi027.stdout:5/758: creat d0/da/db/d46/d4c/d70/feb x:0 0 0 2024-06-06T08:43:50.139 INFO:tasks.workunit.client.0.smithi027.stdout:5/759: fsync d0/da/db/d46/d54/f84 0 2024-06-06T08:43:50.141 INFO:tasks.workunit.client.0.smithi027.stdout:5/760: symlink d0/da/d47/lec 0 2024-06-06T08:43:50.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/761: dwrite d0/da/db/d46/d4c/f82 [0,4194304] 0 2024-06-06T08:43:50.164 INFO:tasks.workunit.client.0.smithi027.stdout:5/762: mkdir d0/da/d47/ded 0 2024-06-06T08:43:50.166 INFO:tasks.workunit.client.0.smithi027.stdout:5/763: unlink d0/da/db/d46/d54/f90 0 2024-06-06T08:43:50.167 INFO:tasks.workunit.client.0.smithi027.stdout:5/764: getdents d0/da/db/d1a/d7e 0 2024-06-06T08:43:50.167 INFO:tasks.workunit.client.0.smithi027.stdout:5/765: write d0/da/db/d46/d7f/d81/d97/db0/fdd [745595,18032] 0 2024-06-06T08:43:50.170 INFO:tasks.workunit.client.0.smithi027.stdout:5/766: dread d0/da/db/d1a/f6f [0,4194304] 0 2024-06-06T08:43:50.170 INFO:tasks.workunit.client.0.smithi027.stdout:5/767: mknod d0/da/db/d46/cee 0 2024-06-06T08:43:50.170 INFO:tasks.workunit.client.0.smithi027.stdout:5/768: mknod d0/da/db/d1a/cef 0 2024-06-06T08:43:50.177 INFO:tasks.workunit.client.0.smithi027.stdout:5/769: dread d0/da/db/d1a/d5a/fb3 [0,4194304] 0 2024-06-06T08:43:50.178 INFO:tasks.workunit.client.0.smithi027.stdout:5/770: truncate d0/da/db/d46/d54/f84 477825 0 2024-06-06T08:43:50.180 INFO:tasks.workunit.client.0.smithi027.stdout:5/771: getdents d0/da/db/d1a/da9 0 2024-06-06T08:43:50.180 INFO:tasks.workunit.client.0.smithi027.stdout:5/772: dread - d0/da/db/d46/d7f/d81/d97/fb1 zero size 2024-06-06T08:43:50.184 INFO:tasks.workunit.client.0.smithi027.stdout:5/773: mkdir d0/da/df0 0 2024-06-06T08:43:50.188 INFO:tasks.workunit.client.0.smithi027.stdout:5/774: dread d0/da/f92 [0,4194304] 0 2024-06-06T08:43:50.198 INFO:tasks.workunit.client.0.smithi027.stdout:5/775: read d0/da/d37/f41 [38683,32761] 0 2024-06-06T08:43:50.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/570: sync 2024-06-06T08:43:50.200 INFO:tasks.workunit.client.0.smithi027.stdout:5/776: creat d0/da/db/d1a/da3/ff1 x:0 0 0 2024-06-06T08:43:50.203 INFO:tasks.workunit.client.0.smithi027.stdout:1/571: unlink dc/de/d14/d80/da6/cac 0 2024-06-06T08:43:50.216 INFO:tasks.workunit.client.0.smithi027.stdout:1/572: read dc/de/d22/f3b [292822,56408] 0 2024-06-06T08:43:50.218 INFO:tasks.workunit.client.0.smithi027.stdout:5/777: dwrite d0/da/d37/fea [0,4194304] 0 2024-06-06T08:43:50.219 INFO:tasks.workunit.client.0.smithi027.stdout:5/778: creat d0/da/db/d46/d7f/d81/d97/db0/dbc/ff2 x:0 0 0 2024-06-06T08:43:50.220 INFO:tasks.workunit.client.0.smithi027.stdout:5/779: creat d0/da/db/d46/d4c/d70/ff3 x:0 0 0 2024-06-06T08:43:50.221 INFO:tasks.workunit.client.0.smithi027.stdout:5/780: mknod d0/da/d47/ded/cf4 0 2024-06-06T08:43:50.223 INFO:tasks.workunit.client.0.smithi027.stdout:5/781: mkdir d0/da/db/d46/d7f/d81/d97/db0/dbc/dcf/df5 0 2024-06-06T08:43:50.231 INFO:tasks.workunit.client.0.smithi027.stdout:5/782: write d0/da/fae [3201491,76434] 0 2024-06-06T08:43:50.233 INFO:tasks.workunit.client.0.smithi027.stdout:1/573: dwrite dc/de/d3d/d4a/d76/f90 [0,4194304] 0 2024-06-06T08:43:50.236 INFO:tasks.workunit.client.0.smithi027.stdout:1/574: rename dc/d9d/laf to dc/de/d14/d80/da6/db0/lb2 0 2024-06-06T08:43:50.238 INFO:tasks.workunit.client.0.smithi027.stdout:1/575: creat dc/fb3 x:0 0 0 2024-06-06T08:43:50.240 INFO:tasks.workunit.client.0.smithi027.stdout:1/576: rmdir dc/d67/d69/d32/d37/d56/d78 39 2024-06-06T08:43:50.243 INFO:tasks.workunit.client.0.smithi027.stdout:1/577: getdents dc/d67/d69/d32/d37 0 2024-06-06T08:43:50.243 INFO:tasks.workunit.client.0.smithi027.stdout:5/783: dread d0/da/db/d46/d4c/f82 [0,4194304] 0 2024-06-06T08:43:50.243 INFO:tasks.workunit.client.0.smithi027.stdout:5/784: readlink d0/da/d20/ld1 0 2024-06-06T08:43:50.243 INFO:tasks.workunit.client.0.smithi027.stdout:5/785: stat d0/da/db/lf 0 2024-06-06T08:43:50.243 INFO:tasks.workunit.client.0.smithi027.stdout:1/578: rmdir dc/d9d 39 2024-06-06T08:43:50.245 INFO:tasks.workunit.client.0.smithi027.stdout:5/786: dread d0/da/f92 [0,4194304] 0 2024-06-06T08:43:50.245 INFO:tasks.workunit.client.0.smithi027.stdout:5/787: truncate d0/da/d37/d2b/f4a 1660679 0 2024-06-06T08:43:50.253 INFO:tasks.workunit.client.0.smithi027.stdout:6/806: sync 2024-06-06T08:43:50.253 INFO:tasks.workunit.client.0.smithi027.stdout:9/583: sync 2024-06-06T08:43:50.253 INFO:tasks.workunit.client.0.smithi027.stdout:4/705: sync 2024-06-06T08:43:50.255 INFO:tasks.workunit.client.0.smithi027.stdout:4/706: mkdir d0/d1/d6/d8/d82/d8b/dc9/da3/de3 0 2024-06-06T08:43:50.256 INFO:tasks.workunit.client.0.smithi027.stdout:9/584: rename d3/d95/d34/f48 to d3/d95/d23/da9/fce 0 2024-06-06T08:43:50.257 INFO:tasks.workunit.client.0.smithi027.stdout:9/585: truncate d3/d95/d23/d36/f94 272683 0 2024-06-06T08:43:50.257 INFO:tasks.workunit.client.0.smithi027.stdout:6/807: dread d4/d1a/d2e/d44/dca/fe5 [0,4194304] 0 2024-06-06T08:43:50.257 INFO:tasks.workunit.client.0.smithi027.stdout:9/586: chown d3/d95/d23/d36/l8b 2510171 1 2024-06-06T08:43:50.259 INFO:tasks.workunit.client.0.smithi027.stdout:6/808: rename d4/d9/dc/fa8 to d4/d9/dc/d18/d10a/f10b 0 2024-06-06T08:43:50.259 INFO:tasks.workunit.client.0.smithi027.stdout:6/809: stat d4/d9/dc/d18 0 2024-06-06T08:43:50.262 INFO:tasks.workunit.client.0.smithi027.stdout:6/810: mknod d4/d9/d6e/dd4/c10c 0 2024-06-06T08:43:50.262 INFO:tasks.workunit.client.0.smithi027.stdout:5/788: dwrite d0/da/f43 [0,4194304] 0 2024-06-06T08:43:50.264 INFO:tasks.workunit.client.0.smithi027.stdout:9/587: rename d3/d95/d23/d36/f5a to d3/d4e/d75/fcf 0 2024-06-06T08:43:50.268 INFO:tasks.workunit.client.0.smithi027.stdout:4/707: dread d0/f66 [0,4194304] 0 2024-06-06T08:43:50.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/588: rename d3/d95/d23/d36/f6b to d3/d95/d23/d36/d3a/d84/db2/fd0 0 2024-06-06T08:43:50.270 INFO:tasks.workunit.client.0.smithi027.stdout:4/708: dread d0/d1d/f69 [0,4194304] 0 2024-06-06T08:43:50.270 INFO:tasks.workunit.client.0.smithi027.stdout:5/789: dread d0/d6a/f8a [0,4194304] 0 2024-06-06T08:43:50.271 INFO:tasks.workunit.client.0.smithi027.stdout:9/589: rmdir d3/d95/d23/d36/d3a/d84/db2 39 2024-06-06T08:43:50.272 INFO:tasks.workunit.client.0.smithi027.stdout:4/709: creat d0/d63/fe4 x:0 0 0 2024-06-06T08:43:50.273 INFO:tasks.workunit.client.0.smithi027.stdout:6/811: unlink d4/d9/dc/d1c/d52/fa5 0 2024-06-06T08:43:50.274 INFO:tasks.workunit.client.0.smithi027.stdout:5/790: stat d0/da/db/d1a/c5e 0 2024-06-06T08:43:50.275 INFO:tasks.workunit.client.0.smithi027.stdout:4/710: creat d0/fe5 x:0 0 0 2024-06-06T08:43:50.280 INFO:tasks.workunit.client.0.smithi027.stdout:9/590: dread d3/d95/d23/d36/d3a/d84/db2/fd0 [0,4194304] 0 2024-06-06T08:43:50.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/591: mkdir d3/d4e/d75/dc/dd1 0 2024-06-06T08:43:50.295 INFO:tasks.workunit.client.0.smithi027.stdout:5/791: dwrite d0/da/f13 [0,4194304] 0 2024-06-06T08:43:50.295 INFO:tasks.workunit.client.0.smithi027.stdout:5/792: stat d0/da/db/d46/d4c/d70/feb 0 2024-06-06T08:43:50.308 INFO:tasks.workunit.client.0.smithi027.stdout:9/592: dwrite d3/f11 [0,4194304] 0 2024-06-06T08:43:50.318 INFO:tasks.workunit.client.0.smithi027.stdout:5/793: dwrite d0/da/db/d1a/dc6/fd8 [0,4194304] 0 2024-06-06T08:43:50.318 INFO:tasks.workunit.client.0.smithi027.stdout:4/711: dwrite d0/d1/d6/d8/f6a [0,4194304] 0 2024-06-06T08:43:50.320 INFO:tasks.workunit.client.0.smithi027.stdout:6/812: rename d4/de/d45/dcf/f109 to d4/d1a/d2e/d44/dce/f10d 0 2024-06-06T08:43:50.320 INFO:tasks.workunit.client.0.smithi027.stdout:6/813: readlink d4/d1a/lfc 0 2024-06-06T08:43:50.321 INFO:tasks.workunit.client.0.smithi027.stdout:9/593: dread d3/d95/d34/fa3 [0,4194304] 0 2024-06-06T08:43:50.322 INFO:tasks.workunit.client.0.smithi027.stdout:4/712: symlink d0/d1/d6/d8/d21/le6 0 2024-06-06T08:43:50.324 INFO:tasks.workunit.client.0.smithi027.stdout:5/794: creat d0/da/db/d46/d4c/ff6 x:0 0 0 2024-06-06T08:43:50.325 INFO:tasks.workunit.client.0.smithi027.stdout:5/795: fdatasync d0/da/db/d1a/da3/fa6 0 2024-06-06T08:43:50.328 INFO:tasks.workunit.client.0.smithi027.stdout:6/814: write d4/de/f11 [3633210,23742] 0 2024-06-06T08:43:50.330 INFO:tasks.workunit.client.0.smithi027.stdout:4/713: creat d0/d1/fe7 x:0 0 0 2024-06-06T08:43:50.331 INFO:tasks.workunit.client.0.smithi027.stdout:5/796: getdents d0/da/db/d1a 0 2024-06-06T08:43:50.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/594: rename d3/d76/fbf to d3/d4e/fd2 0 2024-06-06T08:43:50.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/595: dread - d3/d95/d23/f3d zero size 2024-06-06T08:43:50.334 INFO:tasks.workunit.client.0.smithi027.stdout:5/797: mknod d0/da/db/d1a/cf7 0 2024-06-06T08:43:50.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/596: dread - d3/d4e/d75/dc/fad zero size 2024-06-06T08:43:50.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/597: chown d3/d95/c77 150718300 1 2024-06-06T08:43:50.335 INFO:tasks.workunit.client.0.smithi027.stdout:9/598: creat d3/d95/d34/d41/fd3 x:0 0 0 2024-06-06T08:43:50.336 INFO:tasks.workunit.client.0.smithi027.stdout:9/599: fdatasync d3/d95/f90 0 2024-06-06T08:43:50.336 INFO:tasks.workunit.client.0.smithi027.stdout:9/600: read - d3/d4e/d75/db0/fbb zero size 2024-06-06T08:43:50.337 INFO:tasks.workunit.client.0.smithi027.stdout:9/601: fsync d3/d95/f4f 0 2024-06-06T08:43:50.337 INFO:tasks.workunit.client.0.smithi027.stdout:5/798: chown d0/da/db/d1a/cef 37996830 1 2024-06-06T08:43:50.337 INFO:tasks.workunit.client.0.smithi027.stdout:9/602: truncate d3/d95/d23/f3d 235456 0 2024-06-06T08:43:50.341 INFO:tasks.workunit.client.0.smithi027.stdout:9/603: symlink d3/d4e/ld4 0 2024-06-06T08:43:50.341 INFO:tasks.workunit.client.0.smithi027.stdout:9/604: chown d3/d95/db6/lc2 59 1 2024-06-06T08:43:50.345 INFO:tasks.workunit.client.0.smithi027.stdout:9/605: creat d3/d95/fd5 x:0 0 0 2024-06-06T08:43:50.345 INFO:tasks.workunit.client.0.smithi027.stdout:9/606: fsync d3/d95/f5e 0 2024-06-06T08:43:50.348 INFO:tasks.workunit.client.0.smithi027.stdout:9/607: write d3/d4e/d75/dc/f2e [1122844,60260] 0 2024-06-06T08:43:50.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/608: creat d3/d95/d23/d36/fd6 x:0 0 0 2024-06-06T08:43:50.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/799: dread d0/da/db/d46/d7f/d81/fd2 [0,4194304] 0 2024-06-06T08:43:50.352 INFO:tasks.workunit.client.0.smithi027.stdout:6/815: dwrite d4/d1a/d2e/d44/fc0 [0,4194304] 0 2024-06-06T08:43:50.353 INFO:tasks.workunit.client.0.smithi027.stdout:9/609: symlink d3/d95/d23/ld7 0 2024-06-06T08:43:50.353 INFO:tasks.workunit.client.0.smithi027.stdout:9/610: truncate d3/d95/d23/d36/fd6 317478 0 2024-06-06T08:43:50.353 INFO:tasks.workunit.client.0.smithi027.stdout:9/611: stat d3/d95/f2b 0 2024-06-06T08:43:50.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/816: chown d4/d13/l6f 392000 1 2024-06-06T08:43:50.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/817: write d4/d9/d49/fe2 [1970791,103728] 0 2024-06-06T08:43:50.354 INFO:tasks.workunit.client.0.smithi027.stdout:7/752: sync 2024-06-06T08:43:50.354 INFO:tasks.workunit.client.0.smithi027.stdout:0/629: sync 2024-06-06T08:43:50.354 INFO:tasks.workunit.client.0.smithi027.stdout:2/659: sync 2024-06-06T08:43:50.354 INFO:tasks.workunit.client.0.smithi027.stdout:3/595: sync 2024-06-06T08:43:50.355 INFO:tasks.workunit.client.0.smithi027.stdout:8/595: sync 2024-06-06T08:43:50.355 INFO:tasks.workunit.client.0.smithi027.stdout:3/596: truncate d1/d90/fa2 778636 0 2024-06-06T08:43:50.355 INFO:tasks.workunit.client.0.smithi027.stdout:3/597: write d1/f84 [277660,116931] 0 2024-06-06T08:43:50.355 INFO:tasks.workunit.client.0.smithi027.stdout:5/800: link d0/da/db/d46/d77/f8c d0/da/db/d46/d7f/ff8 0 2024-06-06T08:43:50.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/612: chown d3/d4e/ca6 35728 1 2024-06-06T08:43:50.356 INFO:tasks.workunit.client.0.smithi027.stdout:6/818: mknod d4/d9/c10e 0 2024-06-06T08:43:50.364 INFO:tasks.workunit.client.0.smithi027.stdout:8/596: chown d7/dc/d36/d83/d60/d93/dac/c6c 103791 1 2024-06-06T08:43:50.365 INFO:tasks.workunit.client.0.smithi027.stdout:9/613: fdatasync d3/f52 0 2024-06-06T08:43:50.366 INFO:tasks.workunit.client.0.smithi027.stdout:2/660: creat d6/dd/dd9/d1d/dbc/fda x:0 0 0 2024-06-06T08:43:50.367 INFO:tasks.workunit.client.0.smithi027.stdout:9/614: creat d3/d4e/d75/db0/fd8 x:0 0 0 2024-06-06T08:43:50.367 INFO:tasks.workunit.client.0.smithi027.stdout:2/661: read d6/dd/dd9/d1d/f60 [3663,5845] 0 2024-06-06T08:43:50.368 INFO:tasks.workunit.client.0.smithi027.stdout:3/598: mknod d1/d8/d59/cb7 0 2024-06-06T08:43:50.370 INFO:tasks.workunit.client.0.smithi027.stdout:6/819: dread d4/d9/d49/fe2 [0,4194304] 0 2024-06-06T08:43:50.370 INFO:tasks.workunit.client.0.smithi027.stdout:6/820: stat d4/d9/dc/d18/d10a 0 2024-06-06T08:43:50.374 INFO:tasks.workunit.client.0.smithi027.stdout:5/801: creat d0/da/db/d1a/da9/ff9 x:0 0 0 2024-06-06T08:43:50.374 INFO:tasks.workunit.client.0.smithi027.stdout:0/630: write d15/d18/d23/d38/d68/d85/f8d [713487,105920] 0 2024-06-06T08:43:50.374 INFO:tasks.workunit.client.0.smithi027.stdout:0/631: chown d15/d18/c1f 132379708 1 2024-06-06T08:43:50.377 INFO:tasks.workunit.client.0.smithi027.stdout:7/753: rename l9 to d17/lf6 0 2024-06-06T08:43:50.377 INFO:tasks.workunit.client.0.smithi027.stdout:7/754: fsync d17/d32/d46/d83/d77/d8c/d91/daa/fe3 0 2024-06-06T08:43:50.378 INFO:tasks.workunit.client.0.smithi027.stdout:2/662: creat d6/dd/dd9/d3f/db0/fdb x:0 0 0 2024-06-06T08:43:50.378 INFO:tasks.workunit.client.0.smithi027.stdout:9/615: dread d3/d95/d23/d36/f5d [0,4194304] 0 2024-06-06T08:43:50.378 INFO:tasks.workunit.client.0.smithi027.stdout:7/755: fdatasync d17/d32/d61/d75/fd1 0 2024-06-06T08:43:50.378 INFO:tasks.workunit.client.0.smithi027.stdout:9/616: getdents d3/d95/d23 0 2024-06-06T08:43:50.379 INFO:tasks.workunit.client.0.smithi027.stdout:8/597: dread d7/d2f/f87 [4194304,4194304] 0 2024-06-06T08:43:50.379 INFO:tasks.workunit.client.0.smithi027.stdout:6/821: mkdir d4/de/d10f 0 2024-06-06T08:43:50.380 INFO:tasks.workunit.client.0.smithi027.stdout:3/599: creat d1/d8/d22/d27/d79/fb8 x:0 0 0 2024-06-06T08:43:50.380 INFO:tasks.workunit.client.0.smithi027.stdout:3/600: fsync d1/f6 0 2024-06-06T08:43:50.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/822: dread d4/d9/dc/f9f [0,4194304] 0 2024-06-06T08:43:50.382 INFO:tasks.workunit.client.0.smithi027.stdout:5/802: unlink d0/da/d20/faf 0 2024-06-06T08:43:50.383 INFO:tasks.workunit.client.0.smithi027.stdout:9/617: mkdir d3/d76/dd9 0 2024-06-06T08:43:50.384 INFO:tasks.workunit.client.0.smithi027.stdout:9/618: fdatasync d3/d95/f4f 0 2024-06-06T08:43:50.385 INFO:tasks.workunit.client.0.smithi027.stdout:3/601: creat d1/d1f/d4d/fb9 x:0 0 0 2024-06-06T08:43:50.385 INFO:tasks.workunit.client.0.smithi027.stdout:9/619: creat d3/d95/d23/d36/d3a/d84/db2/fda x:0 0 0 2024-06-06T08:43:50.387 INFO:tasks.workunit.client.0.smithi027.stdout:7/756: dread d17/d32/d46/d83/f9c [0,4194304] 0 2024-06-06T08:43:50.388 INFO:tasks.workunit.client.0.smithi027.stdout:0/632: dwrite d15/d18/dbe/d5c/d83/f86 [0,4194304] 0 2024-06-06T08:43:50.389 INFO:tasks.workunit.client.0.smithi027.stdout:0/633: read d15/d18/dbe/f71 [76466,92063] 0 2024-06-06T08:43:50.392 INFO:tasks.workunit.client.0.smithi027.stdout:6/823: mknod d4/c110 0 2024-06-06T08:43:50.393 INFO:tasks.workunit.client.0.smithi027.stdout:0/634: mkdir d15/d18/d23/d6f/dd1 0 2024-06-06T08:43:50.395 INFO:tasks.workunit.client.0.smithi027.stdout:0/635: creat d15/d18/d23/d38/d3e/fd2 x:0 0 0 2024-06-06T08:43:50.398 INFO:tasks.workunit.client.0.smithi027.stdout:3/602: link d1/d90/d6d/f9a d1/d8/d22/fba 0 2024-06-06T08:43:50.398 INFO:tasks.workunit.client.0.smithi027.stdout:3/603: fsync d1/d90/fa5 0 2024-06-06T08:43:50.398 INFO:tasks.workunit.client.0.smithi027.stdout:3/604: write d1/d90/d44/d54/fa6 [231024,33059] 0 2024-06-06T08:43:50.399 INFO:tasks.workunit.client.0.smithi027.stdout:8/598: dwrite d7/dc/f12 [0,4194304] 0 2024-06-06T08:43:50.400 INFO:tasks.workunit.client.0.smithi027.stdout:7/757: mkdir d17/d32/d46/d51/d52/d70/df7 0 2024-06-06T08:43:50.401 INFO:tasks.workunit.client.0.smithi027.stdout:7/758: chown d17/d32/d46/d51/d7f/db2/cee 24207 1 2024-06-06T08:43:50.401 INFO:tasks.workunit.client.0.smithi027.stdout:7/759: fsync d17/d32/d61/d75/fe5 0 2024-06-06T08:43:50.401 INFO:tasks.workunit.client.0.smithi027.stdout:2/663: mkdir d6/dd/dd9/d1d/d4d/ddc 0 2024-06-06T08:43:50.401 INFO:tasks.workunit.client.0.smithi027.stdout:2/664: stat d6/dd/dd9/d1d/dbc 0 2024-06-06T08:43:50.401 INFO:tasks.workunit.client.0.smithi027.stdout:2/665: stat d6/dd/dd9/d3f/d77/f4c 0 2024-06-06T08:43:50.402 INFO:tasks.workunit.client.0.smithi027.stdout:0/636: chown d15/c53 57 1 2024-06-06T08:43:50.402 INFO:tasks.workunit.client.0.smithi027.stdout:6/824: unlink d4/d13/fff 0 2024-06-06T08:43:50.402 INFO:tasks.workunit.client.0.smithi027.stdout:6/825: chown d4/de/d45/d6a/l7c 10 1 2024-06-06T08:43:50.404 INFO:tasks.workunit.client.0.smithi027.stdout:3/605: creat d1/d8/d22/d2d/fbb x:0 0 0 2024-06-06T08:43:50.404 INFO:tasks.workunit.client.0.smithi027.stdout:5/803: dwrite d0/da/db/d1a/da3/ff1 [0,4194304] 0 2024-06-06T08:43:50.404 INFO:tasks.workunit.client.0.smithi027.stdout:5/804: dread - d0/da/d47/fa8 zero size 2024-06-06T08:43:50.405 INFO:tasks.workunit.client.0.smithi027.stdout:8/599: creat d7/dc/d36/d83/fc7 x:0 0 0 2024-06-06T08:43:50.405 INFO:tasks.workunit.client.0.smithi027.stdout:6/826: creat d4/d1a/f111 x:0 0 0 2024-06-06T08:43:50.407 INFO:tasks.workunit.client.0.smithi027.stdout:5/805: truncate d0/da/db/d1a/da9/faa 1211004 0 2024-06-06T08:43:50.410 INFO:tasks.workunit.client.0.smithi027.stdout:7/760: write d17/d32/d46/d83/d72/f98 [1800876,44672] 0 2024-06-06T08:43:50.413 INFO:tasks.workunit.client.0.smithi027.stdout:8/600: mkdir d7/dc/d37/dc8 0 2024-06-06T08:43:50.413 INFO:tasks.workunit.client.0.smithi027.stdout:8/601: write d7/f4e [2668080,82206] 0 2024-06-06T08:43:50.415 INFO:tasks.workunit.client.0.smithi027.stdout:9/620: rename d3/d95/d34/d73/c83 to d3/d95/d23/d78/cdb 0 2024-06-06T08:43:50.415 INFO:tasks.workunit.client.0.smithi027.stdout:9/621: write d3/d95/d34/d41/fd3 [456570,36682] 0 2024-06-06T08:43:50.415 INFO:tasks.workunit.client.0.smithi027.stdout:9/622: truncate d3/f20 4823875 0 2024-06-06T08:43:50.415 INFO:tasks.workunit.client.0.smithi027.stdout:6/827: mknod d4/da2/c112 0 2024-06-06T08:43:50.420 INFO:tasks.workunit.client.0.smithi027.stdout:9/623: read d3/d4e/d75/fcf [1448466,14447] 0 2024-06-06T08:43:50.420 INFO:tasks.workunit.client.0.smithi027.stdout:9/624: chown d3/d4e/d75/dc/fc9 1576235 1 2024-06-06T08:43:50.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/602: mknod d7/dc/d37/cc9 0 2024-06-06T08:43:50.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/603: truncate d7/dc/f69 324472 0 2024-06-06T08:43:50.422 INFO:tasks.workunit.client.0.smithi027.stdout:5/806: rename d0/da/db/d1a/l50 to d0/da/db/d1a/dc6/lfa 0 2024-06-06T08:43:50.423 INFO:tasks.workunit.client.0.smithi027.stdout:8/604: creat d7/dc/d36/d3c/fca x:0 0 0 2024-06-06T08:43:50.423 INFO:tasks.workunit.client.0.smithi027.stdout:9/625: rmdir d3/d95/db6 39 2024-06-06T08:43:50.424 INFO:tasks.workunit.client.0.smithi027.stdout:8/605: stat d7/dc/d36/d41/d85/la1 0 2024-06-06T08:43:50.424 INFO:tasks.workunit.client.0.smithi027.stdout:5/807: creat d0/da/ffb x:0 0 0 2024-06-06T08:43:50.426 INFO:tasks.workunit.client.0.smithi027.stdout:3/606: dwrite d1/d8/d22/f73 [0,4194304] 0 2024-06-06T08:43:50.426 INFO:tasks.workunit.client.0.smithi027.stdout:3/607: fdatasync d1/d8/d22/d27/f32 0 2024-06-06T08:43:50.427 INFO:tasks.workunit.client.0.smithi027.stdout:5/808: symlink d0/da/db/d46/d7f/d81/lfc 0 2024-06-06T08:43:50.428 INFO:tasks.workunit.client.0.smithi027.stdout:3/608: mkdir d1/d1f/dbc 0 2024-06-06T08:43:50.433 INFO:tasks.workunit.client.0.smithi027.stdout:6/828: write d4/d9/d6e/f80 [494370,104141] 0 2024-06-06T08:43:50.437 INFO:tasks.workunit.client.0.smithi027.stdout:6/829: chown d4/lb2 1634374 1 2024-06-06T08:43:50.438 INFO:tasks.workunit.client.0.smithi027.stdout:0/637: link d15/d18/d23/d38/d68/c72 d15/d18/d23/d28/d31/cd3 0 2024-06-06T08:43:50.440 INFO:tasks.workunit.client.0.smithi027.stdout:6/830: write d4/f102 [284070,107941] 0 2024-06-06T08:43:50.441 INFO:tasks.workunit.client.0.smithi027.stdout:6/831: stat d4/d9/dc/d1c/d52/c7a 0 2024-06-06T08:43:50.442 INFO:tasks.workunit.client.0.smithi027.stdout:6/832: symlink d4/d1a/d2e/df3/l113 0 2024-06-06T08:43:50.443 INFO:tasks.workunit.client.0.smithi027.stdout:6/833: chown d4/de/d45/d6a/c55 0 1 2024-06-06T08:43:50.443 INFO:tasks.workunit.client.0.smithi027.stdout:9/626: link d3/d95/d23/d78/c7f d3/cdc 0 2024-06-06T08:43:50.443 INFO:tasks.workunit.client.0.smithi027.stdout:8/606: dwrite d7/dc/f9f [0,4194304] 0 2024-06-06T08:43:50.455 INFO:tasks.workunit.client.0.smithi027.stdout:0/638: dwrite d15/d18/d64/f8a [4194304,4194304] 0 2024-06-06T08:43:50.457 INFO:tasks.workunit.client.0.smithi027.stdout:0/639: rmdir d15/d18/d23/d6f/dd1 0 2024-06-06T08:43:50.458 INFO:tasks.workunit.client.0.smithi027.stdout:9/627: dread d3/d95/d23/d36/d3a/f68 [0,4194304] 0 2024-06-06T08:43:50.458 INFO:tasks.workunit.client.0.smithi027.stdout:9/628: stat d3/d4e/d75/db0 0 2024-06-06T08:43:50.464 INFO:tasks.workunit.client.0.smithi027.stdout:0/640: dread f9 [0,4194304] 0 2024-06-06T08:43:50.466 INFO:tasks.workunit.client.0.smithi027.stdout:0/641: stat d15/d18/d64/f6d 0 2024-06-06T08:43:50.468 INFO:tasks.workunit.client.0.smithi027.stdout:5/809: rename d0/da/db/d46/d7f/d81/d97/f9f to d0/da/db/d46/d4c/d70/ffd 0 2024-06-06T08:43:50.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/810: fsync d0/da/db/d1a/d5a/f83 0 2024-06-06T08:43:50.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/811: rename d0/da/db/d1a/d5a/fb3 to d0/da/db/d46/d7f/d81/ffe 0 2024-06-06T08:43:50.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/812: symlink d0/da/db/d1a/da9/lff 0 2024-06-06T08:43:50.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/813: symlink d0/da/d47/l100 0 2024-06-06T08:43:50.486 INFO:tasks.workunit.client.0.smithi027.stdout:5/814: symlink d0/da/d20/l101 0 2024-06-06T08:43:50.486 INFO:tasks.workunit.client.0.smithi027.stdout:5/815: chown d0/da/db/c59 339573 1 2024-06-06T08:43:50.487 INFO:tasks.workunit.client.0.smithi027.stdout:5/816: fdatasync d0/da/d37/d2b/f65 0 2024-06-06T08:43:50.489 INFO:tasks.workunit.client.0.smithi027.stdout:5/817: read d0/da/db/d46/d54/f84 [84781,30735] 0 2024-06-06T08:43:50.489 INFO:tasks.workunit.client.0.smithi027.stdout:5/818: stat d0/da/d47/ded/cf4 0 2024-06-06T08:43:50.490 INFO:tasks.workunit.client.0.smithi027.stdout:5/819: dread - d0/da/db/d1a/d5a/de0/fe8 zero size 2024-06-06T08:43:50.490 INFO:tasks.workunit.client.0.smithi027.stdout:5/820: mknod d0/da/d37/d2b/c102 0 2024-06-06T08:43:50.492 INFO:tasks.workunit.client.0.smithi027.stdout:5/821: mkdir d0/da/db/d46/d7f/d81/d103 0 2024-06-06T08:43:50.493 INFO:tasks.workunit.client.0.smithi027.stdout:5/822: chown d0/da/d47/l8b 390473 1 2024-06-06T08:43:50.494 INFO:tasks.workunit.client.0.smithi027.stdout:5/823: creat d0/da/db/d46/d4c/f104 x:0 0 0 2024-06-06T08:43:50.494 INFO:tasks.workunit.client.0.smithi027.stdout:5/824: write d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd/fd6 [392017,88558] 0 2024-06-06T08:43:50.495 INFO:tasks.workunit.client.0.smithi027.stdout:5/825: mkdir d0/da/db/d1a/d5a/de0/d105 0 2024-06-06T08:43:50.506 INFO:tasks.workunit.client.0.smithi027.stdout:5/826: dread d0/da/db/d1a/da3/ff1 [0,4194304] 0 2024-06-06T08:43:50.507 INFO:tasks.workunit.client.0.smithi027.stdout:5/827: rmdir d0/da/db/d1a 39 2024-06-06T08:43:50.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/828: mkdir d0/da/df0/d106 0 2024-06-06T08:43:50.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/829: read - d0/da/db/d46/d7f/d81/d97/fb1 zero size 2024-06-06T08:43:50.519 INFO:tasks.workunit.client.0.smithi027.stdout:5/830: dread d0/da/db/d46/d7f/f8e [0,4194304] 0 2024-06-06T08:43:50.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/831: mkdir d0/da/db/d1a/da3/de5/d107 0 2024-06-06T08:43:50.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/832: truncate d0/da/db/d46/d7f/d81/d97/db0/fdd 1162639 0 2024-06-06T08:43:50.537 INFO:tasks.workunit.client.0.smithi027.stdout:5/833: dwrite d0/da/db/d1a/f71 [4194304,4194304] 0 2024-06-06T08:43:50.552 INFO:tasks.workunit.client.0.smithi027.stdout:5/834: dwrite d0/da/fc8 [0,4194304] 0 2024-06-06T08:43:50.555 INFO:tasks.workunit.client.0.smithi027.stdout:5/835: unlink d0/da/db/d46/d4c/d70/fe3 0 2024-06-06T08:43:50.565 INFO:tasks.workunit.client.0.smithi027.stdout:5/836: dread d0/f29 [0,4194304] 0 2024-06-06T08:43:50.566 INFO:tasks.workunit.client.0.smithi027.stdout:5/837: getdents d0/da/d37/d2b 0 2024-06-06T08:43:50.569 INFO:tasks.workunit.client.0.smithi027.stdout:5/838: truncate d0/da/db/d46/d77/f8f 3050036 0 2024-06-06T08:43:50.578 INFO:tasks.workunit.client.0.smithi027.stdout:5/839: dread d0/fbf [0,4194304] 0 2024-06-06T08:43:50.579 INFO:tasks.workunit.client.0.smithi027.stdout:5/840: read - d0/da/db/d46/d7f/d81/d97/f9c zero size 2024-06-06T08:43:50.580 INFO:tasks.workunit.client.0.smithi027.stdout:5/841: creat d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd/f108 x:0 0 0 2024-06-06T08:43:50.588 INFO:tasks.workunit.client.0.smithi027.stdout:5/842: link d0/da/d20/l101 d0/da/db/d46/d7f/d81/l109 0 2024-06-06T08:43:50.598 INFO:tasks.workunit.client.0.smithi027.stdout:5/843: write d0/da/db/d1a/dc6/fd8 [974628,76045] 0 2024-06-06T08:43:50.600 INFO:tasks.workunit.client.0.smithi027.stdout:5/844: chown d0/da/db/d1a/lbb 37836 1 2024-06-06T08:43:50.602 INFO:tasks.workunit.client.0.smithi027.stdout:5/845: dread d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd/fd6 [0,4194304] 0 2024-06-06T08:43:50.604 INFO:tasks.workunit.client.0.smithi027.stdout:4/714: sync 2024-06-06T08:43:50.604 INFO:tasks.workunit.client.0.smithi027.stdout:1/579: sync 2024-06-06T08:43:50.606 INFO:tasks.workunit.client.0.smithi027.stdout:4/715: dread d0/d1/d6/f9e [0,4194304] 0 2024-06-06T08:43:50.607 INFO:tasks.workunit.client.0.smithi027.stdout:1/580: creat dc/de/d3d/fb4 x:0 0 0 2024-06-06T08:43:50.609 INFO:tasks.workunit.client.0.smithi027.stdout:4/716: truncate d0/d1/f14 959939 0 2024-06-06T08:43:50.610 INFO:tasks.workunit.client.0.smithi027.stdout:1/581: link dc/d67/d69/f52 dc/de/d14/d80/da6/db0/fb5 0 2024-06-06T08:43:50.612 INFO:tasks.workunit.client.0.smithi027.stdout:4/717: creat d0/d1d/dd0/fe8 x:0 0 0 2024-06-06T08:43:50.617 INFO:tasks.workunit.client.0.smithi027.stdout:4/718: creat d0/d1/d6/d8/d82/d8b/dc9/d3b/fe9 x:0 0 0 2024-06-06T08:43:50.617 INFO:tasks.workunit.client.0.smithi027.stdout:4/719: fdatasync d0/d1/d6/d8/d82/d8b/dc9/fd7 0 2024-06-06T08:43:50.620 INFO:tasks.workunit.client.0.smithi027.stdout:4/720: creat d0/d1d/d34/dd8/fea x:0 0 0 2024-06-06T08:43:50.620 INFO:tasks.workunit.client.0.smithi027.stdout:1/582: dread dc/de/d22/f3b [0,4194304] 0 2024-06-06T08:43:50.620 INFO:tasks.workunit.client.0.smithi027.stdout:5/846: dwrite d0/da/db/f64 [0,4194304] 0 2024-06-06T08:43:50.621 INFO:tasks.workunit.client.0.smithi027.stdout:5/847: write d0/da/db/d46/d7f/d81/d97/db0/fdd [861794,57408] 0 2024-06-06T08:43:50.623 INFO:tasks.workunit.client.0.smithi027.stdout:5/848: mkdir d0/da/db/d46/d7f/d81/d103/d10a 0 2024-06-06T08:43:50.624 INFO:tasks.workunit.client.0.smithi027.stdout:1/583: mknod dc/cb6 0 2024-06-06T08:43:50.625 INFO:tasks.workunit.client.0.smithi027.stdout:4/721: creat d0/d1d/d34/d7f/d97/d9c/dbe/feb x:0 0 0 2024-06-06T08:43:50.625 INFO:tasks.workunit.client.0.smithi027.stdout:5/849: link d0/da/db/d46/d7f/ff8 d0/da/db/d1a/dc6/f10b 0 2024-06-06T08:43:50.625 INFO:tasks.workunit.client.0.smithi027.stdout:1/584: mknod dc/da0/cb7 0 2024-06-06T08:43:50.626 INFO:tasks.workunit.client.0.smithi027.stdout:1/585: chown dc/d67/d69/l25 2636051 1 2024-06-06T08:43:50.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/586: write dc/de/d22/f72 [42954,85905] 0 2024-06-06T08:43:50.627 INFO:tasks.workunit.client.0.smithi027.stdout:4/722: mknod d0/d1d/d60/dc6/d6e/cec 0 2024-06-06T08:43:50.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/587: symlink dc/de/d14/d80/da6/lb8 0 2024-06-06T08:43:50.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/588: dread - dc/de/d3d/fb4 zero size 2024-06-06T08:43:50.631 INFO:tasks.workunit.client.0.smithi027.stdout:1/589: dread dc/de/d14/d4e/f7a [0,4194304] 0 2024-06-06T08:43:50.640 INFO:tasks.workunit.client.0.smithi027.stdout:1/590: dread dc/de/d22/f3b [0,4194304] 0 2024-06-06T08:43:50.642 INFO:tasks.workunit.client.0.smithi027.stdout:1/591: creat dc/d67/d69/fb9 x:0 0 0 2024-06-06T08:43:50.643 INFO:tasks.workunit.client.0.smithi027.stdout:1/592: creat dc/d67/d69/fba x:0 0 0 2024-06-06T08:43:50.645 INFO:tasks.workunit.client.0.smithi027.stdout:4/723: dwrite d0/d1d/d2a/f5e [0,4194304] 0 2024-06-06T08:43:50.645 INFO:tasks.workunit.client.0.smithi027.stdout:5/850: dwrite d0/d6a/f8a [0,4194304] 0 2024-06-06T08:43:50.646 INFO:tasks.workunit.client.0.smithi027.stdout:1/593: write dc/de/d3d/d4f/f9e [884951,22084] 0 2024-06-06T08:43:50.647 INFO:tasks.workunit.client.0.smithi027.stdout:1/594: truncate dc/de/d14/d4e/f7a 543018 0 2024-06-06T08:43:50.647 INFO:tasks.workunit.client.0.smithi027.stdout:1/595: chown dc/de/f13 231318 1 2024-06-06T08:43:50.647 INFO:tasks.workunit.client.0.smithi027.stdout:4/724: dread d0/d1d/d34/d7f/d97/d9c/dbe/fc3 [0,4194304] 0 2024-06-06T08:43:50.648 INFO:tasks.workunit.client.0.smithi027.stdout:1/596: truncate dc/de/d14/d4e/f53 3825851 0 2024-06-06T08:43:50.663 INFO:tasks.workunit.client.0.smithi027.stdout:5/851: link d0/da/db/d46/d7f/d81/d97/db0/dbc/le7 d0/l10c 0 2024-06-06T08:43:50.663 INFO:tasks.workunit.client.0.smithi027.stdout:5/852: chown d0/da/db/d46/d7f/d81/d97/db0/fdd 13 1 2024-06-06T08:43:50.664 INFO:tasks.workunit.client.0.smithi027.stdout:5/853: dread - d0/da/db/d46/d4c/d70/f86 zero size 2024-06-06T08:43:50.666 INFO:tasks.workunit.client.0.smithi027.stdout:4/725: dwrite d0/d1d/d2a/f5e [0,4194304] 0 2024-06-06T08:43:50.668 INFO:tasks.workunit.client.0.smithi027.stdout:1/597: dwrite dc/de/d3d/f44 [0,4194304] 0 2024-06-06T08:43:50.670 INFO:tasks.workunit.client.0.smithi027.stdout:1/598: fsync dc/de/d14/f8a 0 2024-06-06T08:43:50.672 INFO:tasks.workunit.client.0.smithi027.stdout:4/726: link d0/d1d/d2a/c37 d0/d1/d6/dd4/dde/ced 0 2024-06-06T08:43:50.673 INFO:tasks.workunit.client.0.smithi027.stdout:4/727: truncate d0/d1d/d34/d7f/fb7 472372 0 2024-06-06T08:43:50.674 INFO:tasks.workunit.client.0.smithi027.stdout:4/728: readlink d0/d1/d6/d8/d21/lc1 0 2024-06-06T08:43:50.674 INFO:tasks.workunit.client.0.smithi027.stdout:4/729: stat d0/d1d/d34/fa2 0 2024-06-06T08:43:50.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/730: write d0/d1/f7c [495321,92182] 0 2024-06-06T08:43:50.676 INFO:tasks.workunit.client.0.smithi027.stdout:4/731: creat d0/d1d/d34/d7f/d97/dad/fee x:0 0 0 2024-06-06T08:43:50.676 INFO:tasks.workunit.client.0.smithi027.stdout:4/732: truncate d0/d1d/f59 4249762 0 2024-06-06T08:43:50.678 INFO:tasks.workunit.client.0.smithi027.stdout:2/666: sync 2024-06-06T08:43:50.678 INFO:tasks.workunit.client.0.smithi027.stdout:7/761: sync 2024-06-06T08:43:50.678 INFO:tasks.workunit.client.0.smithi027.stdout:4/733: truncate d0/d1d/d2a/f5e 1619261 0 2024-06-06T08:43:50.678 INFO:tasks.workunit.client.0.smithi027.stdout:2/667: chown d6/dd/dd9/d1d/c72 2 1 2024-06-06T08:43:50.678 INFO:tasks.workunit.client.0.smithi027.stdout:7/762: chown d17/d47/f4d 1873781542 1 2024-06-06T08:43:50.681 INFO:tasks.workunit.client.0.smithi027.stdout:4/734: fsync d0/d1d/d34/d7f/d97/dad/fee 0 2024-06-06T08:43:50.682 INFO:tasks.workunit.client.0.smithi027.stdout:2/668: unlink d6/d3b/l90 0 2024-06-06T08:43:50.682 INFO:tasks.workunit.client.0.smithi027.stdout:5/854: dwrite d0/da/db/d46/d7f/d81/d97/fb1 [0,4194304] 0 2024-06-06T08:43:50.682 INFO:tasks.workunit.client.0.smithi027.stdout:5/855: dread - d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd/f108 zero size 2024-06-06T08:43:50.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/735: mkdir d0/d1/d6/d8/dd5/def 0 2024-06-06T08:43:50.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/669: symlink d6/d1e/d2b/ldd 0 2024-06-06T08:43:50.686 INFO:tasks.workunit.client.0.smithi027.stdout:2/670: creat d6/dd/dd9/d3f/d48/fde x:0 0 0 2024-06-06T08:43:50.687 INFO:tasks.workunit.client.0.smithi027.stdout:1/599: dread dc/de/d14/f49 [0,4194304] 0 2024-06-06T08:43:50.692 INFO:tasks.workunit.client.0.smithi027.stdout:7/763: dread d17/f1e [0,4194304] 0 2024-06-06T08:43:50.694 INFO:tasks.workunit.client.0.smithi027.stdout:7/764: creat d17/d32/d46/d4f/d5e/ff8 x:0 0 0 2024-06-06T08:43:50.694 INFO:tasks.workunit.client.0.smithi027.stdout:7/765: truncate d17/d32/d46/d4f/d5e/ff8 612819 0 2024-06-06T08:43:50.695 INFO:tasks.workunit.client.0.smithi027.stdout:5/856: read d0/da/f72 [11128813,30815] 0 2024-06-06T08:43:50.696 INFO:tasks.workunit.client.0.smithi027.stdout:1/600: rename dc/de/d3d/d4f/f77 to dc/d67/d69/d32/d71/fbb 0 2024-06-06T08:43:50.698 INFO:tasks.workunit.client.0.smithi027.stdout:7/766: creat d17/d32/d46/d4f/db6/dc8/de0/ff9 x:0 0 0 2024-06-06T08:43:50.698 INFO:tasks.workunit.client.0.smithi027.stdout:7/767: stat d17/d32/d46/d51/d52/d76 0 2024-06-06T08:43:50.699 INFO:tasks.workunit.client.0.smithi027.stdout:1/601: creat dc/fbc x:0 0 0 2024-06-06T08:43:50.699 INFO:tasks.workunit.client.0.smithi027.stdout:7/768: mknod d17/d32/d46/d83/d77/d8c/cfa 0 2024-06-06T08:43:50.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/769: fdatasync d17/d32/d46/d83/d77/fa2 0 2024-06-06T08:43:50.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/736: dwrite d0/d1/d6/d8/fe2 [0,4194304] 0 2024-06-06T08:43:50.708 INFO:tasks.workunit.client.0.smithi027.stdout:2/671: dwrite d6/dd/d6b/fb3 [0,4194304] 0 2024-06-06T08:43:50.710 INFO:tasks.workunit.client.0.smithi027.stdout:4/737: mknod d0/d1d/d60/dc6/d6e/db6/cf0 0 2024-06-06T08:43:50.714 INFO:tasks.workunit.client.0.smithi027.stdout:7/770: write d17/d32/d46/d4f/d5e/fb0 [35639,9051] 0 2024-06-06T08:43:50.715 INFO:tasks.workunit.client.0.smithi027.stdout:2/672: mkdir d6/dd/dd9/ddf 0 2024-06-06T08:43:50.717 INFO:tasks.workunit.client.0.smithi027.stdout:7/771: symlink d17/d32/d46/d83/lfb 0 2024-06-06T08:43:50.719 INFO:tasks.workunit.client.0.smithi027.stdout:7/772: mkdir d17/d32/d46/d83/d77/d8c/d91/dfc 0 2024-06-06T08:43:50.719 INFO:tasks.workunit.client.0.smithi027.stdout:1/602: dwrite dc/de/d14/f91 [0,4194304] 0 2024-06-06T08:43:50.719 INFO:tasks.workunit.client.0.smithi027.stdout:1/603: stat dc/d67/d69/d32/d37/d56/d78/c9a 0 2024-06-06T08:43:50.720 INFO:tasks.workunit.client.0.smithi027.stdout:4/738: dread d0/d1/f2 [0,4194304] 0 2024-06-06T08:43:50.722 INFO:tasks.workunit.client.0.smithi027.stdout:4/739: truncate d0/d1/f14 687766 0 2024-06-06T08:43:50.723 INFO:tasks.workunit.client.0.smithi027.stdout:4/740: mknod d0/d1/d6/dd4/dde/cf1 0 2024-06-06T08:43:50.724 INFO:tasks.workunit.client.0.smithi027.stdout:5/857: dwrite d0/f1 [4194304,4194304] 0 2024-06-06T08:43:50.724 INFO:tasks.workunit.client.0.smithi027.stdout:5/858: write d0/da/d37/f41 [1814671,23921] 0 2024-06-06T08:43:50.724 INFO:tasks.workunit.client.0.smithi027.stdout:5/859: chown d0/da/db/d46/d7f/d81/d97/fb1 41001 1 2024-06-06T08:43:50.724 INFO:tasks.workunit.client.0.smithi027.stdout:4/741: fdatasync d0/d1/f12 0 2024-06-06T08:43:50.727 INFO:tasks.workunit.client.0.smithi027.stdout:5/860: chown d0/da/d20/f5c 252536 1 2024-06-06T08:43:50.727 INFO:tasks.workunit.client.0.smithi027.stdout:5/861: stat d0/da/fae 0 2024-06-06T08:43:50.739 INFO:tasks.workunit.client.0.smithi027.stdout:5/862: dread d0/da/d37/d2b/f4a [0,4194304] 0 2024-06-06T08:43:50.739 INFO:tasks.workunit.client.0.smithi027.stdout:1/604: write dc/d67/d69/d32/d37/f41 [2473192,108721] 0 2024-06-06T08:43:50.740 INFO:tasks.workunit.client.0.smithi027.stdout:5/863: write d0/da/db/d1a/fda [467068,1544] 0 2024-06-06T08:43:50.741 INFO:tasks.workunit.client.0.smithi027.stdout:7/773: write d17/f21 [1156945,5977] 0 2024-06-06T08:43:50.742 INFO:tasks.workunit.client.0.smithi027.stdout:5/864: mknod d0/da/db/d1a/d5a/de0/c10d 0 2024-06-06T08:43:50.742 INFO:tasks.workunit.client.0.smithi027.stdout:5/865: write d0/da/db/d46/d7f/d81/fb5 [327477,68117] 0 2024-06-06T08:43:50.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/774: creat d17/d32/d6c/ffd x:0 0 0 2024-06-06T08:43:50.744 INFO:tasks.workunit.client.0.smithi027.stdout:5/866: getdents d0/da/db 0 2024-06-06T08:43:50.744 INFO:tasks.workunit.client.0.smithi027.stdout:5/867: getdents d0/da/df0/d106 0 2024-06-06T08:43:50.749 INFO:tasks.workunit.client.0.smithi027.stdout:1/605: link dc/de/l6e dc/d67/d69/d32/d37/d56/d78/lbd 0 2024-06-06T08:43:50.749 INFO:tasks.workunit.client.0.smithi027.stdout:1/606: fdatasync dc/f36 0 2024-06-06T08:43:50.749 INFO:tasks.workunit.client.0.smithi027.stdout:2/673: dwrite d6/dd/dd9/d1d/f4b [0,4194304] 0 2024-06-06T08:43:50.750 INFO:tasks.workunit.client.0.smithi027.stdout:7/775: creat d17/d32/d46/d4f/d5e/d7b/d7c/ffe x:0 0 0 2024-06-06T08:43:50.751 INFO:tasks.workunit.client.0.smithi027.stdout:1/607: creat dc/d67/fbe x:0 0 0 2024-06-06T08:43:50.754 INFO:tasks.workunit.client.0.smithi027.stdout:2/674: creat d6/d1e/d2b/fe0 x:0 0 0 2024-06-06T08:43:50.759 INFO:tasks.workunit.client.0.smithi027.stdout:1/608: dread dc/de/d14/d80/fa4 [0,4194304] 0 2024-06-06T08:43:50.760 INFO:tasks.workunit.client.0.smithi027.stdout:7/776: symlink d17/d32/d46/d4f/d5e/d7b/d7c/d92/lff 0 2024-06-06T08:43:50.762 INFO:tasks.workunit.client.0.smithi027.stdout:7/777: symlink d17/d32/d46/d4f/db6/dc8/de8/l100 0 2024-06-06T08:43:50.763 INFO:tasks.workunit.client.0.smithi027.stdout:2/675: unlink d6/d1e/d2b/d5a/d7b/l98 0 2024-06-06T08:43:50.765 INFO:tasks.workunit.client.0.smithi027.stdout:2/676: mkdir d6/dd/dd9/ddf/de1 0 2024-06-06T08:43:50.767 INFO:tasks.workunit.client.0.smithi027.stdout:2/677: truncate d6/d1e/d2b/d5a/d7b/f7e 582978 0 2024-06-06T08:43:50.768 INFO:tasks.workunit.client.0.smithi027.stdout:2/678: chown d6/dd/dd9/d1d/c6a 1353943 1 2024-06-06T08:43:50.774 INFO:tasks.workunit.client.0.smithi027.stdout:7/778: dread d17/d32/f2f [0,4194304] 0 2024-06-06T08:43:50.775 INFO:tasks.workunit.client.0.smithi027.stdout:7/779: creat d17/d32/d46/d4f/db6/dc8/f101 x:0 0 0 2024-06-06T08:43:50.775 INFO:tasks.workunit.client.0.smithi027.stdout:4/742: rename d0/d1/d75/c81 to d0/d1/d6/d8/d82/d8b/dc9/da3/daa/cf2 0 2024-06-06T08:43:50.777 INFO:tasks.workunit.client.0.smithi027.stdout:4/743: mkdir d0/d1/d75/df3 0 2024-06-06T08:43:50.777 INFO:tasks.workunit.client.0.smithi027.stdout:1/609: dwrite dc/f81 [0,4194304] 0 2024-06-06T08:43:50.778 INFO:tasks.workunit.client.0.smithi027.stdout:2/679: dread d6/d3b/f43 [0,4194304] 0 2024-06-06T08:43:50.778 INFO:tasks.workunit.client.0.smithi027.stdout:2/680: write d6/fd3 [231218,44582] 0 2024-06-06T08:43:50.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/834: sync 2024-06-06T08:43:50.779 INFO:tasks.workunit.client.0.smithi027.stdout:0/642: sync 2024-06-06T08:43:50.779 INFO:tasks.workunit.client.0.smithi027.stdout:3/609: sync 2024-06-06T08:43:50.779 INFO:tasks.workunit.client.0.smithi027.stdout:9/629: sync 2024-06-06T08:43:50.779 INFO:tasks.workunit.client.0.smithi027.stdout:8/607: sync 2024-06-06T08:43:50.779 INFO:tasks.workunit.client.0.smithi027.stdout:3/610: write d1/d8/fa7 [179176,73275] 0 2024-06-06T08:43:50.779 INFO:tasks.workunit.client.0.smithi027.stdout:7/780: creat d17/d32/d46/d83/d2a/d3b/dad/f102 x:0 0 0 2024-06-06T08:43:50.779 INFO:tasks.workunit.client.0.smithi027.stdout:7/781: truncate d17/d32/d46/d4f/db6/dc8/f101 208911 0 2024-06-06T08:43:50.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/643: creat d15/d18/dbe/d5c/d83/fd4 x:0 0 0 2024-06-06T08:43:50.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/782: readlink d17/d32/d46/d83/d2a/d3b/l48 0 2024-06-06T08:43:50.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/783: write d17/d47/ff1 [70036,63543] 0 2024-06-06T08:43:50.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/744: rmdir d0/d1d/d60 39 2024-06-06T08:43:50.785 INFO:tasks.workunit.client.0.smithi027.stdout:6/835: rmdir d4/d13 39 2024-06-06T08:43:50.786 INFO:tasks.workunit.client.0.smithi027.stdout:7/784: mkdir d17/d32/d46/d83/d77/d8c/d91/daa/d103 0 2024-06-06T08:43:50.786 INFO:tasks.workunit.client.0.smithi027.stdout:6/836: readlink d4/d9/dc/lb0 0 2024-06-06T08:43:50.786 INFO:tasks.workunit.client.0.smithi027.stdout:6/837: chown d4/de/d45/f89 120123 1 2024-06-06T08:43:50.787 INFO:tasks.workunit.client.0.smithi027.stdout:7/785: creat d17/d32/d46/d4f/db6/dc8/de0/f104 x:0 0 0 2024-06-06T08:43:50.788 INFO:tasks.workunit.client.0.smithi027.stdout:8/608: mknod d7/dc/d36/d83/d60/d93/ccb 0 2024-06-06T08:43:50.788 INFO:tasks.workunit.client.0.smithi027.stdout:0/644: mknod d15/d18/d23/d38/d68/cd5 0 2024-06-06T08:43:50.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/611: unlink d1/d90/d6d/c75 0 2024-06-06T08:43:50.790 INFO:tasks.workunit.client.0.smithi027.stdout:0/645: readlink d15/d18/d23/l5f 0 2024-06-06T08:43:50.790 INFO:tasks.workunit.client.0.smithi027.stdout:6/838: creat d4/de/d45/d6a/f114 x:0 0 0 2024-06-06T08:43:50.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/646: mknod d15/d18/d23/d5b/da0/cd6 0 2024-06-06T08:43:50.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/647: stat d15/d18/d23/d28/d31/f5e 0 2024-06-06T08:43:50.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/648: dread - d15/d18/dbe/d5d/f98 zero size 2024-06-06T08:43:50.792 INFO:tasks.workunit.client.0.smithi027.stdout:3/612: link d1/d90/d44/la1 d1/d90/d44/d54/db4/lbd 0 2024-06-06T08:43:50.792 INFO:tasks.workunit.client.0.smithi027.stdout:3/613: readlink d1/d1f/d5b/l83 0 2024-06-06T08:43:50.792 INFO:tasks.workunit.client.0.smithi027.stdout:7/786: symlink d17/d32/d46/l105 0 2024-06-06T08:43:50.794 INFO:tasks.workunit.client.0.smithi027.stdout:3/614: creat d1/d8/d22/d27/d50/fbe x:0 0 0 2024-06-06T08:43:50.795 INFO:tasks.workunit.client.0.smithi027.stdout:2/681: dread d6/d1e/f68 [0,4194304] 0 2024-06-06T08:43:50.795 INFO:tasks.workunit.client.0.smithi027.stdout:2/682: dread - d6/d1e/d2b/d5a/fd6 zero size 2024-06-06T08:43:50.795 INFO:tasks.workunit.client.0.smithi027.stdout:8/609: creat d7/dc/d36/d41/d85/fcc x:0 0 0 2024-06-06T08:43:50.796 INFO:tasks.workunit.client.0.smithi027.stdout:8/610: symlink d7/dc/d36/d41/d85/db7/lcd 0 2024-06-06T08:43:50.797 INFO:tasks.workunit.client.0.smithi027.stdout:7/787: dread d17/d47/fa9 [0,4194304] 0 2024-06-06T08:43:50.798 INFO:tasks.workunit.client.0.smithi027.stdout:2/683: unlink d6/d3b/f43 0 2024-06-06T08:43:50.800 INFO:tasks.workunit.client.0.smithi027.stdout:8/611: write d7/d13/d15/f1c [4593180,104460] 0 2024-06-06T08:43:50.800 INFO:tasks.workunit.client.0.smithi027.stdout:7/788: mknod d17/d32/d46/d83/d77/d8c/d91/daa/c106 0 2024-06-06T08:43:50.802 INFO:tasks.workunit.client.0.smithi027.stdout:7/789: symlink d17/d32/d46/d51/d52/d70/dce/l107 0 2024-06-06T08:43:50.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/684: mkdir d6/d1e/d2b/d5a/db7/de2 0 2024-06-06T08:43:50.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/685: stat d6/dd/dd9/d1d/f36 0 2024-06-06T08:43:50.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/686: stat d6/dd/dd9/d3f/d77/d55/d69/l95 0 2024-06-06T08:43:50.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/745: dwrite d0/d1d/d34/dd8/fea [0,4194304] 0 2024-06-06T08:43:50.811 INFO:tasks.workunit.client.0.smithi027.stdout:1/610: dwrite dc/de/d14/f49 [0,4194304] 0 2024-06-06T08:43:50.820 INFO:tasks.workunit.client.0.smithi027.stdout:7/790: write d17/f23 [4614361,100500] 0 2024-06-06T08:43:50.820 INFO:tasks.workunit.client.0.smithi027.stdout:9/630: link d3/d95/l63 d3/ldd 0 2024-06-06T08:43:50.820 INFO:tasks.workunit.client.0.smithi027.stdout:4/746: mkdir d0/d1d/d34/d7f/df4 0 2024-06-06T08:43:50.821 INFO:tasks.workunit.client.0.smithi027.stdout:0/649: dwrite d15/d18/d23/d38/d68/d85/f8d [0,4194304] 0 2024-06-06T08:43:50.821 INFO:tasks.workunit.client.0.smithi027.stdout:0/650: fdatasync d15/d18/d40/f9b 0 2024-06-06T08:43:50.822 INFO:tasks.workunit.client.0.smithi027.stdout:4/747: creat d0/d1d/d34/dd8/ff5 x:0 0 0 2024-06-06T08:43:50.822 INFO:tasks.workunit.client.0.smithi027.stdout:9/631: creat d3/d95/d34/fde x:0 0 0 2024-06-06T08:43:50.824 INFO:tasks.workunit.client.0.smithi027.stdout:8/612: dwrite d7/d13/f17 [0,4194304] 0 2024-06-06T08:43:50.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/615: dwrite d1/d8/d22/fba [0,4194304] 0 2024-06-06T08:43:50.828 INFO:tasks.workunit.client.0.smithi027.stdout:6/839: link d4/d1a/d2e/d44/dca/fe5 d4/de/d45/f115 0 2024-06-06T08:43:50.829 INFO:tasks.workunit.client.0.smithi027.stdout:9/632: write d3/d95/f39 [466453,49001] 0 2024-06-06T08:43:50.834 INFO:tasks.workunit.client.0.smithi027.stdout:3/616: creat d1/d8/d22/d27/d50/fbf x:0 0 0 2024-06-06T08:43:50.834 INFO:tasks.workunit.client.0.smithi027.stdout:6/840: fsync d4/fc3 0 2024-06-06T08:43:50.835 INFO:tasks.workunit.client.0.smithi027.stdout:0/651: read d15/d18/d23/d38/d3e/f5a [858381,32948] 0 2024-06-06T08:43:50.837 INFO:tasks.workunit.client.0.smithi027.stdout:3/617: rename d1/d8/d22/d27/f36 to d1/d8/d22/d27/d50/fc0 0 2024-06-06T08:43:50.837 INFO:tasks.workunit.client.0.smithi027.stdout:3/618: chown d1/d8/d59 227215 1 2024-06-06T08:43:50.837 INFO:tasks.workunit.client.0.smithi027.stdout:6/841: creat d4/d9/dc5/f116 x:0 0 0 2024-06-06T08:43:50.839 INFO:tasks.workunit.client.0.smithi027.stdout:0/652: mknod d15/d18/d23/d5b/da0/cd7 0 2024-06-06T08:43:50.839 INFO:tasks.workunit.client.0.smithi027.stdout:3/619: creat d1/d1f/d45/d5f/fc1 x:0 0 0 2024-06-06T08:43:50.839 INFO:tasks.workunit.client.0.smithi027.stdout:3/620: dread - d1/d8/d22/d2d/faa zero size 2024-06-06T08:43:50.839 INFO:tasks.workunit.client.0.smithi027.stdout:3/621: chown d1/d90/d44/f49 52709 1 2024-06-06T08:43:50.839 INFO:tasks.workunit.client.0.smithi027.stdout:3/622: write d1/d1f/d5b/fb3 [951678,59240] 0 2024-06-06T08:43:50.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/842: fsync d4/f29 0 2024-06-06T08:43:50.841 INFO:tasks.workunit.client.0.smithi027.stdout:7/791: write d17/d32/d46/d51/d52/d76/faf [2093036,83032] 0 2024-06-06T08:43:50.841 INFO:tasks.workunit.client.0.smithi027.stdout:7/792: chown d17/d32/d46/d83/d77/d8c/d91/daa/d103 3 1 2024-06-06T08:43:50.841 INFO:tasks.workunit.client.0.smithi027.stdout:0/653: link d15/da7/fb2 d15/dae/fd8 0 2024-06-06T08:43:50.842 INFO:tasks.workunit.client.0.smithi027.stdout:3/623: creat d1/d1f/d25/fc2 x:0 0 0 2024-06-06T08:43:50.844 INFO:tasks.workunit.client.0.smithi027.stdout:8/613: dwrite d7/d2f/f92 [0,4194304] 0 2024-06-06T08:43:50.844 INFO:tasks.workunit.client.0.smithi027.stdout:8/614: write d7/dc/d36/d83/d60/d93/dac/d73/fc2 [605419,107351] 0 2024-06-06T08:43:50.845 INFO:tasks.workunit.client.0.smithi027.stdout:7/793: stat cc 0 2024-06-06T08:43:50.845 INFO:tasks.workunit.client.0.smithi027.stdout:7/794: write d17/d32/d46/d4f/db6/dc8/de0/f104 [526884,70181] 0 2024-06-06T08:43:50.847 INFO:tasks.workunit.client.0.smithi027.stdout:4/748: dwrite d0/d1/ff [0,4194304] 0 2024-06-06T08:43:50.851 INFO:tasks.workunit.client.0.smithi027.stdout:9/633: rmdir d3/d4e/d75 39 2024-06-06T08:43:50.852 INFO:tasks.workunit.client.0.smithi027.stdout:0/654: write f10 [1774588,118595] 0 2024-06-06T08:43:50.853 INFO:tasks.workunit.client.0.smithi027.stdout:4/749: creat d0/d63/ff6 x:0 0 0 2024-06-06T08:43:50.855 INFO:tasks.workunit.client.0.smithi027.stdout:4/750: creat d0/d1/d6/d8/d82/d8b/ff7 x:0 0 0 2024-06-06T08:43:50.856 INFO:tasks.workunit.client.0.smithi027.stdout:9/634: mkdir d3/d95/d23/d78/ddf 0 2024-06-06T08:43:50.857 INFO:tasks.workunit.client.0.smithi027.stdout:8/615: dread d7/dc/f9f [0,4194304] 0 2024-06-06T08:43:50.857 INFO:tasks.workunit.client.0.smithi027.stdout:4/751: symlink d0/d1d/dd0/lf8 0 2024-06-06T08:43:50.859 INFO:tasks.workunit.client.0.smithi027.stdout:7/795: dread d17/d32/d46/d51/d52/d76/fb8 [0,4194304] 0 2024-06-06T08:43:50.860 INFO:tasks.workunit.client.0.smithi027.stdout:3/624: dread d1/d8/d22/d27/f14 [0,4194304] 0 2024-06-06T08:43:50.860 INFO:tasks.workunit.client.0.smithi027.stdout:4/752: getdents d0/d1d/d34 0 2024-06-06T08:43:50.861 INFO:tasks.workunit.client.0.smithi027.stdout:8/616: creat d7/dc/d36/d41/d85/fce x:0 0 0 2024-06-06T08:43:50.869 INFO:tasks.workunit.client.0.smithi027.stdout:0/655: dread d15/d18/d23/d38/d3e/f5a [0,4194304] 0 2024-06-06T08:43:50.869 INFO:tasks.workunit.client.0.smithi027.stdout:3/625: creat d1/d8/d22/fc3 x:0 0 0 2024-06-06T08:43:50.869 INFO:tasks.workunit.client.0.smithi027.stdout:3/626: chown d1/d8/d22/d27/c1b 3989598 1 2024-06-06T08:43:50.870 INFO:tasks.workunit.client.0.smithi027.stdout:0/656: readlink d15/d18/d23/d38/lac 0 2024-06-06T08:43:50.872 INFO:tasks.workunit.client.0.smithi027.stdout:0/657: unlink d15/d18/d23/d28/d2a/ca5 0 2024-06-06T08:43:50.875 INFO:tasks.workunit.client.0.smithi027.stdout:3/627: creat d1/d90/d44/d54/fc4 x:0 0 0 2024-06-06T08:43:50.876 INFO:tasks.workunit.client.0.smithi027.stdout:3/628: creat d1/d8/d59/fc5 x:0 0 0 2024-06-06T08:43:50.877 INFO:tasks.workunit.client.0.smithi027.stdout:3/629: write d1/f29 [4039032,26029] 0 2024-06-06T08:43:50.877 INFO:tasks.workunit.client.0.smithi027.stdout:3/630: chown d1/d8/fa7 144273 1 2024-06-06T08:43:50.877 INFO:tasks.workunit.client.0.smithi027.stdout:0/658: rename d15/d18/d23/c41 to d15/d18/d23/d28/d2a/d74/cd9 0 2024-06-06T08:43:50.883 INFO:tasks.workunit.client.0.smithi027.stdout:4/753: dwrite d0/d1/d6/d8/d82/d8b/dc9/d3b/fe9 [0,4194304] 0 2024-06-06T08:43:50.885 INFO:tasks.workunit.client.0.smithi027.stdout:7/796: dread d17/d32/fca [4194304,4194304] 0 2024-06-06T08:43:50.887 INFO:tasks.workunit.client.0.smithi027.stdout:9/635: dwrite d3/fae [0,4194304] 0 2024-06-06T08:43:50.894 INFO:tasks.workunit.client.0.smithi027.stdout:4/754: read d0/d1d/d34/d7f/f94 [258460,54291] 0 2024-06-06T08:43:50.894 INFO:tasks.workunit.client.0.smithi027.stdout:4/755: write d0/d1/f7c [683810,86786] 0 2024-06-06T08:43:50.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/659: dwrite d15/d18/d23/d28/f4e [0,4194304] 0 2024-06-06T08:43:50.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/660: dread - d15/d18/dbe/d5c/fb8 zero size 2024-06-06T08:43:50.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/661: write d15/d18/d23/d5b/da0/fc8 [508497,33705] 0 2024-06-06T08:43:50.898 INFO:tasks.workunit.client.0.smithi027.stdout:0/662: write d15/d18/d40/d8c/d9c/fa6 [821901,115534] 0 2024-06-06T08:43:50.899 INFO:tasks.workunit.client.0.smithi027.stdout:9/636: dread d3/d95/d23/da9/fce [0,4194304] 0 2024-06-06T08:43:50.901 INFO:tasks.workunit.client.0.smithi027.stdout:0/663: dread d15/d18/d40/fc2 [0,4194304] 0 2024-06-06T08:43:50.901 INFO:tasks.workunit.client.0.smithi027.stdout:0/664: dread - d15/d18/d64/fcf zero size 2024-06-06T08:43:50.902 INFO:tasks.workunit.client.0.smithi027.stdout:3/631: dwrite d1/d8/f92 [0,4194304] 0 2024-06-06T08:43:50.906 INFO:tasks.workunit.client.0.smithi027.stdout:3/632: rename d1/d8/d22/d27/d50/l8f to d1/d8/d59/lc6 0 2024-06-06T08:43:50.907 INFO:tasks.workunit.client.0.smithi027.stdout:9/637: dread d3/d95/f2f [0,4194304] 0 2024-06-06T08:43:50.909 INFO:tasks.workunit.client.0.smithi027.stdout:7/797: dwrite d17/d32/d46/d51/d52/f85 [0,4194304] 0 2024-06-06T08:43:50.909 INFO:tasks.workunit.client.0.smithi027.stdout:9/638: mkdir d3/d95/d23/de0 0 2024-06-06T08:43:50.910 INFO:tasks.workunit.client.0.smithi027.stdout:9/639: chown d3/d95/d34/d73/la5 247780 1 2024-06-06T08:43:50.912 INFO:tasks.workunit.client.0.smithi027.stdout:9/640: creat d3/d95/d23/d78/fe1 x:0 0 0 2024-06-06T08:43:50.913 INFO:tasks.workunit.client.0.smithi027.stdout:7/798: creat d17/d32/d46/d51/d52/d76/f108 x:0 0 0 2024-06-06T08:43:50.914 INFO:tasks.workunit.client.0.smithi027.stdout:9/641: creat d3/d95/db6/fe2 x:0 0 0 2024-06-06T08:43:50.915 INFO:tasks.workunit.client.0.smithi027.stdout:7/799: mknod d17/d32/d46/d83/d77/d8c/d91/daa/c109 0 2024-06-06T08:43:50.918 INFO:tasks.workunit.client.0.smithi027.stdout:9/642: creat d3/fe3 x:0 0 0 2024-06-06T08:43:50.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/756: link d0/d1d/d34/d7f/d97/dad/fee d0/d1/d6/dd4/ff9 0 2024-06-06T08:43:50.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/757: truncate d0/d63/fe4 275507 0 2024-06-06T08:43:50.919 INFO:tasks.workunit.client.0.smithi027.stdout:4/758: creat d0/d1d/d34/d7f/d97/dad/ffa x:0 0 0 2024-06-06T08:43:50.920 INFO:tasks.workunit.client.0.smithi027.stdout:9/643: rmdir d3/d4e 39 2024-06-06T08:43:50.920 INFO:tasks.workunit.client.0.smithi027.stdout:9/644: write d3/d95/fba [1003761,17561] 0 2024-06-06T08:43:50.923 INFO:tasks.workunit.client.0.smithi027.stdout:7/800: write d17/d32/f38 [305183,6456] 0 2024-06-06T08:43:50.929 INFO:tasks.workunit.client.0.smithi027.stdout:4/759: dread d0/d1d/d34/f40 [0,4194304] 0 2024-06-06T08:43:50.932 INFO:tasks.workunit.client.0.smithi027.stdout:4/760: mknod d0/d1/d6/d8/d82/d8b/dc9/d3b/cfb 0 2024-06-06T08:43:50.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/761: write d0/d1d/d34/d7f/fcd [4532093,112342] 0 2024-06-06T08:43:50.935 INFO:tasks.workunit.client.0.smithi027.stdout:4/762: truncate d0/d1d/d34/dd8/fea 2390689 0 2024-06-06T08:43:50.944 INFO:tasks.workunit.client.0.smithi027.stdout:7/801: dwrite d17/d32/d61/d75/fec [0,4194304] 0 2024-06-06T08:43:50.944 INFO:tasks.workunit.client.0.smithi027.stdout:7/802: stat d17/d32/d46/d4f/db6 0 2024-06-06T08:43:50.944 INFO:tasks.workunit.client.0.smithi027.stdout:7/803: write d17/d32/d46/d83/d2a/d3b/dad/f102 [11781,8614] 0 2024-06-06T08:43:50.948 INFO:tasks.workunit.client.0.smithi027.stdout:4/763: dread d0/d1/d6/f33 [0,4194304] 0 2024-06-06T08:43:50.948 INFO:tasks.workunit.client.0.smithi027.stdout:7/804: read fd [718338,23103] 0 2024-06-06T08:43:50.950 INFO:tasks.workunit.client.0.smithi027.stdout:7/805: chown d17/d32/d46/d51/d52/d70/d87/fa3 96192244 1 2024-06-06T08:43:50.950 INFO:tasks.workunit.client.0.smithi027.stdout:7/806: dread - d17/d32/d6c/ffd zero size 2024-06-06T08:43:50.953 INFO:tasks.workunit.client.0.smithi027.stdout:7/807: dread d17/d32/d46/d51/f35 [0,4194304] 0 2024-06-06T08:43:50.955 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:50 smithi027 ceph-mon[32335]: pgmap v110: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 695 op/s 2024-06-06T08:43:50.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:50 smithi203 ceph-mon[36579]: pgmap v110: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 695 op/s 2024-06-06T08:43:50.991 INFO:tasks.workunit.client.0.smithi027.stdout:0/665: dwrite d15/d18/d23/d6f/fcd [0,4194304] 0 2024-06-06T08:43:50.994 INFO:tasks.workunit.client.0.smithi027.stdout:9/645: dwrite d3/d95/d23/d36/d3a/f68 [0,4194304] 0 2024-06-06T08:43:51.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:50 smithi176 ceph-mon[36786]: pgmap v110: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 695 op/s 2024-06-06T08:43:51.006 INFO:tasks.workunit.client.0.smithi027.stdout:9/646: write d3/d95/d34/fa3 [1558258,118459] 0 2024-06-06T08:43:51.008 INFO:tasks.workunit.client.0.smithi027.stdout:7/808: dread d17/f26 [4194304,4194304] 0 2024-06-06T08:43:51.009 INFO:tasks.workunit.client.0.smithi027.stdout:9/647: fdatasync d3/d95/d34/fa3 0 2024-06-06T08:43:51.009 INFO:tasks.workunit.client.0.smithi027.stdout:9/648: stat d3/d4e/d75/dc/d9b/f91 0 2024-06-06T08:43:51.011 INFO:tasks.workunit.client.0.smithi027.stdout:9/649: creat d3/d95/d23/d36/d3a/fe4 x:0 0 0 2024-06-06T08:43:51.011 INFO:tasks.workunit.client.0.smithi027.stdout:9/650: fdatasync d3/f11 0 2024-06-06T08:43:51.012 INFO:tasks.workunit.client.0.smithi027.stdout:9/651: dread d3/d95/d23/d36/fd6 [0,4194304] 0 2024-06-06T08:43:51.014 INFO:tasks.workunit.client.0.smithi027.stdout:0/666: dwrite d15/f22 [0,4194304] 0 2024-06-06T08:43:51.015 INFO:tasks.workunit.client.0.smithi027.stdout:0/667: symlink d15/dae/lda 0 2024-06-06T08:43:51.016 INFO:tasks.workunit.client.0.smithi027.stdout:7/809: read d17/f21 [1042514,34463] 0 2024-06-06T08:43:51.017 INFO:tasks.workunit.client.0.smithi027.stdout:7/810: rename d17/d32 to d17/d32/d46/d4f/d5e/d7b/d7c/d92/d10a 22 2024-06-06T08:43:51.018 INFO:tasks.workunit.client.0.smithi027.stdout:7/811: read d17/d32/d46/d4f/db6/dc8/de0/fe9 [139420,103641] 0 2024-06-06T08:43:51.018 INFO:tasks.workunit.client.0.smithi027.stdout:7/812: truncate d17/d47/ff1 53325 0 2024-06-06T08:43:51.020 INFO:tasks.workunit.client.0.smithi027.stdout:9/652: dread d3/d95/d23/da9/fb9 [0,4194304] 0 2024-06-06T08:43:51.021 INFO:tasks.workunit.client.0.smithi027.stdout:9/653: readlink d3/d4e/l6e 0 2024-06-06T08:43:51.021 INFO:tasks.workunit.client.0.smithi027.stdout:9/654: fdatasync d3/d95/d23/d36/fbd 0 2024-06-06T08:43:51.022 INFO:tasks.workunit.client.0.smithi027.stdout:7/813: dread d17/f60 [0,4194304] 0 2024-06-06T08:43:51.022 INFO:tasks.workunit.client.0.smithi027.stdout:9/655: mkdir d3/d95/d23/d78/ddf/de5 0 2024-06-06T08:43:51.023 INFO:tasks.workunit.client.0.smithi027.stdout:9/656: creat d3/d95/d23/d36/fe6 x:0 0 0 2024-06-06T08:43:51.024 INFO:tasks.workunit.client.0.smithi027.stdout:9/657: creat d3/d95/d34/fe7 x:0 0 0 2024-06-06T08:43:51.024 INFO:tasks.workunit.client.0.smithi027.stdout:7/814: mkdir d17/d32/d46/d51/d52/d10b 0 2024-06-06T08:43:51.026 INFO:tasks.workunit.client.0.smithi027.stdout:9/658: unlink d3/d4e/d75/dc/f2e 0 2024-06-06T08:43:51.027 INFO:tasks.workunit.client.0.smithi027.stdout:9/659: truncate d3/d4e/d75/fcf 325856 0 2024-06-06T08:43:51.029 INFO:tasks.workunit.client.0.smithi027.stdout:9/660: creat d3/d76/dd9/fe8 x:0 0 0 2024-06-06T08:43:51.030 INFO:tasks.workunit.client.0.smithi027.stdout:0/668: write d15/d18/d23/d38/f51 [333566,98069] 0 2024-06-06T08:43:51.030 INFO:tasks.workunit.client.0.smithi027.stdout:0/669: write d15/d18/d40/d8c/d9c/fa6 [1740662,35001] 0 2024-06-06T08:43:51.031 INFO:tasks.workunit.client.0.smithi027.stdout:9/661: rmdir d3/d95/d23/de0 0 2024-06-06T08:43:51.031 INFO:tasks.workunit.client.0.smithi027.stdout:0/670: truncate d15/d18/d64/f8a 7417083 0 2024-06-06T08:43:51.031 INFO:tasks.workunit.client.0.smithi027.stdout:0/671: fsync d15/d18/d40/fc2 0 2024-06-06T08:43:51.032 INFO:tasks.workunit.client.0.smithi027.stdout:9/662: creat d3/d95/d23/d78/ddf/fe9 x:0 0 0 2024-06-06T08:43:51.036 INFO:tasks.workunit.client.0.smithi027.stdout:0/672: mknod d15/d18/d23/d38/d68/cdb 0 2024-06-06T08:43:51.036 INFO:tasks.workunit.client.0.smithi027.stdout:0/673: chown d15/d18/d23/d5b/cc1 0 1 2024-06-06T08:43:51.036 INFO:tasks.workunit.client.0.smithi027.stdout:7/815: write d17/d32/d46/d4f/d5e/f8b [3301491,124067] 0 2024-06-06T08:43:51.037 INFO:tasks.workunit.client.0.smithi027.stdout:0/674: creat d15/d18/d23/d38/fdc x:0 0 0 2024-06-06T08:43:51.041 INFO:tasks.workunit.client.0.smithi027.stdout:9/663: dread d3/d95/d23/d36/f5d [0,4194304] 0 2024-06-06T08:43:51.046 INFO:tasks.workunit.client.0.smithi027.stdout:7/816: link d17/d32/fca d17/d32/d46/d51/d52/d70/d87/dc0/f10c 0 2024-06-06T08:43:51.046 INFO:tasks.workunit.client.0.smithi027.stdout:0/675: dread d15/f21 [0,4194304] 0 2024-06-06T08:43:51.049 INFO:tasks.workunit.client.0.smithi027.stdout:0/676: creat d15/d18/dbe/d5c/fdd x:0 0 0 2024-06-06T08:43:51.052 INFO:tasks.workunit.client.0.smithi027.stdout:0/677: symlink d15/d18/d23/lde 0 2024-06-06T08:43:51.056 INFO:tasks.workunit.client.0.smithi027.stdout:9/664: write d3/d95/d23/d36/f37 [3155828,125858] 0 2024-06-06T08:43:51.067 INFO:tasks.workunit.client.0.smithi027.stdout:7/817: link d17/f27 d17/d32/d46/d83/d77/f10d 0 2024-06-06T08:43:51.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/665: dread d3/d95/d23/d36/d3a/f68 [0,4194304] 0 2024-06-06T08:43:51.068 INFO:tasks.workunit.client.0.smithi027.stdout:9/666: truncate d3/f4 4038279 0 2024-06-06T08:43:51.068 INFO:tasks.workunit.client.0.smithi027.stdout:9/667: dread - d3/d4e/d75/dc/fc9 zero size 2024-06-06T08:43:51.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/678: rename d15/d18/d23/d28/d31/l7c to d15/d18/d40/ldf 0 2024-06-06T08:43:51.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/679: write fb [5715763,106152] 0 2024-06-06T08:43:51.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/680: fsync d15/d18/dbe/d5c/d83/fb6 0 2024-06-06T08:43:51.076 INFO:tasks.workunit.client.0.smithi027.stdout:0/681: mkdir d15/d18/de0 0 2024-06-06T08:43:51.082 INFO:tasks.workunit.client.0.smithi027.stdout:9/668: dread d3/d95/d34/f51 [0,4194304] 0 2024-06-06T08:43:51.085 INFO:tasks.workunit.client.0.smithi027.stdout:7/818: dread d17/f58 [0,4194304] 0 2024-06-06T08:43:51.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/669: link d3/d4e/d75/c4d d3/d76/cea 0 2024-06-06T08:43:51.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/670: read - d3/d4e/d75/f50 zero size 2024-06-06T08:43:51.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/671: fdatasync d3/d95/d34/d41/f9c 0 2024-06-06T08:43:51.094 INFO:tasks.workunit.client.0.smithi027.stdout:9/672: creat d3/d95/feb x:0 0 0 2024-06-06T08:43:51.094 INFO:tasks.workunit.client.0.smithi027.stdout:9/673: write d3/d95/f2f [87314,45688] 0 2024-06-06T08:43:51.095 INFO:tasks.workunit.client.0.smithi027.stdout:9/674: truncate d3/d95/f4f 3803261 0 2024-06-06T08:43:51.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/819: read d17/f3f [115699,101040] 0 2024-06-06T08:43:51.096 INFO:tasks.workunit.client.0.smithi027.stdout:0/682: dwrite d15/d18/d23/d6f/fcd [0,4194304] 0 2024-06-06T08:43:51.096 INFO:tasks.workunit.client.0.smithi027.stdout:9/675: creat d3/d95/fec x:0 0 0 2024-06-06T08:43:51.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/820: stat d17/d32/d46/d51/l40 0 2024-06-06T08:43:51.099 INFO:tasks.workunit.client.0.smithi027.stdout:9/676: creat d3/d95/d23/d36/d3a/d84/db2/fed x:0 0 0 2024-06-06T08:43:51.100 INFO:tasks.workunit.client.0.smithi027.stdout:0/683: symlink d15/d18/d64/le1 0 2024-06-06T08:43:51.101 INFO:tasks.workunit.client.0.smithi027.stdout:0/684: read d15/d18/d23/d28/d2a/d74/fb0 [693541,21062] 0 2024-06-06T08:43:51.101 INFO:tasks.workunit.client.0.smithi027.stdout:0/685: write d15/d18/d23/d38/d3e/f5a [480649,123051] 0 2024-06-06T08:43:51.101 INFO:tasks.workunit.client.0.smithi027.stdout:0/686: fsync d15/d18/d23/fba 0 2024-06-06T08:43:51.101 INFO:tasks.workunit.client.0.smithi027.stdout:7/821: rmdir d17/d32/d46/d83/d77 39 2024-06-06T08:43:51.105 INFO:tasks.workunit.client.0.smithi027.stdout:0/687: rmdir d15/d18/dbe/d5d 39 2024-06-06T08:43:51.106 INFO:tasks.workunit.client.0.smithi027.stdout:0/688: creat d15/d18/d23/d3c/fe2 x:0 0 0 2024-06-06T08:43:51.107 INFO:tasks.workunit.client.0.smithi027.stdout:0/689: symlink d15/d18/d23/d28/d2a/d74/le3 0 2024-06-06T08:43:51.109 INFO:tasks.workunit.client.0.smithi027.stdout:0/690: unlink d15/d18/d40/f76 0 2024-06-06T08:43:51.110 INFO:tasks.workunit.client.0.smithi027.stdout:0/691: dread d15/d18/d23/d28/d2a/f33 [0,4194304] 0 2024-06-06T08:43:51.112 INFO:tasks.workunit.client.0.smithi027.stdout:0/692: mknod d15/d18/d23/d28/d2a/ce4 0 2024-06-06T08:43:51.112 INFO:tasks.workunit.client.0.smithi027.stdout:0/693: fdatasync d15/d18/d23/d28/d2a/f33 0 2024-06-06T08:43:51.114 INFO:tasks.workunit.client.0.smithi027.stdout:9/677: dwrite d3/d95/d23/d36/fe6 [0,4194304] 0 2024-06-06T08:43:51.114 INFO:tasks.workunit.client.0.smithi027.stdout:9/678: fdatasync d3/d95/d23/d36/f5d 0 2024-06-06T08:43:51.116 INFO:tasks.workunit.client.0.smithi027.stdout:9/679: symlink d3/d95/d23/d78/lee 0 2024-06-06T08:43:51.116 INFO:tasks.workunit.client.0.smithi027.stdout:9/680: write d3/d4e/fa8 [1053169,103832] 0 2024-06-06T08:43:51.118 INFO:tasks.workunit.client.0.smithi027.stdout:9/681: mkdir d3/d95/d23/def 0 2024-06-06T08:43:51.121 INFO:tasks.workunit.client.0.smithi027.stdout:9/682: mkdir d3/d95/d23/d36/d3a/d84/df0 0 2024-06-06T08:43:51.123 INFO:tasks.workunit.client.0.smithi027.stdout:9/683: mkdir d3/d95/d23/d36/d3a/df1 0 2024-06-06T08:43:51.126 INFO:tasks.workunit.client.0.smithi027.stdout:7/822: rename d17/d32/d46/d83/d2a/lbe to d17/d32/d46/d4f/db6/dc8/de8/l10e 0 2024-06-06T08:43:51.128 INFO:tasks.workunit.client.0.smithi027.stdout:0/694: dwrite d15/d18/d23/d38/d68/f8f [4194304,4194304] 0 2024-06-06T08:43:51.129 INFO:tasks.workunit.client.0.smithi027.stdout:0/695: chown d15/d18/d23/d28/d31/f5e 1605334 1 2024-06-06T08:43:51.129 INFO:tasks.workunit.client.0.smithi027.stdout:7/823: symlink d17/d32/d46/d4f/d5e/d7b/l10f 0 2024-06-06T08:43:51.131 INFO:tasks.workunit.client.0.smithi027.stdout:0/696: unlink d15/d18/d23/d38/lac 0 2024-06-06T08:43:51.132 INFO:tasks.workunit.client.0.smithi027.stdout:7/824: stat d17/d32/d46/d83/d72/ld8 0 2024-06-06T08:43:51.134 INFO:tasks.workunit.client.0.smithi027.stdout:9/684: write d3/d95/d23/d36/d3a/d84/db2/fd0 [968,62600] 0 2024-06-06T08:43:51.135 INFO:tasks.workunit.client.0.smithi027.stdout:9/685: chown d3/d4e/d75/dc/f2a 161 1 2024-06-06T08:43:51.135 INFO:tasks.workunit.client.0.smithi027.stdout:9/686: readlink d3/d95/d23/d36/d3a/lca 0 2024-06-06T08:43:51.146 INFO:tasks.workunit.client.0.smithi027.stdout:7/825: dread d17/d32/f6d [0,4194304] 0 2024-06-06T08:43:51.148 INFO:tasks.workunit.client.0.smithi027.stdout:7/826: symlink d17/d32/d61/d75/l110 0 2024-06-06T08:43:51.149 INFO:tasks.workunit.client.0.smithi027.stdout:0/697: dwrite d15/f22 [4194304,4194304] 0 2024-06-06T08:43:51.149 INFO:tasks.workunit.client.0.smithi027.stdout:0/698: dread - d15/d18/d64/fcf zero size 2024-06-06T08:43:51.149 INFO:tasks.workunit.client.0.smithi027.stdout:7/827: truncate d17/d32/d6c/dc2/fbc 979909 0 2024-06-06T08:43:51.153 INFO:tasks.workunit.client.0.smithi027.stdout:0/699: mknod d15/ce5 0 2024-06-06T08:43:51.153 INFO:tasks.workunit.client.0.smithi027.stdout:0/700: chown d15/d18/d23/d38/d3e/f5a 23298544 1 2024-06-06T08:43:51.158 INFO:tasks.workunit.client.0.smithi027.stdout:9/687: dwrite d3/d95/f2f [0,4194304] 0 2024-06-06T08:43:51.160 INFO:tasks.workunit.client.0.smithi027.stdout:7/828: write d17/f26 [4957211,48591] 0 2024-06-06T08:43:51.166 INFO:tasks.workunit.client.0.smithi027.stdout:7/829: mkdir d17/d32/d46/d83/d77/d8c/dae/d111 0 2024-06-06T08:43:51.166 INFO:tasks.workunit.client.0.smithi027.stdout:0/701: write d15/d18/d23/d38/d68/f7b [3516064,43729] 0 2024-06-06T08:43:51.167 INFO:tasks.workunit.client.0.smithi027.stdout:7/830: truncate d17/d47/f49 101313 0 2024-06-06T08:43:51.167 INFO:tasks.workunit.client.0.smithi027.stdout:7/831: write d17/d32/d46/d83/d2a/f69 [2645142,66404] 0 2024-06-06T08:43:51.168 INFO:tasks.workunit.client.0.smithi027.stdout:8/617: sync 2024-06-06T08:43:51.169 INFO:tasks.workunit.client.0.smithi027.stdout:3/633: sync 2024-06-06T08:43:51.169 INFO:tasks.workunit.client.0.smithi027.stdout:4/764: sync 2024-06-06T08:43:51.169 INFO:tasks.workunit.client.0.smithi027.stdout:8/618: chown d7/dc/d36/d83/d60/d93/dac/d73 45864685 1 2024-06-06T08:43:51.169 INFO:tasks.workunit.client.0.smithi027.stdout:6/843: sync 2024-06-06T08:43:51.169 INFO:tasks.workunit.client.0.smithi027.stdout:2/687: sync 2024-06-06T08:43:51.169 INFO:tasks.workunit.client.0.smithi027.stdout:5/868: sync 2024-06-06T08:43:51.169 INFO:tasks.workunit.client.0.smithi027.stdout:2/688: stat d6/dd/dd9/d1d/c72 0 2024-06-06T08:43:51.169 INFO:tasks.workunit.client.0.smithi027.stdout:5/869: fsync d0/da/db/d46/d4c/f104 0 2024-06-06T08:43:51.169 INFO:tasks.workunit.client.0.smithi027.stdout:1/611: sync 2024-06-06T08:43:51.170 INFO:tasks.workunit.client.0.smithi027.stdout:4/765: chown d0/d1d/l2c 7 1 2024-06-06T08:43:51.171 INFO:tasks.workunit.client.0.smithi027.stdout:9/688: dread d3/d95/d34/fa3 [0,4194304] 0 2024-06-06T08:43:51.171 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3a2bb295', total seconds elapsed: 40.7 2024-06-06T08:43:51.171 INFO:tasks.workunit.client.0.smithi027.stdout:6/844: mknod d4/d9/df2/c117 0 2024-06-06T08:43:51.183 INFO:tasks.workunit.client.0.smithi027.stdout:5/870: creat d0/da/db/d1a/da3/de5/d107/f10e x:0 0 0 2024-06-06T08:43:51.188 INFO:tasks.workunit.client.0.smithi027.stdout:1/612: truncate dc/d67/d69/d32/d37/d56/f93 510147 0 2024-06-06T08:43:51.188 INFO:tasks.workunit.client.0.smithi027.stdout:1/613: readlink dc/de/d14/d4e/l88 0 2024-06-06T08:43:51.188 INFO:tasks.workunit.client.0.smithi027.stdout:1/614: dread - dc/d67/d69/fba zero size 2024-06-06T08:43:51.188 INFO:tasks.workunit.client.0.smithi027.stdout:8/619: write d7/f9 [512124,83011] 0 2024-06-06T08:43:51.190 INFO:tasks.workunit.client.0.smithi027.stdout:8/620: mknod d7/dc/d36/d41/dbf/ccf 0 2024-06-06T08:43:51.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/615: truncate dc/f36 1491037 0 2024-06-06T08:43:51.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/616: write dc/de/f75 [475409,116416] 0 2024-06-06T08:43:51.194 INFO:tasks.workunit.client.0.smithi027.stdout:3/634: dwrite d1/d8/d22/d2d/fbb [0,4194304] 0 2024-06-06T08:43:51.194 INFO:tasks.workunit.client.0.smithi027.stdout:5/871: link d0/da/fc8 d0/da/db/d46/d7f/f10f 0 2024-06-06T08:43:51.194 INFO:tasks.workunit.client.0.smithi027.stdout:0/702: dread d15/d18/d64/f6d [0,4194304] 0 2024-06-06T08:43:51.194 INFO:tasks.workunit.client.0.smithi027.stdout:7/832: dwrite d17/d32/d46/d51/d52/fd3 [0,4194304] 0 2024-06-06T08:43:51.199 INFO:tasks.workunit.client.0.smithi027.stdout:4/766: write d0/f5c [42816,34506] 0 2024-06-06T08:43:51.200 INFO:tasks.workunit.client.0.smithi027.stdout:8/621: dread d7/dc/d36/d83/fbc [0,4194304] 0 2024-06-06T08:43:51.201 INFO:tasks.workunit.client.0.smithi027.stdout:0/703: link d15/d18/d64/le1 d15/d18/d23/d28/d31/dbb/le6 0 2024-06-06T08:43:51.203 INFO:tasks.workunit.client.0.smithi027.stdout:0/704: mknod d15/d18/d40/ce7 0 2024-06-06T08:43:51.203 INFO:tasks.workunit.client.0.smithi027.stdout:9/689: dread d3/d95/d23/d36/d3a/f68 [0,4194304] 0 2024-06-06T08:43:51.203 INFO:tasks.workunit.client.0.smithi027.stdout:2/689: dwrite d6/fe [0,4194304] 0 2024-06-06T08:43:51.204 INFO:tasks.workunit.client.0.smithi027.stdout:9/690: chown d3/f15 295767 1 2024-06-06T08:43:51.204 INFO:tasks.workunit.client.0.smithi027.stdout:9/691: fdatasync d3/d4e/d75/fcf 0 2024-06-06T08:43:51.204 INFO:tasks.workunit.client.0.smithi027.stdout:9/692: dread - d3/d76/dd9/fe8 zero size 2024-06-06T08:43:51.206 INFO:tasks.workunit.client.0.smithi027.stdout:5/872: symlink d0/d6a/l110 0 2024-06-06T08:43:51.211 INFO:tasks.workunit.client.0.smithi027.stdout:2/690: read d6/dd/dd9/f12 [930877,55999] 0 2024-06-06T08:43:51.211 INFO:tasks.workunit.client.0.smithi027.stdout:6/845: dwrite d4/de/d45/d6a/f76 [0,4194304] 0 2024-06-06T08:43:51.212 INFO:tasks.workunit.client.0.smithi027.stdout:5/873: truncate d0/da/db/d1a/f71 487975 0 2024-06-06T08:43:51.218 INFO:tasks.workunit.client.0.smithi027.stdout:2/691: truncate d6/dd/f23 5090170 0 2024-06-06T08:43:51.219 INFO:tasks.workunit.client.0.smithi027.stdout:4/767: dread d0/d1d/d34/f6b [0,4194304] 0 2024-06-06T08:43:51.219 INFO:tasks.workunit.client.0.smithi027.stdout:1/617: dwrite dc/de/d14/d80/fa4 [0,4194304] 0 2024-06-06T08:43:51.220 INFO:tasks.workunit.client.0.smithi027.stdout:1/618: read - dc/d67/d69/fb9 zero size 2024-06-06T08:43:51.221 INFO:tasks.workunit.client.0.smithi027.stdout:9/693: rename d3/d4e/d75/l1e to d3/d95/d23/d36/d3a/d84/df0/lf2 0 2024-06-06T08:43:51.222 INFO:tasks.workunit.client.0.smithi027.stdout:5/874: mkdir d0/da/db/d1a/dc6/d111 0 2024-06-06T08:43:51.222 INFO:tasks.workunit.client.0.smithi027.stdout:7/833: write d17/d32/d61/f6a [938,14699] 0 2024-06-06T08:43:51.223 INFO:tasks.workunit.client.0.smithi027.stdout:9/694: creat d3/d4e/d75/dc/dd1/ff3 x:0 0 0 2024-06-06T08:43:51.224 INFO:tasks.workunit.client.0.smithi027.stdout:6/846: mkdir d4/d13/dba/d118 0 2024-06-06T08:43:51.224 INFO:tasks.workunit.client.0.smithi027.stdout:6/847: write d4/de/d45/f75 [2600957,84873] 0 2024-06-06T08:43:51.225 INFO:tasks.workunit.client.0.smithi027.stdout:6/848: write d4/d9/d49/f8d [5195368,67161] 0 2024-06-06T08:43:51.225 INFO:tasks.workunit.client.0.smithi027.stdout:5/875: dread d0/da/db/fa0 [0,4194304] 0 2024-06-06T08:43:51.225 INFO:tasks.workunit.client.0.smithi027.stdout:2/692: mkdir d6/dd/dd9/d3f/d48/d83/de3 0 2024-06-06T08:43:51.227 INFO:tasks.workunit.client.0.smithi027.stdout:5/876: creat d0/da/db/d46/d7f/d81/d97/db0/dbc/dc4/f112 x:0 0 0 2024-06-06T08:43:51.227 INFO:tasks.workunit.client.0.smithi027.stdout:2/693: readlink d6/dd/dd9/d3f/d48/d83/l9e 0 2024-06-06T08:43:51.229 INFO:tasks.workunit.client.0.smithi027.stdout:2/694: mkdir d6/dd/dd9/d3f/d48/d83/de3/de4 0 2024-06-06T08:43:51.230 INFO:tasks.workunit.client.0.smithi027.stdout:4/768: rename d0/d1/d6/d8/d82/d8b/dc9/da3/de3 to d0/d1d/d34/d7f/dfc 0 2024-06-06T08:43:51.230 INFO:tasks.workunit.client.0.smithi027.stdout:5/877: symlink d0/da/db/d46/d7f/d81/d97/db0/dbc/dcf/l113 0 2024-06-06T08:43:51.233 INFO:tasks.workunit.client.0.smithi027.stdout:5/878: mknod d0/da/db/d46/d7f/d81/d97/db0/dbc/c114 0 2024-06-06T08:43:51.235 INFO:tasks.workunit.client.0.smithi027.stdout:2/695: rename d6/dd/dd9/d3f/dd1 to d6/dd/dd9/d3f/d77/de5 0 2024-06-06T08:43:51.235 INFO:tasks.workunit.client.0.smithi027.stdout:3/635: dwrite d1/d1f/d25/f3a [0,4194304] 0 2024-06-06T08:43:51.235 INFO:tasks.workunit.client.0.smithi027.stdout:3/636: chown d1/d8/d22/fba 23 1 2024-06-06T08:43:51.236 INFO:tasks.workunit.client.0.smithi027.stdout:1/619: truncate dc/de/d14/d80/fa4 1688651 0 2024-06-06T08:43:51.238 INFO:tasks.workunit.client.0.smithi027.stdout:4/769: mknod d0/d1d/d34/dd8/cfd 0 2024-06-06T08:43:51.238 INFO:tasks.workunit.client.0.smithi027.stdout:4/770: chown d0/d1d/dd0/fd6 3097 1 2024-06-06T08:43:51.241 INFO:tasks.workunit.client.0.smithi027.stdout:7/834: dread d17/d32/d46/d83/d77/d8c/d91/daa/fdb [0,4194304] 0 2024-06-06T08:43:51.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/705: link d15/d18/d23/d28/cc9 d15/d18/dbe/d5d/ce8 0 2024-06-06T08:43:51.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/706: truncate d15/d18/d23/d6f/f94 779641 0 2024-06-06T08:43:51.244 INFO:tasks.workunit.client.0.smithi027.stdout:0/707: creat d15/d18/d23/d38/d68/d85/fe9 x:0 0 0 2024-06-06T08:43:51.245 INFO:tasks.workunit.client.0.smithi027.stdout:9/695: write d3/d95/d23/d36/fe6 [2963807,17456] 0 2024-06-06T08:43:51.247 INFO:tasks.workunit.client.0.smithi027.stdout:2/696: creat d6/dd/dd9/d3f/d48/d83/fe6 x:0 0 0 2024-06-06T08:43:51.247 INFO:tasks.workunit.client.0.smithi027.stdout:0/708: link d15/d18/d23/d28/d2a/d2d/f34 d15/d18/d23/d38/d68/d85/fea 0 2024-06-06T08:43:51.247 INFO:tasks.workunit.client.0.smithi027.stdout:4/771: dread d0/d1d/d34/d7f/f94 [0,4194304] 0 2024-06-06T08:43:51.249 INFO:tasks.workunit.client.0.smithi027.stdout:0/709: mkdir d15/d18/d23/d6f/deb 0 2024-06-06T08:43:51.249 INFO:tasks.workunit.client.0.smithi027.stdout:0/710: fsync d15/d18/d23/d6f/f94 0 2024-06-06T08:43:51.251 INFO:tasks.workunit.client.0.smithi027.stdout:3/637: dread d1/f85 [0,4194304] 0 2024-06-06T08:43:51.255 INFO:tasks.workunit.client.0.smithi027.stdout:1/620: rmdir dc/de/d14/d80 39 2024-06-06T08:43:51.255 INFO:tasks.workunit.client.0.smithi027.stdout:4/772: truncate d0/d1d/d34/d7f/d97/d9c/fbd 56707 0 2024-06-06T08:43:51.261 INFO:tasks.workunit.client.0.smithi027.stdout:4/773: getdents d0/d1d/d60/dc6/d6e 0 2024-06-06T08:43:51.262 INFO:tasks.workunit.client.0.smithi027.stdout:3/638: creat d1/d1f/d45/fc7 x:0 0 0 2024-06-06T08:43:51.262 INFO:tasks.workunit.client.0.smithi027.stdout:3/639: chown d1/d8/l28 136386 1 2024-06-06T08:43:51.263 INFO:tasks.workunit.client.0.smithi027.stdout:3/640: fsync d1/d1f/d45/d5f/f7c 0 2024-06-06T08:43:51.266 INFO:tasks.workunit.client.0.smithi027.stdout:3/641: creat d1/d8/d22/fc8 x:0 0 0 2024-06-06T08:43:51.266 INFO:tasks.workunit.client.0.smithi027.stdout:3/642: dread - d1/d8/fab zero size 2024-06-06T08:43:51.268 INFO:tasks.workunit.client.0.smithi027.stdout:3/643: dread d1/d8/d59/f94 [0,4194304] 0 2024-06-06T08:43:51.268 INFO:tasks.workunit.client.0.smithi027.stdout:3/644: write d1/f1e [1856329,46861] 0 2024-06-06T08:43:51.268 INFO:tasks.workunit.client.0.smithi027.stdout:3/645: chown d1/d8/d22/d2d/f99 102403 1 2024-06-06T08:43:51.268 INFO:tasks.workunit.client.0.smithi027.stdout:2/697: dwrite d6/dd/dd9/fd4 [0,4194304] 0 2024-06-06T08:43:51.273 INFO:tasks.workunit.client.0.smithi027.stdout:5/879: rename d0/da/f92 to d0/da/db/d46/d7f/d81/d97/db0/f115 0 2024-06-06T08:43:51.273 INFO:tasks.workunit.client.0.smithi027.stdout:5/880: chown d0/da/db/d46/d77/f9d 6087 1 2024-06-06T08:43:51.277 INFO:tasks.workunit.client.0.smithi027.stdout:1/621: dwrite dc/d67/d69/fb9 [0,4194304] 0 2024-06-06T08:43:51.278 INFO:tasks.workunit.client.0.smithi027.stdout:3/646: write d1/d1f/d25/f76 [4017984,13553] 0 2024-06-06T08:43:51.280 INFO:tasks.workunit.client.0.smithi027.stdout:1/622: rmdir dc/de/d14/d80/da6 39 2024-06-06T08:43:51.282 INFO:tasks.workunit.client.0.smithi027.stdout:1/623: mknod dc/d67/d69/d32/d37/cbf 0 2024-06-06T08:43:51.283 INFO:tasks.workunit.client.0.smithi027.stdout:3/647: link d1/d8/c23 d1/d90/d44/d54/db4/cc9 0 2024-06-06T08:43:51.283 INFO:tasks.workunit.client.0.smithi027.stdout:3/648: chown d1/f84 588 1 2024-06-06T08:43:51.283 INFO:tasks.workunit.client.0.smithi027.stdout:4/774: dwrite d0/d1d/d34/d7f/d97/dad/fee [0,4194304] 0 2024-06-06T08:43:51.283 INFO:tasks.workunit.client.0.smithi027.stdout:4/775: fsync d0/d1/f7c 0 2024-06-06T08:43:51.283 INFO:tasks.workunit.client.0.smithi027.stdout:4/776: stat d0/d1/d6/d8/d82/d8b/dc9/fc8 0 2024-06-06T08:43:51.284 INFO:tasks.workunit.client.0.smithi027.stdout:3/649: mkdir d1/d1f/d45/d5f/dca 0 2024-06-06T08:43:51.285 INFO:tasks.workunit.client.0.smithi027.stdout:3/650: creat d1/d90/fcb x:0 0 0 2024-06-06T08:43:51.287 INFO:tasks.workunit.client.0.smithi027.stdout:4/777: link d0/d1/d6/f33 d0/d1d/ffe 0 2024-06-06T08:43:51.288 INFO:tasks.workunit.client.0.smithi027.stdout:4/778: chown d0/f55 126853263 1 2024-06-06T08:43:51.290 INFO:tasks.workunit.client.0.smithi027.stdout:5/881: dwrite d0/da/db/d1a/fde [0,4194304] 0 2024-06-06T08:43:51.290 INFO:tasks.workunit.client.0.smithi027.stdout:5/882: dread - d0/da/fb4 zero size 2024-06-06T08:43:51.290 INFO:tasks.workunit.client.0.smithi027.stdout:5/883: truncate d0/da/db/d46/d7f/d81/d97/db0/f115 1361114 0 2024-06-06T08:43:51.293 INFO:tasks.workunit.client.0.smithi027.stdout:3/651: getdents d1/d90 0 2024-06-06T08:43:51.294 INFO:tasks.workunit.client.0.smithi027.stdout:3/652: chown d1/d8/f61 5 1 2024-06-06T08:43:51.295 INFO:tasks.workunit.client.0.smithi027.stdout:4/779: symlink d0/d1d/d34/d7f/lff 0 2024-06-06T08:43:51.295 INFO:tasks.workunit.client.0.smithi027.stdout:4/780: write d0/d1d/dd0/fd6 [1080383,81449] 0 2024-06-06T08:43:51.295 INFO:tasks.workunit.client.0.smithi027.stdout:5/884: truncate d0/da/db/d1a/f6f 576048 0 2024-06-06T08:43:51.296 INFO:tasks.workunit.client.0.smithi027.stdout:3/653: creat d1/d1f/dbc/fcc x:0 0 0 2024-06-06T08:43:51.298 INFO:tasks.workunit.client.0.smithi027.stdout:3/654: creat d1/d8/da9/fcd x:0 0 0 2024-06-06T08:43:51.306 INFO:tasks.workunit.client.0.smithi027.stdout:4/781: dread d0/d1/d6/fae [0,4194304] 0 2024-06-06T08:43:51.308 INFO:tasks.workunit.client.0.smithi027.stdout:4/782: truncate d0/d1d/f54 5031260 0 2024-06-06T08:43:51.309 INFO:tasks.workunit.client.0.smithi027.stdout:7/835: rename d17/d32/d46/d83/d2a/d3b/dad/f102 to d17/d32/d46/d4f/db6/dc8/de0/f112 0 2024-06-06T08:43:51.309 INFO:tasks.workunit.client.0.smithi027.stdout:7/836: fsync d17/fa4 0 2024-06-06T08:43:51.310 INFO:tasks.workunit.client.0.smithi027.stdout:7/837: creat d17/d32/d46/d83/d77/d8c/d91/dfc/f113 x:0 0 0 2024-06-06T08:43:51.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/838: creat d17/d32/d46/d83/f114 x:0 0 0 2024-06-06T08:43:51.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/839: chown d17/d32/d46/d83/d77/d8c/d90/lc4 2 1 2024-06-06T08:43:51.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/840: chown d17/d32/d46/d51/d52/d70/dd6 1 1 2024-06-06T08:43:51.314 INFO:tasks.workunit.client.0.smithi027.stdout:7/841: creat d17/d32/d46/d51/d52/d70/d87/dc0/f115 x:0 0 0 2024-06-06T08:43:51.316 INFO:tasks.workunit.client.0.smithi027.stdout:7/842: mkdir d17/d32/d46/d4f/db6/dc8/de8/d116 0 2024-06-06T08:43:51.319 INFO:tasks.workunit.client.0.smithi027.stdout:7/843: dread d17/d32/d46/d83/d2a/fbf [0,4194304] 0 2024-06-06T08:43:51.320 INFO:tasks.workunit.client.0.smithi027.stdout:7/844: unlink d17/d32/d46/d83/d72/f98 0 2024-06-06T08:43:51.326 INFO:tasks.workunit.client.0.smithi027.stdout:9/696: rename d3/d95/l62 to d3/lf4 0 2024-06-06T08:43:51.328 INFO:tasks.workunit.client.0.smithi027.stdout:9/697: write d3/d4e/d75/dc/f8d [311282,18152] 0 2024-06-06T08:43:51.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/698: rename d6/d1e/f73 to d6/dd/fe7 0 2024-06-06T08:43:51.331 INFO:tasks.workunit.client.0.smithi027.stdout:2/699: truncate d6/dd/dd9/d3f/d77/d55/fb6 305440 0 2024-06-06T08:43:51.332 INFO:tasks.workunit.client.0.smithi027.stdout:9/698: dread d3/d95/d23/da9/fab [0,4194304] 0 2024-06-06T08:43:51.332 INFO:tasks.workunit.client.0.smithi027.stdout:9/699: write d3/f15 [323075,44198] 0 2024-06-06T08:43:51.332 INFO:tasks.workunit.client.0.smithi027.stdout:9/700: dread - d3/d95/d23/d78/fe1 zero size 2024-06-06T08:43:51.333 INFO:tasks.workunit.client.0.smithi027.stdout:1/624: rename dc/de/d14/f73 to dc/fc0 0 2024-06-06T08:43:51.334 INFO:tasks.workunit.client.0.smithi027.stdout:2/700: truncate d6/dd/f18 1812960 0 2024-06-06T08:43:51.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/701: write d3/d4e/d75/f50 [473641,98107] 0 2024-06-06T08:43:51.336 INFO:tasks.workunit.client.0.smithi027.stdout:5/885: rename d0/da/db/d1a/lbb to d0/da/db/d1a/da9/l116 0 2024-06-06T08:43:51.337 INFO:tasks.workunit.client.0.smithi027.stdout:5/886: write d0/f1 [1866650,23296] 0 2024-06-06T08:43:51.337 INFO:tasks.workunit.client.0.smithi027.stdout:5/887: stat d0/da/db/d46/d7f/d81/d97 0 2024-06-06T08:43:51.337 INFO:tasks.workunit.client.0.smithi027.stdout:5/888: stat d0/da/db/d46/d4c/d70/f9e 0 2024-06-06T08:43:51.337 INFO:tasks.workunit.client.0.smithi027.stdout:5/889: chown d0/da/db/d46/d54/c69 415055543 1 2024-06-06T08:43:51.338 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:43:51.338 INFO:tasks.workunit.client.0.smithi027.stdout:4/783: dwrite d0/d1/fe7 [0,4194304] 0 2024-06-06T08:43:51.338 INFO:tasks.workunit.client.0.smithi027.stdout:4/784: stat d0/d1/d6/d8/d82/d8b/dc9/d3b 0 2024-06-06T08:43:51.340 INFO:tasks.workunit.client.0.smithi027.stdout:7/845: dwrite d17/d32/d46/d83/d77/d8c/d91/daa/fe3 [0,4194304] 0 2024-06-06T08:43:51.341 INFO:tasks.workunit.client.0.smithi027.stdout:7/846: fsync d17/d32/f6d 0 2024-06-06T08:43:51.341 INFO:tasks.workunit.client.0.smithi027.stdout:5/890: rename d0/da/d37/d2b/le4 to d0/da/db/d1a/dc6/l117 0 2024-06-06T08:43:51.341 INFO:tasks.workunit.client.0.smithi027.stdout:2/701: mknod d6/dd/dd9/d3f/d77/d55/d69/ce8 0 2024-06-06T08:43:51.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/847: creat d17/d32/d46/d4f/db6/f117 x:0 0 0 2024-06-06T08:43:51.346 INFO:tasks.workunit.client.0.smithi027.stdout:1/625: dread dc/fd [0,4194304] 0 2024-06-06T08:43:51.346 INFO:tasks.workunit.client.0.smithi027.stdout:1/626: chown dc/de/d14/d80/fa4 40155 1 2024-06-06T08:43:51.349 INFO:tasks.workunit.client.0.smithi027.stdout:7/848: rmdir d17/d32/d46/d83/d77/d8c/d90 39 2024-06-06T08:43:51.349 INFO:tasks.workunit.client.0.smithi027.stdout:7/849: fsync d17/d32/d46/d4f/db6/dc8/f101 0 2024-06-06T08:43:51.349 INFO:tasks.workunit.client.0.smithi027.stdout:1/627: mknod dc/d67/d69/d32/d37/d98/cc1 0 2024-06-06T08:43:51.352 INFO:tasks.workunit.client.0.smithi027.stdout:7/850: stat d17/d32/d46/d83/d2a/d3b/cb1 0 2024-06-06T08:43:51.353 INFO:tasks.workunit.client.0.smithi027.stdout:5/891: link d0/da/db/d1a/da3/ff1 d0/da/db/d46/d4c/d70/f118 0 2024-06-06T08:43:51.353 INFO:tasks.workunit.client.0.smithi027.stdout:5/892: truncate d0/da/db/d1a/d5a/de0/fe8 980231 0 2024-06-06T08:43:51.353 INFO:tasks.workunit.client.0.smithi027.stdout:1/628: readlink dc/d67/d69/l4c 0 2024-06-06T08:43:51.356 INFO:tasks.workunit.client.0.smithi027.stdout:5/893: symlink d0/da/d37/l119 0 2024-06-06T08:43:51.358 INFO:tasks.workunit.client.0.smithi027.stdout:1/629: mknod dc/de/d3d/d4f/cc2 0 2024-06-06T08:43:51.359 INFO:tasks.workunit.client.0.smithi027.stdout:2/702: dread d6/dd/f18 [0,4194304] 0 2024-06-06T08:43:51.364 INFO:tasks.workunit.client.0.smithi027.stdout:7/851: write d17/f3f [155916,38078] 0 2024-06-06T08:43:51.367 INFO:tasks.workunit.client.0.smithi027.stdout:9/702: dwrite d3/d95/d34/d73/f85 [0,4194304] 0 2024-06-06T08:43:51.367 INFO:tasks.workunit.client.0.smithi027.stdout:9/703: read d3/d4e/d75/dc/f2d [2318343,93977] 0 2024-06-06T08:43:51.367 INFO:tasks.workunit.client.0.smithi027.stdout:9/704: read - d3/d4e/f9d zero size 2024-06-06T08:43:51.368 INFO:tasks.workunit.client.0.smithi027.stdout:4/785: write d0/d1/d6/d8/fe2 [3698230,33257] 0 2024-06-06T08:43:51.368 INFO:tasks.workunit.client.0.smithi027.stdout:4/786: chown d0/d1/d6/d8/f58 462636 1 2024-06-06T08:43:51.370 INFO:tasks.workunit.client.0.smithi027.stdout:4/787: getdents d0/d1/d6/d8/d82/d8b/dc9 0 2024-06-06T08:43:51.371 INFO:tasks.workunit.client.0.smithi027.stdout:4/788: creat d0/d1/d6/dd4/dde/f100 x:0 0 0 2024-06-06T08:43:51.373 INFO:tasks.workunit.client.0.smithi027.stdout:9/705: rename d3/d95/d23/da9/fb9 to d3/d95/d34/ff5 0 2024-06-06T08:43:51.375 INFO:tasks.workunit.client.0.smithi027.stdout:4/789: mknod d0/d1/d6/d8/d82/d8b/c101 0 2024-06-06T08:43:51.376 INFO:tasks.workunit.client.0.smithi027.stdout:9/706: mkdir d3/d4e/d75/dc/df6 0 2024-06-06T08:43:51.380 INFO:tasks.workunit.client.0.smithi027.stdout:5/894: dwrite d0/da/db/d46/d7f/d81/fb5 [0,4194304] 0 2024-06-06T08:43:51.380 INFO:tasks.workunit.client.0.smithi027.stdout:5/895: write d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd/f108 [856751,19658] 0 2024-06-06T08:43:51.385 INFO:tasks.workunit.client.0.smithi027.stdout:7/852: dwrite d17/d32/d46/d4f/db6/dc8/de0/f112 [0,4194304] 0 2024-06-06T08:43:51.386 INFO:tasks.workunit.client.0.smithi027.stdout:1/630: write dc/d67/d69/d32/f5b [6231578,123425] 0 2024-06-06T08:43:51.386 INFO:tasks.workunit.client.0.smithi027.stdout:2/703: dwrite d6/dd/dd9/d1d/d4d/f5f [0,4194304] 0 2024-06-06T08:43:51.387 INFO:tasks.workunit.client.0.smithi027.stdout:4/790: dread d0/d1/d6/d8/f6a [0,4194304] 0 2024-06-06T08:43:51.389 INFO:tasks.workunit.client.0.smithi027.stdout:5/896: dread d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd/f108 [0,4194304] 0 2024-06-06T08:43:51.391 INFO:tasks.workunit.client.0.smithi027.stdout:7/853: creat d17/d32/d61/f118 x:0 0 0 2024-06-06T08:43:51.397 INFO:tasks.workunit.client.0.smithi027.stdout:1/631: write dc/de/d14/f15 [63220,21351] 0 2024-06-06T08:43:51.398 INFO:tasks.workunit.client.0.smithi027.stdout:8/622: sync 2024-06-06T08:43:51.398 INFO:tasks.workunit.client.0.smithi027.stdout:9/707: link d3/d95/d34/d73/lbc d3/d76/dd9/lf7 0 2024-06-06T08:43:51.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/704: mknod d6/dd/dd9/d1d/d4d/d8e/ce9 0 2024-06-06T08:43:51.399 INFO:tasks.workunit.client.0.smithi027.stdout:6/849: sync 2024-06-06T08:43:51.400 INFO:tasks.workunit.client.0.smithi027.stdout:7/854: dread d17/d32/d46/d4f/d5e/f8b [0,4194304] 0 2024-06-06T08:43:51.400 INFO:tasks.workunit.client.0.smithi027.stdout:7/855: dread - d17/d32/d46/d83/d77/fdd zero size 2024-06-06T08:43:51.401 INFO:tasks.workunit.client.0.smithi027.stdout:1/632: creat dc/de/d3d/d4a/d76/fc3 x:0 0 0 2024-06-06T08:43:51.401 INFO:tasks.workunit.client.0.smithi027.stdout:1/633: truncate dc/de/fab 284895 0 2024-06-06T08:43:51.402 INFO:tasks.workunit.client.0.smithi027.stdout:4/791: truncate d0/d1/d6/d8/d21/f74 267089 0 2024-06-06T08:43:51.402 INFO:tasks.workunit.client.0.smithi027.stdout:4/792: fsync d0/d1d/f54 0 2024-06-06T08:43:51.402 INFO:tasks.workunit.client.0.smithi027.stdout:5/897: link d0/da/db/d1a/d5a/l96 d0/da/df0/d106/l11a 0 2024-06-06T08:43:51.402 INFO:tasks.workunit.client.0.smithi027.stdout:7/856: creat d17/d32/d46/d83/d2a/d3b/dc7/f119 x:0 0 0 2024-06-06T08:43:51.403 INFO:tasks.workunit.client.0.smithi027.stdout:9/708: write d3/f89 [178282,49087] 0 2024-06-06T08:43:51.403 INFO:tasks.workunit.client.0.smithi027.stdout:6/850: read - d4/de/f5d zero size 2024-06-06T08:43:51.404 INFO:tasks.workunit.client.0.smithi027.stdout:5/898: creat d0/da/db/d46/d7f/d81/d97/f11b x:0 0 0 2024-06-06T08:43:51.404 INFO:tasks.workunit.client.0.smithi027.stdout:9/709: chown d3/d95/d23/d36/d3a/f96 0 1 2024-06-06T08:43:51.406 INFO:tasks.workunit.client.0.smithi027.stdout:5/899: creat d0/da/d47/ded/f11c x:0 0 0 2024-06-06T08:43:51.407 INFO:tasks.workunit.client.0.smithi027.stdout:2/705: mknod d6/d1e/cea 0 2024-06-06T08:43:51.407 INFO:tasks.workunit.client.0.smithi027.stdout:8/623: dread d7/f2c [0,4194304] 0 2024-06-06T08:43:51.409 INFO:tasks.workunit.client.0.smithi027.stdout:1/634: symlink dc/de/d3d/d4a/lc4 0 2024-06-06T08:43:51.410 INFO:tasks.workunit.client.0.smithi027.stdout:4/793: symlink d0/l102 0 2024-06-06T08:43:51.412 INFO:tasks.workunit.client.0.smithi027.stdout:2/706: mknod d6/dd/dd9/d3f/d48/d83/de3/de4/ceb 0 2024-06-06T08:43:51.412 INFO:tasks.workunit.client.0.smithi027.stdout:2/707: write d6/dd/dd9/d3f/d77/d55/fb4 [1070831,43617] 0 2024-06-06T08:43:51.412 INFO:tasks.workunit.client.0.smithi027.stdout:9/710: dread d3/d95/d23/d36/f37 [0,4194304] 0 2024-06-06T08:43:51.412 INFO:tasks.workunit.client.0.smithi027.stdout:9/711: write d3/d95/d23/d36/d3a/fe4 [956151,42622] 0 2024-06-06T08:43:51.413 INFO:tasks.workunit.client.0.smithi027.stdout:8/624: getdents d7/dc/d36/d83/d60/d93 0 2024-06-06T08:43:51.413 INFO:tasks.workunit.client.0.smithi027.stdout:9/712: truncate d3/d95/f39 44406 0 2024-06-06T08:43:51.414 INFO:tasks.workunit.client.0.smithi027.stdout:6/851: getdents d4/d13/dba 0 2024-06-06T08:43:51.416 INFO:tasks.workunit.client.0.smithi027.stdout:1/635: mknod dc/da0/cc5 0 2024-06-06T08:43:51.416 INFO:tasks.workunit.client.0.smithi027.stdout:5/900: dread d0/da/db/d1a/fde [0,4194304] 0 2024-06-06T08:43:51.419 INFO:tasks.workunit.client.0.smithi027.stdout:2/708: creat d6/dd/dd9/ddf/de1/fec x:0 0 0 2024-06-06T08:43:51.419 INFO:tasks.workunit.client.0.smithi027.stdout:2/709: getdents d6/dd/dd9/d1d/d4d/ddc 0 2024-06-06T08:43:51.419 INFO:tasks.workunit.client.0.smithi027.stdout:8/625: fsync d7/dc/d36/d83/f91 0 2024-06-06T08:43:51.420 INFO:tasks.workunit.client.0.smithi027.stdout:4/794: dread d0/d1/d6/d8/d21/f2d [0,4194304] 0 2024-06-06T08:43:51.421 INFO:tasks.workunit.client.0.smithi027.stdout:6/852: creat d4/da2/f119 x:0 0 0 2024-06-06T08:43:51.423 INFO:tasks.workunit.client.0.smithi027.stdout:1/636: creat dc/de/d14/d80/fc6 x:0 0 0 2024-06-06T08:43:51.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/626: truncate d7/dc/d36/d83/f6e 5208901 0 2024-06-06T08:43:51.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/627: write d7/f4e [3603663,85301] 0 2024-06-06T08:43:51.427 INFO:tasks.workunit.client.0.smithi027.stdout:7/857: link d17/d32/d46/d83/d2a/d3b/cb1 d17/d47/c11a 0 2024-06-06T08:43:51.427 INFO:tasks.workunit.client.0.smithi027.stdout:7/858: stat d17/f26 0 2024-06-06T08:43:51.427 INFO:tasks.workunit.client.0.smithi027.stdout:7/859: truncate d17/f1e 4760899 0 2024-06-06T08:43:51.428 INFO:tasks.workunit.client.0.smithi027.stdout:7/860: creat d17/d32/d46/d83/d77/f11b x:0 0 0 2024-06-06T08:43:51.428 INFO:tasks.workunit.client.0.smithi027.stdout:7/861: chown d17/d32/d61/f74 64989645 1 2024-06-06T08:43:51.429 INFO:tasks.workunit.client.0.smithi027.stdout:6/853: rmdir d4/d9/d49 39 2024-06-06T08:43:51.429 INFO:tasks.workunit.client.0.smithi027.stdout:6/854: chown d4/d13 3999 1 2024-06-06T08:43:51.429 INFO:tasks.workunit.client.0.smithi027.stdout:1/637: readlink dc/de/d22/l1f 0 2024-06-06T08:43:51.429 INFO:tasks.workunit.client.0.smithi027.stdout:1/638: write dc/de/d14/f15 [74592,57870] 0 2024-06-06T08:43:51.431 INFO:tasks.workunit.client.0.smithi027.stdout:5/901: symlink d0/da/db/d1a/d7e/l11d 0 2024-06-06T08:43:51.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/795: truncate d0/d1d/d2a/f57 924905 0 2024-06-06T08:43:51.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/796: dread - d0/d1/d6/d8/d82/f9f zero size 2024-06-06T08:43:51.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/855: mknod d4/d9/dc/d1c/dfa/c11a 0 2024-06-06T08:43:51.435 INFO:tasks.workunit.client.0.smithi027.stdout:1/639: getdents dc/de/d14 0 2024-06-06T08:43:51.437 INFO:tasks.workunit.client.0.smithi027.stdout:5/902: creat d0/da/db/d46/d4c/d70/f11e x:0 0 0 2024-06-06T08:43:51.441 INFO:tasks.workunit.client.0.smithi027.stdout:5/903: creat d0/da/db/d1a/da3/de5/f11f x:0 0 0 2024-06-06T08:43:51.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/640: mknod dc/d67/d69/d32/d71/cc7 0 2024-06-06T08:43:51.444 INFO:tasks.workunit.client.0.smithi027.stdout:8/628: rmdir d7/d13/d15/dbe 0 2024-06-06T08:43:51.445 INFO:tasks.workunit.client.0.smithi027.stdout:1/641: mkdir dc/de/d14/dc8 0 2024-06-06T08:43:51.446 INFO:tasks.workunit.client.0.smithi027.stdout:7/862: dwrite d17/f60 [0,4194304] 0 2024-06-06T08:43:51.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/629: unlink d7/d2f/cbd 0 2024-06-06T08:43:51.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/630: truncate d7/d2f/fb8 187693 0 2024-06-06T08:43:51.447 INFO:tasks.workunit.client.0.smithi027.stdout:4/797: dread d0/d1/d6/fae [0,4194304] 0 2024-06-06T08:43:51.448 INFO:tasks.workunit.client.0.smithi027.stdout:7/863: write d17/d32/d46/d83/d77/fa2 [72993,84712] 0 2024-06-06T08:43:51.448 INFO:tasks.workunit.client.0.smithi027.stdout:4/798: getdents d0/d1/d6/d8/d82/d8b/dc9/da3/daa 0 2024-06-06T08:43:51.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/799: creat d0/d1d/d34/d7f/dfc/f103 x:0 0 0 2024-06-06T08:43:51.453 INFO:tasks.workunit.client.0.smithi027.stdout:8/631: creat d7/fd0 x:0 0 0 2024-06-06T08:43:51.454 INFO:tasks.workunit.client.0.smithi027.stdout:7/864: creat d17/d32/d46/d51/d52/d70/dce/f11c x:0 0 0 2024-06-06T08:43:51.454 INFO:tasks.workunit.client.0.smithi027.stdout:4/800: mknod d0/d1/d6/d8/d82/d8b/dc9/da3/daa/c104 0 2024-06-06T08:43:51.454 INFO:tasks.workunit.client.0.smithi027.stdout:4/801: dread - d0/d1/d6/d8/d82/fda zero size 2024-06-06T08:43:51.455 INFO:tasks.workunit.client.0.smithi027.stdout:7/865: symlink d17/d32/d46/d51/d52/d10b/l11d 0 2024-06-06T08:43:51.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/802: write d0/d1/d6/d8/d82/d8b/dc9/fd7 [209300,60998] 0 2024-06-06T08:43:51.457 INFO:tasks.workunit.client.0.smithi027.stdout:7/866: mknod d17/d32/d46/d4f/db6/dc8/c11e 0 2024-06-06T08:43:51.459 INFO:tasks.workunit.client.0.smithi027.stdout:6/856: dwrite d4/d9/dc/d8f/fa3 [0,4194304] 0 2024-06-06T08:43:51.459 INFO:tasks.workunit.client.0.smithi027.stdout:6/857: write d4/d9/d6e/dd4/fde [317749,30066] 0 2024-06-06T08:43:51.460 INFO:tasks.workunit.client.0.smithi027.stdout:4/803: rmdir d0/d1/d6/d8/dd5/def 0 2024-06-06T08:43:51.460 INFO:tasks.workunit.client.0.smithi027.stdout:4/804: fsync d0/d1d/d2a/fdd 0 2024-06-06T08:43:51.460 INFO:tasks.workunit.client.0.smithi027.stdout:7/867: rename d17/d32/d46/l105 to d17/d32/d46/d51/d7f/db2/l11f 0 2024-06-06T08:43:51.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/858: creat d4/de/d45/d6a/f11b x:0 0 0 2024-06-06T08:43:51.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/859: chown d4/d9/dc/d18/fd0 683791 1 2024-06-06T08:43:51.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/860: write d4/d9/dc/d18/f24 [1044158,4244] 0 2024-06-06T08:43:51.468 INFO:tasks.workunit.client.0.smithi027.stdout:6/861: dread d4/d1a/d2e/d44/dce/fd5 [0,4194304] 0 2024-06-06T08:43:51.469 INFO:tasks.workunit.client.0.smithi027.stdout:6/862: creat d4/d9/dc/d18/f11c x:0 0 0 2024-06-06T08:43:51.471 INFO:tasks.workunit.client.0.smithi027.stdout:6/863: creat d4/d1a/d2e/f11d x:0 0 0 2024-06-06T08:43:51.472 INFO:tasks.workunit.client.0.smithi027.stdout:1/642: write dc/d67/d69/d32/d37/d56/f93 [955572,87285] 0 2024-06-06T08:43:51.476 INFO:tasks.workunit.client.0.smithi027.stdout:1/643: getdents dc/d67/d69/d32/d37/d56 0 2024-06-06T08:43:51.480 INFO:tasks.workunit.client.0.smithi027.stdout:8/632: dwrite d7/d2f/f92 [4194304,4194304] 0 2024-06-06T08:43:51.487 INFO:tasks.workunit.client.0.smithi027.stdout:7/868: dwrite d17/d32/d61/f74 [0,4194304] 0 2024-06-06T08:43:51.488 INFO:tasks.workunit.client.0.smithi027.stdout:5/904: rmdir d0/da/db/d46/d7f/d81 39 2024-06-06T08:43:51.488 INFO:tasks.workunit.client.0.smithi027.stdout:5/905: chown d0/da/d37/d2b/c61 24666 1 2024-06-06T08:43:51.489 INFO:tasks.workunit.client.0.smithi027.stdout:7/869: chown d17/d32/d46/d83/d2a/d3b/c62 2283256 1 2024-06-06T08:43:51.489 INFO:tasks.workunit.client.0.smithi027.stdout:4/805: dwrite d0/d63/ff6 [0,4194304] 0 2024-06-06T08:43:51.490 INFO:tasks.workunit.client.0.smithi027.stdout:8/633: creat d7/dc/fd1 x:0 0 0 2024-06-06T08:43:51.491 INFO:tasks.workunit.client.0.smithi027.stdout:5/906: dread d0/da/db/d1a/da9/fe1 [0,4194304] 0 2024-06-06T08:43:51.492 INFO:tasks.workunit.client.0.smithi027.stdout:6/864: write d4/f79 [419774,77123] 0 2024-06-06T08:43:51.492 INFO:tasks.workunit.client.0.smithi027.stdout:5/907: creat d0/da/db/d46/d7f/d81/d97/db0/dbc/dcf/f120 x:0 0 0 2024-06-06T08:43:51.494 INFO:tasks.workunit.client.0.smithi027.stdout:6/865: symlink d4/d9/df2/l11e 0 2024-06-06T08:43:51.494 INFO:tasks.workunit.client.0.smithi027.stdout:6/866: write d4/d9/dc5/f103 [585198,65826] 0 2024-06-06T08:43:51.494 INFO:tasks.workunit.client.0.smithi027.stdout:6/867: chown d4/de/d45/d6a/l7c 49 1 2024-06-06T08:43:51.497 INFO:tasks.workunit.client.0.smithi027.stdout:7/870: rmdir d17/d32/d46/d51/d52/d70/d87/dc0 39 2024-06-06T08:43:51.497 INFO:tasks.workunit.client.0.smithi027.stdout:7/871: truncate d17/d32/d46/d83/d2a/d3b/f56 5138277 0 2024-06-06T08:43:51.503 INFO:tasks.workunit.client.0.smithi027.stdout:3/655: sync 2024-06-06T08:43:51.503 INFO:tasks.workunit.client.0.smithi027.stdout:0/711: sync 2024-06-06T08:43:51.503 INFO:tasks.workunit.client.0.smithi027.stdout:3/656: write d1/d1f/d25/f8d [1367751,94151] 0 2024-06-06T08:43:51.504 INFO:tasks.workunit.client.0.smithi027.stdout:3/657: creat d1/d90/d44/d54/fce x:0 0 0 2024-06-06T08:43:51.505 INFO:tasks.workunit.client.0.smithi027.stdout:0/712: mknod d15/d18/d23/d38/cec 0 2024-06-06T08:43:51.505 INFO:tasks.workunit.client.0.smithi027.stdout:0/713: chown d15/d18/dbe/d5c/d83/fb6 9435245 1 2024-06-06T08:43:51.505 INFO:tasks.workunit.client.0.smithi027.stdout:0/714: fsync d15/d18/d23/d38/d3e/f5a 0 2024-06-06T08:43:51.506 INFO:tasks.workunit.client.0.smithi027.stdout:7/872: dread d17/d32/d46/d51/d7f/f7a [0,4194304] 0 2024-06-06T08:43:51.506 INFO:tasks.workunit.client.0.smithi027.stdout:0/715: dread d15/d18/d23/d28/d2a/f33 [0,4194304] 0 2024-06-06T08:43:51.508 INFO:tasks.workunit.client.0.smithi027.stdout:3/658: unlink d1/d90/f18 0 2024-06-06T08:43:51.509 INFO:tasks.workunit.client.0.smithi027.stdout:3/659: stat d1/d8/d22/d27/d79 0 2024-06-06T08:43:51.510 INFO:tasks.workunit.client.0.smithi027.stdout:5/908: symlink d0/da/db/d46/d7f/d81/l121 0 2024-06-06T08:43:51.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/716: creat d15/d18/d64/fed x:0 0 0 2024-06-06T08:43:51.515 INFO:tasks.workunit.client.0.smithi027.stdout:0/717: fsync d15/d18/d23/d38/d68/f8f 0 2024-06-06T08:43:51.515 INFO:tasks.workunit.client.0.smithi027.stdout:6/868: dwrite d4/d9/dc/d18/fcc [0,4194304] 0 2024-06-06T08:43:51.516 INFO:tasks.workunit.client.0.smithi027.stdout:5/909: creat d0/da/db/d1a/da3/de5/d107/f122 x:0 0 0 2024-06-06T08:43:51.517 INFO:tasks.workunit.client.0.smithi027.stdout:6/869: truncate d4/de/d45/d6a/f76 577393 0 2024-06-06T08:43:51.517 INFO:tasks.workunit.client.0.smithi027.stdout:5/910: write d0/da/db/d46/d7f/d81/fd2 [4536171,20113] 0 2024-06-06T08:43:51.517 INFO:tasks.workunit.client.0.smithi027.stdout:5/911: stat d0/da/db/d46/d7f/d81/d97/db0/dbc/dcd/f108 0 2024-06-06T08:43:51.518 INFO:tasks.workunit.client.0.smithi027.stdout:1/644: dwrite dc/de/d14/f1a [0,4194304] 0 2024-06-06T08:43:51.520 INFO:tasks.workunit.client.0.smithi027.stdout:1/645: symlink dc/de/d14/dc8/lc9 0 2024-06-06T08:43:51.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/646: dread - dc/d67/fbe zero size 2024-06-06T08:43:51.522 INFO:tasks.workunit.client.0.smithi027.stdout:1/647: creat dc/d6d/fca x:0 0 0 2024-06-06T08:43:51.526 INFO:tasks.workunit.client.0.smithi027.stdout:0/718: write d15/d18/d23/d38/d68/d85/fea [1794916,60769] 0 2024-06-06T08:43:51.536 INFO:tasks.workunit.client.0.smithi027.stdout:4/806: dwrite d0/d63/f6d [0,4194304] 0 2024-06-06T08:43:51.536 INFO:tasks.workunit.client.0.smithi027.stdout:1/648: dread dc/d67/d69/f52 [0,4194304] 0 2024-06-06T08:43:51.536 INFO:tasks.workunit.client.0.smithi027.stdout:4/807: stat d0/d1/d6/d8/d21/f4a 0 2024-06-06T08:43:51.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/660: dwrite d1/f1e [0,4194304] 0 2024-06-06T08:43:51.538 INFO:tasks.workunit.client.0.smithi027.stdout:4/808: truncate d0/d1d/d34/fa2 4054603 0 2024-06-06T08:43:51.541 INFO:tasks.workunit.client.0.smithi027.stdout:4/809: mknod d0/d1/d6/d8/c105 0 2024-06-06T08:43:51.541 INFO:tasks.workunit.client.0.smithi027.stdout:4/810: dread - d0/d1/d6/d8/d82/d8b/dc9/fc8 zero size 2024-06-06T08:43:51.541 INFO:tasks.workunit.client.0.smithi027.stdout:1/649: symlink dc/de/d3d/d4f/lcb 0 2024-06-06T08:43:51.542 INFO:tasks.workunit.client.0.smithi027.stdout:1/650: chown dc/de/d14/d80/da6 3465748 1 2024-06-06T08:43:51.547 INFO:tasks.workunit.client.0.smithi027.stdout:4/811: rmdir d0/d1d/d34 39 2024-06-06T08:43:51.547 INFO:tasks.workunit.client.0.smithi027.stdout:6/870: rename d4/d9/dc/lb0 to d4/d9/dc/d1c/l11f 0 2024-06-06T08:43:51.549 INFO:tasks.workunit.client.0.smithi027.stdout:5/912: rename d0/da/d47/cdf to d0/da/db/d46/d77/c123 0 2024-06-06T08:43:51.550 INFO:tasks.workunit.client.0.smithi027.stdout:6/871: mknod d4/de/d45/d6a/c120 0 2024-06-06T08:43:51.550 INFO:tasks.workunit.client.0.smithi027.stdout:5/913: readlink d0/da/db/d1a/lb6 0 2024-06-06T08:43:51.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/914: symlink d0/da/db/d1a/dc6/d111/l124 0 2024-06-06T08:43:51.561 INFO:tasks.workunit.client.0.smithi027.stdout:1/651: dwrite dc/de/f75 [0,4194304] 0 2024-06-06T08:43:51.561 INFO:tasks.workunit.client.0.smithi027.stdout:6/872: dread d4/d9/d49/fe2 [0,4194304] 0 2024-06-06T08:43:51.562 INFO:tasks.workunit.client.0.smithi027.stdout:6/873: symlink d4/d9/dc/d1c/dfa/l121 0 2024-06-06T08:43:51.567 INFO:tasks.workunit.client.0.smithi027.stdout:0/719: rename d15/l36 to d15/d18/d23/d5b/da0/lee 0 2024-06-06T08:43:51.568 INFO:tasks.workunit.client.0.smithi027.stdout:0/720: write d15/d18/d23/d28/d2a/d2d/f58 [1760976,73903] 0 2024-06-06T08:43:51.569 INFO:tasks.workunit.client.0.smithi027.stdout:1/652: write dc/fd [1038608,32764] 0 2024-06-06T08:43:51.570 INFO:tasks.workunit.client.0.smithi027.stdout:3/661: rename d1/d1f/d25/l89 to d1/d1f/dbc/lcf 0 2024-06-06T08:43:51.570 INFO:tasks.workunit.client.0.smithi027.stdout:0/721: creat d15/d18/d23/fef x:0 0 0 2024-06-06T08:43:51.573 INFO:tasks.workunit.client.0.smithi027.stdout:0/722: dread d15/d18/d23/d3c/f67 [0,4194304] 0 2024-06-06T08:43:51.573 INFO:tasks.workunit.client.0.smithi027.stdout:4/812: rename d0/d1d/f79 to d0/d1/d6/f106 0 2024-06-06T08:43:51.575 INFO:tasks.workunit.client.0.smithi027.stdout:6/874: rename d4/de/d45/d6a/cbc to d4/de/d45/d6a/de8/c122 0 2024-06-06T08:43:51.575 INFO:tasks.workunit.client.0.smithi027.stdout:6/875: fdatasync d4/d1a/d2e/fc1 0 2024-06-06T08:43:51.577 INFO:tasks.workunit.client.0.smithi027.stdout:6/876: symlink d4/d9/d49/l123 0 2024-06-06T08:43:51.579 INFO:tasks.workunit.client.0.smithi027.stdout:6/877: creat d4/de/d45/d6a/de8/f124 x:0 0 0 2024-06-06T08:43:51.579 INFO:tasks.workunit.client.0.smithi027.stdout:6/878: chown d4/d9/dc/d1c/d52/c7a 137093050 1 2024-06-06T08:43:51.579 INFO:tasks.workunit.client.0.smithi027.stdout:1/653: write dc/de/d3d/f44 [1044097,25451] 0 2024-06-06T08:43:51.586 INFO:tasks.workunit.client.0.smithi027.stdout:0/723: dread d15/d18/d23/d28/d2a/d2d/f58 [0,4194304] 0 2024-06-06T08:43:51.586 INFO:tasks.workunit.client.0.smithi027.stdout:0/724: getdents d15/d18/d23/d6f/deb 0 2024-06-06T08:43:51.590 INFO:tasks.workunit.client.0.smithi027.stdout:0/725: link d15/d18/d23/d28/d2a/cb3 d15/d18/dbe/d5c/cf0 0 2024-06-06T08:43:51.591 INFO:tasks.workunit.client.0.smithi027.stdout:6/879: dread d4/d9/dc/d18/fd0 [0,4194304] 0 2024-06-06T08:43:51.592 INFO:tasks.workunit.client.0.smithi027.stdout:0/726: creat d15/d18/dbe/d5d/ff1 x:0 0 0 2024-06-06T08:43:51.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/727: truncate d15/d18/d23/f2e 1551654 0 2024-06-06T08:43:51.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/728: fdatasync d15/d18/f73 0 2024-06-06T08:43:51.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/729: getdents d15/d18/d23/d6f/deb 0 2024-06-06T08:43:51.594 INFO:tasks.workunit.client.0.smithi027.stdout:0/730: creat d15/d18/d23/d38/d68/ff2 x:0 0 0 2024-06-06T08:43:51.599 INFO:tasks.workunit.client.0.smithi027.stdout:1/654: dwrite dc/d67/d69/d32/f4d [0,4194304] 0 2024-06-06T08:43:51.599 INFO:tasks.workunit.client.0.smithi027.stdout:1/655: readlink dc/de/d3d/d4a/lc4 0 2024-06-06T08:43:51.599 INFO:tasks.workunit.client.0.smithi027.stdout:4/813: rename d0/d1d/d34/d7f/d97 to d0/d1/d6/d8/d107 0 2024-06-06T08:43:51.601 INFO:tasks.workunit.client.0.smithi027.stdout:4/814: creat d0/d1/d6/d8/d107/d9c/f108 x:0 0 0 2024-06-06T08:43:51.601 INFO:tasks.workunit.client.0.smithi027.stdout:1/656: symlink dc/de/d3d/d4f/lcc 0 2024-06-06T08:43:51.601 INFO:tasks.workunit.client.0.smithi027.stdout:1/657: readlink dc/d67/d69/d32/d37/l97 0 2024-06-06T08:43:51.613 INFO:tasks.workunit.client.0.smithi027.stdout:0/731: dwrite d15/d18/d23/d38/d68/f8f [0,4194304] 0 2024-06-06T08:43:51.617 INFO:tasks.workunit.client.0.smithi027.stdout:4/815: read d0/d1/d6/d8/d82/d8b/dc9/f89 [1371341,80861] 0 2024-06-06T08:43:51.621 INFO:tasks.workunit.client.0.smithi027.stdout:4/816: rmdir d0/d1/d6/d8/dd5 0 2024-06-06T08:43:51.622 INFO:tasks.workunit.client.0.smithi027.stdout:6/880: rename d4/d9/dc to d4/d125 0 2024-06-06T08:43:51.622 INFO:tasks.workunit.client.0.smithi027.stdout:4/817: stat d0/d1d/d60/dc6/c67 0 2024-06-06T08:43:51.623 INFO:tasks.workunit.client.0.smithi027.stdout:4/818: write d0/d1/d6/d8/d107/fe0 [765515,12040] 0 2024-06-06T08:43:51.623 INFO:tasks.workunit.client.0.smithi027.stdout:0/732: write d15/d18/d23/d28/f4e [1247129,19134] 0 2024-06-06T08:43:51.624 INFO:tasks.workunit.client.0.smithi027.stdout:4/819: truncate d0/d1/fb 7329586 0 2024-06-06T08:43:51.624 INFO:tasks.workunit.client.0.smithi027.stdout:1/658: dwrite dc/de/f13 [4194304,4194304] 0 2024-06-06T08:43:51.624 INFO:tasks.workunit.client.0.smithi027.stdout:1/659: truncate dc/de/d22/f72 1020792 0 2024-06-06T08:43:51.624 INFO:tasks.workunit.client.0.smithi027.stdout:1/660: chown dc/de/d14/d4e/f5d 0 1 2024-06-06T08:43:51.626 INFO:tasks.workunit.client.0.smithi027.stdout:1/661: creat dc/da0/fcd x:0 0 0 2024-06-06T08:43:51.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/662: rename dc/de/d14/f91 to dc/d6d/fce 0 2024-06-06T08:43:51.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/663: fsync dc/f81 0 2024-06-06T08:43:51.629 INFO:tasks.workunit.client.0.smithi027.stdout:0/733: dread d15/d18/d23/d28/d2a/d2d/f34 [0,4194304] 0 2024-06-06T08:43:51.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/664: rename dc/de/d14/l1b to dc/d9d/lcf 0 2024-06-06T08:43:51.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/665: write dc/de/d22/f3b [1122059,38873] 0 2024-06-06T08:43:51.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/666: write dc/f81 [5226478,96017] 0 2024-06-06T08:43:51.631 INFO:tasks.workunit.client.0.smithi027.stdout:0/734: stat d15/l92 0 2024-06-06T08:43:51.632 INFO:tasks.workunit.client.0.smithi027.stdout:0/735: creat d15/d18/d23/d38/d68/d85/ff3 x:0 0 0 2024-06-06T08:43:51.635 INFO:tasks.workunit.client.0.smithi027.stdout:4/820: write d0/d1d/d60/fc5 [877892,64702] 0 2024-06-06T08:43:51.637 INFO:tasks.workunit.client.0.smithi027.stdout:1/667: dread dc/de/d3d/d4f/f9e [0,4194304] 0 2024-06-06T08:43:51.638 INFO:tasks.workunit.client.0.smithi027.stdout:0/736: creat d15/d18/d64/ff4 x:0 0 0 2024-06-06T08:43:51.640 INFO:tasks.workunit.client.0.smithi027.stdout:0/737: dread - d15/d18/d23/d38/d3e/f99 zero size 2024-06-06T08:43:51.641 INFO:tasks.workunit.client.0.smithi027.stdout:4/821: mknod d0/d1/c109 0 2024-06-06T08:43:51.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/881: dwrite d4/de/d45/d6a/f76 [0,4194304] 0 2024-06-06T08:43:51.642 INFO:tasks.workunit.client.0.smithi027.stdout:6/882: dread - d4/d125/d1c/d52/f108 zero size 2024-06-06T08:43:51.643 INFO:tasks.workunit.client.0.smithi027.stdout:0/738: stat d15/d18/d23/d28/d2a/d2d/l47 0 2024-06-06T08:43:51.645 INFO:tasks.workunit.client.0.smithi027.stdout:4/822: symlink d0/d1/d6/d8/d107/d9c/l10a 0 2024-06-06T08:43:51.654 INFO:tasks.workunit.client.0.smithi027.stdout:0/739: mknod d15/d18/d23/d28/d2a/d74/cf5 0 2024-06-06T08:43:51.657 INFO:tasks.workunit.client.0.smithi027.stdout:0/740: creat d15/d18/ff6 x:0 0 0 2024-06-06T08:43:51.657 INFO:tasks.workunit.client.0.smithi027.stdout:6/883: dread d4/d1a/d2e/d44/fc0 [0,4194304] 0 2024-06-06T08:43:51.657 INFO:tasks.workunit.client.0.smithi027.stdout:0/741: write d15/d18/d23/d28/d2a/d2d/f34 [670462,127917] 0 2024-06-06T08:43:51.658 INFO:tasks.workunit.client.0.smithi027.stdout:0/742: truncate d15/d18/d23/d28/f55 845714 0 2024-06-06T08:43:51.658 INFO:tasks.workunit.client.0.smithi027.stdout:1/668: dwrite dc/de/d3d/fb4 [0,4194304] 0 2024-06-06T08:43:51.660 INFO:tasks.workunit.client.0.smithi027.stdout:6/884: mkdir d4/d125/d8f/d126 0 2024-06-06T08:43:51.661 INFO:tasks.workunit.client.0.smithi027.stdout:1/669: truncate dc/fd 318577 0 2024-06-06T08:43:51.661 INFO:tasks.workunit.client.0.smithi027.stdout:1/670: fdatasync dc/f81 0 2024-06-06T08:43:51.662 INFO:tasks.workunit.client.0.smithi027.stdout:6/885: unlink d4/d125/d1c/dfa/l121 0 2024-06-06T08:43:51.663 INFO:tasks.workunit.client.0.smithi027.stdout:6/886: rmdir d4/de/d45/df7 39 2024-06-06T08:43:51.663 INFO:tasks.workunit.client.0.smithi027.stdout:6/887: truncate d4/d125/f9f 42390 0 2024-06-06T08:43:51.663 INFO:tasks.workunit.client.0.smithi027.stdout:6/888: fsync d4/de/fe6 0 2024-06-06T08:43:51.664 INFO:tasks.workunit.client.0.smithi027.stdout:6/889: dread - d4/da2/fc6 zero size 2024-06-06T08:43:51.673 INFO:tasks.workunit.client.0.smithi027.stdout:1/671: write dc/de/d3d/f85 [466157,123911] 0 2024-06-06T08:43:51.674 INFO:tasks.workunit.client.0.smithi027.stdout:1/672: mknod dc/de/d14/d80/da6/cd0 0 2024-06-06T08:43:51.681 INFO:tasks.workunit.client.0.smithi027.stdout:4/823: rename d0/d1/d75/df3 to d0/d1/d6/d8/d82/d8b/dc9/da3/d10b 0 2024-06-06T08:43:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:1/673: dwrite dc/de/d22/f79 [0,4194304] 0 2024-06-06T08:43:51.690 INFO:tasks.workunit.client.0.smithi027.stdout:1/674: read - dc/da0/fcd zero size 2024-06-06T08:43:51.693 INFO:tasks.workunit.client.0.smithi027.stdout:1/675: mkdir dc/d67/d69/d32/d71/dd1 0 2024-06-06T08:43:51.698 INFO:tasks.workunit.client.0.smithi027.stdout:4/824: dwrite d0/d1/fe7 [0,4194304] 0 2024-06-06T08:43:51.698 INFO:tasks.workunit.client.0.smithi027.stdout:0/743: rename d15/d18/d23/d5b/da0/fc8 to d15/d18/d23/d28/d31/ff7 0 2024-06-06T08:43:51.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/676: rename dc/d67/d69/d32/d37/d56/d78/lbd to dc/d67/d69/d32/d37/d56/ld2 0 2024-06-06T08:43:51.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/677: write dc/fbc [253339,79416] 0 2024-06-06T08:43:51.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/678: chown dc/de/d3d/f3e 15566992 1 2024-06-06T08:43:51.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/825: mkdir d0/d63/d10c 0 2024-06-06T08:43:51.706 INFO:tasks.workunit.client.0.smithi027.stdout:1/679: mknod dc/cd3 0 2024-06-06T08:43:51.708 INFO:tasks.workunit.client.0.smithi027.stdout:4/826: symlink d0/d1d/d2a/d98/l10d 0 2024-06-06T08:43:51.708 INFO:tasks.workunit.client.0.smithi027.stdout:4/827: truncate d0/d1/d6/d8/f6a 4513684 0 2024-06-06T08:43:51.709 INFO:tasks.workunit.client.0.smithi027.stdout:1/680: creat dc/de/d14/d80/fd4 x:0 0 0 2024-06-06T08:43:51.712 INFO:tasks.workunit.client.0.smithi027.stdout:1/681: creat dc/d67/d69/d32/d71/fd5 x:0 0 0 2024-06-06T08:43:51.717 INFO:tasks.workunit.client.0.smithi027.stdout:1/682: getdents dc 0 2024-06-06T08:43:51.725 INFO:tasks.workunit.client.0.smithi027.stdout:4/828: dwrite d0/d1/d6/f2f [4194304,4194304] 0 2024-06-06T08:43:51.725 INFO:tasks.workunit.client.0.smithi027.stdout:4/829: fdatasync d0/d1/d6/f106 0 2024-06-06T08:43:51.726 INFO:tasks.workunit.client.0.smithi027.stdout:4/830: rmdir d0/d1d/d34 39 2024-06-06T08:43:51.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/744: rename d15/d18/d23/d28/d2a/d2d/f34 to d15/da7/ff8 0 2024-06-06T08:43:51.729 INFO:tasks.workunit.client.0.smithi027.stdout:4/831: rmdir d0/d1d/d60/dc6/d6e/db6 39 2024-06-06T08:43:51.729 INFO:tasks.workunit.client.0.smithi027.stdout:0/745: dread d15/d18/f1c [4194304,4194304] 0 2024-06-06T08:43:51.731 INFO:tasks.workunit.client.0.smithi027.stdout:0/746: symlink d15/d18/d23/d6f/dcb/lf9 0 2024-06-06T08:43:51.736 INFO:tasks.workunit.client.0.smithi027.stdout:1/683: dwrite dc/d67/d69/fb9 [0,4194304] 0 2024-06-06T08:43:51.738 INFO:tasks.workunit.client.0.smithi027.stdout:0/747: write d15/d18/d64/f6d [1612056,4684] 0 2024-06-06T08:43:51.739 INFO:tasks.workunit.client.0.smithi027.stdout:1/684: dread dc/de/d22/f72 [0,4194304] 0 2024-06-06T08:43:51.739 INFO:tasks.workunit.client.0.smithi027.stdout:0/748: unlink d15/d18/d23/d28/lb4 0 2024-06-06T08:43:51.739 INFO:tasks.workunit.client.0.smithi027.stdout:1/685: readlink dc/de/l3a 0 2024-06-06T08:43:51.739 INFO:tasks.workunit.client.0.smithi027.stdout:1/686: truncate dc/fb3 611304 0 2024-06-06T08:43:51.740 INFO:tasks.workunit.client.0.smithi027.stdout:4/832: dread d0/d1/d6/fae [4194304,4194304] 0 2024-06-06T08:43:51.740 INFO:tasks.workunit.client.0.smithi027.stdout:4/833: stat d0/d1d/d60/l71 0 2024-06-06T08:43:51.741 INFO:tasks.workunit.client.0.smithi027.stdout:0/749: mkdir d15/d18/dbe/d5c/dfa 0 2024-06-06T08:43:51.742 INFO:tasks.workunit.client.0.smithi027.stdout:0/750: creat d15/d18/d23/d6f/ffb x:0 0 0 2024-06-06T08:43:51.743 INFO:tasks.workunit.client.0.smithi027.stdout:0/751: unlink d15/d18/d23/d38/d3e/f5a 0 2024-06-06T08:43:51.743 INFO:tasks.workunit.client.0.smithi027.stdout:0/752: write fb [1881117,113681] 0 2024-06-06T08:43:51.744 INFO:tasks.workunit.client.0.smithi027.stdout:0/753: creat d15/ffc x:0 0 0 2024-06-06T08:43:51.745 INFO:tasks.workunit.client.0.smithi027.stdout:0/754: creat d15/ffd x:0 0 0 2024-06-06T08:43:51.745 INFO:tasks.workunit.client.0.smithi027.stdout:0/755: write d15/d18/d23/d38/d3e/f99 [484675,108569] 0 2024-06-06T08:43:51.746 INFO:tasks.workunit.client.0.smithi027.stdout:0/756: mknod d15/d18/d23/d38/d68/cfe 0 2024-06-06T08:43:51.747 INFO:tasks.workunit.client.0.smithi027.stdout:0/757: creat d15/d18/d23/d6f/deb/fff x:0 0 0 2024-06-06T08:43:51.750 INFO:tasks.workunit.client.0.smithi027.stdout:4/834: link d0/d1d/d34/f40 d0/d1/d6/d8/d21/f10e 0 2024-06-06T08:43:51.750 INFO:tasks.workunit.client.0.smithi027.stdout:1/687: dread dc/de/f13 [0,4194304] 0 2024-06-06T08:43:51.751 INFO:tasks.workunit.client.0.smithi027.stdout:1/688: creat dc/fd6 x:0 0 0 2024-06-06T08:43:51.751 INFO:tasks.workunit.client.0.smithi027.stdout:1/689: dread - dc/de/d14/d80/fd4 zero size 2024-06-06T08:43:51.752 INFO:tasks.workunit.client.0.smithi027.stdout:1/690: mkdir dc/de/d3d/d4a/d76/dd7 0 2024-06-06T08:43:51.754 INFO:tasks.workunit.client.0.smithi027.stdout:1/691: symlink dc/de/d14/d4e/ld8 0 2024-06-06T08:43:51.756 INFO:tasks.workunit.client.0.smithi027.stdout:4/835: dread d0/d1/f4f [0,4194304] 0 2024-06-06T08:43:51.756 INFO:tasks.workunit.client.0.smithi027.stdout:0/758: getdents d15/d18 0 2024-06-06T08:43:51.757 INFO:tasks.workunit.client.0.smithi027.stdout:4/836: creat d0/d1d/d34/d7f/df4/f10f x:0 0 0 2024-06-06T08:43:51.757 INFO:tasks.workunit.client.0.smithi027.stdout:4/837: write d0/d1/d6/d8/fe2 [3536407,130997] 0 2024-06-06T08:43:51.757 INFO:tasks.workunit.client.0.smithi027.stdout:4/838: chown d0/d1d/d2a/d98/l10d 6313114 1 2024-06-06T08:43:51.757 INFO:tasks.workunit.client.0.smithi027.stdout:4/839: write d0/d1d/d34/dd8/ff5 [584294,98254] 0 2024-06-06T08:43:51.758 INFO:tasks.workunit.client.0.smithi027.stdout:0/759: creat d15/d18/dbe/d5c/f100 x:0 0 0 2024-06-06T08:43:51.758 INFO:tasks.workunit.client.0.smithi027.stdout:0/760: fsync d15/d18/ff6 0 2024-06-06T08:43:51.758 INFO:tasks.workunit.client.0.smithi027.stdout:4/840: symlink d0/d1d/d34/d7f/dfc/l110 0 2024-06-06T08:43:51.760 INFO:tasks.workunit.client.0.smithi027.stdout:0/761: symlink d15/d18/d64/l101 0 2024-06-06T08:43:51.762 INFO:tasks.workunit.client.0.smithi027.stdout:0/762: mknod d15/d18/d23/c102 0 2024-06-06T08:43:51.775 INFO:tasks.workunit.client.0.smithi027.stdout:1/692: dwrite dc/de/d14/d4e/f9b [0,4194304] 0 2024-06-06T08:43:51.777 INFO:tasks.workunit.client.0.smithi027.stdout:1/693: rmdir dc/d9d 39 2024-06-06T08:43:51.777 INFO:tasks.workunit.client.0.smithi027.stdout:1/694: write dc/de/f47 [1322541,29814] 0 2024-06-06T08:43:51.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/841: dwrite d0/fe5 [0,4194304] 0 2024-06-06T08:43:51.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/842: fdatasync d0/d63/f6d 0 2024-06-06T08:43:51.779 INFO:tasks.workunit.client.0.smithi027.stdout:1/695: dread dc/d67/d69/d32/f8b [0,4194304] 0 2024-06-06T08:43:51.779 INFO:tasks.workunit.client.0.smithi027.stdout:0/763: rename fc to d15/d18/d23/f103 0 2024-06-06T08:43:51.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/764: stat d15/d18/d23/d3c/f95 0 2024-06-06T08:43:51.782 INFO:tasks.workunit.client.0.smithi027.stdout:4/843: mknod d0/d1/d6/d8/d82/d8b/dc9/da3/daa/c111 0 2024-06-06T08:43:51.783 INFO:tasks.workunit.client.0.smithi027.stdout:4/844: creat d0/d1d/d34/dd8/f112 x:0 0 0 2024-06-06T08:43:51.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/765: read d15/fa1 [1069672,12065] 0 2024-06-06T08:43:51.801 INFO:tasks.workunit.client.0.smithi027.stdout:1/696: dwrite dc/d67/d69/f27 [0,4194304] 0 2024-06-06T08:43:51.808 INFO:tasks.workunit.client.0.smithi027.stdout:4/845: dwrite d0/d1/d6/d8/d82/d8b/dc9/d3b/fe9 [0,4194304] 0 2024-06-06T08:43:51.808 INFO:tasks.workunit.client.0.smithi027.stdout:0/766: link d15/d18/d23/d38/d68/cfe d15/d18/d40/d8c/d9c/c104 0 2024-06-06T08:43:51.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/846: mkdir d0/d63/d10c/d113 0 2024-06-06T08:43:51.812 INFO:tasks.workunit.client.0.smithi027.stdout:0/767: mknod d15/d18/dbe/c105 0 2024-06-06T08:43:51.826 INFO:tasks.workunit.client.0.smithi027.stdout:4/847: dread d0/d1/d6/f2f [0,4194304] 0 2024-06-06T08:43:51.827 INFO:tasks.workunit.client.0.smithi027.stdout:1/697: dwrite dc/fd6 [0,4194304] 0 2024-06-06T08:43:51.829 INFO:tasks.workunit.client.0.smithi027.stdout:1/698: mknod dc/d67/d69/d32/d37/d56/d78/cd9 0 2024-06-06T08:43:51.830 INFO:tasks.workunit.client.0.smithi027.stdout:0/768: rename d15/d18/d23/d3c/c4d to d15/d18/d23/d28/d31/c106 0 2024-06-06T08:43:51.831 INFO:tasks.workunit.client.0.smithi027.stdout:0/769: symlink d15/dae/l107 0 2024-06-06T08:43:51.831 INFO:tasks.workunit.client.0.smithi027.stdout:0/770: dread - d15/d18/d23/d28/d31/dbb/fcc zero size 2024-06-06T08:43:51.833 INFO:tasks.workunit.client.0.smithi027.stdout:0/771: mkdir d15/d18/d108 0 2024-06-06T08:43:51.842 INFO:tasks.workunit.client.0.smithi027.stdout:4/848: dwrite d0/d1/d6/f106 [0,4194304] 0 2024-06-06T08:43:51.843 INFO:tasks.workunit.client.0.smithi027.stdout:4/849: write d0/d1/d6/d8/d82/d8b/dc9/d3b/f68 [1037233,1107] 0 2024-06-06T08:43:51.843 INFO:tasks.workunit.client.0.smithi027.stdout:4/850: write d0/d1d/d2a/fdd [571471,107831] 0 2024-06-06T08:43:51.847 INFO:tasks.workunit.client.0.smithi027.stdout:1/699: dwrite dc/de/d14/f15 [0,4194304] 0 2024-06-06T08:43:51.850 INFO:tasks.workunit.client.0.smithi027.stdout:1/700: symlink dc/da0/lda 0 2024-06-06T08:43:51.850 INFO:tasks.workunit.client.0.smithi027.stdout:1/701: write dc/de/d14/f49 [4513203,129347] 0 2024-06-06T08:43:51.851 INFO:tasks.workunit.client.0.smithi027.stdout:1/702: creat dc/da0/fdb x:0 0 0 2024-06-06T08:43:51.853 INFO:tasks.workunit.client.0.smithi027.stdout:0/772: dwrite d15/d18/d64/f8a [0,4194304] 0 2024-06-06T08:43:51.855 INFO:tasks.workunit.client.0.smithi027.stdout:0/773: creat d15/d18/dbe/d5c/f109 x:0 0 0 2024-06-06T08:43:51.856 INFO:tasks.workunit.client.0.smithi027.stdout:4/851: dread d0/d1d/d2a/d98/fa8 [0,4194304] 0 2024-06-06T08:43:51.856 INFO:tasks.workunit.client.0.smithi027.stdout:0/774: creat d15/d18/dbe/d5c/dfa/f10a x:0 0 0 2024-06-06T08:43:51.857 INFO:tasks.workunit.client.0.smithi027.stdout:4/852: getdents d0/d1/d75 0 2024-06-06T08:43:51.857 INFO:tasks.workunit.client.0.smithi027.stdout:0/775: mkdir d15/d18/d23/d28/d10b 0 2024-06-06T08:43:51.862 INFO:tasks.workunit.client.0.smithi027.stdout:1/703: dread dc/de/d14/d4e/f9b [0,4194304] 0 2024-06-06T08:43:51.863 INFO:tasks.workunit.client.0.smithi027.stdout:1/704: creat dc/d67/fdc x:0 0 0 2024-06-06T08:43:51.865 INFO:tasks.workunit.client.0.smithi027.stdout:1/705: link dc/l23 dc/de/d22/ldd 0 2024-06-06T08:43:51.865 INFO:tasks.workunit.client.0.smithi027.stdout:1/706: truncate dc/fd6 1617432 0 2024-06-06T08:43:51.867 INFO:tasks.workunit.client.0.smithi027.stdout:1/707: link dc/d6d/fce dc/d6d/fde 0 2024-06-06T08:43:51.873 INFO:tasks.workunit.client.0.smithi027.stdout:4/853: dwrite d0/d1/d6/fae [8388608,4194304] 0 2024-06-06T08:43:51.875 INFO:tasks.workunit.client.0.smithi027.stdout:4/854: creat d0/d1/d6/d8/d82/d8b/f114 x:0 0 0 2024-06-06T08:43:51.878 INFO:tasks.workunit.client.0.smithi027.stdout:4/855: symlink d0/d1d/d34/d7f/l115 0 2024-06-06T08:43:51.880 INFO:tasks.workunit.client.0.smithi027.stdout:9/713: sync 2024-06-06T08:43:51.880 INFO:tasks.workunit.client.0.smithi027.stdout:9/714: write d3/d95/d23/d36/fe6 [1149893,95691] 0 2024-06-06T08:43:51.880 INFO:tasks.workunit.client.0.smithi027.stdout:9/715: dread - d3/d95/d23/d36/d3a/f96 zero size 2024-06-06T08:43:51.880 INFO:tasks.workunit.client.0.smithi027.stdout:9/716: write d3/d95/d23/d36/d3a/f6d [1277220,91530] 0 2024-06-06T08:43:51.881 INFO:tasks.workunit.client.0.smithi027.stdout:0/776: dwrite d15/d18/d40/f9e [0,4194304] 0 2024-06-06T08:43:51.881 INFO:tasks.workunit.client.0.smithi027.stdout:0/777: write d15/f21 [7776644,34106] 0 2024-06-06T08:43:51.884 INFO:tasks.workunit.client.0.smithi027.stdout:9/717: write d3/d95/d23/d36/d3a/fe4 [819025,11303] 0 2024-06-06T08:43:51.885 INFO:tasks.workunit.client.0.smithi027.stdout:0/778: creat d15/d18/d40/d8c/d9c/f10c x:0 0 0 2024-06-06T08:43:51.885 INFO:tasks.workunit.client.0.smithi027.stdout:9/718: mknod d3/d4e/d75/db4/cf8 0 2024-06-06T08:43:51.887 INFO:tasks.workunit.client.0.smithi027.stdout:0/779: creat d15/d18/f10d x:0 0 0 2024-06-06T08:43:51.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/708: dwrite dc/de/f13 [4194304,4194304] 0 2024-06-06T08:43:51.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/709: write dc/de/d14/d80/fc6 [436245,92648] 0 2024-06-06T08:43:51.896 INFO:tasks.workunit.client.0.smithi027.stdout:1/710: mkdir dc/d67/d69/d32/d37/ddf 0 2024-06-06T08:43:51.897 INFO:tasks.workunit.client.0.smithi027.stdout:1/711: truncate dc/d6d/fca 252672 0 2024-06-06T08:43:51.897 INFO:tasks.workunit.client.0.smithi027.stdout:1/712: readlink dc/d67/d69/d32/d37/l6b 0 2024-06-06T08:43:51.897 INFO:tasks.workunit.client.0.smithi027.stdout:1/713: write dc/de/d3d/f42 [1572758,8415] 0 2024-06-06T08:43:51.899 INFO:tasks.workunit.client.0.smithi027.stdout:1/714: symlink dc/d67/d69/d32/d37/d98/le0 0 2024-06-06T08:43:51.899 INFO:tasks.workunit.client.0.smithi027.stdout:1/715: truncate dc/de/d3d/f85 2524980 0 2024-06-06T08:43:51.905 INFO:tasks.workunit.client.0.smithi027.stdout:9/719: link d3/d95/d23/d78/l8f d3/d4e/d75/db4/lf9 0 2024-06-06T08:43:51.910 INFO:tasks.workunit.client.0.smithi027.stdout:1/716: write dc/d67/d69/f27 [3539649,77088] 0 2024-06-06T08:43:51.911 INFO:tasks.workunit.client.0.smithi027.stdout:1/717: chown dc/d67/d69/d32/c48 1303779 1 2024-06-06T08:43:51.911 INFO:tasks.workunit.client.0.smithi027.stdout:1/718: write dc/fd [300076,122698] 0 2024-06-06T08:43:51.912 INFO:tasks.workunit.client.0.smithi027.stdout:1/719: mkdir dc/d67/d69/d32/d37/d56/de1 0 2024-06-06T08:43:51.913 INFO:tasks.workunit.client.0.smithi027.stdout:1/720: truncate dc/de/d14/d4e/f9b 801830 0 2024-06-06T08:43:51.914 INFO:tasks.workunit.client.0.smithi027.stdout:1/721: mknod dc/de/d3d/d4a/d76/ce2 0 2024-06-06T08:43:51.915 INFO:tasks.workunit.client.0.smithi027.stdout:1/722: unlink dc/de/d14/d4e/f7a 0 2024-06-06T08:43:51.917 INFO:tasks.workunit.client.0.smithi027.stdout:1/723: creat dc/de/d14/dc8/fe3 x:0 0 0 2024-06-06T08:43:51.918 INFO:tasks.workunit.client.0.smithi027.stdout:1/724: truncate dc/fd 117393 0 2024-06-06T08:43:51.921 INFO:tasks.workunit.client.0.smithi027.stdout:1/725: mkdir dc/de/de4 0 2024-06-06T08:43:51.921 INFO:tasks.workunit.client.0.smithi027.stdout:1/726: write dc/de/d14/d80/da6/db0/fb5 [1420539,46747] 0 2024-06-06T08:43:51.922 INFO:tasks.workunit.client.0.smithi027.stdout:1/727: mknod dc/d67/d69/d32/d37/d98/ce5 0 2024-06-06T08:43:51.941 INFO:tasks.workunit.client.0.smithi027.stdout:4/856: dwrite d0/d1/d6/d4b/f80 [0,4194304] 0 2024-06-06T08:43:51.941 INFO:tasks.workunit.client.0.smithi027.stdout:4/857: chown d0/d1d/d60/dc6 526770063 1 2024-06-06T08:43:51.944 INFO:tasks.workunit.client.0.smithi027.stdout:9/720: dwrite d3/d4e/d75/dc/f2d [0,4194304] 0 2024-06-06T08:43:51.945 INFO:tasks.workunit.client.0.smithi027.stdout:9/721: creat d3/d95/d23/d36/d3a/d84/db2/ffa x:0 0 0 2024-06-06T08:43:51.945 INFO:tasks.workunit.client.0.smithi027.stdout:9/722: getdents d3/d95/d23/d78/ddf/de5 0 2024-06-06T08:43:51.945 INFO:tasks.workunit.client.0.smithi027.stdout:9/723: readlink d3/d76/dd9/lf7 0 2024-06-06T08:43:51.945 INFO:tasks.workunit.client.0.smithi027.stdout:1/728: dwrite dc/fd6 [0,4194304] 0 2024-06-06T08:43:51.945 INFO:tasks.workunit.client.0.smithi027.stdout:1/729: chown dc/de/d14/d4e/l88 13658896 1 2024-06-06T08:43:51.950 INFO:tasks.workunit.client.0.smithi027.stdout:1/730: creat dc/de/d14/d80/fe6 x:0 0 0 2024-06-06T08:43:51.951 INFO:tasks.workunit.client.0.smithi027.stdout:1/731: creat dc/d9d/fe7 x:0 0 0 2024-06-06T08:43:51.953 INFO:tasks.workunit.client.0.smithi027.stdout:9/724: mknod d3/cfb 0 2024-06-06T08:43:51.953 INFO:tasks.workunit.client.0.smithi027.stdout:1/732: getdents dc/de/d3d/d4f 0 2024-06-06T08:43:51.955 INFO:tasks.workunit.client.0.smithi027.stdout:1/733: mkdir dc/de/d14/d80/de8 0 2024-06-06T08:43:51.959 INFO:tasks.workunit.client.0.smithi027.stdout:4/858: rmdir d0/d1d/d34/d7f 39 2024-06-06T08:43:51.962 INFO:tasks.workunit.client.0.smithi027.stdout:9/725: dread d3/f5f [0,4194304] 0 2024-06-06T08:43:51.964 INFO:tasks.workunit.client.0.smithi027.stdout:9/726: symlink d3/d95/d34/d41/lfc 0 2024-06-06T08:43:51.966 INFO:tasks.workunit.client.0.smithi027.stdout:1/734: dread dc/de/d14/d4e/f5d [0,4194304] 0 2024-06-06T08:43:51.966 INFO:tasks.workunit.client.0.smithi027.stdout:1/735: fsync dc/de/d14/f63 0 2024-06-06T08:43:51.967 INFO:tasks.workunit.client.0.smithi027.stdout:1/736: creat dc/d67/d69/d32/d37/d56/de1/fe9 x:0 0 0 2024-06-06T08:43:51.974 INFO:tasks.workunit.client.0.smithi027.stdout:4/859: rename d0/fd to d0/d1/d6/d8/d82/d8b/f116 0 2024-06-06T08:43:51.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/727: rename d3/d95/d23/c7b to d3/d95/d23/d36/d3a/d84/cfd 0 2024-06-06T08:43:51.977 INFO:tasks.workunit.client.0.smithi027.stdout:9/728: mkdir d3/d95/d23/d36/d3a/df1/dfe 0 2024-06-06T08:43:51.978 INFO:tasks.workunit.client.0.smithi027.stdout:4/860: mknod d0/d1/d6/d8/d82/d8b/dc9/da3/d10b/c117 0 2024-06-06T08:43:51.980 INFO:tasks.workunit.client.0.smithi027.stdout:4/861: creat d0/d1d/d60/dc6/f118 x:0 0 0 2024-06-06T08:43:51.981 INFO:tasks.workunit.client.0.smithi027.stdout:4/862: mkdir d0/d1d/d60/d119 0 2024-06-06T08:43:51.982 INFO:tasks.workunit.client.0.smithi027.stdout:1/737: dwrite dc/d6d/fde [0,4194304] 0 2024-06-06T08:43:51.983 INFO:tasks.workunit.client.0.smithi027.stdout:4/863: creat d0/d1/d6/d8/d82/d8b/dc9/f11a x:0 0 0 2024-06-06T08:43:51.984 INFO:tasks.workunit.client.0.smithi027.stdout:4/864: mkdir d0/d1d/d34/d11b 0 2024-06-06T08:43:51.984 INFO:tasks.workunit.client.0.smithi027.stdout:9/729: dread d3/d95/f4f [0,4194304] 0 2024-06-06T08:43:51.985 INFO:tasks.workunit.client.0.smithi027.stdout:4/865: creat d0/d63/d10c/d113/f11c x:0 0 0 2024-06-06T08:43:51.986 INFO:tasks.workunit.client.0.smithi027.stdout:1/738: rename dc/de/d22/f72 to dc/d67/d69/d32/d37/d56/d78/fea 0 2024-06-06T08:43:51.987 INFO:tasks.workunit.client.0.smithi027.stdout:4/866: creat d0/d1d/dd0/f11d x:0 0 0 2024-06-06T08:43:51.988 INFO:tasks.workunit.client.0.smithi027.stdout:1/739: link dc/de/d14/f19 dc/feb 0 2024-06-06T08:43:51.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/730: write d3/d95/f28 [2507269,30861] 0 2024-06-06T08:43:51.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/731: chown d3/d95/d23/d36/d3a/d84/db2/fd0 7 1 2024-06-06T08:43:51.989 INFO:tasks.workunit.client.0.smithi027.stdout:1/740: link dc/d6d/fde dc/de/d3d/d4a/d76/fec 0 2024-06-06T08:43:51.990 INFO:tasks.workunit.client.0.smithi027.stdout:9/732: unlink d3/d4e/d75/db4/cc8 0 2024-06-06T08:43:51.991 INFO:tasks.workunit.client.0.smithi027.stdout:9/733: mkdir d3/d4e/d75/dc/df6/dff 0 2024-06-06T08:43:52.012 INFO:tasks.workunit.client.0.smithi027.stdout:9/734: dread d3/f15 [4194304,4194304] 0 2024-06-06T08:43:52.012 INFO:tasks.workunit.client.0.smithi027.stdout:4/867: dwrite d0/d1/d6/d4b/fce [0,4194304] 0 2024-06-06T08:43:52.012 INFO:tasks.workunit.client.0.smithi027.stdout:9/735: chown d3/d4e/d75/dc/df6 1275 1 2024-06-06T08:43:52.013 INFO:tasks.workunit.client.0.smithi027.stdout:1/741: dwrite dc/d67/fbe [0,4194304] 0 2024-06-06T08:43:52.014 INFO:tasks.workunit.client.0.smithi027.stdout:4/868: symlink d0/d1d/d34/l11e 0 2024-06-06T08:43:52.014 INFO:tasks.workunit.client.0.smithi027.stdout:9/736: mkdir d3/d95/d23/d78/d100 0 2024-06-06T08:43:52.014 INFO:tasks.workunit.client.0.smithi027.stdout:9/737: write d3/d95/f57 [1545393,89146] 0 2024-06-06T08:43:52.021 INFO:tasks.workunit.client.0.smithi027.stdout:1/742: link dc/de/d14/d80/fa4 dc/d67/d69/d32/fed 0 2024-06-06T08:43:52.021 INFO:tasks.workunit.client.0.smithi027.stdout:9/738: symlink d3/d4e/d75/dc/l101 0 2024-06-06T08:43:52.022 INFO:tasks.workunit.client.0.smithi027.stdout:9/739: symlink d3/d4e/d75/dc/df6/l102 0 2024-06-06T08:43:52.024 INFO:tasks.workunit.client.0.smithi027.stdout:9/740: creat d3/d4e/d75/dc/f103 x:0 0 0 2024-06-06T08:43:52.024 INFO:tasks.workunit.client.0.smithi027.stdout:1/743: rmdir dc/d67 39 2024-06-06T08:43:52.025 INFO:tasks.workunit.client.0.smithi027.stdout:8/634: sync 2024-06-06T08:43:52.025 INFO:tasks.workunit.client.0.smithi027.stdout:2/710: sync 2024-06-06T08:43:52.027 INFO:tasks.workunit.client.0.smithi027.stdout:9/741: creat d3/d95/d23/da9/f104 x:0 0 0 2024-06-06T08:43:52.028 INFO:tasks.workunit.client.0.smithi027.stdout:1/744: symlink dc/lee 0 2024-06-06T08:43:52.031 INFO:tasks.workunit.client.0.smithi027.stdout:1/745: creat dc/de/de4/fef x:0 0 0 2024-06-06T08:43:52.033 INFO:tasks.workunit.client.0.smithi027.stdout:8/635: rename d7/dc/d36/d3c/l48 to d7/d9c/daf/ld2 0 2024-06-06T08:43:52.034 INFO:tasks.workunit.client.0.smithi027.stdout:1/746: mknod dc/de/d3d/d4a/d76/dd7/cf0 0 2024-06-06T08:43:52.034 INFO:tasks.workunit.client.0.smithi027.stdout:8/636: mkdir d7/dc/d36/d41/d85/db7/dd3 0 2024-06-06T08:43:52.034 INFO:tasks.workunit.client.0.smithi027.stdout:8/637: truncate d7/dc/d36/d41/d85/fce 1006588 0 2024-06-06T08:43:52.035 INFO:tasks.workunit.client.0.smithi027.stdout:8/638: chown d7/dc/d37/l7e 304 1 2024-06-06T08:43:52.035 INFO:tasks.workunit.client.0.smithi027.stdout:8/639: truncate d7/dc/d36/d83/d60/fad 1244964 0 2024-06-06T08:43:52.036 INFO:tasks.workunit.client.0.smithi027.stdout:9/742: write d3/d95/d34/f60 [1134187,123140] 0 2024-06-06T08:43:52.037 INFO:tasks.workunit.client.0.smithi027.stdout:9/743: symlink d3/d95/d34/l105 0 2024-06-06T08:43:52.040 INFO:tasks.workunit.client.0.smithi027.stdout:9/744: mkdir d3/d95/d23/d36/d3a/d106 0 2024-06-06T08:43:52.042 INFO:tasks.workunit.client.0.smithi027.stdout:9/745: unlink d3/d95/d23/d36/d3a/d84/db2/fda 0 2024-06-06T08:43:52.043 INFO:tasks.workunit.client.0.smithi027.stdout:9/746: creat d3/d95/d34/f107 x:0 0 0 2024-06-06T08:43:52.043 INFO:tasks.workunit.client.0.smithi027.stdout:9/747: getdents d3/d95/d23/d36/d3a/df1/dfe 0 2024-06-06T08:43:52.043 INFO:tasks.workunit.client.0.smithi027.stdout:9/748: chown d3/d95/f5e 3739 1 2024-06-06T08:43:52.044 INFO:tasks.workunit.client.0.smithi027.stdout:9/749: creat d3/d4e/d75/dc/f108 x:0 0 0 2024-06-06T08:43:52.046 INFO:tasks.workunit.client.0.smithi027.stdout:9/750: symlink d3/d95/d23/d36/d3a/d84/l109 0 2024-06-06T08:43:52.048 INFO:tasks.workunit.client.0.smithi027.stdout:2/711: dwrite d6/dd/dd9/fb2 [0,4194304] 0 2024-06-06T08:43:52.048 INFO:tasks.workunit.client.0.smithi027.stdout:2/712: fdatasync d6/d1e/d2b/fe0 0 2024-06-06T08:43:52.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/751: getdents d3/d95/d23/d36/d3a/d84/db2 0 2024-06-06T08:43:52.053 INFO:tasks.workunit.client.0.smithi027.stdout:8/640: dwrite d7/d2f/f98 [0,4194304] 0 2024-06-06T08:43:52.053 INFO:tasks.workunit.client.0.smithi027.stdout:8/641: write d7/d13/d15/f1c [2142205,86285] 0 2024-06-06T08:43:52.057 INFO:tasks.workunit.client.0.smithi027.stdout:8/642: chown d7/dc/d37/d46/f52 5 1 2024-06-06T08:43:52.061 INFO:tasks.workunit.client.0.smithi027.stdout:9/752: dread d3/d95/d23/d36/fe6 [0,4194304] 0 2024-06-06T08:43:52.073 INFO:tasks.workunit.client.0.smithi027.stdout:2/713: dwrite d6/d1e/d2b/d5a/d7b/f7e [0,4194304] 0 2024-06-06T08:43:52.074 INFO:tasks.workunit.client.0.smithi027.stdout:8/643: dwrite d7/dc/d36/d83/d60/f68 [0,4194304] 0 2024-06-06T08:43:52.074 INFO:tasks.workunit.client.0.smithi027.stdout:8/644: chown d7/dc/d36/d3c/f61 2781 1 2024-06-06T08:43:52.078 INFO:tasks.workunit.client.0.smithi027.stdout:2/714: link d6/dd/dd9/d1d/f5b d6/dd/dd9/d3f/db0/fed 0 2024-06-06T08:43:52.082 INFO:tasks.workunit.client.0.smithi027.stdout:9/753: dwrite d3/d95/db6/fe2 [0,4194304] 0 2024-06-06T08:43:52.086 INFO:tasks.workunit.client.0.smithi027.stdout:9/754: mknod d3/d4e/d75/dc/c10a 0 2024-06-06T08:43:52.086 INFO:tasks.workunit.client.0.smithi027.stdout:9/755: stat d3/f11 0 2024-06-06T08:43:52.086 INFO:tasks.workunit.client.0.smithi027.stdout:9/756: dread - d3/d95/d34/f107 zero size 2024-06-06T08:43:52.089 INFO:tasks.workunit.client.0.smithi027.stdout:9/757: symlink d3/d95/d34/d73/l10b 0 2024-06-06T08:43:52.092 INFO:tasks.workunit.client.0.smithi027.stdout:2/715: dread d6/d1e/d2b/d5a/d7b/f7e [0,4194304] 0 2024-06-06T08:43:52.094 INFO:tasks.workunit.client.0.smithi027.stdout:2/716: symlink d6/d3b/d80/lee 0 2024-06-06T08:43:52.095 INFO:tasks.workunit.client.0.smithi027.stdout:2/717: creat d6/d1e/d2b/d5a/fef x:0 0 0 2024-06-06T08:43:52.096 INFO:tasks.workunit.client.0.smithi027.stdout:8/645: dwrite d7/dc/d36/d83/d60/f7a [0,4194304] 0 2024-06-06T08:43:52.096 INFO:tasks.workunit.client.0.smithi027.stdout:8/646: getdents d7/dc/d36/d41/dbf/dc3 0 2024-06-06T08:43:52.099 INFO:tasks.workunit.client.0.smithi027.stdout:9/758: read d3/d95/d23/d36/d3a/f68 [1058480,74852] 0 2024-06-06T08:43:52.099 INFO:tasks.workunit.client.0.smithi027.stdout:8/647: mknod d7/dc/d36/d83/d60/cd4 0 2024-06-06T08:43:52.100 INFO:tasks.workunit.client.0.smithi027.stdout:9/759: symlink d3/d95/d23/d36/d3a/d84/db2/l10c 0 2024-06-06T08:43:52.112 INFO:tasks.workunit.client.0.smithi027.stdout:9/760: write d3/d95/d34/d41/f79 [374983,59057] 0 2024-06-06T08:43:52.112 INFO:tasks.workunit.client.0.smithi027.stdout:2/718: dwrite d6/d1e/f27 [0,4194304] 0 2024-06-06T08:43:52.114 INFO:tasks.workunit.client.0.smithi027.stdout:2/719: symlink d6/dd/dd9/d3f/db0/lf0 0 2024-06-06T08:43:52.115 INFO:tasks.workunit.client.0.smithi027.stdout:9/761: dread d3/d95/fba [0,4194304] 0 2024-06-06T08:43:52.117 INFO:tasks.workunit.client.0.smithi027.stdout:9/762: symlink d3/d4e/d75/l10d 0 2024-06-06T08:43:52.119 INFO:tasks.workunit.client.0.smithi027.stdout:9/763: rename d3/d95/d23/d36/d3a/d84/l109 to d3/d95/d23/d36/l10e 0 2024-06-06T08:43:52.120 INFO:tasks.workunit.client.0.smithi027.stdout:9/764: creat d3/d95/d23/d78/d100/f10f x:0 0 0 2024-06-06T08:43:52.134 INFO:tasks.workunit.client.0.smithi027.stdout:2/720: dwrite d6/dd/dd9/d3f/d77/d55/fa4 [0,4194304] 0 2024-06-06T08:43:52.141 INFO:tasks.workunit.client.0.smithi027.stdout:9/765: dwrite d3/d95/d23/d36/d3a/f6d [0,4194304] 0 2024-06-06T08:43:52.143 INFO:tasks.workunit.client.0.smithi027.stdout:9/766: rmdir d3/d95/d23/d36/d3a/df1 39 2024-06-06T08:43:52.146 INFO:tasks.workunit.client.0.smithi027.stdout:9/767: rename d3/d95/d34/l54 to d3/d4e/d75/dc/df6/dff/l110 0 2024-06-06T08:43:52.160 INFO:tasks.workunit.client.0.smithi027.stdout:2/721: dwrite d6/dd/dd9/f22 [0,4194304] 0 2024-06-06T08:43:52.160 INFO:tasks.workunit.client.0.smithi027.stdout:2/722: fdatasync d6/d1e/f27 0 2024-06-06T08:43:52.161 INFO:tasks.workunit.client.0.smithi027.stdout:9/768: link d3/d95/d23/d36/fe6 d3/f111 0 2024-06-06T08:43:52.162 INFO:tasks.workunit.client.0.smithi027.stdout:9/769: creat d3/d95/d23/d36/d3a/df1/dfe/f112 x:0 0 0 2024-06-06T08:43:52.164 INFO:tasks.workunit.client.0.smithi027.stdout:2/723: creat d6/dd/dd9/d3f/d48/ff1 x:0 0 0 2024-06-06T08:43:52.173 INFO:tasks.workunit.client.0.smithi027.stdout:2/724: write d6/dd/dd9/d3f/d48/f62 [3171233,106054] 0 2024-06-06T08:43:52.176 INFO:tasks.workunit.client.0.smithi027.stdout:2/725: mknod d6/dd/dd9/d3f/d48/d83/cf2 0 2024-06-06T08:43:52.182 INFO:tasks.workunit.client.0.smithi027.stdout:9/770: dwrite d3/d95/d34/ff5 [0,4194304] 0 2024-06-06T08:43:52.183 INFO:tasks.workunit.client.0.smithi027.stdout:9/771: creat d3/d95/d23/da9/f113 x:0 0 0 2024-06-06T08:43:52.184 INFO:tasks.workunit.client.0.smithi027.stdout:9/772: mkdir d3/d95/d23/def/d114 0 2024-06-06T08:43:52.184 INFO:tasks.workunit.client.0.smithi027.stdout:9/773: chown d3/d95/d23/d78/ddf/de5 227225 1 2024-06-06T08:43:52.193 INFO:tasks.workunit.client.0.smithi027.stdout:2/726: dwrite d6/dd/dd9/d1d/f86 [0,4194304] 0 2024-06-06T08:43:52.194 INFO:tasks.workunit.client.0.smithi027.stdout:2/727: read - d6/dd/dd9/d1d/d4d/d8e/fa8 zero size 2024-06-06T08:43:52.200 INFO:tasks.workunit.client.0.smithi027.stdout:9/774: rename d3/d95/d23/c66 to d3/d76/dd9/c115 0 2024-06-06T08:43:52.200 INFO:tasks.workunit.client.0.smithi027.stdout:9/775: write d3/d95/d23/d36/d3a/df1/dfe/f112 [850975,85881] 0 2024-06-06T08:43:52.200 INFO:tasks.workunit.client.0.smithi027.stdout:9/776: read - d3/d4e/d75/dc/f103 zero size 2024-06-06T08:43:52.200 INFO:tasks.workunit.client.0.smithi027.stdout:9/777: chown d3/d95/d34/d73/l10b 118 1 2024-06-06T08:43:52.201 INFO:tasks.workunit.client.0.smithi027.stdout:9/778: mkdir d3/d95/d23/def/d116 0 2024-06-06T08:43:52.208 INFO:tasks.workunit.client.0.smithi027.stdout:7/873: sync 2024-06-06T08:43:52.208 INFO:tasks.workunit.client.0.smithi027.stdout:5/915: sync 2024-06-06T08:43:52.208 INFO:tasks.workunit.client.0.smithi027.stdout:6/890: sync 2024-06-06T08:43:52.208 INFO:tasks.workunit.client.0.smithi027.stdout:3/662: sync 2024-06-06T08:43:52.208 INFO:tasks.workunit.client.0.smithi027.stdout:3/663: write d1/d1f/d45/d5f/fc1 [879761,37] 0 2024-06-06T08:43:52.213 INFO:tasks.workunit.client.0.smithi027.stdout:5/916: link d0/c4b d0/c125 0 2024-06-06T08:43:52.214 INFO:tasks.workunit.client.0.smithi027.stdout:5/917: truncate d0/da/db/d1a/dc6/fd8 4109664 0 2024-06-06T08:43:52.217 INFO:tasks.workunit.client.0.smithi027.stdout:9/779: dread d3/d95/faa [0,4194304] 0 2024-06-06T08:43:52.218 INFO:tasks.workunit.client.0.smithi027.stdout:9/780: mkdir d3/d95/db6/d117 0 2024-06-06T08:43:52.222 INFO:tasks.workunit.client.0.smithi027.stdout:2/728: dwrite d6/dd/dd9/f54 [0,4194304] 0 2024-06-06T08:43:52.226 INFO:tasks.workunit.client.0.smithi027.stdout:2/729: rmdir d6/dd/dd9/d3f/d48 39 2024-06-06T08:43:52.229 INFO:tasks.workunit.client.0.smithi027.stdout:6/891: dread d4/de/d45/d6a/f4b [0,4194304] 0 2024-06-06T08:43:52.229 INFO:tasks.workunit.client.0.smithi027.stdout:2/730: fdatasync d6/dd/dd9/d3f/d77/d55/d69/d82/fa9 0 2024-06-06T08:43:52.231 INFO:tasks.workunit.client.0.smithi027.stdout:9/781: dread d3/d95/d23/d36/f5d [0,4194304] 0 2024-06-06T08:43:52.231 INFO:tasks.workunit.client.0.smithi027.stdout:7/874: dwrite d17/d32/d61/d75/fe5 [8388608,4194304] 0 2024-06-06T08:43:52.233 INFO:tasks.workunit.client.0.smithi027.stdout:2/731: chown d6/fa7 7497 1 2024-06-06T08:43:52.236 INFO:tasks.workunit.client.0.smithi027.stdout:7/875: mknod d17/d32/d46/d51/d52/c120 0 2024-06-06T08:43:52.236 INFO:tasks.workunit.client.0.smithi027.stdout:7/876: dread - d17/d32/d46/d83/f114 zero size 2024-06-06T08:43:52.236 INFO:tasks.workunit.client.0.smithi027.stdout:7/877: chown d17/d32/d61/l9f 4 1 2024-06-06T08:43:52.237 INFO:tasks.workunit.client.0.smithi027.stdout:2/732: creat d6/d3b/d80/ff3 x:0 0 0 2024-06-06T08:43:52.238 INFO:tasks.workunit.client.0.smithi027.stdout:7/878: creat d17/d32/d6c/dc2/f121 x:0 0 0 2024-06-06T08:43:52.239 INFO:tasks.workunit.client.0.smithi027.stdout:5/918: dwrite d0/da/db/f1b [4194304,4194304] 0 2024-06-06T08:43:52.240 INFO:tasks.workunit.client.0.smithi027.stdout:2/733: unlink d6/dd/dd9/d3f/d77/d55/cc6 0 2024-06-06T08:43:52.240 INFO:tasks.workunit.client.0.smithi027.stdout:3/664: dwrite d1/f29 [0,4194304] 0 2024-06-06T08:43:52.242 INFO:tasks.workunit.client.0.smithi027.stdout:5/919: creat d0/da/db/d46/f126 x:0 0 0 2024-06-06T08:43:52.243 INFO:tasks.workunit.client.0.smithi027.stdout:2/734: chown d6/dd/dd9/d3f/d48/f62 0 1 2024-06-06T08:43:52.245 INFO:tasks.workunit.client.0.smithi027.stdout:7/879: mkdir d17/d32/d46/d4f/db6/dc8/d122 0 2024-06-06T08:43:52.245 INFO:tasks.workunit.client.0.smithi027.stdout:5/920: chown d0/da/db/d46/d4c/l87 35630 1 2024-06-06T08:43:52.245 INFO:tasks.workunit.client.0.smithi027.stdout:5/921: stat d0/da/db/d1a/lb6 0 2024-06-06T08:43:52.246 INFO:tasks.workunit.client.0.smithi027.stdout:7/880: symlink d17/d32/d46/d83/d2a/d3b/dad/l123 0 2024-06-06T08:43:52.247 INFO:tasks.workunit.client.0.smithi027.stdout:7/881: read d17/d47/ff1 [47741,65970] 0 2024-06-06T08:43:52.247 INFO:tasks.workunit.client.0.smithi027.stdout:7/882: truncate d17/d32/d46/d83/f114 414852 0 2024-06-06T08:43:52.248 INFO:tasks.workunit.client.0.smithi027.stdout:5/922: rename d0/da/db/d46/d4c/d70/f9e to d0/da/db/f127 0 2024-06-06T08:43:52.249 INFO:tasks.workunit.client.0.smithi027.stdout:2/735: symlink d6/dd/dd9/d3f/d48/lf4 0 2024-06-06T08:43:52.249 INFO:tasks.workunit.client.0.smithi027.stdout:2/736: chown d6/dd/dd9/d3f/db0/fcc 56747740 1 2024-06-06T08:43:52.250 INFO:tasks.workunit.client.0.smithi027.stdout:5/923: chown d0/l23 37528 1 2024-06-06T08:43:52.250 INFO:tasks.workunit.client.0.smithi027.stdout:5/924: chown d0/da/db/d46/d4c/d70 418045 1 2024-06-06T08:43:52.251 INFO:tasks.workunit.client.0.smithi027.stdout:2/737: chown d6/dd/f26 7694 1 2024-06-06T08:43:52.252 INFO:tasks.workunit.client.0.smithi027.stdout:5/925: creat d0/da/db/d1a/da9/f128 x:0 0 0 2024-06-06T08:43:52.255 INFO:tasks.workunit.client.0.smithi027.stdout:3/665: dwrite d1/d8/d22/d27/f14 [4194304,4194304] 0 2024-06-06T08:43:52.255 INFO:tasks.workunit.client.0.smithi027.stdout:3/666: chown d1/d90/d44/d54/fc4 63826 1 2024-06-06T08:43:52.259 INFO:tasks.workunit.client.0.smithi027.stdout:5/926: dread d0/da/d37/d2b/f4a [0,4194304] 0 2024-06-06T08:43:52.259 INFO:tasks.workunit.client.0.smithi027.stdout:5/927: fdatasync d0/da/db/d1a/da3/de5/f11f 0 2024-06-06T08:43:52.260 INFO:tasks.workunit.client.0.smithi027.stdout:5/928: mkdir d0/da/db/d1a/dc6/d111/d129 0 2024-06-06T08:43:52.262 INFO:tasks.workunit.client.0.smithi027.stdout:6/892: dwrite d4/fef [0,4194304] 0 2024-06-06T08:43:52.267 INFO:tasks.workunit.client.0.smithi027.stdout:2/738: write d6/dd/dd9/d1d/f86 [1754296,82168] 0 2024-06-06T08:43:52.267 INFO:tasks.workunit.client.0.smithi027.stdout:2/739: truncate d6/dd/dd9/d3f/d77/d55/d69/d82/fa9 815150 0 2024-06-06T08:43:52.267 INFO:tasks.workunit.client.0.smithi027.stdout:7/883: dwrite d17/d32/d46/d83/d77/fa2 [0,4194304] 0 2024-06-06T08:43:52.269 INFO:tasks.workunit.client.0.smithi027.stdout:6/893: getdents d4/de 0 2024-06-06T08:43:52.269 INFO:tasks.workunit.client.0.smithi027.stdout:6/894: fdatasync d4/d125/d18/d10a/f10b 0 2024-06-06T08:43:52.270 INFO:tasks.workunit.client.0.smithi027.stdout:6/895: chown d4/d1a/d2e 875695490 1 2024-06-06T08:43:52.270 INFO:tasks.workunit.client.0.smithi027.stdout:6/896: write d4/d1a/d2e/f11d [123894,97908] 0 2024-06-06T08:43:52.270 INFO:tasks.workunit.client.0.smithi027.stdout:6/897: write d4/f79 [1356419,108852] 0 2024-06-06T08:43:52.271 INFO:tasks.workunit.client.0.smithi027.stdout:7/884: stat d17/d32/d46/d83/d77/d8c/dae 0 2024-06-06T08:43:52.278 INFO:tasks.workunit.client.0.smithi027.stdout:7/885: dread d17/d47/f4d [0,4194304] 0 2024-06-06T08:43:52.282 INFO:tasks.workunit.client.0.smithi027.stdout:5/929: dwrite d0/da/db/d1a/da3/de5/d107/f10e [0,4194304] 0 2024-06-06T08:43:52.283 INFO:tasks.workunit.client.0.smithi027.stdout:3/667: dwrite d1/d1f/d25/f7d [0,4194304] 0 2024-06-06T08:43:52.284 INFO:tasks.workunit.client.0.smithi027.stdout:5/930: mknod d0/da/db/d46/d4c/c12a 0 2024-06-06T08:43:52.286 INFO:tasks.workunit.client.0.smithi027.stdout:3/668: creat d1/d8/d22/d2d/fd0 x:0 0 0 2024-06-06T08:43:52.286 INFO:tasks.workunit.client.0.smithi027.stdout:3/669: dread - d1/d1f/dbc/fcc zero size 2024-06-06T08:43:52.288 INFO:tasks.workunit.client.0.smithi027.stdout:3/670: unlink d1/d90/fa5 0 2024-06-06T08:43:52.288 INFO:tasks.workunit.client.0.smithi027.stdout:3/671: dread - d1/d8/d22/fc3 zero size 2024-06-06T08:43:52.292 INFO:tasks.workunit.client.0.smithi027.stdout:3/672: rmdir d1/d8/d22/d2d 39 2024-06-06T08:43:52.293 INFO:tasks.workunit.client.0.smithi027.stdout:2/740: dwrite d6/f2c [0,4194304] 0 2024-06-06T08:43:52.297 INFO:tasks.workunit.client.0.smithi027.stdout:2/741: dread d6/dd/dd9/d1d/f5b [0,4194304] 0 2024-06-06T08:43:52.298 INFO:tasks.workunit.client.0.smithi027.stdout:2/742: mknod d6/dd/dd9/d1d/dbc/cf5 0 2024-06-06T08:43:52.299 INFO:tasks.workunit.client.0.smithi027.stdout:2/743: readlink d6/dd/dd9/d1d/lb1 0 2024-06-06T08:43:52.299 INFO:tasks.workunit.client.0.smithi027.stdout:6/898: dwrite d4/d125/d18/fcc [0,4194304] 0 2024-06-06T08:43:52.300 INFO:tasks.workunit.client.0.smithi027.stdout:2/744: write d6/dd/dd9/d1d/f45 [6145028,102593] 0 2024-06-06T08:43:52.300 INFO:tasks.workunit.client.0.smithi027.stdout:2/745: write d6/dd/dd9/d3f/d48/f62 [1434089,37623] 0 2024-06-06T08:43:52.300 INFO:tasks.workunit.client.0.smithi027.stdout:7/886: dwrite d17/d32/d46/d4f/d5e/fb0 [0,4194304] 0 2024-06-06T08:43:52.301 INFO:tasks.workunit.client.0.smithi027.stdout:3/673: dread d1/f84 [0,4194304] 0 2024-06-06T08:43:52.302 INFO:tasks.workunit.client.0.smithi027.stdout:7/887: mkdir d17/d32/d46/d83/d2a/d3b/dc7/d124 0 2024-06-06T08:43:52.303 INFO:tasks.workunit.client.0.smithi027.stdout:7/888: mkdir d17/d32/d46/d83/d77/d8c/d125 0 2024-06-06T08:43:52.304 INFO:tasks.workunit.client.0.smithi027.stdout:7/889: getdents d17/d32/d46/d83/d77 0 2024-06-06T08:43:52.306 INFO:tasks.workunit.client.0.smithi027.stdout:7/890: write d17/f1b [163270,113727] 0 2024-06-06T08:43:52.308 INFO:tasks.workunit.client.0.smithi027.stdout:3/674: mknod d1/d8/d59/cd1 0 2024-06-06T08:43:52.308 INFO:tasks.workunit.client.0.smithi027.stdout:7/891: mknod d17/d32/d46/d51/d7f/db2/c126 0 2024-06-06T08:43:52.310 INFO:tasks.workunit.client.0.smithi027.stdout:3/675: truncate d1/d8/f92 463878 0 2024-06-06T08:43:52.310 INFO:tasks.workunit.client.0.smithi027.stdout:7/892: symlink d17/d32/d46/d4f/db6/dc8/de8/d116/l127 0 2024-06-06T08:43:52.310 INFO:tasks.workunit.client.0.smithi027.stdout:0/780: sync 2024-06-06T08:43:52.310 INFO:tasks.workunit.client.0.smithi027.stdout:0/781: truncate d15/ffd 864639 0 2024-06-06T08:43:52.311 INFO:tasks.workunit.client.0.smithi027.stdout:3/676: symlink d1/d8/ld2 0 2024-06-06T08:43:52.313 INFO:tasks.workunit.client.0.smithi027.stdout:0/782: symlink d15/d18/dbe/d5c/d83/l10e 0 2024-06-06T08:43:52.313 INFO:tasks.workunit.client.0.smithi027.stdout:0/783: write d15/d18/d64/fed [276903,79339] 0 2024-06-06T08:43:52.313 INFO:tasks.workunit.client.0.smithi027.stdout:0/784: dread - d15/d18/dbe/d5c/dfa/f10a zero size 2024-06-06T08:43:52.316 INFO:tasks.workunit.client.0.smithi027.stdout:3/677: rename d1/d1f/d4d/f52 to d1/d90/d44/d54/db4/fd3 0 2024-06-06T08:43:52.316 INFO:tasks.workunit.client.0.smithi027.stdout:5/931: dwrite d0/da/db/d1a/f6f [0,4194304] 0 2024-06-06T08:43:52.317 INFO:tasks.workunit.client.0.smithi027.stdout:0/785: unlink d15/d18/d23/d38/cab 0 2024-06-06T08:43:52.319 INFO:tasks.workunit.client.0.smithi027.stdout:7/893: link d17/d32/d46/d4f/d5e/f67 d17/d32/d46/d4f/db6/f128 0 2024-06-06T08:43:52.322 INFO:tasks.workunit.client.0.smithi027.stdout:3/678: unlink d1/d8/d59/l7b 0 2024-06-06T08:43:52.322 INFO:tasks.workunit.client.0.smithi027.stdout:3/679: read - d1/d1f/d4d/fb9 zero size 2024-06-06T08:43:52.323 INFO:tasks.workunit.client.0.smithi027.stdout:5/932: mknod d0/da/db/d1a/da3/de5/d107/c12b 0 2024-06-06T08:43:52.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/746: dwrite d6/fbd [0,4194304] 0 2024-06-06T08:43:52.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/747: getdents d6/dd/dd9/d1d/d4d/ddc 0 2024-06-06T08:43:52.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/748: dread - d6/dd/dd9/d3f/d48/d83/fce zero size 2024-06-06T08:43:52.329 INFO:tasks.workunit.client.0.smithi027.stdout:6/899: dwrite d4/de/d45/f75 [0,4194304] 0 2024-06-06T08:43:52.331 INFO:tasks.workunit.client.0.smithi027.stdout:3/680: write d1/d90/f77 [3896938,36576] 0 2024-06-06T08:43:52.332 INFO:tasks.workunit.client.0.smithi027.stdout:3/681: write d1/d8/d22/d2d/f7f [5071531,101957] 0 2024-06-06T08:43:52.336 INFO:tasks.workunit.client.0.smithi027.stdout:5/933: truncate d0/da/db/d1a/d5a/f9b 691296 0 2024-06-06T08:43:52.338 INFO:tasks.workunit.client.0.smithi027.stdout:3/682: fdatasync d1/d1f/d25/f76 0 2024-06-06T08:43:52.341 INFO:tasks.workunit.client.0.smithi027.stdout:7/894: write d17/d32/d61/d75/fec [193272,100644] 0 2024-06-06T08:43:52.344 INFO:tasks.workunit.client.0.smithi027.stdout:5/934: rename d0/da/db/d46/d4c/d70 to d0/da/db/d1a/d7e/d12c 0 2024-06-06T08:43:52.344 INFO:tasks.workunit.client.0.smithi027.stdout:0/786: dwrite d15/d18/dbe/d5d/ff1 [0,4194304] 0 2024-06-06T08:43:52.346 INFO:tasks.workunit.client.0.smithi027.stdout:3/683: rename d1/d8/f8a to d1/d1f/d25/fd4 0 2024-06-06T08:43:52.347 INFO:tasks.workunit.client.0.smithi027.stdout:3/684: write d1/d1f/d25/f76 [2680393,114255] 0 2024-06-06T08:43:52.347 INFO:tasks.workunit.client.0.smithi027.stdout:2/749: dwrite f5 [4194304,4194304] 0 2024-06-06T08:43:52.351 INFO:tasks.workunit.client.0.smithi027.stdout:2/750: symlink d6/dd/dd9/d3f/d77/d55/d69/d82/lf6 0 2024-06-06T08:43:52.351 INFO:tasks.workunit.client.0.smithi027.stdout:7/895: write d17/d32/d46/d4f/db6/dc8/de0/f112 [3003373,45681] 0 2024-06-06T08:43:52.351 INFO:tasks.workunit.client.0.smithi027.stdout:7/896: dread - d17/d32/d61/f118 zero size 2024-06-06T08:43:52.351 INFO:tasks.workunit.client.0.smithi027.stdout:0/787: mknod d15/d18/d108/c10f 0 2024-06-06T08:43:52.352 INFO:tasks.workunit.client.0.smithi027.stdout:2/751: creat d6/dd/dd9/d1d/d4d/ff7 x:0 0 0 2024-06-06T08:43:52.353 INFO:tasks.workunit.client.0.smithi027.stdout:2/752: fdatasync d6/dd/dd9/f25 0 2024-06-06T08:43:52.355 INFO:tasks.workunit.client.0.smithi027.stdout:6/900: read d4/d125/d1c/d52/fa9 [589030,121116] 0 2024-06-06T08:43:52.356 INFO:tasks.workunit.client.0.smithi027.stdout:3/685: rename d1/d1f/d5b to d1/d1f/d4d/dd5 0 2024-06-06T08:43:52.372 INFO:tasks.workunit.client.0.smithi027.stdout:5/935: dwrite d0/da/db/d46/d77/f9d [4194304,4194304] 0 2024-06-06T08:43:52.383 INFO:tasks.workunit.client.0.smithi027.stdout:0/788: dwrite d15/d18/d64/fbd [0,4194304] 0 2024-06-06T08:43:52.386 INFO:tasks.workunit.client.0.smithi027.stdout:7/897: dwrite d17/d32/d46/d51/d52/d70/dce/f11c [0,4194304] 0 2024-06-06T08:43:52.386 INFO:tasks.workunit.client.0.smithi027.stdout:0/789: dread d15/d18/d23/d3c/f67 [0,4194304] 0 2024-06-06T08:43:52.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/753: dwrite d6/dd/f24 [0,4194304] 0 2024-06-06T08:43:52.389 INFO:tasks.workunit.client.0.smithi027.stdout:3/686: write d1/d90/d44/d54/db4/fd3 [9067,119727] 0 2024-06-06T08:43:52.389 INFO:tasks.workunit.client.0.smithi027.stdout:3/687: write d1/d1f/d45/fc7 [925055,74522] 0 2024-06-06T08:43:52.393 INFO:tasks.workunit.client.0.smithi027.stdout:3/688: mkdir d1/d1f/d45/d5f/dd6 0 2024-06-06T08:43:52.394 INFO:tasks.workunit.client.0.smithi027.stdout:5/936: dread d0/da/db/d46/d7f/d81/fd2 [0,4194304] 0 2024-06-06T08:43:52.394 INFO:tasks.workunit.client.0.smithi027.stdout:5/937: fdatasync d0/da/d37/d2b/f5b 0 2024-06-06T08:43:52.394 INFO:tasks.workunit.client.0.smithi027.stdout:3/689: readlink d1/d90/d44/l4b 0 2024-06-06T08:43:52.394 INFO:tasks.workunit.client.0.smithi027.stdout:3/690: stat d1/d1f/d45/fc7 0 2024-06-06T08:43:52.395 INFO:tasks.workunit.client.0.smithi027.stdout:5/938: truncate d0/da/d20/fa1 229590 0 2024-06-06T08:43:52.396 INFO:tasks.workunit.client.0.smithi027.stdout:2/754: dread f5 [0,4194304] 0 2024-06-06T08:43:52.401 INFO:tasks.workunit.client.0.smithi027.stdout:5/939: rename d0/da/db/d1a/d7e/c98 to d0/da/db/d1a/da3/de5/d107/c12d 0 2024-06-06T08:43:52.403 INFO:tasks.workunit.client.0.smithi027.stdout:0/790: dwrite d15/dae/fbf [0,4194304] 0 2024-06-06T08:43:52.407 INFO:tasks.workunit.client.0.smithi027.stdout:2/755: link d6/d3b/cc4 d6/dd/dd9/d1d/dbc/cf8 0 2024-06-06T08:43:52.408 INFO:tasks.workunit.client.0.smithi027.stdout:7/898: dwrite d17/d32/d46/d51/d52/d76/f108 [0,4194304] 0 2024-06-06T08:43:52.410 INFO:tasks.workunit.client.0.smithi027.stdout:0/791: dread d15/d18/d23/f2e [0,4194304] 0 2024-06-06T08:43:52.410 INFO:tasks.workunit.client.0.smithi027.stdout:0/792: fdatasync d15/fa1 0 2024-06-06T08:43:52.415 INFO:tasks.workunit.client.0.smithi027.stdout:5/940: write d0/da/d37/d2b/f4a [495261,253] 0 2024-06-06T08:43:52.419 INFO:tasks.workunit.client.0.smithi027.stdout:2/756: read d6/f14 [478392,113441] 0 2024-06-06T08:43:52.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/757: fsync d6/dd/f18 0 2024-06-06T08:43:52.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/758: truncate d6/d1e/f27 479730 0 2024-06-06T08:43:52.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/941: dread d0/da/db/d1a/d5a/f9b [0,4194304] 0 2024-06-06T08:43:52.422 INFO:tasks.workunit.client.0.smithi027.stdout:7/899: dread d17/fa4 [0,4194304] 0 2024-06-06T08:43:52.424 INFO:tasks.workunit.client.0.smithi027.stdout:2/759: link d6/d1e/d2b/d5a/f84 d6/d1e/d2b/d5a/ff9 0 2024-06-06T08:43:52.427 INFO:tasks.workunit.client.0.smithi027.stdout:2/760: mknod d6/d3b/d61/cfa 0 2024-06-06T08:43:52.427 INFO:tasks.workunit.client.0.smithi027.stdout:5/942: link d0/da/db/d46/d7f/lbd d0/da/db/d1a/dc6/l12e 0 2024-06-06T08:43:52.428 INFO:tasks.workunit.client.0.smithi027.stdout:7/900: truncate d17/d32/d46/d51/d52/d70/d87/dc0/f10c 1303639 0 2024-06-06T08:43:52.430 INFO:tasks.workunit.client.0.smithi027.stdout:7/901: creat d17/d32/d46/d51/d7f/db2/f129 x:0 0 0 2024-06-06T08:43:52.431 INFO:tasks.workunit.client.0.smithi027.stdout:2/761: mkdir d6/d1e/dfb 0 2024-06-06T08:43:52.433 INFO:tasks.workunit.client.0.smithi027.stdout:5/943: unlink d0/da/db/d1a/f66 0 2024-06-06T08:43:52.434 INFO:tasks.workunit.client.0.smithi027.stdout:5/944: write d0/da/db/d1a/f6f [4720718,75613] 0 2024-06-06T08:43:52.440 INFO:tasks.workunit.client.0.smithi027.stdout:5/945: rename d0/da/d20/l101 to d0/da/db/d46/d7f/l12f 0 2024-06-06T08:43:52.440 INFO:tasks.workunit.client.0.smithi027.stdout:5/946: dread - d0/da/db/d1a/d7e/d12c/fc9 zero size 2024-06-06T08:43:52.444 INFO:tasks.workunit.client.0.smithi027.stdout:2/762: read d6/dd/dd9/f22 [2779520,125389] 0 2024-06-06T08:43:52.450 INFO:tasks.workunit.client.0.smithi027.stdout:5/947: write d0/da/f13 [5569117,25948] 0 2024-06-06T08:43:52.453 INFO:tasks.workunit.client.0.smithi027.stdout:7/902: dwrite f11 [0,4194304] 0 2024-06-06T08:43:52.457 INFO:tasks.workunit.client.0.smithi027.stdout:1/747: sync 2024-06-06T08:43:52.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/869: sync 2024-06-06T08:43:52.457 INFO:tasks.workunit.client.0.smithi027.stdout:8/648: sync 2024-06-06T08:43:52.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/870: fsync d0/d63/ff6 0 2024-06-06T08:43:52.459 INFO:tasks.workunit.client.0.smithi027.stdout:4/871: mkdir d0/d1/d6/d8/d11f 0 2024-06-06T08:43:52.463 INFO:tasks.workunit.client.0.smithi027.stdout:1/748: rename dc/de/d14 to dc/d67/d69/d32/d37/d56/df1 0 2024-06-06T08:43:52.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/749: chown dc/d67/d69/d32/d37/d56/df1/l8d 439554 1 2024-06-06T08:43:52.467 INFO:tasks.workunit.client.0.smithi027.stdout:5/948: read d0/f1 [7118907,117993] 0 2024-06-06T08:43:52.467 INFO:tasks.workunit.client.0.smithi027.stdout:5/949: chown d0/da/db/d1a/dc6 150256629 1 2024-06-06T08:43:52.468 INFO:tasks.workunit.client.0.smithi027.stdout:5/950: mknod d0/da/db/d1a/da3/c130 0 2024-06-06T08:43:52.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/872: dread d0/d1/d6/f106 [0,4194304] 0 2024-06-06T08:43:52.469 INFO:tasks.workunit.client.0.smithi027.stdout:8/649: write d7/dc/d37/f80 [536799,40960] 0 2024-06-06T08:43:52.469 INFO:tasks.workunit.client.0.smithi027.stdout:8/650: dread - d7/d9c/fa9 zero size 2024-06-06T08:43:52.471 INFO:tasks.workunit.client.0.smithi027.stdout:4/873: mknod d0/d1d/d34/c120 0 2024-06-06T08:43:52.472 INFO:tasks.workunit.client.0.smithi027.stdout:8/651: rename d7/d9c/fa9 to d7/dc/d36/d83/d60/d93/fd5 0 2024-06-06T08:43:52.473 INFO:tasks.workunit.client.0.smithi027.stdout:8/652: symlink d7/dc/ld6 0 2024-06-06T08:43:52.474 INFO:tasks.workunit.client.0.smithi027.stdout:4/874: creat d0/d1/d6/d8/d82/d8b/dc9/da3/d10b/f121 x:0 0 0 2024-06-06T08:43:52.474 INFO:tasks.workunit.client.0.smithi027.stdout:8/653: write d7/d13/f24 [1029206,85248] 0 2024-06-06T08:43:52.476 INFO:tasks.workunit.client.0.smithi027.stdout:8/654: mkdir d7/dc/d36/d41/dbf/dd7 0 2024-06-06T08:43:52.476 INFO:tasks.workunit.client.0.smithi027.stdout:8/655: fdatasync d7/dc/d36/d3c/f79 0 2024-06-06T08:43:52.477 INFO:tasks.workunit.client.0.smithi027.stdout:5/951: dread d0/da/d37/d2b/f65 [0,4194304] 0 2024-06-06T08:43:52.477 INFO:tasks.workunit.client.0.smithi027.stdout:5/952: readlink d0/da/db/d46/d7f/lbd 0 2024-06-06T08:43:52.477 INFO:tasks.workunit.client.0.smithi027.stdout:5/953: chown d0/da/db/d46/d7f/d81/d97/lb7 4 1 2024-06-06T08:43:52.477 INFO:tasks.workunit.client.0.smithi027.stdout:5/954: readlink d0/da/db/d1a/da3/le6 0 2024-06-06T08:43:52.478 INFO:tasks.workunit.client.0.smithi027.stdout:7/903: dwrite d17/d32/d6c/dc2/fc5 [0,4194304] 0 2024-06-06T08:43:52.478 INFO:tasks.workunit.client.0.smithi027.stdout:8/656: symlink d7/dc/d36/d83/d58/ld8 0 2024-06-06T08:43:52.478 INFO:tasks.workunit.client.0.smithi027.stdout:8/657: write d7/dc/d36/d83/d60/d93/dac/fb2 [864857,69397] 0 2024-06-06T08:43:52.478 INFO:tasks.workunit.client.0.smithi027.stdout:8/658: write d7/dc/d36/d83/fbc [1658322,95410] 0 2024-06-06T08:43:52.480 INFO:tasks.workunit.client.0.smithi027.stdout:5/955: rmdir d0/da/db/d1a/d7e/d12c 39 2024-06-06T08:43:52.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/956: mkdir d0/da/db/d46/d7f/d81/d97/db0/dbc/d131 0 2024-06-06T08:43:52.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/957: write d0/da/db/d46/d54/f84 [1220321,30802] 0 2024-06-06T08:43:52.484 INFO:tasks.workunit.client.0.smithi027.stdout:1/750: dwrite dc/da0/fdb [0,4194304] 0 2024-06-06T08:43:52.484 INFO:tasks.workunit.client.0.smithi027.stdout:4/875: dread d0/d1d/d34/d7f/fcd [0,4194304] 0 2024-06-06T08:43:52.487 INFO:tasks.workunit.client.0.smithi027.stdout:7/904: symlink d17/d32/d6c/l12a 0 2024-06-06T08:43:52.490 INFO:tasks.workunit.client.0.smithi027.stdout:1/751: rmdir dc/d67/d69/d32/d71 39 2024-06-06T08:43:52.490 INFO:tasks.workunit.client.0.smithi027.stdout:1/752: fdatasync dc/d9d/fa2 0 2024-06-06T08:43:52.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/958: truncate d0/da/d37/d2b/f5b 3347299 0 2024-06-06T08:43:52.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/959: dread - d0/da/db/d1a/dc6/fc7 zero size 2024-06-06T08:43:52.500 INFO:tasks.workunit.client.0.smithi027.stdout:1/753: write f5 [2009030,29383] 0 2024-06-06T08:43:52.500 INFO:tasks.workunit.client.0.smithi027.stdout:1/754: stat dc/d67/d69/d32/d37/d98/ce5 0 2024-06-06T08:43:52.501 INFO:tasks.workunit.client.0.smithi027.stdout:7/905: dread d17/f26 [4194304,4194304] 0 2024-06-06T08:43:52.507 INFO:tasks.workunit.client.0.smithi027.stdout:5/960: write d0/da/db/d46/d7f/f10f [2973712,72769] 0 2024-06-06T08:43:52.507 INFO:tasks.workunit.client.0.smithi027.stdout:5/961: rename d0 to d0/da/db/d46/d7f/d81/d97/db0/dbc/dc4/d132 22 2024-06-06T08:43:52.507 INFO:tasks.workunit.client.0.smithi027.stdout:5/962: chown d0/da/db/d46/d54/c7a 15 1 2024-06-06T08:43:52.509 INFO:tasks.workunit.client.0.smithi027.stdout:4/876: dwrite d0/d1/d6/d4b/f80 [0,4194304] 0 2024-06-06T08:43:52.512 INFO:tasks.workunit.client.0.smithi027.stdout:4/877: getdents d0/d1d/dd0 0 2024-06-06T08:43:52.514 INFO:tasks.workunit.client.0.smithi027.stdout:4/878: creat d0/d1d/d34/f122 x:0 0 0 2024-06-06T08:43:52.514 INFO:tasks.workunit.client.0.smithi027.stdout:4/879: stat d0/d1/d6/d8/d107/d9c/dbe 0 2024-06-06T08:43:52.514 INFO:tasks.workunit.client.0.smithi027.stdout:8/659: dwrite d7/dc/d36/d83/d60/fad [0,4194304] 0 2024-06-06T08:43:52.515 INFO:tasks.workunit.client.0.smithi027.stdout:7/906: dread d17/f2e [0,4194304] 0 2024-06-06T08:43:52.515 INFO:tasks.workunit.client.0.smithi027.stdout:7/907: fdatasync d17/d32/d46/d4f/db6/f117 0 2024-06-06T08:43:52.517 INFO:tasks.workunit.client.0.smithi027.stdout:8/660: creat d7/dc/d36/d41/dbf/dd7/fd9 x:0 0 0 2024-06-06T08:43:52.517 INFO:tasks.workunit.client.0.smithi027.stdout:8/661: stat d7/dc/d36/d83/d60/d93/dac/c6c 0 2024-06-06T08:43:52.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/755: dwrite dc/de/d3d/d4a/d76/fc3 [0,4194304] 0 2024-06-06T08:43:52.521 INFO:tasks.workunit.client.0.smithi027.stdout:4/880: rmdir d0/d1/d6/d8/d107/d9c 39 2024-06-06T08:43:52.522 INFO:tasks.workunit.client.0.smithi027.stdout:8/662: creat d7/dc/d37/fda x:0 0 0 2024-06-06T08:43:52.527 INFO:tasks.workunit.client.0.smithi027.stdout:1/756: fsync dc/de/d22/f82 0 2024-06-06T08:43:52.529 INFO:tasks.workunit.client.0.smithi027.stdout:4/881: truncate d0/d1d/d2a/d98/fa8 1617074 0 2024-06-06T08:43:52.537 INFO:tasks.workunit.client.0.smithi027.stdout:8/663: truncate d7/dc/d36/d41/f34 118866 0 2024-06-06T08:43:52.537 INFO:tasks.workunit.client.0.smithi027.stdout:8/664: stat d7/dc/d36/d41/d85/cc5 0 2024-06-06T08:43:52.537 INFO:tasks.workunit.client.0.smithi027.stdout:1/757: getdents dc/d67/d69/d32/d37/d56/df1/d4e 0 2024-06-06T08:43:52.538 INFO:tasks.workunit.client.0.smithi027.stdout:1/758: chown dc/d67/d69/d32/d37/d56/df1/d80/da6/laa 1170799 1 2024-06-06T08:43:52.538 INFO:tasks.workunit.client.0.smithi027.stdout:7/908: unlink d17/c20 0 2024-06-06T08:43:52.538 INFO:tasks.workunit.client.0.smithi027.stdout:7/909: write d17/f60 [4923222,49284] 0 2024-06-06T08:43:52.538 INFO:tasks.workunit.client.0.smithi027.stdout:8/665: dread d7/d2f/fb8 [0,4194304] 0 2024-06-06T08:43:52.539 INFO:tasks.workunit.client.0.smithi027.stdout:1/759: getdents dc/d67/d69 0 2024-06-06T08:43:52.539 INFO:tasks.workunit.client.0.smithi027.stdout:7/910: mknod d17/d32/d46/d83/d77/d8c/d91/daa/d103/c12b 0 2024-06-06T08:43:52.541 INFO:tasks.workunit.client.0.smithi027.stdout:1/760: getdents dc/d6d 0 2024-06-06T08:43:52.541 INFO:tasks.workunit.client.0.smithi027.stdout:1/761: fsync dc/d67/d69/d32/d37/d56/de1/fe9 0 2024-06-06T08:43:52.545 INFO:tasks.workunit.client.0.smithi027.stdout:1/762: write dc/d67/d69/d32/fed [1497256,57940] 0 2024-06-06T08:43:52.548 INFO:tasks.workunit.client.0.smithi027.stdout:4/882: link d0/d1d/c61 d0/d1/d6/d8/d82/d8b/c123 0 2024-06-06T08:43:52.553 INFO:tasks.workunit.client.0.smithi027.stdout:4/883: dread d0/d1/d6/d8/fe2 [0,4194304] 0 2024-06-06T08:43:52.555 INFO:tasks.workunit.client.0.smithi027.stdout:4/884: mknod d0/d1d/d60/dc6/c124 0 2024-06-06T08:43:52.557 INFO:tasks.workunit.client.0.smithi027.stdout:4/885: symlink d0/d1/d6/d8/d107/dad/l125 0 2024-06-06T08:43:52.571 INFO:tasks.workunit.client.0.smithi027.stdout:4/886: dwrite d0/d1/d6/fc2 [0,4194304] 0 2024-06-06T08:43:52.574 INFO:tasks.workunit.client.0.smithi027.stdout:4/887: symlink d0/d1/d6/d8/d82/d8b/dc9/da3/d10b/l126 0 2024-06-06T08:43:52.586 INFO:tasks.workunit.client.0.smithi027.stdout:4/888: rename d0/c6c to d0/d1/d6/d8/d82/d8b/c127 0 2024-06-06T08:43:52.586 INFO:tasks.workunit.client.0.smithi027.stdout:4/889: chown d0/d1/d6/f2f 239159 1 2024-06-06T08:43:52.586 INFO:tasks.workunit.client.0.smithi027.stdout:4/890: chown d0/d1/d6/d8/d82/d8b/dc9/f11a 314329799 1 2024-06-06T08:43:52.587 INFO:tasks.workunit.client.0.smithi027.stdout:9/782: sync 2024-06-06T08:43:52.596 INFO:tasks.workunit.client.0.smithi027.stdout:9/783: write d3/d4e/d75/dc/f2d [4134133,127358] 0 2024-06-06T08:43:52.597 INFO:tasks.workunit.client.0.smithi027.stdout:9/784: read - d3/fe3 zero size 2024-06-06T08:43:52.597 INFO:tasks.workunit.client.0.smithi027.stdout:9/785: fsync d3/d4e/d75/dc/dd1/ff3 0 2024-06-06T08:43:52.599 INFO:tasks.workunit.client.0.smithi027.stdout:9/786: mkdir d3/d95/d118 0 2024-06-06T08:43:52.599 INFO:tasks.workunit.client.0.smithi027.stdout:9/787: write d3/d95/f39 [463083,34184] 0 2024-06-06T08:43:52.599 INFO:tasks.workunit.client.0.smithi027.stdout:9/788: rmdir d3/d4e/d75/dc/df6/dff 39 2024-06-06T08:43:52.601 INFO:tasks.workunit.client.0.smithi027.stdout:9/789: mknod d3/d95/d23/d78/ddf/de5/c119 0 2024-06-06T08:43:52.763 INFO:tasks.workunit.client.0.smithi027.stdout:6/901: sync 2024-06-06T08:43:52.765 INFO:tasks.workunit.client.0.smithi027.stdout:6/902: unlink d4/da2/fd9 0 2024-06-06T08:43:52.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/903: dwrite d4/de/d45/f115 [0,4194304] 0 2024-06-06T08:43:52.780 INFO:tasks.workunit.client.0.smithi027.stdout:6/904: creat d4/d9/dc5/f127 x:0 0 0 2024-06-06T08:43:52.801 INFO:tasks.workunit.client.0.smithi027.stdout:6/905: dwrite d4/d9/fad [0,4194304] 0 2024-06-06T08:43:52.801 INFO:tasks.workunit.client.0.smithi027.stdout:6/906: readlink d4/d125/d8f/la7 0 2024-06-06T08:43:52.809 INFO:tasks.workunit.client.0.smithi027.stdout:0/793: sync 2024-06-06T08:43:52.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/691: sync 2024-06-06T08:43:52.810 INFO:tasks.workunit.client.0.smithi027.stdout:3/692: creat d1/d8/d22/d2d/fd7 x:0 0 0 2024-06-06T08:43:52.811 INFO:tasks.workunit.client.0.smithi027.stdout:3/693: creat d1/d1f/fd8 x:0 0 0 2024-06-06T08:43:52.811 INFO:tasks.workunit.client.0.smithi027.stdout:3/694: stat d1/d1f/d45/d5f 0 2024-06-06T08:43:52.813 INFO:tasks.workunit.client.0.smithi027.stdout:6/907: dread d4/f5 [4194304,4194304] 0 2024-06-06T08:43:52.814 INFO:tasks.workunit.client.0.smithi027.stdout:6/908: creat d4/d9/d6e/f128 x:0 0 0 2024-06-06T08:43:52.814 INFO:tasks.workunit.client.0.smithi027.stdout:6/909: chown d4/d1a/d2e/d44 1080698187 1 2024-06-06T08:43:52.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/910: creat d4/d125/d18/f129 x:0 0 0 2024-06-06T08:43:52.816 INFO:tasks.workunit.client.0.smithi027.stdout:3/695: write d1/d1f/d25/f8d [1099520,23788] 0 2024-06-06T08:43:52.816 INFO:tasks.workunit.client.0.smithi027.stdout:3/696: chown d1/d1f/d45/f69 479351 1 2024-06-06T08:43:52.816 INFO:tasks.workunit.client.0.smithi027.stdout:3/697: chown d1/d1f/laf 3492 1 2024-06-06T08:43:52.816 INFO:tasks.workunit.client.0.smithi027.stdout:3/698: truncate d1/d90/fcb 973778 0 2024-06-06T08:43:52.819 INFO:tasks.workunit.client.0.smithi027.stdout:0/794: dread d15/d18/dbe/d5c/d83/f86 [0,4194304] 0 2024-06-06T08:43:52.819 INFO:tasks.workunit.client.0.smithi027.stdout:0/795: readlink d15/d18/d23/d28/d2a/d2d/l47 0 2024-06-06T08:43:52.821 INFO:tasks.workunit.client.0.smithi027.stdout:0/796: mkdir d15/d18/d40/d8c/d110 0 2024-06-06T08:43:52.822 INFO:tasks.workunit.client.0.smithi027.stdout:0/797: rmdir d15/d18/d23/d6f 39 2024-06-06T08:43:52.822 INFO:tasks.workunit.client.0.smithi027.stdout:0/798: truncate d15/d18/f81 458311 0 2024-06-06T08:43:52.824 INFO:tasks.workunit.client.0.smithi027.stdout:6/911: write d4/d1a/d2e/f5c [1544068,86025] 0 2024-06-06T08:43:52.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/799: link d15/d18/c1f d15/d18/d23/d6f/c111 0 2024-06-06T08:43:52.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/800: write d15/d18/d23/d28/d31/f96 [1015333,87144] 0 2024-06-06T08:43:52.825 INFO:tasks.workunit.client.0.smithi027.stdout:0/801: readlink d15/d18/d23/d38/l52 0 2024-06-06T08:43:52.825 INFO:tasks.workunit.client.0.smithi027.stdout:0/802: chown d15/d18/d64/l101 686544952 1 2024-06-06T08:43:52.828 INFO:tasks.workunit.client.0.smithi027.stdout:6/912: unlink d4/d13/l8b 0 2024-06-06T08:43:52.828 INFO:tasks.workunit.client.0.smithi027.stdout:6/913: rename d4 to d4/d125/d1c/dd2/d12a 22 2024-06-06T08:43:52.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/914: creat d4/d1a/d2e/d44/dce/f12b x:0 0 0 2024-06-06T08:43:52.832 INFO:tasks.workunit.client.0.smithi027.stdout:6/915: creat d4/de/d45/d6a/f12c x:0 0 0 2024-06-06T08:43:52.834 INFO:tasks.workunit.client.0.smithi027.stdout:3/699: dwrite d1/f1e [0,4194304] 0 2024-06-06T08:43:52.837 INFO:tasks.workunit.client.0.smithi027.stdout:3/700: rename d1/d8/d22/d27/d50/fbf to d1/d8/d59/fd9 0 2024-06-06T08:43:52.838 INFO:tasks.workunit.client.0.smithi027.stdout:3/701: mknod d1/d1f/d25/cda 0 2024-06-06T08:43:52.840 INFO:tasks.workunit.client.0.smithi027.stdout:3/702: unlink d1/d8/f5c 0 2024-06-06T08:43:52.845 INFO:tasks.workunit.client.0.smithi027.stdout:0/803: dwrite d15/d18/dbe/f71 [0,4194304] 0 2024-06-06T08:43:52.850 INFO:tasks.workunit.client.0.smithi027.stdout:6/916: dwrite d4/f29 [0,4194304] 0 2024-06-06T08:43:52.861 INFO:tasks.workunit.client.0.smithi027.stdout:3/703: dwrite d1/d8/d22/d27/f78 [0,4194304] 0 2024-06-06T08:43:52.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/804: dwrite d15/f22 [0,4194304] 0 2024-06-06T08:43:52.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/805: dread - d15/d18/d23/d38/d68/d85/ff3 zero size 2024-06-06T08:43:52.866 INFO:tasks.workunit.client.0.smithi027.stdout:0/806: mknod d15/d18/c112 0 2024-06-06T08:43:52.867 INFO:tasks.workunit.client.0.smithi027.stdout:6/917: dwrite d4/d125/d18/fd0 [0,4194304] 0 2024-06-06T08:43:52.869 INFO:tasks.workunit.client.0.smithi027.stdout:3/704: rename d1/d1f/d25 to d1/d90/d6d/dac/ddb 0 2024-06-06T08:43:52.869 INFO:tasks.workunit.client.0.smithi027.stdout:5/963: sync 2024-06-06T08:43:52.869 INFO:tasks.workunit.client.0.smithi027.stdout:8/666: sync 2024-06-06T08:43:52.869 INFO:tasks.workunit.client.0.smithi027.stdout:2/763: sync 2024-06-06T08:43:52.869 INFO:tasks.workunit.client.0.smithi027.stdout:9/790: sync 2024-06-06T08:43:52.869 INFO:tasks.workunit.client.0.smithi027.stdout:7/911: sync 2024-06-06T08:43:52.871 INFO:tasks.workunit.client.0.smithi027.stdout:5/964: symlink d0/da/db/d1a/d5a/de0/l133 0 2024-06-06T08:43:52.871 INFO:tasks.workunit.client.0.smithi027.stdout:5/965: getdents d0/da/db/d46/d7f/d81/d97/db0/dbc/dd9 0 2024-06-06T08:43:52.871 INFO:tasks.workunit.client.0.smithi027.stdout:5/966: fdatasync d0/da/db/d1a/da3/fa6 0 2024-06-06T08:43:52.871 INFO:tasks.workunit.client.0.smithi027.stdout:9/791: mkdir d3/d95/d23/d78/d11a 0 2024-06-06T08:43:52.872 INFO:tasks.workunit.client.0.smithi027.stdout:3/705: mkdir d1/d1f/d4d/ddc 0 2024-06-06T08:43:52.873 INFO:tasks.workunit.client.0.smithi027.stdout:0/807: link d15/d18/d23/d28/d2a/d2d/c2f d15/d18/d23/d28/d2a/c113 0 2024-06-06T08:43:52.873 INFO:tasks.workunit.client.0.smithi027.stdout:0/808: write d15/fa1 [2654084,94911] 0 2024-06-06T08:43:52.873 INFO:tasks.workunit.client.0.smithi027.stdout:0/809: dread - d15/d18/d23/d28/d31/f5e zero size 2024-06-06T08:43:52.873 INFO:tasks.workunit.client.0.smithi027.stdout:0/810: chown d15/da7/ff8 121 1 2024-06-06T08:43:52.874 INFO:tasks.workunit.client.0.smithi027.stdout:5/967: rmdir d0/d6a 39 2024-06-06T08:43:52.874 INFO:tasks.workunit.client.0.smithi027.stdout:0/811: getdents d15/d18/d23/d3c 0 2024-06-06T08:43:52.874 INFO:tasks.workunit.client.0.smithi027.stdout:0/812: fsync d15/f22 0 2024-06-06T08:43:52.875 INFO:tasks.workunit.client.0.smithi027.stdout:5/968: mknod d0/da/db/d46/d54/c134 0 2024-06-06T08:43:52.877 INFO:tasks.workunit.client.0.smithi027.stdout:5/969: creat d0/da/db/d46/d7f/d81/f135 x:0 0 0 2024-06-06T08:43:52.877 INFO:tasks.workunit.client.0.smithi027.stdout:5/970: stat d0/da/db/l33 0 2024-06-06T08:43:52.877 INFO:tasks.workunit.client.0.smithi027.stdout:5/971: write d0/da/ffb [296974,51024] 0 2024-06-06T08:43:52.878 INFO:tasks.workunit.client.0.smithi027.stdout:6/918: rename d4/d13/dba/df5 to d4/d125/d12d 0 2024-06-06T08:43:52.879 INFO:tasks.workunit.client.0.smithi027.stdout:5/972: creat d0/da/d47/f136 x:0 0 0 2024-06-06T08:43:52.881 INFO:tasks.workunit.client.0.smithi027.stdout:8/667: dread d7/dc/d36/d83/d60/fad [0,4194304] 0 2024-06-06T08:43:52.882 INFO:tasks.workunit.client.0.smithi027.stdout:7/912: write d17/d32/d46/d51/f71 [109395,100143] 0 2024-06-06T08:43:52.882 INFO:tasks.workunit.client.0.smithi027.stdout:5/973: creat d0/da/db/d1a/f137 x:0 0 0 2024-06-06T08:43:52.883 INFO:tasks.workunit.client.0.smithi027.stdout:5/974: write d0/da/db/d1a/d5a/de0/fe8 [102073,119212] 0 2024-06-06T08:43:52.883 INFO:tasks.workunit.client.0.smithi027.stdout:2/764: dread d6/d1e/d2b/f2f [0,4194304] 0 2024-06-06T08:43:52.883 INFO:tasks.workunit.client.0.smithi027.stdout:7/913: creat d17/d32/d46/d4f/d5e/d7b/f12c x:0 0 0 2024-06-06T08:43:52.885 INFO:tasks.workunit.client.0.smithi027.stdout:0/813: dread d15/f21 [0,4194304] 0 2024-06-06T08:43:52.890 INFO:tasks.workunit.client.0.smithi027.stdout:8/668: dread d7/dc/d37/f80 [0,4194304] 0 2024-06-06T08:43:52.891 INFO:tasks.workunit.client.0.smithi027.stdout:5/975: write d0/da/d37/d2b/f65 [1020569,127228] 0 2024-06-06T08:43:52.896 INFO:tasks.workunit.client.0.smithi027.stdout:3/706: dwrite d1/d90/d44/d54/f63 [0,4194304] 0 2024-06-06T08:43:52.898 INFO:tasks.workunit.client.0.smithi027.stdout:0/814: write d15/d18/f1c [398478,54485] 0 2024-06-06T08:43:52.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/815: chown d15/d18/d23/d38/d68/l7a 139973 1 2024-06-06T08:43:52.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/816: write d15/d18/d64/fed [622151,125950] 0 2024-06-06T08:43:52.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/817: chown d15/d18/d23/d6f 296 1 2024-06-06T08:43:52.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/818: chown d15/d18/d23/d28/d2a/c46 322 1 2024-06-06T08:43:52.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/819: truncate d15/d18/d23/d28/d2a/d2d/f56 1307469 0 2024-06-06T08:43:52.901 INFO:tasks.workunit.client.0.smithi027.stdout:5/976: dread d0/da/d37/d2b/f67 [0,4194304] 0 2024-06-06T08:43:52.902 INFO:tasks.workunit.client.0.smithi027.stdout:9/792: rename d3/d95/d23/def/d116 to d3/d76/d11b 0 2024-06-06T08:43:52.903 INFO:tasks.workunit.client.0.smithi027.stdout:6/919: read d4/d125/d18/f24 [4268509,39606] 0 2024-06-06T08:43:52.903 INFO:tasks.workunit.client.0.smithi027.stdout:5/977: symlink d0/da/df0/l138 0 2024-06-06T08:43:52.905 INFO:tasks.workunit.client.0.smithi027.stdout:3/707: mknod d1/d1f/d4d/cdd 0 2024-06-06T08:43:52.905 INFO:tasks.workunit.client.0.smithi027.stdout:3/708: chown d1/d8/d22/d27/f14 27934 1 2024-06-06T08:43:52.905 INFO:tasks.workunit.client.0.smithi027.stdout:6/920: truncate d4/d125/d8f/fa3 2147108 0 2024-06-06T08:43:52.905 INFO:tasks.workunit.client.0.smithi027.stdout:6/921: fsync d4/fc3 0 2024-06-06T08:43:52.905 INFO:tasks.workunit.client.0.smithi027.stdout:6/922: read - d4/de/d45/d6a/fbd zero size 2024-06-06T08:43:52.905 INFO:tasks.workunit.client.0.smithi027.stdout:5/978: truncate d0/da/db/d1a/d5a/f9b 1696149 0 2024-06-06T08:43:52.906 INFO:tasks.workunit.client.0.smithi027.stdout:6/923: chown d4/d125/fd3 26485 1 2024-06-06T08:43:52.906 INFO:tasks.workunit.client.0.smithi027.stdout:6/924: chown d4/d1a/d2e/d44/f99 62 1 2024-06-06T08:43:52.906 INFO:tasks.workunit.client.0.smithi027.stdout:3/709: rmdir d1/d8/d22/d27/d79 39 2024-06-06T08:43:52.906 INFO:tasks.workunit.client.0.smithi027.stdout:3/710: read - d1/d8/d22/d2d/fd0 zero size 2024-06-06T08:43:52.906 INFO:tasks.workunit.client.0.smithi027.stdout:3/711: write d1/d8/d22/fc8 [720437,98887] 0 2024-06-06T08:43:52.908 INFO:tasks.workunit.client.0.smithi027.stdout:6/925: creat d4/d9/d6e/f12e x:0 0 0 2024-06-06T08:43:52.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/765: rename d6/d1e/d2b/d5a/fba to d6/dd/dd9/d1d/d4d/ffc 0 2024-06-06T08:43:52.909 INFO:tasks.workunit.client.0.smithi027.stdout:5/979: mkdir d0/da/db/d46/d139 0 2024-06-06T08:43:52.911 INFO:tasks.workunit.client.0.smithi027.stdout:6/926: stat d4/d125/d1c/lc9 0 2024-06-06T08:43:52.911 INFO:tasks.workunit.client.0.smithi027.stdout:6/927: write d4/d125/d1c/f9e [476411,110085] 0 2024-06-06T08:43:52.913 INFO:tasks.workunit.client.0.smithi027.stdout:9/793: dread d3/d4e/d75/dc/d9b/f91 [0,4194304] 0 2024-06-06T08:43:52.914 INFO:tasks.workunit.client.0.smithi027.stdout:9/794: creat d3/d4e/d75/dc/d9b/f11c x:0 0 0 2024-06-06T08:43:52.914 INFO:tasks.workunit.client.0.smithi027.stdout:5/980: unlink d0/l23 0 2024-06-06T08:43:52.916 INFO:tasks.workunit.client.0.smithi027.stdout:8/669: dwrite d7/dc/d36/d3c/f72 [0,4194304] 0 2024-06-06T08:43:52.916 INFO:tasks.workunit.client.0.smithi027.stdout:8/670: chown d7/dc/d36/d83/d60/d93/ccb 1 1 2024-06-06T08:43:52.917 INFO:tasks.workunit.client.0.smithi027.stdout:2/766: link d6/d1e/d2b/d5a/d7b/cc7 d6/dd/dd9/d3f/cfd 0 2024-06-06T08:43:52.918 INFO:tasks.workunit.client.0.smithi027.stdout:9/795: creat d3/d95/d118/f11d x:0 0 0 2024-06-06T08:43:52.918 INFO:tasks.workunit.client.0.smithi027.stdout:6/928: write d4/d125/d1c/f2a [1102296,100944] 0 2024-06-06T08:43:52.918 INFO:tasks.workunit.client.0.smithi027.stdout:6/929: chown d4/d9/d6e/f12e 0 1 2024-06-06T08:43:52.918 INFO:tasks.workunit.client.0.smithi027.stdout:8/671: read d7/dc/d36/d83/d60/d93/dac/d73/fc2 [88385,92180] 0 2024-06-06T08:43:52.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/672: truncate d7/dc/d36/d3c/fca 986044 0 2024-06-06T08:43:52.919 INFO:tasks.workunit.client.0.smithi027.stdout:5/981: symlink d0/da/db/d46/d7f/l13a 0 2024-06-06T08:43:52.919 INFO:tasks.workunit.client.0.smithi027.stdout:5/982: readlink d0/da/df0/l138 0 2024-06-06T08:43:52.920 INFO:tasks.workunit.client.0.smithi027.stdout:8/673: chown d7/dc/d36/l71 770 1 2024-06-06T08:43:52.920 INFO:tasks.workunit.client.0.smithi027.stdout:8/674: fdatasync d7/dc/d36/d83/d60/d93/dac/fb2 0 2024-06-06T08:43:52.921 INFO:tasks.workunit.client.0.smithi027.stdout:9/796: symlink d3/d95/d118/l11e 0 2024-06-06T08:43:52.921 INFO:tasks.workunit.client.0.smithi027.stdout:9/797: fdatasync d3/d95/d23/d36/f37 0 2024-06-06T08:43:52.922 INFO:tasks.workunit.client.0.smithi027.stdout:9/798: symlink d3/d76/l11f 0 2024-06-06T08:43:52.922 INFO:tasks.workunit.client.0.smithi027.stdout:9/799: truncate d3/d95/d23/d36/f94 958093 0 2024-06-06T08:43:52.923 INFO:tasks.workunit.client.0.smithi027.stdout:9/800: dread d3/d95/d23/d36/fd6 [0,4194304] 0 2024-06-06T08:43:52.925 INFO:tasks.workunit.client.0.smithi027.stdout:5/983: write d0/da/d20/fa1 [656758,8074] 0 2024-06-06T08:43:52.926 INFO:tasks.workunit.client.0.smithi027.stdout:5/984: write d0/da/db/d46/d7f/d81/d97/db0/f115 [1171628,24784] 0 2024-06-06T08:43:52.927 INFO:tasks.workunit.client.0.smithi027.stdout:9/801: creat d3/d95/f120 x:0 0 0 2024-06-06T08:43:52.927 INFO:tasks.workunit.client.0.smithi027.stdout:2/767: read d6/d1e/d2b/f58 [1558122,38389] 0 2024-06-06T08:43:52.927 INFO:tasks.workunit.client.0.smithi027.stdout:9/802: getdents d3/d95/db6 0 2024-06-06T08:43:52.928 INFO:tasks.workunit.client.0.smithi027.stdout:5/985: creat d0/da/db/d46/d4c/f13b x:0 0 0 2024-06-06T08:43:52.932 INFO:tasks.workunit.client.0.smithi027.stdout:2/768: dread d6/f2e [0,4194304] 0 2024-06-06T08:43:52.934 INFO:tasks.workunit.client.0.smithi027.stdout:2/769: creat d6/d1e/dfb/ffe x:0 0 0 2024-06-06T08:43:52.934 INFO:tasks.workunit.client.0.smithi027.stdout:7/914: rename d17/d32/d46/d83/d2a/c3c to d17/d32/d46/d83/d2a/d3b/c12d 0 2024-06-06T08:43:52.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:52 smithi027 ceph-mon[32335]: pgmap v111: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 250 op/s 2024-06-06T08:43:52.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/770: rmdir d6/dd/dd9/d3f/d48/d83/de3 39 2024-06-06T08:43:52.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/771: getdents d6/d1e/d2b/d5a/db7/de2 0 2024-06-06T08:43:52.936 INFO:tasks.workunit.client.0.smithi027.stdout:0/820: rename d15/d18/d23/d3c/f67 to d15/d18/d23/d38/d68/f114 0 2024-06-06T08:43:52.937 INFO:tasks.workunit.client.0.smithi027.stdout:0/821: creat d15/d18/d40/d8c/d9c/f115 x:0 0 0 2024-06-06T08:43:52.940 INFO:tasks.workunit.client.0.smithi027.stdout:2/772: creat d6/dd/dd9/d1d/d4d/fff x:0 0 0 2024-06-06T08:43:52.940 INFO:tasks.workunit.client.0.smithi027.stdout:2/773: truncate d6/fd3 988461 0 2024-06-06T08:43:52.940 INFO:tasks.workunit.client.0.smithi027.stdout:2/774: chown d6/d1e/d2b/d5a/db7 1204948 1 2024-06-06T08:43:52.942 INFO:tasks.workunit.client.0.smithi027.stdout:3/712: rename d1/d90/f42 to d1/d1f/d4d/ddc/fde 0 2024-06-06T08:43:52.942 INFO:tasks.workunit.client.0.smithi027.stdout:3/713: stat d1/f29 0 2024-06-06T08:43:52.944 INFO:tasks.workunit.client.0.smithi027.stdout:2/775: getdents d6/dd/dd9/d3f/d48/d83 0 2024-06-06T08:43:52.945 INFO:tasks.workunit.client.0.smithi027.stdout:9/803: dwrite d3/f4 [0,4194304] 0 2024-06-06T08:43:52.946 INFO:tasks.workunit.client.0.smithi027.stdout:8/675: rmdir d7/dc/d36/d41 39 2024-06-06T08:43:52.946 INFO:tasks.workunit.client.0.smithi027.stdout:8/676: dread - d7/dc/d36/d83/d60/d93/fd5 zero size 2024-06-06T08:43:52.947 INFO:tasks.workunit.client.0.smithi027.stdout:8/677: read d7/f4e [70199,67300] 0 2024-06-06T08:43:52.949 INFO:tasks.workunit.client.0.smithi027.stdout:9/804: rmdir d3/d95/d118 39 2024-06-06T08:43:52.949 INFO:tasks.workunit.client.0.smithi027.stdout:9/805: read d3/d95/f4f [2880492,62272] 0 2024-06-06T08:43:52.949 INFO:tasks.workunit.client.0.smithi027.stdout:0/822: symlink d15/d18/d64/l116 0 2024-06-06T08:43:52.949 INFO:tasks.workunit.client.0.smithi027.stdout:0/823: chown d15/d18/d40 781 1 2024-06-06T08:43:52.950 INFO:tasks.workunit.client.0.smithi027.stdout:9/806: creat d3/d95/d23/d78/ddf/f121 x:0 0 0 2024-06-06T08:43:52.950 INFO:tasks.workunit.client.0.smithi027.stdout:9/807: read - d3/d95/d23/da9/f104 zero size 2024-06-06T08:43:52.951 INFO:tasks.workunit.client.0.smithi027.stdout:2/776: unlink d6/d1e/d2b/d5a/f84 0 2024-06-06T08:43:52.952 INFO:tasks.workunit.client.0.smithi027.stdout:3/714: write d1/d90/d44/f49 [2799871,111991] 0 2024-06-06T08:43:52.953 INFO:tasks.workunit.client.0.smithi027.stdout:2/777: mkdir d6/dd/dd9/d3f/d48/d83/de3/d100 0 2024-06-06T08:43:52.953 INFO:tasks.workunit.client.0.smithi027.stdout:2/778: chown d6/dd/dd9/d1d/dbc/cf5 2 1 2024-06-06T08:43:52.954 INFO:tasks.workunit.client.0.smithi027.stdout:8/678: rmdir d7 39 2024-06-06T08:43:52.955 INFO:tasks.workunit.client.0.smithi027.stdout:8/679: fdatasync d7/dc/d36/d83/fc7 0 2024-06-06T08:43:52.958 INFO:tasks.workunit.client.0.smithi027.stdout:2/779: mknod d6/dd/dd9/d3f/d77/d55/d69/d82/c101 0 2024-06-06T08:43:52.961 INFO:tasks.workunit.client.0.smithi027.stdout:9/808: link d3/d95/d23/d36/f5d d3/d95/d23/d78/d11a/f122 0 2024-06-06T08:43:52.963 INFO:tasks.workunit.client.0.smithi027.stdout:0/824: getdents d15/d18 0 2024-06-06T08:43:52.963 INFO:tasks.workunit.client.0.smithi027.stdout:0/825: readlink d15/d18/l7d 0 2024-06-06T08:43:52.965 INFO:tasks.workunit.client.0.smithi027.stdout:6/930: rename d4/d1a/d2e/d44/dce/fe3 to d4/d9/f12f 0 2024-06-06T08:43:52.965 INFO:tasks.workunit.client.0.smithi027.stdout:6/931: write d4/d125/d18/ff1 [835089,30394] 0 2024-06-06T08:43:52.965 INFO:tasks.workunit.client.0.smithi027.stdout:6/932: dread - d4/de/d45/d6a/f11b zero size 2024-06-06T08:43:52.965 INFO:tasks.workunit.client.0.smithi027.stdout:6/933: fdatasync d4/d9/d49/f8d 0 2024-06-06T08:43:52.965 INFO:tasks.workunit.client.0.smithi027.stdout:2/780: symlink d6/dd/dd9/d3f/d77/d55/d69/d82/l102 0 2024-06-06T08:43:52.966 INFO:tasks.workunit.client.0.smithi027.stdout:0/826: mknod d15/d18/dbe/d5c/d83/c117 0 2024-06-06T08:43:52.967 INFO:tasks.workunit.client.0.smithi027.stdout:0/827: fsync d15/d18/d23/d38/d68/d85/ff3 0 2024-06-06T08:43:52.967 INFO:tasks.workunit.client.0.smithi027.stdout:7/915: link c8 d17/d32/d46/d83/d77/d8c/d91/daa/c12e 0 2024-06-06T08:43:52.968 INFO:tasks.workunit.client.0.smithi027.stdout:5/986: rename d0/da/db/d46/d54/c7a to d0/da/db/d1a/da3/de5/d107/c13c 0 2024-06-06T08:43:52.969 INFO:tasks.workunit.client.0.smithi027.stdout:3/715: dread d1/d8/d22/f95 [0,4194304] 0 2024-06-06T08:43:52.969 INFO:tasks.workunit.client.0.smithi027.stdout:4/891: sync 2024-06-06T08:43:52.969 INFO:tasks.workunit.client.0.smithi027.stdout:1/763: sync 2024-06-06T08:43:52.972 INFO:tasks.workunit.client.0.smithi027.stdout:4/892: fsync d0/d1/d6/d8/d107/d9c/dbe/fc3 0 2024-06-06T08:43:52.977 INFO:tasks.workunit.client.0.smithi027.stdout:3/716: mknod d1/d1f/dbc/cdf 0 2024-06-06T08:43:52.981 INFO:tasks.workunit.client.0.smithi027.stdout:3/717: dread d1/d90/d44/d54/fa6 [0,4194304] 0 2024-06-06T08:43:52.981 INFO:tasks.workunit.client.0.smithi027.stdout:3/718: fdatasync d1/d90/d44/d54/fa6 0 2024-06-06T08:43:52.983 INFO:tasks.workunit.client.0.smithi027.stdout:5/987: fsync d0/da/db/d1a/d5a/de0/fe8 0 2024-06-06T08:43:52.983 INFO:tasks.workunit.client.0.smithi027.stdout:8/680: rename d7/fb4 to d7/dc/d36/d83/d58/fdb 0 2024-06-06T08:43:52.986 INFO:tasks.workunit.client.0.smithi027.stdout:9/809: dwrite d3/d4e/d75/dc/f2d [0,4194304] 0 2024-06-06T08:43:52.988 INFO:tasks.workunit.client.0.smithi027.stdout:9/810: mknod d3/d95/d23/d78/ddf/c123 0 2024-06-06T08:43:52.988 INFO:tasks.workunit.client.0.smithi027.stdout:9/811: write d3/f5f [2172856,109807] 0 2024-06-06T08:43:52.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:52 smithi203 ceph-mon[36579]: pgmap v111: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 250 op/s 2024-06-06T08:43:52.990 INFO:tasks.workunit.client.0.smithi027.stdout:8/681: dread d7/f9 [0,4194304] 0 2024-06-06T08:43:52.991 INFO:tasks.workunit.client.0.smithi027.stdout:9/812: mknod d3/d76/c124 0 2024-06-06T08:43:52.992 INFO:tasks.workunit.client.0.smithi027.stdout:9/813: write d3/d95/d23/d78/ddf/f121 [112584,124418] 0 2024-06-06T08:43:52.993 INFO:tasks.workunit.client.0.smithi027.stdout:9/814: creat d3/d95/d34/f125 x:0 0 0 2024-06-06T08:43:52.994 INFO:tasks.workunit.client.0.smithi027.stdout:9/815: creat d3/d95/d34/d73/f126 x:0 0 0 2024-06-06T08:43:52.995 INFO:tasks.workunit.client.0.smithi027.stdout:3/719: dread d1/d8/f55 [0,4194304] 0 2024-06-06T08:43:52.995 INFO:tasks.workunit.client.0.smithi027.stdout:8/682: mkdir d7/d13/d15/ddc 0 2024-06-06T08:43:52.998 INFO:tasks.workunit.client.0.smithi027.stdout:7/916: dwrite d17/d32/d46/d4f/d5e/f67 [0,4194304] 0 2024-06-06T08:43:52.998 INFO:tasks.workunit.client.0.smithi027.stdout:2/781: rename d6/dd/dd9/d3f/db0/lb5 to d6/dd/dd9/d1d/dbc/l103 0 2024-06-06T08:43:53.001 INFO:tasks.workunit.client.0.smithi027.stdout:8/683: getdents d7/d2f 0 2024-06-06T08:43:53.001 INFO:tasks.workunit.client.0.smithi027.stdout:8/684: dread - d7/dc/d36/d83/fc7 zero size 2024-06-06T08:43:53.002 INFO:tasks.workunit.client.0.smithi027.stdout:1/764: rename dc/fd6 to dc/d67/d69/d32/d37/d56/df1/d80/da6/ff2 0 2024-06-06T08:43:53.003 INFO:tasks.workunit.client.0.smithi027.stdout:2/782: truncate d6/dd/f23 1502183 0 2024-06-06T08:43:53.003 INFO:tasks.workunit.client.0.smithi027.stdout:2/783: readlink d6/dd/dd9/d1d/dbc/l103 0 2024-06-06T08:43:53.004 INFO:tasks.workunit.client.0.smithi027.stdout:8/685: truncate d7/dc/d36/d83/fa4 3325369 0 2024-06-06T08:43:53.005 INFO:tasks.workunit.client.0.smithi027.stdout:1/765: creat dc/de/d3d/d4a/ff3 x:0 0 0 2024-06-06T08:43:53.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:52 smithi176 ceph-mon[36786]: pgmap v111: 129 pgs: 129 active+clean; 1.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 250 op/s 2024-06-06T08:43:53.007 INFO:tasks.workunit.client.0.smithi027.stdout:5/988: dwrite d0/da/db/f64 [0,4194304] 0 2024-06-06T08:43:53.008 INFO:tasks.workunit.client.0.smithi027.stdout:2/784: link d6/dd/dd9/d3f/d48/lf4 d6/d3b/l104 0 2024-06-06T08:43:53.011 INFO:tasks.workunit.client.0.smithi027.stdout:2/785: mkdir d6/d3b/d61/d105 0 2024-06-06T08:43:53.012 INFO:tasks.workunit.client.0.smithi027.stdout:2/786: readlink d6/dd/dd9/d1d/dbc/lc2 0 2024-06-06T08:43:53.013 INFO:tasks.workunit.client.0.smithi027.stdout:3/720: dread d1/d8/d22/f37 [0,4194304] 0 2024-06-06T08:43:53.015 INFO:tasks.workunit.client.0.smithi027.stdout:9/816: dwrite d3/d95/d23/d36/d3a/fac [0,4194304] 0 2024-06-06T08:43:53.017 INFO:tasks.workunit.client.0.smithi027.stdout:7/917: write f11 [3197390,72198] 0 2024-06-06T08:43:53.017 INFO:tasks.workunit.client.0.smithi027.stdout:2/787: unlink d6/f2e 0 2024-06-06T08:43:53.017 INFO:tasks.workunit.client.0.smithi027.stdout:8/686: symlink d7/d13/ldd 0 2024-06-06T08:43:53.019 INFO:tasks.workunit.client.0.smithi027.stdout:9/817: mkdir d3/d95/d118/d127 0 2024-06-06T08:43:53.019 INFO:tasks.workunit.client.0.smithi027.stdout:9/818: dread - d3/d95/d23/d36/fbd zero size 2024-06-06T08:43:53.022 INFO:tasks.workunit.client.0.smithi027.stdout:8/687: mkdir d7/dc/dde 0 2024-06-06T08:43:53.022 INFO:tasks.workunit.client.0.smithi027.stdout:8/688: chown d7/dc/d37/cc9 6250326 1 2024-06-06T08:43:53.022 INFO:tasks.workunit.client.0.smithi027.stdout:8/689: write d7/d13/d15/f1c [2318827,125413] 0 2024-06-06T08:43:53.023 INFO:tasks.workunit.client.0.smithi027.stdout:2/788: symlink d6/dd/dd9/d1d/l106 0 2024-06-06T08:43:53.026 INFO:tasks.workunit.client.0.smithi027.stdout:8/690: unlink d7/dc/d36/d83/d60/d93/dac/d73/fc2 0 2024-06-06T08:43:53.028 INFO:tasks.workunit.client.0.smithi027.stdout:1/766: dwrite dc/de/d22/f82 [0,4194304] 0 2024-06-06T08:43:53.029 INFO:tasks.workunit.client.0.smithi027.stdout:3/721: write d1/d8/d22/d27/f32 [3960827,45916] 0 2024-06-06T08:43:53.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/722: mkdir d1/d90/d6d/dac/de0 0 2024-06-06T08:43:53.035 INFO:tasks.workunit.client.0.smithi027.stdout:9/819: read d3/d4e/d75/fcf [3005276,18526] 0 2024-06-06T08:43:53.035 INFO:tasks.workunit.client.0.smithi027.stdout:9/820: chown d3/d95/d23 46 1 2024-06-06T08:43:53.036 INFO:tasks.workunit.client.0.smithi027.stdout:7/918: write d17/d32/d6c/dc2/fc5 [2586888,67411] 0 2024-06-06T08:43:53.036 INFO:tasks.workunit.client.0.smithi027.stdout:9/821: creat d3/d95/d23/f128 x:0 0 0 2024-06-06T08:43:53.042 INFO:tasks.workunit.client.0.smithi027.stdout:8/691: dwrite d7/dc/d36/d83/d60/f68 [0,4194304] 0 2024-06-06T08:43:53.045 INFO:tasks.workunit.client.0.smithi027.stdout:7/919: readlink d17/d32/d46/d83/d2a/la5 0 2024-06-06T08:43:53.046 INFO:tasks.workunit.client.0.smithi027.stdout:7/920: write d17/d32/d46/d4f/db6/dc8/fcd [1101295,93392] 0 2024-06-06T08:43:53.048 INFO:tasks.workunit.client.0.smithi027.stdout:9/822: mknod d3/d76/d11b/c129 0 2024-06-06T08:43:53.048 INFO:tasks.workunit.client.0.smithi027.stdout:9/823: chown d3/d4e/d75/dc/fad 7 1 2024-06-06T08:43:53.048 INFO:tasks.workunit.client.0.smithi027.stdout:9/824: write d3/d4e/d75/dc/d9b/f11c [375872,124684] 0 2024-06-06T08:43:53.049 INFO:tasks.workunit.client.0.smithi027.stdout:9/825: creat d3/d4e/d75/dc/d9b/f12a x:0 0 0 2024-06-06T08:43:53.049 INFO:tasks.workunit.client.0.smithi027.stdout:9/826: stat d3/d4e/d75/dc/c1c 0 2024-06-06T08:43:53.050 INFO:tasks.workunit.client.0.smithi027.stdout:2/789: write d6/dd/dd9/d3f/d77/d55/d69/d82/f79 [1303949,22094] 0 2024-06-06T08:43:53.050 INFO:tasks.workunit.client.0.smithi027.stdout:9/827: chown d3/d4e/d75/c10 0 1 2024-06-06T08:43:53.053 INFO:tasks.workunit.client.0.smithi027.stdout:8/692: dread d7/dc/d36/f7b [0,4194304] 0 2024-06-06T08:43:53.055 INFO:tasks.workunit.client.0.smithi027.stdout:8/693: creat d7/dc/d36/fdf x:0 0 0 2024-06-06T08:43:53.056 INFO:tasks.workunit.client.0.smithi027.stdout:4/893: rename d0/d1/d6/l65 to d0/d1d/d34/d7f/l128 0 2024-06-06T08:43:53.060 INFO:tasks.workunit.client.0.smithi027.stdout:8/694: mknod d7/d13/d15/ce0 0 2024-06-06T08:43:53.060 INFO:tasks.workunit.client.0.smithi027.stdout:8/695: chown d7/dc/d36/d83/d60/d93/lb9 12674114 1 2024-06-06T08:43:53.061 INFO:tasks.workunit.client.0.smithi027.stdout:8/696: mknod d7/dc/d36/d41/dbf/dc3/ce1 0 2024-06-06T08:43:53.062 INFO:tasks.workunit.client.0.smithi027.stdout:2/790: dread d6/d1e/d2b/f58 [0,4194304] 0 2024-06-06T08:43:53.064 INFO:tasks.workunit.client.0.smithi027.stdout:8/697: mkdir d7/dc/d36/d41/de2 0 2024-06-06T08:43:53.065 INFO:tasks.workunit.client.0.smithi027.stdout:8/698: write d7/d13/d15/f1c [5261474,84601] 0 2024-06-06T08:43:53.065 INFO:tasks.workunit.client.0.smithi027.stdout:8/699: creat d7/d9c/daf/fe3 x:0 0 0 2024-06-06T08:43:53.068 INFO:tasks.workunit.client.0.smithi027.stdout:4/894: dread d0/d1/d6/d8/d21/f10e [0,4194304] 0 2024-06-06T08:43:53.070 INFO:tasks.workunit.client.0.smithi027.stdout:4/895: dread d0/d1/d6/d8/d82/d8b/f116 [0,4194304] 0 2024-06-06T08:43:53.071 INFO:tasks.workunit.client.0.smithi027.stdout:4/896: creat d0/d1/d6/d8/d82/d8b/f129 x:0 0 0 2024-06-06T08:43:53.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/989: rename d0/da/db/d46/d7f/d81/d97/f11b to d0/da/d20/f13d 0 2024-06-06T08:43:53.078 INFO:tasks.workunit.client.0.smithi027.stdout:1/767: rename dc/d67/d69/d32/d37/d56/df1/f63 to dc/d67/d69/d32/d37/ff4 0 2024-06-06T08:43:53.078 INFO:tasks.workunit.client.0.smithi027.stdout:1/768: fsync dc/d9d/fa2 0 2024-06-06T08:43:53.080 INFO:tasks.workunit.client.0.smithi027.stdout:3/723: rename d1/f84 to d1/d90/d6d/dac/fe1 0 2024-06-06T08:43:53.089 INFO:tasks.workunit.client.0.smithi027.stdout:4/897: dwrite d0/d1/d6/f9e [0,4194304] 0 2024-06-06T08:43:53.089 INFO:tasks.workunit.client.0.smithi027.stdout:4/898: chown d0/d1/d6/d8/d107/dad/fee 748916 1 2024-06-06T08:43:53.090 INFO:tasks.workunit.client.0.smithi027.stdout:4/899: mknod d0/d1/d6/d8/d107/dad/c12a 0 2024-06-06T08:43:53.090 INFO:tasks.workunit.client.0.smithi027.stdout:4/900: dread - d0/d1/d6/d8/d107/dad/ffa zero size 2024-06-06T08:43:53.093 INFO:tasks.workunit.client.0.smithi027.stdout:1/769: dread dc/de/d22/f3b [0,4194304] 0 2024-06-06T08:43:53.093 INFO:tasks.workunit.client.0.smithi027.stdout:1/770: fsync dc/d67/d69/d32/d37/d56/df1/f1a 0 2024-06-06T08:43:53.093 INFO:tasks.workunit.client.0.smithi027.stdout:4/901: mknod d0/d63/c12b 0 2024-06-06T08:43:53.093 INFO:tasks.workunit.client.0.smithi027.stdout:4/902: fsync d0/d1d/d2a/f5e 0 2024-06-06T08:43:53.094 INFO:tasks.workunit.client.0.smithi027.stdout:4/903: mknod d0/d1d/d34/d7f/df4/c12c 0 2024-06-06T08:43:53.096 INFO:tasks.workunit.client.0.smithi027.stdout:1/771: creat dc/de/d3d/d4a/d76/dd7/ff5 x:0 0 0 2024-06-06T08:43:53.097 INFO:tasks.workunit.client.0.smithi027.stdout:1/772: chown dc/c12 63 1 2024-06-06T08:43:53.098 INFO:tasks.workunit.client.0.smithi027.stdout:4/904: creat d0/d1d/d60/d119/f12d x:0 0 0 2024-06-06T08:43:53.098 INFO:tasks.workunit.client.0.smithi027.stdout:4/905: stat d0/d1d/d2a/d98 0 2024-06-06T08:43:53.104 INFO:tasks.workunit.client.0.smithi027.stdout:1/773: dread f5 [0,4194304] 0 2024-06-06T08:43:53.104 INFO:tasks.workunit.client.0.smithi027.stdout:1/774: chown dc/d67/d69/d32/d37/d56/l6a 79 1 2024-06-06T08:43:53.106 INFO:tasks.workunit.client.0.smithi027.stdout:3/724: dwrite d1/d90/f77 [0,4194304] 0 2024-06-06T08:43:53.107 INFO:tasks.workunit.client.0.smithi027.stdout:3/725: chown d1/d90/d6d/dac/ddb/f3a 1387241828 1 2024-06-06T08:43:53.109 INFO:tasks.workunit.client.0.smithi027.stdout:3/726: mknod d1/d1f/d45/d5f/dca/ce2 0 2024-06-06T08:43:53.109 INFO:tasks.workunit.client.0.smithi027.stdout:3/727: readlink d1/d90/d44/d54/db4/lbd 0 2024-06-06T08:43:53.109 INFO:tasks.workunit.client.0.smithi027.stdout:3/728: chown d1/d90/d44/fb0 1 1 2024-06-06T08:43:53.111 INFO:tasks.workunit.client.0.smithi027.stdout:7/921: rename d17/d32/d46/d83/d77/d8c/dae/d111 to d17/d32/d46/d4f/d5e/d7b/d7c/d92/d12f 0 2024-06-06T08:43:53.111 INFO:tasks.workunit.client.0.smithi027.stdout:7/922: chown d17/d32/d46/d4f/db6/dc8/d122 1265 1 2024-06-06T08:43:53.112 INFO:tasks.workunit.client.0.smithi027.stdout:7/923: fsync d17/d32/d61/f74 0 2024-06-06T08:43:53.112 INFO:tasks.workunit.client.0.smithi027.stdout:7/924: readlink d17/d32/d6c/ld2 0 2024-06-06T08:43:53.115 INFO:tasks.workunit.client.0.smithi027.stdout:1/775: read dc/d67/d69/d32/d37/d56/df1/f49 [950397,11682] 0 2024-06-06T08:43:53.115 INFO:tasks.workunit.client.0.smithi027.stdout:1/776: write dc/de/d22/f79 [5063488,28041] 0 2024-06-06T08:43:53.115 INFO:tasks.workunit.client.0.smithi027.stdout:2/791: rename d6/dd/dd9/d1d/lb1 to d6/dd/dd9/d1d/dbc/l107 0 2024-06-06T08:43:53.116 INFO:tasks.workunit.client.0.smithi027.stdout:7/925: symlink d17/d32/d46/d4f/d5e/d7b/l130 0 2024-06-06T08:43:53.117 INFO:tasks.workunit.client.0.smithi027.stdout:1/777: truncate dc/d67/d69/fb9 2450448 0 2024-06-06T08:43:53.118 INFO:tasks.workunit.client.0.smithi027.stdout:4/906: dwrite d0/d1/f12 [0,4194304] 0 2024-06-06T08:43:53.119 INFO:tasks.workunit.client.0.smithi027.stdout:7/926: stat d17/d32/d46/d83/c7d 0 2024-06-06T08:43:53.121 INFO:tasks.workunit.client.0.smithi027.stdout:4/907: mkdir d0/d63/d10c/d12e 0 2024-06-06T08:43:53.121 INFO:tasks.workunit.client.0.smithi027.stdout:4/908: stat d0/d1/d6/d8/d82/d8b/c101 0 2024-06-06T08:43:53.121 INFO:tasks.workunit.client.0.smithi027.stdout:4/909: fdatasync d0/d1/fe7 0 2024-06-06T08:43:53.121 INFO:tasks.workunit.client.0.smithi027.stdout:3/729: dread d1/d90/d44/d54/db4/fd3 [0,4194304] 0 2024-06-06T08:43:53.121 INFO:tasks.workunit.client.0.smithi027.stdout:4/910: dread - d0/d1d/dd0/f11d zero size 2024-06-06T08:43:53.121 INFO:tasks.workunit.client.0.smithi027.stdout:2/792: mkdir d6/dd/dd9/d3f/db0/d108 0 2024-06-06T08:43:53.122 INFO:tasks.workunit.client.0.smithi027.stdout:1/778: rmdir dc/de/d3d 39 2024-06-06T08:43:53.122 INFO:tasks.workunit.client.0.smithi027.stdout:1/779: readlink dc/da0/la5 0 2024-06-06T08:43:53.126 INFO:tasks.workunit.client.0.smithi027.stdout:1/780: mknod dc/d67/d69/d32/d37/d56/df1/d80/da6/cf6 0 2024-06-06T08:43:53.128 INFO:tasks.workunit.client.0.smithi027.stdout:4/911: write d0/d1/d6/d8/f6a [907626,31835] 0 2024-06-06T08:43:53.130 INFO:tasks.workunit.client.0.smithi027.stdout:3/730: symlink d1/d8/d22/le3 0 2024-06-06T08:43:53.130 INFO:tasks.workunit.client.0.smithi027.stdout:1/781: creat dc/d67/d69/d32/d37/d56/ff7 x:0 0 0 2024-06-06T08:43:53.131 INFO:tasks.workunit.client.0.smithi027.stdout:3/731: write d1/d8/fa7 [763655,33758] 0 2024-06-06T08:43:53.131 INFO:tasks.workunit.client.0.smithi027.stdout:3/732: dread - d1/d8/d59/fc5 zero size 2024-06-06T08:43:53.131 INFO:tasks.workunit.client.0.smithi027.stdout:4/912: symlink d0/d63/d10c/d12e/l12f 0 2024-06-06T08:43:53.131 INFO:tasks.workunit.client.0.smithi027.stdout:4/913: chown d0/ld3 0 1 2024-06-06T08:43:53.133 INFO:tasks.workunit.client.0.smithi027.stdout:1/782: stat dc/l23 0 2024-06-06T08:43:53.133 INFO:tasks.workunit.client.0.smithi027.stdout:1/783: stat dc/d67/d69/d32/d37/d56/df1/d80/da6/db0 0 2024-06-06T08:43:53.133 INFO:tasks.workunit.client.0.smithi027.stdout:2/793: getdents d6/d3b/d80 0 2024-06-06T08:43:53.134 INFO:tasks.workunit.client.0.smithi027.stdout:3/733: creat d1/d8/d22/d27/d50/fe4 x:0 0 0 2024-06-06T08:43:53.137 INFO:tasks.workunit.client.0.smithi027.stdout:2/794: mkdir d6/d1e/d2b/d5a/db7/d109 0 2024-06-06T08:43:53.139 INFO:tasks.workunit.client.0.smithi027.stdout:2/795: rmdir d6/dd/dd9/d3f/d77 39 2024-06-06T08:43:53.141 INFO:tasks.workunit.client.0.smithi027.stdout:4/914: dread d0/fe5 [0,4194304] 0 2024-06-06T08:43:53.141 INFO:tasks.workunit.client.0.smithi027.stdout:4/915: stat d0/d1/d6/d4b 0 2024-06-06T08:43:53.143 INFO:tasks.workunit.client.0.smithi027.stdout:4/916: getdents d0/d63/d10c/d113 0 2024-06-06T08:43:53.143 INFO:tasks.workunit.client.0.smithi027.stdout:4/917: chown d0/d1/d6/d8/d82/d8b/c101 689 1 2024-06-06T08:43:53.143 INFO:tasks.workunit.client.0.smithi027.stdout:4/918: fsync d0/d1/d6/d8/d21/f2d 0 2024-06-06T08:43:53.143 INFO:tasks.workunit.client.0.smithi027.stdout:4/919: dread - d0/d1/d6/d8/d82/fda zero size 2024-06-06T08:43:53.143 INFO:tasks.workunit.client.0.smithi027.stdout:7/927: dwrite d17/d32/f2f [0,4194304] 0 2024-06-06T08:43:53.144 INFO:tasks.workunit.client.0.smithi027.stdout:4/920: read d0/d1/d6/d8/d82/fab [103300,100995] 0 2024-06-06T08:43:53.144 INFO:tasks.workunit.client.0.smithi027.stdout:7/928: symlink d17/d32/d46/d83/d77/d8c/d91/daa/d103/l131 0 2024-06-06T08:43:53.147 INFO:tasks.workunit.client.0.smithi027.stdout:7/929: creat d17/d32/d46/d51/d52/d70/f132 x:0 0 0 2024-06-06T08:43:53.148 INFO:tasks.workunit.client.0.smithi027.stdout:1/784: write dc/de/d22/f79 [1320014,121977] 0 2024-06-06T08:43:53.149 INFO:tasks.workunit.client.0.smithi027.stdout:1/785: write dc/d67/d69/d32/d37/d56/ff7 [795883,37404] 0 2024-06-06T08:43:53.149 INFO:tasks.workunit.client.0.smithi027.stdout:1/786: chown dc/d67/d69/d32/d37/d56/df1/d4e 33 1 2024-06-06T08:43:53.149 INFO:tasks.workunit.client.0.smithi027.stdout:4/921: mkdir d0/d1/d6/d8/d130 0 2024-06-06T08:43:53.151 INFO:tasks.workunit.client.0.smithi027.stdout:1/787: dread dc/d67/d69/d32/d37/ff4 [0,4194304] 0 2024-06-06T08:43:53.151 INFO:tasks.workunit.client.0.smithi027.stdout:1/788: read - dc/d67/d69/d32/d37/d56/de1/fe9 zero size 2024-06-06T08:43:53.154 INFO:tasks.workunit.client.0.smithi027.stdout:4/922: mknod d0/d1/d6/d8/d82/d8b/dc9/d3b/c131 0 2024-06-06T08:43:53.157 INFO:tasks.workunit.client.0.smithi027.stdout:1/789: symlink dc/de/d22/lf8 0 2024-06-06T08:43:53.157 INFO:tasks.workunit.client.0.smithi027.stdout:1/790: chown dc/d67/d69/d32/d37/d56/df1/d80/da6/laa 1927019 1 2024-06-06T08:43:53.157 INFO:tasks.workunit.client.0.smithi027.stdout:1/791: fdatasync dc/d67/d69/d32/d37/d56/df1/f19 0 2024-06-06T08:43:53.157 INFO:tasks.workunit.client.0.smithi027.stdout:1/792: getdents dc/d67/d69/d32/d37/d56/df1/d4e 0 2024-06-06T08:43:53.158 INFO:tasks.workunit.client.0.smithi027.stdout:5/990: rename d0/da/db/d46/d7f/d81/l121 to d0/d6a/l13e 0 2024-06-06T08:43:53.159 INFO:tasks.workunit.client.0.smithi027.stdout:2/796: dwrite d6/dd/dd9/d3f/d77/f4c [0,4194304] 0 2024-06-06T08:43:53.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/991: getdents d0/da/db/d1a/dc6 0 2024-06-06T08:43:53.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/992: write d0/da/fb4 [275602,114349] 0 2024-06-06T08:43:53.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/993: write d0/da/d20/f13d [93628,21463] 0 2024-06-06T08:43:53.162 INFO:tasks.workunit.client.0.smithi027.stdout:4/923: symlink d0/d1/l132 0 2024-06-06T08:43:53.163 INFO:tasks.workunit.client.0.smithi027.stdout:1/793: unlink dc/d67/d69/d32/d37/d56/de1/fe9 0 2024-06-06T08:43:53.165 INFO:tasks.workunit.client.0.smithi027.stdout:5/994: getdents d0/da/d20 0 2024-06-06T08:43:53.166 INFO:tasks.workunit.client.0.smithi027.stdout:5/995: chown d0/da/d37/d2b/f91 1132381211 1 2024-06-06T08:43:53.169 INFO:tasks.workunit.client.0.smithi027.stdout:5/996: dread d0/da/db/d1a/da9/fe1 [0,4194304] 0 2024-06-06T08:43:53.169 INFO:tasks.workunit.client.0.smithi027.stdout:4/924: mknod d0/d1/d6/d4b/c133 0 2024-06-06T08:43:53.171 INFO:tasks.workunit.client.0.smithi027.stdout:5/997: symlink d0/da/db/d46/l13f 0 2024-06-06T08:43:53.172 INFO:tasks.workunit.client.0.smithi027.stdout:4/925: mkdir d0/d1/d6/d134 0 2024-06-06T08:43:53.176 INFO:tasks.workunit.client.0.smithi027.stdout:2/797: write d6/dd/dd9/d3f/d77/d55/f74 [630765,53926] 0 2024-06-06T08:43:53.176 INFO:tasks.workunit.client.0.smithi027.stdout:2/798: chown d6/dd/dd9/d3f/d48/fde 339 1 2024-06-06T08:43:53.176 INFO:tasks.workunit.client.0.smithi027.stdout:7/930: dwrite d17/f58 [0,4194304] 0 2024-06-06T08:43:53.176 INFO:tasks.workunit.client.0.smithi027.stdout:1/794: write dc/d67/fbe [829482,45114] 0 2024-06-06T08:43:53.176 INFO:tasks.workunit.client.0.smithi027.stdout:1/795: read - dc/de/de4/fef zero size 2024-06-06T08:43:53.177 INFO:tasks.workunit.client.0.smithi027.stdout:2/799: truncate d6/d1e/d2b/f91 1087182 0 2024-06-06T08:43:53.178 INFO:tasks.workunit.client.0.smithi027.stdout:1/796: dread dc/d67/d69/d32/d37/d56/d78/fea [0,4194304] 0 2024-06-06T08:43:53.181 INFO:tasks.workunit.client.0.smithi027.stdout:2/800: unlink d6/dd/dd9/d1d/d4d/fcf 0 2024-06-06T08:43:53.181 INFO:tasks.workunit.client.0.smithi027.stdout:1/797: creat dc/d67/d69/ff9 x:0 0 0 2024-06-06T08:43:53.181 INFO:tasks.workunit.client.0.smithi027.stdout:1/798: write dc/d67/d69/d32/d37/d56/df1/d80/fa4 [2448341,9201] 0 2024-06-06T08:43:53.183 INFO:tasks.workunit.client.0.smithi027.stdout:1/799: mkdir dc/d67/d69/d32/d37/ddf/dfa 0 2024-06-06T08:43:53.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/801: mkdir d6/dd/dd9/d1d/d4d/d8e/d10a 0 2024-06-06T08:43:53.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/802: read - d6/dd/dd9/d3f/db0/fcc zero size 2024-06-06T08:43:53.185 INFO:tasks.workunit.client.0.smithi027.stdout:1/800: truncate dc/de/d3d/fb4 605638 0 2024-06-06T08:43:53.185 INFO:tasks.workunit.client.0.smithi027.stdout:1/801: getdents dc/de/d3d/d4f 0 2024-06-06T08:43:53.185 INFO:tasks.workunit.client.0.smithi027.stdout:4/926: dread d0/d63/f6d [0,4194304] 0 2024-06-06T08:43:53.185 INFO:tasks.workunit.client.0.smithi027.stdout:1/802: symlink dc/d9d/lfb 0 2024-06-06T08:43:53.186 INFO:tasks.workunit.client.0.smithi027.stdout:1/803: mkdir dc/d67/d69/d32/d37/d56/df1/dc8/dfc 0 2024-06-06T08:43:53.187 INFO:tasks.workunit.client.0.smithi027.stdout:7/931: write d17/d32/d46/d51/d7f/f7a [1762305,71043] 0 2024-06-06T08:43:53.191 INFO:tasks.workunit.client.0.smithi027.stdout:7/932: rmdir d17/d32/d46/d83/d2a/d3b/dad 39 2024-06-06T08:43:53.191 INFO:tasks.workunit.client.0.smithi027.stdout:7/933: chown d17/d32/d46/d51/d52/d70/d87/dc0/f115 33138 1 2024-06-06T08:43:53.194 INFO:tasks.workunit.client.0.smithi027.stdout:4/927: unlink d0/d1/d6/d8/d21/f74 0 2024-06-06T08:43:53.194 INFO:tasks.workunit.client.0.smithi027.stdout:4/928: fdatasync d0/d1/d6/dd4/ff9 0 2024-06-06T08:43:53.198 INFO:tasks.workunit.client.0.smithi027.stdout:5/998: dwrite d0/f29 [0,4194304] 0 2024-06-06T08:43:53.200 INFO:tasks.workunit.client.0.smithi027.stdout:2/803: dread d6/dd/dd9/d3f/f71 [4194304,4194304] 0 2024-06-06T08:43:53.200 INFO:tasks.workunit.client.0.smithi027.stdout:2/804: getdents d6/d1e/d2b/d5a/db7/d109 0 2024-06-06T08:43:53.200 INFO:tasks.workunit.client.0.smithi027.stdout:2/805: stat d6/dd/dd9/d1d/d4d/ff7 0 2024-06-06T08:43:53.208 INFO:tasks.workunit.client.0.smithi027.stdout:1/804: dwrite dc/d67/d69/d32/d37/d56/df1/f19 [4194304,4194304] 0 2024-06-06T08:43:53.209 INFO:tasks.workunit.client.0.smithi027.stdout:1/805: chown dc/d67/d69/d32/d37/c40 403657 1 2024-06-06T08:43:53.209 INFO:tasks.workunit.client.0.smithi027.stdout:1/806: write dc/d67/d69/d32/d37/d56/df1/d4e/f5d [2171318,649] 0 2024-06-06T08:43:53.215 INFO:tasks.workunit.client.0.smithi027.stdout:5/999: dread d0/da/db/d1a/fde [0,4194304] 0 2024-06-06T08:43:53.217 INFO:tasks.workunit.client.0.smithi027.stdout:4/929: write d0/d63/f6d [3236820,14669] 0 2024-06-06T08:43:53.218 INFO:tasks.workunit.client.0.smithi027.stdout:4/930: mknod d0/d1d/d34/dd8/c135 0 2024-06-06T08:43:53.219 INFO:tasks.workunit.client.0.smithi027.stdout:4/931: write d0/d1/d6/d8/d82/d8b/f116 [647970,101337] 0 2024-06-06T08:43:53.219 INFO:tasks.workunit.client.0.smithi027.stdout:7/934: dwrite d17/d32/d46/d4f/db6/dc8/f101 [0,4194304] 0 2024-06-06T08:43:53.219 INFO:tasks.workunit.client.0.smithi027.stdout:7/935: write d17/d32/d46/d51/d52/fd3 [5095863,78064] 0 2024-06-06T08:43:53.220 INFO:tasks.workunit.client.0.smithi027.stdout:4/932: mknod d0/d1d/d60/dc6/d6e/db6/c136 0 2024-06-06T08:43:53.222 INFO:tasks.workunit.client.0.smithi027.stdout:7/936: creat d17/d32/d46/d51/d52/d10b/f133 x:0 0 0 2024-06-06T08:43:53.222 INFO:tasks.workunit.client.0.smithi027.stdout:4/933: creat d0/d1/f137 x:0 0 0 2024-06-06T08:43:53.223 INFO:tasks.workunit.client.0.smithi027.stdout:7/937: creat d17/d32/d46/d4f/f134 x:0 0 0 2024-06-06T08:43:53.225 INFO:tasks.workunit.client.0.smithi027.stdout:4/934: mknod d0/d1/d6/dd4/dde/c138 0 2024-06-06T08:43:53.225 INFO:tasks.workunit.client.0.smithi027.stdout:4/935: readlink d0/d1d/l2c 0 2024-06-06T08:43:53.225 INFO:tasks.workunit.client.0.smithi027.stdout:4/936: write d0/d1/d6/d8/d82/d8b/f114 [523453,12501] 0 2024-06-06T08:43:53.225 INFO:tasks.workunit.client.0.smithi027.stdout:7/938: symlink d17/d32/d46/d51/d52/d10b/l135 0 2024-06-06T08:43:53.226 INFO:tasks.workunit.client.0.smithi027.stdout:1/807: dwrite dc/d67/d69/d32/d37/d56/df1/d80/fd4 [0,4194304] 0 2024-06-06T08:43:53.227 INFO:tasks.workunit.client.0.smithi027.stdout:7/939: dread d17/d32/d46/d51/f35 [0,4194304] 0 2024-06-06T08:43:53.231 INFO:tasks.workunit.client.0.smithi027.stdout:4/937: mkdir d0/d1/d6/d8/d82/d8b/dc9/da3/d10b/d139 0 2024-06-06T08:43:53.231 INFO:tasks.workunit.client.0.smithi027.stdout:4/938: chown d0/d1d/d34/d11b 13 1 2024-06-06T08:43:53.231 INFO:tasks.workunit.client.0.smithi027.stdout:4/939: dread - d0/d1d/d34/dd8/f112 zero size 2024-06-06T08:43:53.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/940: write d0/d1/f2 [1667392,79574] 0 2024-06-06T08:43:53.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/941: truncate d0/d1/f14 1772573 0 2024-06-06T08:43:53.238 INFO:tasks.workunit.client.0.smithi027.stdout:4/942: creat d0/d63/d10c/d113/f13a x:0 0 0 2024-06-06T08:43:53.238 INFO:tasks.workunit.client.0.smithi027.stdout:4/943: mknod d0/c13b 0 2024-06-06T08:43:53.246 INFO:tasks.workunit.client.0.smithi027.stdout:7/940: read d17/d32/d46/d4f/db6/dc8/de0/f104 [520434,45108] 0 2024-06-06T08:43:53.246 INFO:tasks.workunit.client.0.smithi027.stdout:1/808: dwrite dc/de/d3d/d4f/f64 [0,4194304] 0 2024-06-06T08:43:53.246 INFO:tasks.workunit.client.0.smithi027.stdout:1/809: read dc/d67/d69/d32/d37/d56/df1/d4e/f5d [2740408,58667] 0 2024-06-06T08:43:53.253 INFO:tasks.workunit.client.0.smithi027.stdout:1/810: dread dc/de/d22/f3b [4194304,4194304] 0 2024-06-06T08:43:53.258 INFO:tasks.workunit.client.0.smithi027.stdout:1/811: unlink dc/de/d3d/d4f/c60 0 2024-06-06T08:43:53.259 INFO:tasks.workunit.client.0.smithi027.stdout:4/944: dwrite d0/d1/d6/f9e [0,4194304] 0 2024-06-06T08:43:53.264 INFO:tasks.workunit.client.0.smithi027.stdout:7/941: dwrite d17/d32/f6d [4194304,4194304] 0 2024-06-06T08:43:53.269 INFO:tasks.workunit.client.0.smithi027.stdout:7/942: rename d17/d32/d46/d51/d7f/db2/c126 to d17/d32/d6c/dc2/c136 0 2024-06-06T08:43:53.269 INFO:tasks.workunit.client.0.smithi027.stdout:7/943: dread - d17/d32/d46/d4f/db6/dc8/de0/ff9 zero size 2024-06-06T08:43:53.274 INFO:tasks.workunit.client.0.smithi027.stdout:1/812: write dc/d6d/fce [1965990,31129] 0 2024-06-06T08:43:53.275 INFO:tasks.workunit.client.0.smithi027.stdout:7/944: mknod d17/d32/d61/c137 0 2024-06-06T08:43:53.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/945: mkdir d17/d32/d46/d4f/db6/dc8/d138 0 2024-06-06T08:43:53.278 INFO:tasks.workunit.client.0.smithi027.stdout:4/945: dwrite d0/d1d/f69 [0,4194304] 0 2024-06-06T08:43:53.278 INFO:tasks.workunit.client.0.smithi027.stdout:4/946: write d0/d1/f137 [952662,103078] 0 2024-06-06T08:43:53.278 INFO:tasks.workunit.client.0.smithi027.stdout:4/947: chown d0/d1d/d34/d7f/fcd 219 1 2024-06-06T08:43:53.288 INFO:tasks.workunit.client.0.smithi027.stdout:7/946: link d17/d32/d46/d4f/db6/dc8/f101 d17/d32/d46/d83/d2a/d3b/dc7/d124/f139 0 2024-06-06T08:43:53.288 INFO:tasks.workunit.client.0.smithi027.stdout:7/947: getdents d17/d32/d46/d51/d52/d70/df7 0 2024-06-06T08:43:53.288 INFO:tasks.workunit.client.0.smithi027.stdout:7/948: chown d17/d32/d46/d4f/f8f 2977 1 2024-06-06T08:43:53.289 INFO:tasks.workunit.client.0.smithi027.stdout:7/949: creat d17/d32/d46/d83/d77/d8c/d91/f13a x:0 0 0 2024-06-06T08:43:53.298 INFO:tasks.workunit.client.0.smithi027.stdout:1/813: dwrite dc/d6d/fde [0,4194304] 0 2024-06-06T08:43:53.299 INFO:tasks.workunit.client.0.smithi027.stdout:1/814: rmdir dc/d67/d69/d32/d37/ddf 39 2024-06-06T08:43:53.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/948: dwrite d0/d1/d6/d8/d82/d8b/dc9/f89 [0,4194304] 0 2024-06-06T08:43:53.302 INFO:tasks.workunit.client.0.smithi027.stdout:4/949: mknod d0/d1d/d60/dc6/d6e/db6/c13c 0 2024-06-06T08:43:53.304 INFO:tasks.workunit.client.0.smithi027.stdout:4/950: mknod d0/d1/d6/d8/d107/c13d 0 2024-06-06T08:43:53.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/950: dwrite d17/d32/d46/d4f/db6/f117 [0,4194304] 0 2024-06-06T08:43:53.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/951: fdatasync d17/f2e 0 2024-06-06T08:43:53.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/952: chown d17/d32/d46/d83/lfb 1762142 1 2024-06-06T08:43:53.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/953: chown d17/d32/d46/d83/d77/f11b 26 1 2024-06-06T08:43:53.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/954: stat d17/d32/d46/d4f/db6/dc8/de8/l10e 0 2024-06-06T08:43:53.318 INFO:tasks.workunit.client.0.smithi027.stdout:4/951: rename d0/d1/d6/d8/d82/f9f to d0/d1/d6/d8/d107/d9c/f13e 0 2024-06-06T08:43:53.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/952: write d0/d1/d6/d8/d82/d8b/dc9/d3b/fa9 [973130,28786] 0 2024-06-06T08:43:53.324 INFO:tasks.workunit.client.0.smithi027.stdout:6/934: sync 2024-06-06T08:43:53.326 INFO:tasks.workunit.client.0.smithi027.stdout:6/935: rmdir d4/de/d45 39 2024-06-06T08:43:53.327 INFO:tasks.workunit.client.0.smithi027.stdout:6/936: truncate d4/da2/fc6 606979 0 2024-06-06T08:43:53.327 INFO:tasks.workunit.client.0.smithi027.stdout:6/937: write d4/da2/fc6 [1638272,6043] 0 2024-06-06T08:43:53.327 INFO:tasks.workunit.client.0.smithi027.stdout:6/938: chown d4/de/d45/dcf/cd6 1 1 2024-06-06T08:43:53.327 INFO:tasks.workunit.client.0.smithi027.stdout:6/939: creat d4/d1a/d2e/f130 x:0 0 0 2024-06-06T08:43:53.327 INFO:tasks.workunit.client.0.smithi027.stdout:6/940: truncate d4/d1a/d2e/fc1 366204 0 2024-06-06T08:43:53.328 INFO:tasks.workunit.client.0.smithi027.stdout:6/941: write d4/d125/d18/f24 [2972555,20766] 0 2024-06-06T08:43:53.333 INFO:tasks.workunit.client.0.smithi027.stdout:7/955: dwrite d17/d32/d61/f74 [0,4194304] 0 2024-06-06T08:43:53.333 INFO:tasks.workunit.client.0.smithi027.stdout:7/956: write d17/d32/d46/d4f/d5e/f8b [950104,99919] 0 2024-06-06T08:43:53.345 INFO:tasks.workunit.client.0.smithi027.stdout:6/942: dwrite d4/de/d45/f115 [0,4194304] 0 2024-06-06T08:43:53.345 INFO:tasks.workunit.client.0.smithi027.stdout:6/943: chown d4/d125/f43 12 1 2024-06-06T08:43:53.346 INFO:tasks.workunit.client.0.smithi027.stdout:7/957: read d17/d32/d46/d4f/d5e/fb0 [3454654,75228] 0 2024-06-06T08:43:53.346 INFO:tasks.workunit.client.0.smithi027.stdout:4/953: dwrite d0/d1/ff [0,4194304] 0 2024-06-06T08:43:53.348 INFO:tasks.workunit.client.0.smithi027.stdout:4/954: rename d0/d1d/f69 to d0/d1d/d34/dd8/f13f 0 2024-06-06T08:43:53.348 INFO:tasks.workunit.client.0.smithi027.stdout:4/955: read d0/d63/f6d [805123,8856] 0 2024-06-06T08:43:53.348 INFO:tasks.workunit.client.0.smithi027.stdout:4/956: truncate d0/d1d/d2a/fdc 376047 0 2024-06-06T08:43:53.348 INFO:tasks.workunit.client.0.smithi027.stdout:4/957: chown d0/d1/f12 6839175 1 2024-06-06T08:43:53.349 INFO:tasks.workunit.client.0.smithi027.stdout:4/958: creat d0/d1d/d60/dc6/d6e/db6/f140 x:0 0 0 2024-06-06T08:43:53.350 INFO:tasks.workunit.client.0.smithi027.stdout:7/958: creat d17/d32/d46/d51/f13b x:0 0 0 2024-06-06T08:43:53.350 INFO:tasks.workunit.client.0.smithi027.stdout:7/959: chown f11 3510859 1 2024-06-06T08:43:53.350 INFO:tasks.workunit.client.0.smithi027.stdout:4/959: getdents d0/d1d/d2a/d98 0 2024-06-06T08:43:53.350 INFO:tasks.workunit.client.0.smithi027.stdout:4/960: fdatasync d0/d1/d6/d8/d82/fda 0 2024-06-06T08:43:53.351 INFO:tasks.workunit.client.0.smithi027.stdout:7/960: mknod d17/d32/d46/c13c 0 2024-06-06T08:43:53.360 INFO:tasks.workunit.client.0.smithi027.stdout:7/961: dread f15 [0,4194304] 0 2024-06-06T08:43:53.366 INFO:tasks.workunit.client.0.smithi027.stdout:6/944: dwrite d4/de/fb4 [0,4194304] 0 2024-06-06T08:43:53.368 INFO:tasks.workunit.client.0.smithi027.stdout:6/945: read - d4/d1a/d2e/fe9 zero size 2024-06-06T08:43:53.368 INFO:tasks.workunit.client.0.smithi027.stdout:4/961: dwrite d0/d1d/ffe [8388608,4194304] 0 2024-06-06T08:43:53.379 INFO:tasks.workunit.client.0.smithi027.stdout:6/946: link d4/da2/ldf d4/d9/d49/l131 0 2024-06-06T08:43:53.379 INFO:tasks.workunit.client.0.smithi027.stdout:6/947: write d4/da2/fa6 [508776,119708] 0 2024-06-06T08:43:53.381 INFO:tasks.workunit.client.0.smithi027.stdout:7/962: dwrite d17/d32/d46/d51/f54 [0,4194304] 0 2024-06-06T08:43:53.381 INFO:tasks.workunit.client.0.smithi027.stdout:7/963: dread - d17/d32/d6c/dc2/f121 zero size 2024-06-06T08:43:53.381 INFO:tasks.workunit.client.0.smithi027.stdout:7/964: chown f11 68098255 1 2024-06-06T08:43:53.385 INFO:tasks.workunit.client.0.smithi027.stdout:6/948: fsync d4/da2/fc6 0 2024-06-06T08:43:53.386 INFO:tasks.workunit.client.0.smithi027.stdout:6/949: creat d4/d9/dc5/f132 x:0 0 0 2024-06-06T08:43:53.390 INFO:tasks.workunit.client.0.smithi027.stdout:6/950: dread d4/d1a/d2e/d44/dca/fec [0,4194304] 0 2024-06-06T08:43:53.390 INFO:tasks.workunit.client.0.smithi027.stdout:4/962: dwrite d0/d1/d6/f106 [4194304,4194304] 0 2024-06-06T08:43:53.394 INFO:tasks.workunit.client.0.smithi027.stdout:4/963: creat d0/d1d/d60/dc6/f141 x:0 0 0 2024-06-06T08:43:53.394 INFO:tasks.workunit.client.0.smithi027.stdout:4/964: chown d0/d1/d6/la6 2809228 1 2024-06-06T08:43:53.396 INFO:tasks.workunit.client.0.smithi027.stdout:6/951: unlink d4/f83 0 2024-06-06T08:43:53.400 INFO:tasks.workunit.client.0.smithi027.stdout:6/952: creat d4/d125/d12d/f133 x:0 0 0 2024-06-06T08:43:53.401 INFO:tasks.workunit.client.0.smithi027.stdout:6/953: truncate d4/d125/d18/fe7 2867709 0 2024-06-06T08:43:53.408 INFO:tasks.workunit.client.0.smithi027.stdout:4/965: dwrite d0/d1/d6/d8/d82/d8b/f116 [0,4194304] 0 2024-06-06T08:43:53.411 INFO:tasks.workunit.client.0.smithi027.stdout:4/966: unlink d0/d1d/c61 0 2024-06-06T08:43:53.411 INFO:tasks.workunit.client.0.smithi027.stdout:4/967: fsync d0/d1/d6/d8/d82/d8b/f116 0 2024-06-06T08:43:53.421 INFO:tasks.workunit.client.0.smithi027.stdout:6/954: dwrite d4/d1a/d2e/f5c [4194304,4194304] 0 2024-06-06T08:43:53.430 INFO:tasks.workunit.client.0.smithi027.stdout:4/968: dwrite d0/d1/d6/f9e [0,4194304] 0 2024-06-06T08:43:53.430 INFO:tasks.workunit.client.0.smithi027.stdout:4/969: write d0/d1/f4f [2779818,43605] 0 2024-06-06T08:43:53.430 INFO:tasks.workunit.client.0.smithi027.stdout:6/955: write d4/de/d45/f75 [3193829,80391] 0 2024-06-06T08:43:53.431 INFO:tasks.workunit.client.0.smithi027.stdout:4/970: mknod d0/d1d/d60/d119/c142 0 2024-06-06T08:43:53.431 INFO:tasks.workunit.client.0.smithi027.stdout:4/971: fsync d0/d1/d6/d8/d21/f2d 0 2024-06-06T08:43:53.432 INFO:tasks.workunit.client.0.smithi027.stdout:4/972: creat d0/d1d/d2a/d98/f143 x:0 0 0 2024-06-06T08:43:53.432 INFO:tasks.workunit.client.0.smithi027.stdout:6/956: mkdir d4/d9/dc5/d134 0 2024-06-06T08:43:53.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/973: mknod d0/d1/c144 0 2024-06-06T08:43:53.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/974: readlink d0/d63/d10c/d12e/l12f 0 2024-06-06T08:43:53.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/957: creat d4/d13/f135 x:0 0 0 2024-06-06T08:43:53.435 INFO:tasks.workunit.client.0.smithi027.stdout:6/958: getdents d4/d9/df2 0 2024-06-06T08:43:53.437 INFO:tasks.workunit.client.0.smithi027.stdout:6/959: symlink d4/d125/d12d/l136 0 2024-06-06T08:43:53.437 INFO:tasks.workunit.client.0.smithi027.stdout:6/960: chown d4/d13/dba/d118 3344293 1 2024-06-06T08:43:53.451 INFO:tasks.workunit.client.0.smithi027.stdout:4/975: link d0/d1d/d34/l11e d0/d1/d6/d8/d82/l145 0 2024-06-06T08:43:53.455 INFO:tasks.workunit.client.0.smithi027.stdout:4/976: read d0/d1d/d2a/fdd [17526,26384] 0 2024-06-06T08:43:53.459 INFO:tasks.workunit.client.0.smithi027.stdout:6/961: dwrite d4/d1a/d2e/d44/dca/fec [0,4194304] 0 2024-06-06T08:43:53.460 INFO:tasks.workunit.client.0.smithi027.stdout:6/962: write d4/d13/dba/fdd [845587,9586] 0 2024-06-06T08:43:53.464 INFO:tasks.workunit.client.0.smithi027.stdout:6/963: symlink d4/d125/l137 0 2024-06-06T08:43:53.467 INFO:tasks.workunit.client.0.smithi027.stdout:6/964: symlink d4/d9/l138 0 2024-06-06T08:43:53.473 INFO:tasks.workunit.client.0.smithi027.stdout:4/977: write d0/d1/d6/d8/d107/dad/fee [2559293,104050] 0 2024-06-06T08:43:53.485 INFO:tasks.workunit.client.0.smithi027.stdout:6/965: dwrite d4/de/d45/d6a/de8/f124 [0,4194304] 0 2024-06-06T08:43:53.487 INFO:tasks.workunit.client.0.smithi027.stdout:4/978: write d0/d1/d6/f9e [1766944,117064] 0 2024-06-06T08:43:53.487 INFO:tasks.workunit.client.0.smithi027.stdout:6/966: creat d4/d13/dba/f139 x:0 0 0 2024-06-06T08:43:53.487 INFO:tasks.workunit.client.0.smithi027.stdout:6/967: chown d4/d125/d1c/d52/cac 3 1 2024-06-06T08:43:53.499 INFO:tasks.workunit.client.0.smithi027.stdout:6/968: rename d4/de/d45/d6a/cae to d4/d125/d1c/d52/c13a 0 2024-06-06T08:43:53.499 INFO:tasks.workunit.client.0.smithi027.stdout:6/969: write d4/d125/d18/f11c [1034421,81287] 0 2024-06-06T08:43:53.502 INFO:tasks.workunit.client.0.smithi027.stdout:4/979: dwrite d0/d1/d6/d8/d21/f2d [4194304,4194304] 0 2024-06-06T08:43:53.503 INFO:tasks.workunit.client.0.smithi027.stdout:4/980: fsync d0/d1d/d60/d119/f12d 0 2024-06-06T08:43:53.504 INFO:tasks.workunit.client.0.smithi027.stdout:4/981: creat d0/d1/d6/d8/d82/d8b/dc9/da3/f146 x:0 0 0 2024-06-06T08:43:53.504 INFO:tasks.workunit.client.0.smithi027.stdout:4/982: write d0/d1d/d2a/fdc [456894,110444] 0 2024-06-06T08:43:53.506 INFO:tasks.workunit.client.0.smithi027.stdout:4/983: link d0/d1d/d60/dc6/c124 d0/d63/d10c/c147 0 2024-06-06T08:43:53.506 INFO:tasks.workunit.client.0.smithi027.stdout:4/984: fsync d0/d1/d6/d8/d21/f4a 0 2024-06-06T08:43:53.506 INFO:tasks.workunit.client.0.smithi027.stdout:4/985: fsync d0/d1/d6/d8/d21/f2d 0 2024-06-06T08:43:53.506 INFO:tasks.workunit.client.0.smithi027.stdout:4/986: truncate d0/d1/d6/dd4/dde/f100 442622 0 2024-06-06T08:43:53.507 INFO:tasks.workunit.client.0.smithi027.stdout:4/987: mkdir d0/d1/d6/d8/d82/d8b/dc9/da3/d10b/d139/d148 0 2024-06-06T08:43:53.509 INFO:tasks.workunit.client.0.smithi027.stdout:6/970: dread d4/d13/dba/fdd [0,4194304] 0 2024-06-06T08:43:53.509 INFO:tasks.workunit.client.0.smithi027.stdout:6/971: chown d4/d9/d49/f77 468932781 1 2024-06-06T08:43:53.509 INFO:tasks.workunit.client.0.smithi027.stdout:4/988: link d0/d1/f2 d0/d1/d6/dd4/dde/f149 0 2024-06-06T08:43:53.510 INFO:tasks.workunit.client.0.smithi027.stdout:4/989: creat d0/f14a x:0 0 0 2024-06-06T08:43:53.510 INFO:tasks.workunit.client.0.smithi027.stdout:6/972: creat d4/de/d45/d6a/de8/f13b x:0 0 0 2024-06-06T08:43:53.511 INFO:tasks.workunit.client.0.smithi027.stdout:6/973: creat d4/d13/f13c x:0 0 0 2024-06-06T08:43:53.516 INFO:tasks.workunit.client.0.smithi027.stdout:4/990: mknod d0/d1/d6/d8/d82/d8b/dc9/c14b 0 2024-06-06T08:43:53.527 INFO:tasks.workunit.client.0.smithi027.stdout:6/974: dwrite d4/de/d45/d6a/f12c [0,4194304] 0 2024-06-06T08:43:53.528 INFO:tasks.workunit.client.0.smithi027.stdout:6/975: mkdir d4/de/d10f/d13d 0 2024-06-06T08:43:53.530 INFO:tasks.workunit.client.0.smithi027.stdout:6/976: creat d4/d9/df2/f13e x:0 0 0 2024-06-06T08:43:53.530 INFO:tasks.workunit.client.0.smithi027.stdout:6/977: write d4/d13/f21 [1102289,74152] 0 2024-06-06T08:43:53.532 INFO:tasks.workunit.client.0.smithi027.stdout:6/978: symlink d4/d1a/d2e/l13f 0 2024-06-06T08:43:53.534 INFO:tasks.workunit.client.0.smithi027.stdout:6/979: rmdir d4/d1a/d2e/d44/dce 39 2024-06-06T08:43:53.539 INFO:tasks.workunit.client.0.smithi027.stdout:4/991: dwrite d0/d1d/d2a/f57 [0,4194304] 0 2024-06-06T08:43:53.539 INFO:tasks.workunit.client.0.smithi027.stdout:4/992: chown d0/d63/d10c/d12e/l12f 6 1 2024-06-06T08:43:53.543 INFO:tasks.workunit.client.0.smithi027.stdout:4/993: symlink d0/d1/d6/d8/d82/d8b/dc9/da3/d10b/l14c 0 2024-06-06T08:43:53.546 INFO:tasks.workunit.client.0.smithi027.stdout:6/980: write d4/de/fb4 [3493777,31290] 0 2024-06-06T08:43:53.555 INFO:tasks.workunit.client.0.smithi027.stdout:4/994: link d0/d1d/d2a/d98/l10d d0/d1/d6/d8/d82/d8b/dc9/da3/daa/l14d 0 2024-06-06T08:43:53.558 INFO:tasks.workunit.client.0.smithi027.stdout:6/981: rename d4/de/d45/l85 to d4/d125/d18/l140 0 2024-06-06T08:43:53.559 INFO:tasks.workunit.client.0.smithi027.stdout:6/982: mknod d4/d125/d8f/d126/c141 0 2024-06-06T08:43:53.561 INFO:tasks.workunit.client.0.smithi027.stdout:6/983: link d4/d1a/d2e/fc1 d4/d1a/d2e/df3/f142 0 2024-06-06T08:43:53.561 INFO:tasks.workunit.client.0.smithi027.stdout:6/984: truncate d4/de/d45/d6a/f76 4392178 0 2024-06-06T08:43:53.571 INFO:tasks.workunit.client.0.smithi027.stdout:6/985: dread d4/d125/fd3 [0,4194304] 0 2024-06-06T08:43:53.571 INFO:tasks.workunit.client.0.smithi027.stdout:6/986: fsync d4/d125/d18/fcc 0 2024-06-06T08:43:53.572 INFO:tasks.workunit.client.0.smithi027.stdout:6/987: creat d4/de/d10f/d13d/f143 x:0 0 0 2024-06-06T08:43:53.573 INFO:tasks.workunit.client.0.smithi027.stdout:6/988: getdents d4/d9/df2 0 2024-06-06T08:43:53.575 INFO:tasks.workunit.client.0.smithi027.stdout:6/989: dread d4/d9/d49/f71 [0,4194304] 0 2024-06-06T08:43:53.578 INFO:tasks.workunit.client.0.smithi027.stdout:4/995: rename d0/d1d/d60/dc6/c67 to d0/d1d/d34/d7f/df4/c14e 0 2024-06-06T08:43:53.580 INFO:tasks.workunit.client.0.smithi027.stdout:4/996: mkdir d0/d1/d6/d8/d11f/d14f 0 2024-06-06T08:43:53.588 INFO:tasks.workunit.client.0.smithi027.stdout:0/828: sync 2024-06-06T08:43:53.588 INFO:tasks.workunit.client.0.smithi027.stdout:9/828: sync 2024-06-06T08:43:53.589 INFO:tasks.workunit.client.0.smithi027.stdout:0/829: truncate d15/d18/f1c 3113428 0 2024-06-06T08:43:53.590 INFO:tasks.workunit.client.0.smithi027.stdout:4/997: write d0/d1/d6/f2f [4547389,30353] 0 2024-06-06T08:43:53.590 INFO:tasks.workunit.client.0.smithi027.stdout:0/830: creat d15/d18/d23/d28/d31/dbb/f118 x:0 0 0 2024-06-06T08:43:53.590 INFO:tasks.workunit.client.0.smithi027.stdout:4/998: fdatasync d0/d63/fe4 0 2024-06-06T08:43:53.590 INFO:tasks.workunit.client.0.smithi027.stdout:4/999: readlink d0/d1d/d60/dc6/l5f 0 2024-06-06T08:43:53.591 INFO:tasks.workunit.client.0.smithi027.stdout:0/831: fsync d15/d18/d23/d28/d2a/d2d/f3a 0 2024-06-06T08:43:53.591 INFO:tasks.workunit.client.0.smithi027.stdout:6/990: dwrite d4/d125/d18/f129 [0,4194304] 0 2024-06-06T08:43:53.591 INFO:tasks.workunit.client.0.smithi027.stdout:9/829: dread d3/d4e/d75/dc/f8d [0,4194304] 0 2024-06-06T08:43:53.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/832: truncate f9 1787990 0 2024-06-06T08:43:53.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/833: write d15/d18/d40/d8c/fca [878060,23613] 0 2024-06-06T08:43:53.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/834: write d15/d18/d23/fba [721322,86817] 0 2024-06-06T08:43:53.594 INFO:tasks.workunit.client.0.smithi027.stdout:0/835: fsync d15/d18/d23/d6f/deb/fff 0 2024-06-06T08:43:53.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/830: chown d3/c26 0 1 2024-06-06T08:43:53.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/831: chown d3/ldd 4 1 2024-06-06T08:43:53.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/832: chown d3/lf4 800017769 1 2024-06-06T08:43:53.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/833: read - d3/d95/fec zero size 2024-06-06T08:43:53.596 INFO:tasks.workunit.client.0.smithi027.stdout:9/834: rmdir d3/d95/d34 39 2024-06-06T08:43:53.596 INFO:tasks.workunit.client.0.smithi027.stdout:9/835: chown d3/d4e/d75/dc/d9b/f11c 825328350 1 2024-06-06T08:43:53.596 INFO:tasks.workunit.client.0.smithi027.stdout:0/836: symlink d15/d18/d23/d28/d31/l119 0 2024-06-06T08:43:53.598 INFO:tasks.workunit.client.0.smithi027.stdout:0/837: creat d15/d18/d108/f11a x:0 0 0 2024-06-06T08:43:53.600 INFO:tasks.workunit.client.0.smithi027.stdout:9/836: symlink d3/d95/d34/d73/l12b 0 2024-06-06T08:43:53.602 INFO:tasks.workunit.client.0.smithi027.stdout:9/837: mkdir d3/d95/d23/d36/d3a/d84/db2/d12c 0 2024-06-06T08:43:53.603 INFO:tasks.workunit.client.0.smithi027.stdout:9/838: chown d3/d95/d34/l105 542597 1 2024-06-06T08:43:53.605 INFO:tasks.workunit.client.0.smithi027.stdout:9/839: mknod d3/d4e/d75/dc/d9b/c12d 0 2024-06-06T08:43:53.610 INFO:tasks.workunit.client.0.smithi027.stdout:6/991: dwrite d4/d125/d18/d10a/f10b [0,4194304] 0 2024-06-06T08:43:53.615 INFO:tasks.workunit.client.0.smithi027.stdout:0/838: dwrite d15/d18/dbe/f71 [0,4194304] 0 2024-06-06T08:43:53.632 INFO:tasks.workunit.client.0.smithi027.stdout:3/734: sync 2024-06-06T08:43:53.633 INFO:tasks.workunit.client.0.smithi027.stdout:2/806: sync 2024-06-06T08:43:53.633 INFO:tasks.workunit.client.0.smithi027.stdout:2/807: readlink d6/dd/dd9/d3f/d77/d55/d69/l95 0 2024-06-06T08:43:53.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/815: sync 2024-06-06T08:43:53.633 INFO:tasks.workunit.client.0.smithi027.stdout:8/700: sync 2024-06-06T08:43:53.633 INFO:tasks.workunit.client.0.smithi027.stdout:7/965: sync 2024-06-06T08:43:53.633 INFO:tasks.workunit.client.0.smithi027.stdout:7/966: chown d17/d32/d61/d75/fe5 38407 1 2024-06-06T08:43:53.633 INFO:tasks.workunit.client.0.smithi027.stdout:3/735: rmdir d1/d1f/d45/d5f/dca 39 2024-06-06T08:43:53.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/992: dwrite d4/d1a/d2e/d44/f74 [0,4194304] 0 2024-06-06T08:43:53.634 INFO:tasks.workunit.client.0.smithi027.stdout:7/967: creat d17/d32/d46/d83/d77/d8c/d91/f13d x:0 0 0 2024-06-06T08:43:53.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/816: getdents dc/da0 0 2024-06-06T08:43:53.635 INFO:tasks.workunit.client.0.smithi027.stdout:7/968: chown d17/d32/d6c/dc2/lb7 131 1 2024-06-06T08:43:53.635 INFO:tasks.workunit.client.0.smithi027.stdout:7/969: readlink d17/d32/d46/d83/l2b 0 2024-06-06T08:43:53.639 INFO:tasks.workunit.client.0.smithi027.stdout:7/970: dread d17/d32/d46/d83/d2a/fbf [0,4194304] 0 2024-06-06T08:43:53.640 INFO:tasks.workunit.client.0.smithi027.stdout:7/971: creat d17/d32/d46/d51/d52/d76/f13e x:0 0 0 2024-06-06T08:43:53.640 INFO:tasks.workunit.client.0.smithi027.stdout:7/972: fsync d17/d32/d46/d83/d77/d8c/d91/f13a 0 2024-06-06T08:43:53.640 INFO:tasks.workunit.client.0.smithi027.stdout:7/973: truncate d17/d32/d46/d83/d77/f11b 868414 0 2024-06-06T08:43:53.640 INFO:tasks.workunit.client.0.smithi027.stdout:7/974: read d17/d32/d46/d4f/d5e/f79 [1105234,64943] 0 2024-06-06T08:43:53.641 INFO:tasks.workunit.client.0.smithi027.stdout:7/975: creat d17/d32/d46/d4f/db6/dc8/f13f x:0 0 0 2024-06-06T08:43:53.642 INFO:tasks.workunit.client.0.smithi027.stdout:7/976: stat l16 0 2024-06-06T08:43:53.643 INFO:tasks.workunit.client.0.smithi027.stdout:7/977: mknod d17/d32/d46/d51/d52/d70/dce/de6/c140 0 2024-06-06T08:43:53.645 INFO:tasks.workunit.client.0.smithi027.stdout:8/701: dread d7/dc/d36/d83/f6e [0,4194304] 0 2024-06-06T08:43:53.650 INFO:tasks.workunit.client.0.smithi027.stdout:2/808: dwrite d6/dd/dd9/d1d/f45 [0,4194304] 0 2024-06-06T08:43:53.650 INFO:tasks.workunit.client.0.smithi027.stdout:2/809: chown d6/dd/dd9/d3f/cfd 134725464 1 2024-06-06T08:43:53.657 INFO:tasks.workunit.client.0.smithi027.stdout:6/993: rename d4/d1a/d2e/f5c to d4/d9/df2/f144 0 2024-06-06T08:43:53.657 INFO:tasks.workunit.client.0.smithi027.stdout:3/736: dwrite d1/d90/d44/d54/db4/fd3 [4194304,4194304] 0 2024-06-06T08:43:53.657 INFO:tasks.workunit.client.0.smithi027.stdout:3/737: stat d1/d8/d22/f37 0 2024-06-06T08:43:53.657 INFO:tasks.workunit.client.0.smithi027.stdout:2/810: mknod d6/dd/d6b/c10b 0 2024-06-06T08:43:53.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/811: symlink d6/d1e/d2b/d5a/db7/de2/l10c 0 2024-06-06T08:43:53.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/812: chown d6/dd/dd9/d3f/d77/d55/d69/d82/fa9 33491071 1 2024-06-06T08:43:53.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/813: stat d6/dd/dd9/d1d/f5b 0 2024-06-06T08:43:53.660 INFO:tasks.workunit.client.0.smithi027.stdout:6/994: mkdir d4/de/d45/d6a/d145 0 2024-06-06T08:43:53.660 INFO:tasks.workunit.client.0.smithi027.stdout:6/995: write d4/de/fb4 [4878254,111544] 0 2024-06-06T08:43:53.660 INFO:tasks.workunit.client.0.smithi027.stdout:6/996: stat d4/d1a/d2e/d44/dca/fec 0 2024-06-06T08:43:53.662 INFO:tasks.workunit.client.0.smithi027.stdout:1/817: rename dc/d67/d69/d32/d37/d56/df1/d4e/l88 to dc/d67/d69/d32/d37/d56/df1/lfd 0 2024-06-06T08:43:53.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/814: mkdir d6/d1e/dfb/d10d 0 2024-06-06T08:43:53.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/815: chown d6/dd/dd9/ddf/de1/fec 0 1 2024-06-06T08:43:53.665 INFO:tasks.workunit.client.0.smithi027.stdout:2/816: symlink d6/d1e/d2b/l10e 0 2024-06-06T08:43:53.666 INFO:tasks.workunit.client.0.smithi027.stdout:6/997: write d4/d1a/d2e/f3c [1207540,89208] 0 2024-06-06T08:43:53.667 INFO:tasks.workunit.client.0.smithi027.stdout:2/817: creat d6/d1e/d2b/dc5/f10f x:0 0 0 2024-06-06T08:43:53.667 INFO:tasks.workunit.client.0.smithi027.stdout:8/702: dwrite d7/dc/f69 [0,4194304] 0 2024-06-06T08:43:53.667 INFO:tasks.workunit.client.0.smithi027.stdout:8/703: chown d7/d2f/f98 0 1 2024-06-06T08:43:53.667 INFO:tasks.workunit.client.0.smithi027.stdout:6/998: creat d4/d13/dba/f146 x:0 0 0 2024-06-06T08:43:53.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/818: mknod d6/dd/dd9/d3f/d48/d83/c110 0 2024-06-06T08:43:53.671 INFO:tasks.workunit.client.0.smithi027.stdout:1/818: dread dc/d67/d69/d32/d37/d56/df1/f87 [0,4194304] 0 2024-06-06T08:43:53.672 INFO:tasks.workunit.client.0.smithi027.stdout:8/704: link d7/dc/f12 d7/dc/d36/d83/d60/d93/fe4 0 2024-06-06T08:43:53.673 INFO:tasks.workunit.client.0.smithi027.stdout:8/705: dread d7/d2f/fb8 [0,4194304] 0 2024-06-06T08:43:53.675 INFO:tasks.workunit.client.0.smithi027.stdout:3/738: dwrite d1/d8/d22/d2d/fbb [4194304,4194304] 0 2024-06-06T08:43:53.675 INFO:tasks.workunit.client.0.smithi027.stdout:3/739: fdatasync d1/d8/f15 0 2024-06-06T08:43:53.675 INFO:tasks.workunit.client.0.smithi027.stdout:2/819: readlink d6/dd/dd9/d3f/d77/d55/d69/l76 0 2024-06-06T08:43:53.680 INFO:tasks.workunit.client.0.smithi027.stdout:6/999: symlink d4/d13/l147 0 2024-06-06T08:43:53.681 INFO:tasks.workunit.client.0.smithi027.stdout:8/706: dread d7/d2f/f77 [0,4194304] 0 2024-06-06T08:43:53.681 INFO:tasks.workunit.client.0.smithi027.stdout:3/740: rmdir d1/d1f/dbc 39 2024-06-06T08:43:53.684 INFO:tasks.workunit.client.0.smithi027.stdout:3/741: mknod d1/d1f/d45/d5f/dd6/ce5 0 2024-06-06T08:43:53.685 INFO:tasks.workunit.client.0.smithi027.stdout:3/742: truncate d1/d8/d22/d27/d50/f51 3956741 0 2024-06-06T08:43:53.686 INFO:tasks.workunit.client.0.smithi027.stdout:3/743: mknod d1/d8/d22/d27/d79/ce6 0 2024-06-06T08:43:53.688 INFO:tasks.workunit.client.0.smithi027.stdout:3/744: mknod d1/d1f/d45/d5f/dca/ce7 0 2024-06-06T08:43:53.688 INFO:tasks.workunit.client.0.smithi027.stdout:3/745: stat d1/d1f/d45/d5f/dd6 0 2024-06-06T08:43:53.694 INFO:tasks.workunit.client.0.smithi027.stdout:1/819: write dc/d67/d69/d32/d71/fbb [1999297,22224] 0 2024-06-06T08:43:53.700 INFO:tasks.workunit.client.0.smithi027.stdout:8/707: link d7/dc/d36/fdf d7/dc/d36/d41/dbf/fe5 0 2024-06-06T08:43:53.701 INFO:tasks.workunit.client.0.smithi027.stdout:7/978: rename d17/d32/d46/d51/d52/d70/dce/l107 to d17/d32/d46/d4f/d5e/d7b/d7c/d92/d12f/l141 0 2024-06-06T08:43:53.701 INFO:tasks.workunit.client.0.smithi027.stdout:7/979: write d17/d32/d46/d4f/f50 [3700304,129162] 0 2024-06-06T08:43:53.703 INFO:tasks.workunit.client.0.smithi027.stdout:7/980: mkdir d17/d32/d46/d83/d77/d8c/d91/dfc/d142 0 2024-06-06T08:43:53.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/820: rename d6/dd/dd9/ddf to d6/dd/dd9/d3f/d111 0 2024-06-06T08:43:53.704 INFO:tasks.workunit.client.0.smithi027.stdout:3/746: dwrite d1/d8/f92 [0,4194304] 0 2024-06-06T08:43:53.706 INFO:tasks.workunit.client.0.smithi027.stdout:7/981: mknod d17/d32/d46/d4f/db6/c143 0 2024-06-06T08:43:53.708 INFO:tasks.workunit.client.0.smithi027.stdout:3/747: symlink d1/d8/d22/d27/d79/le8 0 2024-06-06T08:43:53.710 INFO:tasks.workunit.client.0.smithi027.stdout:3/748: link d1/d1f/l35 d1/d90/d6d/dac/le9 0 2024-06-06T08:43:53.710 INFO:tasks.workunit.client.0.smithi027.stdout:3/749: fsync d1/d90/d44/fb0 0 2024-06-06T08:43:53.710 INFO:tasks.workunit.client.0.smithi027.stdout:3/750: stat d1/f41 0 2024-06-06T08:43:53.711 INFO:tasks.workunit.client.0.smithi027.stdout:2/821: mknod d6/d3b/d80/c112 0 2024-06-06T08:43:53.711 INFO:tasks.workunit.client.0.smithi027.stdout:7/982: mknod d17/d32/d46/d51/d52/d70/c144 0 2024-06-06T08:43:53.712 INFO:tasks.workunit.client.0.smithi027.stdout:3/751: mkdir d1/d90/d6d/dac/dea 0 2024-06-06T08:43:53.712 INFO:tasks.workunit.client.0.smithi027.stdout:7/983: mknod d17/d32/d46/d83/d2a/d3b/dc7/d124/c145 0 2024-06-06T08:43:53.713 INFO:tasks.workunit.client.0.smithi027.stdout:7/984: truncate d17/d32/d46/d83/d77/d8c/d90/fcf 207259 0 2024-06-06T08:43:53.714 INFO:tasks.workunit.client.0.smithi027.stdout:2/822: link d6/d1e/d2b/d5a/db7/lcd d6/d1e/dfb/l113 0 2024-06-06T08:43:53.714 INFO:tasks.workunit.client.0.smithi027.stdout:2/823: readlink d6/d3b/d80/lee 0 2024-06-06T08:43:53.714 INFO:tasks.workunit.client.0.smithi027.stdout:3/752: mkdir d1/d90/d44/d54/db4/deb 0 2024-06-06T08:43:53.716 INFO:tasks.workunit.client.0.smithi027.stdout:7/985: getdents d17/d32/d46/d51/d52/d70/dce/de6 0 2024-06-06T08:43:53.716 INFO:tasks.workunit.client.0.smithi027.stdout:2/824: mknod d6/dd/dd9/d3f/d77/de5/c114 0 2024-06-06T08:43:53.716 INFO:tasks.workunit.client.0.smithi027.stdout:2/825: fdatasync d6/dd/dd9/d3f/db0/fdb 0 2024-06-06T08:43:53.717 INFO:tasks.workunit.client.0.smithi027.stdout:3/753: creat d1/d90/d44/d54/db4/deb/fec x:0 0 0 2024-06-06T08:43:53.724 INFO:tasks.workunit.client.0.smithi027.stdout:8/708: read d7/dc/d36/d83/d60/f7a [720732,125983] 0 2024-06-06T08:43:53.724 INFO:tasks.workunit.client.0.smithi027.stdout:1/820: dwrite dc/d67/d69/d32/d37/d56/ff7 [0,4194304] 0 2024-06-06T08:43:53.724 INFO:tasks.workunit.client.0.smithi027.stdout:8/709: getdents d7/dc/d36/d41/dbf/dd7 0 2024-06-06T08:43:53.725 INFO:tasks.workunit.client.0.smithi027.stdout:3/754: write d1/f41 [3206790,62747] 0 2024-06-06T08:43:53.725 INFO:tasks.workunit.client.0.smithi027.stdout:1/821: chown dc/de/d3d/l4b 15 1 2024-06-06T08:43:53.727 INFO:tasks.workunit.client.0.smithi027.stdout:8/710: getdents d7/dc/d36/d83/d60/d93/dac 0 2024-06-06T08:43:53.727 INFO:tasks.workunit.client.0.smithi027.stdout:2/826: dread d6/f14 [0,4194304] 0 2024-06-06T08:43:53.728 INFO:tasks.workunit.client.0.smithi027.stdout:3/755: mknod d1/d90/d44/d54/ced 0 2024-06-06T08:43:53.740 INFO:tasks.workunit.client.0.smithi027.stdout:3/756: write d1/d90/f77 [2107658,57519] 0 2024-06-06T08:43:53.741 INFO:tasks.workunit.client.0.smithi027.stdout:3/757: rmdir d1/d90 39 2024-06-06T08:43:53.742 INFO:tasks.workunit.client.0.smithi027.stdout:8/711: write d7/d13/f17 [2866007,28374] 0 2024-06-06T08:43:53.742 INFO:tasks.workunit.client.0.smithi027.stdout:1/822: write dc/d6d/fde [1356800,59975] 0 2024-06-06T08:43:53.742 INFO:tasks.workunit.client.0.smithi027.stdout:7/986: dwrite d17/d32/d46/d83/f45 [0,4194304] 0 2024-06-06T08:43:53.743 INFO:tasks.workunit.client.0.smithi027.stdout:3/758: rename d1/d90/f77 to d1/d90/d6d/dac/fee 0 2024-06-06T08:43:53.743 INFO:tasks.workunit.client.0.smithi027.stdout:8/712: chown d7/dc/d36/f75 449424 1 2024-06-06T08:43:53.745 INFO:tasks.workunit.client.0.smithi027.stdout:7/987: chown d17/d32/d46/d51/f6f 9594131 1 2024-06-06T08:43:53.747 INFO:tasks.workunit.client.0.smithi027.stdout:8/713: creat d7/dc/d37/d46/fe6 x:0 0 0 2024-06-06T08:43:53.748 INFO:tasks.workunit.client.0.smithi027.stdout:8/714: chown d7/dc/d36/d83/d60/cd4 3146061 1 2024-06-06T08:43:53.748 INFO:tasks.workunit.client.0.smithi027.stdout:1/823: rmdir dc 39 2024-06-06T08:43:53.751 INFO:tasks.workunit.client.0.smithi027.stdout:1/824: dread dc/d67/d69/d32/d37/ff4 [0,4194304] 0 2024-06-06T08:43:53.751 INFO:tasks.workunit.client.0.smithi027.stdout:2/827: dwrite d6/dd/dd9/d1d/f86 [0,4194304] 0 2024-06-06T08:43:53.754 INFO:tasks.workunit.client.0.smithi027.stdout:3/759: mknod d1/d1f/d45/d5f/cef 0 2024-06-06T08:43:53.754 INFO:tasks.workunit.client.0.smithi027.stdout:3/760: chown d1 4863 1 2024-06-06T08:43:53.757 INFO:tasks.workunit.client.0.smithi027.stdout:8/715: link d7/dc/d36/d83/f91 d7/fe7 0 2024-06-06T08:43:53.759 INFO:tasks.workunit.client.0.smithi027.stdout:2/828: mkdir d6/d1e/d2b/d5a/db7/d115 0 2024-06-06T08:43:53.760 INFO:tasks.workunit.client.0.smithi027.stdout:3/761: mkdir d1/d1f/d45/df0 0 2024-06-06T08:43:53.763 INFO:tasks.workunit.client.0.smithi027.stdout:7/988: dwrite d17/d32/d46/d83/d2a/d3b/dc7/f119 [0,4194304] 0 2024-06-06T08:43:53.764 INFO:tasks.workunit.client.0.smithi027.stdout:2/829: mknod d6/dd/dd9/d1d/d4d/c116 0 2024-06-06T08:43:53.766 INFO:tasks.workunit.client.0.smithi027.stdout:2/830: creat d6/dd/dd9/d3f/d77/d55/f117 x:0 0 0 2024-06-06T08:43:53.768 INFO:tasks.workunit.client.0.smithi027.stdout:7/989: mknod d17/d32/d46/d51/d52/c146 0 2024-06-06T08:43:53.769 INFO:tasks.workunit.client.0.smithi027.stdout:2/831: mknod d6/dd/c118 0 2024-06-06T08:43:53.773 INFO:tasks.workunit.client.0.smithi027.stdout:8/716: dwrite d7/dc/d36/d3c/f61 [0,4194304] 0 2024-06-06T08:43:53.773 INFO:tasks.workunit.client.0.smithi027.stdout:8/717: dread - d7/dc/d37/d46/fe6 zero size 2024-06-06T08:43:53.773 INFO:tasks.workunit.client.0.smithi027.stdout:7/990: unlink d17/d32/d61/d75/fec 0 2024-06-06T08:43:53.773 INFO:tasks.workunit.client.0.smithi027.stdout:3/762: read d1/d8/f9 [497088,40915] 0 2024-06-06T08:43:53.775 INFO:tasks.workunit.client.0.smithi027.stdout:8/718: mkdir d7/dc/d36/d83/d60/d93/dac/de8 0 2024-06-06T08:43:53.775 INFO:tasks.workunit.client.0.smithi027.stdout:3/763: mkdir d1/d1f/d45/d5f/dd6/df1 0 2024-06-06T08:43:53.778 INFO:tasks.workunit.client.0.smithi027.stdout:3/764: link d1/d8/d22/d27/d50/l7a d1/d90/d6d/dac/lf2 0 2024-06-06T08:43:53.781 INFO:tasks.workunit.client.0.smithi027.stdout:3/765: creat d1/d1f/d45/ff3 x:0 0 0 2024-06-06T08:43:53.795 INFO:tasks.workunit.client.0.smithi027.stdout:8/719: dread d7/dc/f9f [0,4194304] 0 2024-06-06T08:43:53.797 INFO:tasks.workunit.client.0.smithi027.stdout:8/720: dread d7/dc/d36/d83/f6e [4194304,4194304] 0 2024-06-06T08:43:53.801 INFO:tasks.workunit.client.0.smithi027.stdout:7/991: rename d17/d32/d46/d83/d77/d8c/d91/ca6 to d17/d32/d46/d51/d52/c147 0 2024-06-06T08:43:53.803 INFO:tasks.workunit.client.0.smithi027.stdout:8/721: mkdir d7/d13/d15/de9 0 2024-06-06T08:43:53.804 INFO:tasks.workunit.client.0.smithi027.stdout:7/992: dread d17/d47/ff1 [0,4194304] 0 2024-06-06T08:43:53.804 INFO:tasks.workunit.client.0.smithi027.stdout:8/722: symlink d7/dc/d36/d41/d85/db7/dd3/lea 0 2024-06-06T08:43:53.805 INFO:tasks.workunit.client.0.smithi027.stdout:8/723: truncate d7/d2f/f3b 4744141 0 2024-06-06T08:43:53.805 INFO:tasks.workunit.client.0.smithi027.stdout:3/766: dwrite d1/d8/d22/d2d/fbb [0,4194304] 0 2024-06-06T08:43:53.808 INFO:tasks.workunit.client.0.smithi027.stdout:3/767: mkdir d1/d1f/df4 0 2024-06-06T08:43:53.809 INFO:tasks.workunit.client.0.smithi027.stdout:3/768: creat d1/d90/d6d/dac/dea/ff5 x:0 0 0 2024-06-06T08:43:53.810 INFO:tasks.workunit.client.0.smithi027.stdout:2/832: dwrite d6/f6e [0,4194304] 0 2024-06-06T08:43:53.813 INFO:tasks.workunit.client.0.smithi027.stdout:2/833: dread d6/dd/dd9/d3f/d77/d55/d69/d82/fa3 [0,4194304] 0 2024-06-06T08:43:53.815 INFO:tasks.workunit.client.0.smithi027.stdout:3/769: unlink d1/d1f/d4d/dd5/l83 0 2024-06-06T08:43:53.820 INFO:tasks.workunit.client.0.smithi027.stdout:3/770: link d1/d8/d22/d27/c1b d1/d90/d44/cf6 0 2024-06-06T08:43:53.824 INFO:tasks.workunit.client.0.smithi027.stdout:7/993: dwrite d17/d32/d46/d4f/d5e/fb0 [4194304,4194304] 0 2024-06-06T08:43:53.831 INFO:tasks.workunit.client.0.smithi027.stdout:3/771: dread d1/f20 [0,4194304] 0 2024-06-06T08:43:53.831 INFO:tasks.workunit.client.0.smithi027.stdout:3/772: fdatasync d1/d8/d22/d27/f78 0 2024-06-06T08:43:53.831 INFO:tasks.workunit.client.0.smithi027.stdout:8/724: dwrite d7/dc/d36/d83/f6e [0,4194304] 0 2024-06-06T08:43:53.832 INFO:tasks.workunit.client.0.smithi027.stdout:3/773: mknod d1/d1f/d45/d5f/dca/cf7 0 2024-06-06T08:43:53.832 INFO:tasks.workunit.client.0.smithi027.stdout:2/834: dwrite d6/dd/dd9/d3f/d77/d55/f6d [0,4194304] 0 2024-06-06T08:43:53.832 INFO:tasks.workunit.client.0.smithi027.stdout:8/725: write d7/d13/fa0 [169794,33191] 0 2024-06-06T08:43:53.837 INFO:tasks.workunit.client.0.smithi027.stdout:2/835: creat d6/d1e/dfb/f119 x:0 0 0 2024-06-06T08:43:53.837 INFO:tasks.workunit.client.0.smithi027.stdout:2/836: chown d6/dd/dd9/d3f/fd8 2 1 2024-06-06T08:43:53.840 INFO:tasks.workunit.client.0.smithi027.stdout:2/837: unlink d6/dd/dd9/d1d/d4d/ffc 0 2024-06-06T08:43:53.842 INFO:tasks.workunit.client.0.smithi027.stdout:8/726: mknod d7/dc/d37/ceb 0 2024-06-06T08:43:53.844 INFO:tasks.workunit.client.0.smithi027.stdout:2/838: link d6/d1e/d2b/ldd d6/dd/dd9/d3f/db0/l11a 0 2024-06-06T08:43:53.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/774: dwrite d1/d8/d22/f73 [4194304,4194304] 0 2024-06-06T08:43:53.846 INFO:tasks.workunit.client.0.smithi027.stdout:3/775: write d1/d90/d6d/dac/dea/ff5 [352692,42672] 0 2024-06-06T08:43:53.848 INFO:tasks.workunit.client.0.smithi027.stdout:8/727: dread d7/d13/f24 [0,4194304] 0 2024-06-06T08:43:53.848 INFO:tasks.workunit.client.0.smithi027.stdout:3/776: truncate d1/d8/d22/f46 1382007 0 2024-06-06T08:43:53.848 INFO:tasks.workunit.client.0.smithi027.stdout:3/777: chown d1/d90/d6d/dac/lf2 147 1 2024-06-06T08:43:53.848 INFO:tasks.workunit.client.0.smithi027.stdout:3/778: fdatasync d1/d8/d22/f73 0 2024-06-06T08:43:53.849 INFO:tasks.workunit.client.0.smithi027.stdout:8/728: link d7/dc/d36/d83/d60/d93/lb3 d7/dc/d36/d83/d60/d93/dac/de8/lec 0 2024-06-06T08:43:53.850 INFO:tasks.workunit.client.0.smithi027.stdout:3/779: mknod d1/d1f/d45/df0/cf8 0 2024-06-06T08:43:53.857 INFO:tasks.workunit.client.0.smithi027.stdout:2/839: write d6/dd/dd9/d3f/d77/d55/d69/f6c [1108002,66581] 0 2024-06-06T08:43:53.857 INFO:tasks.workunit.client.0.smithi027.stdout:2/840: write d6/d1e/dfb/ffe [297204,117958] 0 2024-06-06T08:43:53.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/780: dwrite d1/d1f/f6f [0,4194304] 0 2024-06-06T08:43:53.870 INFO:tasks.workunit.client.0.smithi027.stdout:3/781: mknod d1/d90/d6d/dac/de0/cf9 0 2024-06-06T08:43:53.871 INFO:tasks.workunit.client.0.smithi027.stdout:3/782: mknod d1/d1f/df4/cfa 0 2024-06-06T08:43:53.871 INFO:tasks.workunit.client.0.smithi027.stdout:3/783: truncate d1/d90/d6d/f9a 5035525 0 2024-06-06T08:43:53.873 INFO:tasks.workunit.client.0.smithi027.stdout:3/784: symlink d1/lfb 0 2024-06-06T08:43:53.873 INFO:tasks.workunit.client.0.smithi027.stdout:8/729: dwrite d7/dc/d36/fdf [0,4194304] 0 2024-06-06T08:43:53.876 INFO:tasks.workunit.client.0.smithi027.stdout:8/730: truncate d7/d2f/f87 4275028 0 2024-06-06T08:43:53.876 INFO:tasks.workunit.client.0.smithi027.stdout:8/731: write d7/d9c/daf/fe3 [644666,37142] 0 2024-06-06T08:43:53.878 INFO:tasks.workunit.client.0.smithi027.stdout:8/732: symlink d7/dc/d36/d41/dbf/dc3/led 0 2024-06-06T08:43:53.878 INFO:tasks.workunit.client.0.smithi027.stdout:8/733: write d7/dc/d36/d83/f91 [278533,128339] 0 2024-06-06T08:43:53.878 INFO:tasks.workunit.client.0.smithi027.stdout:8/734: mkdir d7/dc/d36/d41/d85/db7/dd3/dee 0 2024-06-06T08:43:53.879 INFO:tasks.workunit.client.0.smithi027.stdout:2/841: dwrite d6/dd/dd9/d3f/d77/d55/f74 [0,4194304] 0 2024-06-06T08:43:53.879 INFO:tasks.workunit.client.0.smithi027.stdout:2/842: chown d6/d1e/d2b/fe0 10612 1 2024-06-06T08:43:53.880 INFO:tasks.workunit.client.0.smithi027.stdout:8/735: creat d7/d13/d15/fef x:0 0 0 2024-06-06T08:43:53.883 INFO:tasks.workunit.client.0.smithi027.stdout:2/843: getdents d6/dd/dd9/d3f/db0 0 2024-06-06T08:43:53.887 INFO:tasks.workunit.client.0.smithi027.stdout:2/844: creat d6/d1e/d2b/d5a/db7/de2/f11b x:0 0 0 2024-06-06T08:43:53.891 INFO:tasks.workunit.client.0.smithi027.stdout:3/785: dwrite d1/d8/d22/d27/d50/fc0 [0,4194304] 0 2024-06-06T08:43:53.892 INFO:tasks.workunit.client.0.smithi027.stdout:8/736: unlink d7/dc/d37/d46/c8f 0 2024-06-06T08:43:53.895 INFO:tasks.workunit.client.0.smithi027.stdout:8/737: dread d7/fe7 [0,4194304] 0 2024-06-06T08:43:53.900 INFO:tasks.workunit.client.0.smithi027.stdout:3/786: creat d1/d1f/d4d/dd5/ffc x:0 0 0 2024-06-06T08:43:53.902 INFO:tasks.workunit.client.0.smithi027.stdout:2/845: rename d6/dd/dd9/d3f/f49 to d6/d1e/d2b/d5a/db7/d115/f11c 0 2024-06-06T08:43:53.904 INFO:tasks.workunit.client.0.smithi027.stdout:8/738: creat d7/dc/d36/d83/d60/d93/dac/ff0 x:0 0 0 2024-06-06T08:43:53.905 INFO:tasks.workunit.client.0.smithi027.stdout:3/787: write d1/d90/d6d/dac/ddb/fb6 [434805,103729] 0 2024-06-06T08:43:53.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/788: stat d1/d8/d22/l5a 0 2024-06-06T08:43:53.910 INFO:tasks.workunit.client.0.smithi027.stdout:8/739: mknod d7/dc/d36/d3c/cf1 0 2024-06-06T08:43:53.911 INFO:tasks.workunit.client.0.smithi027.stdout:8/740: readlink d7/dc/d36/d3c/lbb 0 2024-06-06T08:43:53.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/789: chown d1/d90/d6d/dac/ddb/l4a 8329107 1 2024-06-06T08:43:53.913 INFO:tasks.workunit.client.0.smithi027.stdout:8/741: creat d7/dc/d36/d83/ff2 x:0 0 0 2024-06-06T08:43:53.916 INFO:tasks.workunit.client.0.smithi027.stdout:8/742: symlink d7/dc/d37/lf3 0 2024-06-06T08:43:53.918 INFO:tasks.workunit.client.0.smithi027.stdout:2/846: dread d6/dd/dd9/f29 [0,4194304] 0 2024-06-06T08:43:53.918 INFO:tasks.workunit.client.0.smithi027.stdout:8/743: creat d7/dc/d36/d41/ff4 x:0 0 0 2024-06-06T08:43:53.918 INFO:tasks.workunit.client.0.smithi027.stdout:8/744: read d7/f4e [1654250,99017] 0 2024-06-06T08:43:53.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/745: symlink d7/dc/d36/d41/d85/db7/dd3/lf5 0 2024-06-06T08:43:53.919 INFO:tasks.workunit.client.0.smithi027.stdout:2/847: mknod d6/d1e/dfb/c11d 0 2024-06-06T08:43:53.921 INFO:tasks.workunit.client.0.smithi027.stdout:2/848: write d6/dd/dd9/f30 [643974,70694] 0 2024-06-06T08:43:53.921 INFO:tasks.workunit.client.0.smithi027.stdout:3/790: rename d1/d8/d22/d27/d50 to d1/d8/d22/d27/d79/dfd 0 2024-06-06T08:43:53.923 INFO:tasks.workunit.client.0.smithi027.stdout:3/791: creat d1/d8/da9/ffe x:0 0 0 2024-06-06T08:43:53.923 INFO:tasks.workunit.client.0.smithi027.stdout:3/792: chown d1/d1f/d45 824593 1 2024-06-06T08:43:53.924 INFO:tasks.workunit.client.0.smithi027.stdout:8/746: mkdir d7/dc/d36/d41/d85/df6 0 2024-06-06T08:43:53.924 INFO:tasks.workunit.client.0.smithi027.stdout:2/849: mknod d6/d3b/d61/c11e 0 2024-06-06T08:43:53.924 INFO:tasks.workunit.client.0.smithi027.stdout:2/850: truncate d6/fc0 790988 0 2024-06-06T08:43:53.925 INFO:tasks.workunit.client.0.smithi027.stdout:3/793: creat d1/d1f/d45/df0/fff x:0 0 0 2024-06-06T08:43:53.926 INFO:tasks.workunit.client.0.smithi027.stdout:3/794: write d1/d8/d59/fc5 [434617,78907] 0 2024-06-06T08:43:53.927 INFO:tasks.workunit.client.0.smithi027.stdout:8/747: creat d7/dc/d37/ff7 x:0 0 0 2024-06-06T08:43:53.928 INFO:tasks.workunit.client.0.smithi027.stdout:8/748: symlink d7/dc/d37/dc8/lf8 0 2024-06-06T08:43:53.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/851: write d6/dd/dd9/d3f/d77/d55/f6d [4122194,130713] 0 2024-06-06T08:43:53.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/852: stat d6/d1e/cea 0 2024-06-06T08:43:53.939 INFO:tasks.workunit.client.0.smithi027.stdout:2/853: creat d6/dd/dd9/d3f/d111/de1/f11f x:0 0 0 2024-06-06T08:43:53.943 INFO:tasks.workunit.client.0.smithi027.stdout:8/749: dwrite d7/dc/d36/d83/fc7 [0,4194304] 0 2024-06-06T08:43:53.948 INFO:tasks.workunit.client.0.smithi027.stdout:3/795: write d1/d8/d22/f37 [235160,19789] 0 2024-06-06T08:43:53.948 INFO:tasks.workunit.client.0.smithi027.stdout:3/796: write d1/d90/d6d/dac/fe1 [958876,52671] 0 2024-06-06T08:43:53.948 INFO:tasks.workunit.client.0.smithi027.stdout:3/797: chown d1/d8/d22/d27/d79/dfd/fbe 1242 1 2024-06-06T08:43:53.951 INFO:tasks.workunit.client.0.smithi027.stdout:3/798: mknod d1/d90/d44/c100 0 2024-06-06T08:43:53.952 INFO:tasks.workunit.client.0.smithi027.stdout:2/854: dread d6/dd/dd9/d3f/d77/d55/f74 [0,4194304] 0 2024-06-06T08:43:53.952 INFO:tasks.workunit.client.0.smithi027.stdout:2/855: truncate d6/dd/dd9/d3f/d48/ff1 123888 0 2024-06-06T08:43:53.952 INFO:tasks.workunit.client.0.smithi027.stdout:2/856: chown d6/d1e/dfb/c11d 60249 1 2024-06-06T08:43:53.953 INFO:tasks.workunit.client.0.smithi027.stdout:3/799: mknod d1/d8/d22/d2d/c101 0 2024-06-06T08:43:53.954 INFO:tasks.workunit.client.0.smithi027.stdout:2/857: fdatasync d6/dd/f17 0 2024-06-06T08:43:53.957 INFO:tasks.workunit.client.0.smithi027.stdout:3/800: mkdir d1/d1f/d45/d102 0 2024-06-06T08:43:53.957 INFO:tasks.workunit.client.0.smithi027.stdout:3/801: chown d1/d90/d6d/f9b 1 1 2024-06-06T08:43:53.959 INFO:tasks.workunit.client.0.smithi027.stdout:2/858: mkdir d6/dd/dd9/d3f/d77/d55/d120 0 2024-06-06T08:43:53.959 INFO:tasks.workunit.client.0.smithi027.stdout:8/750: dwrite d7/dc/d36/f7b [4194304,4194304] 0 2024-06-06T08:43:53.961 INFO:tasks.workunit.client.0.smithi027.stdout:8/751: mknod d7/dc/dde/cf9 0 2024-06-06T08:43:53.962 INFO:tasks.workunit.client.0.smithi027.stdout:3/802: mknod d1/d1f/d45/d5f/c103 0 2024-06-06T08:43:53.963 INFO:tasks.workunit.client.0.smithi027.stdout:3/803: truncate d1/d1f/d45/df0/fff 498631 0 2024-06-06T08:43:53.963 INFO:tasks.workunit.client.0.smithi027.stdout:2/859: rename d6/dd/c41 to d6/d1e/d2b/d5a/db7/d115/c121 0 2024-06-06T08:43:53.964 INFO:tasks.workunit.client.0.smithi027.stdout:8/752: truncate d7/dc/d36/d83/d60/d93/dac/f7f 127289 0 2024-06-06T08:43:53.967 INFO:tasks.workunit.client.0.smithi027.stdout:2/860: mknod d6/d1e/d2b/d5a/d7b/c122 0 2024-06-06T08:43:53.970 INFO:tasks.workunit.client.0.smithi027.stdout:2/861: dread d6/f8b [0,4194304] 0 2024-06-06T08:43:53.972 INFO:tasks.workunit.client.0.smithi027.stdout:2/862: getdents d6/d3b/d61 0 2024-06-06T08:43:53.972 INFO:tasks.workunit.client.0.smithi027.stdout:2/863: chown d6/d3b/d80/lee 56401887 1 2024-06-06T08:43:53.978 INFO:tasks.workunit.client.0.smithi027.stdout:2/864: rename d6/d3b/d61 to d6/dd/dd9/d3f/db0/d123 0 2024-06-06T08:43:53.979 INFO:tasks.workunit.client.0.smithi027.stdout:2/865: link d6/dd/dd9/d3f/d48/d83/de3/de4/ceb d6/dd/dd9/d3f/d111/de1/c124 0 2024-06-06T08:43:53.981 INFO:tasks.workunit.client.0.smithi027.stdout:8/753: dread d7/dc/d36/d3c/f79 [0,4194304] 0 2024-06-06T08:43:53.982 INFO:tasks.workunit.client.0.smithi027.stdout:3/804: dwrite d1/d8/d22/d2d/faa [0,4194304] 0 2024-06-06T08:43:53.982 INFO:tasks.workunit.client.0.smithi027.stdout:3/805: write d1/d8/d22/f46 [1936407,79393] 0 2024-06-06T08:43:53.982 INFO:tasks.workunit.client.0.smithi027.stdout:3/806: fdatasync d1/d8/d22/d27/f32 0 2024-06-06T08:43:53.983 INFO:tasks.workunit.client.0.smithi027.stdout:8/754: read d7/dc/d36/d83/f1d [2312589,32056] 0 2024-06-06T08:43:53.985 INFO:tasks.workunit.client.0.smithi027.stdout:8/755: getdents d7/dc/dde 0 2024-06-06T08:43:53.988 INFO:tasks.workunit.client.0.smithi027.stdout:3/807: dread d1/d90/d6d/dac/ddb/f76 [0,4194304] 0 2024-06-06T08:43:53.989 INFO:tasks.workunit.client.0.smithi027.stdout:3/808: rmdir d1/d1f/df4 39 2024-06-06T08:43:53.993 INFO:tasks.workunit.client.0.smithi027.stdout:8/756: write d7/dc/f69 [3052452,107990] 0 2024-06-06T08:43:53.993 INFO:tasks.workunit.client.0.smithi027.stdout:8/757: write d7/fd0 [455666,52457] 0 2024-06-06T08:43:53.993 INFO:tasks.workunit.client.0.smithi027.stdout:8/758: dread - d7/dc/d37/ff7 zero size 2024-06-06T08:43:53.994 INFO:tasks.workunit.client.0.smithi027.stdout:8/759: symlink d7/dc/dde/lfa 0 2024-06-06T08:43:53.999 INFO:tasks.workunit.client.0.smithi027.stdout:2/866: dwrite d6/d1e/d2b/d5a/fef [0,4194304] 0 2024-06-06T08:43:54.039 INFO:tasks.workunit.client.0.smithi027.stdout:3/809: dwrite d1/d90/d6d/dac/ddb/f8d [0,4194304] 0 2024-06-06T08:43:54.039 INFO:tasks.workunit.client.0.smithi027.stdout:3/810: fsync d1/d90/d6d/dac/ddb/f3a 0 2024-06-06T08:43:54.039 INFO:tasks.workunit.client.0.smithi027.stdout:0/839: sync 2024-06-06T08:43:54.040 INFO:tasks.workunit.client.0.smithi027.stdout:7/994: sync 2024-06-06T08:43:54.040 INFO:tasks.workunit.client.0.smithi027.stdout:1/825: sync 2024-06-06T08:43:54.040 INFO:tasks.workunit.client.0.smithi027.stdout:1/826: fdatasync dc/d67/d69/d32/d37/d56/df1/d80/da6/ff2 0 2024-06-06T08:43:54.040 INFO:tasks.workunit.client.0.smithi027.stdout:9/840: sync 2024-06-06T08:43:54.041 INFO:tasks.workunit.client.0.smithi027.stdout:3/811: stat d1/d90/d6d/f81 0 2024-06-06T08:43:54.042 INFO:tasks.workunit.client.0.smithi027.stdout:0/840: link d15/d18/d23/d38/d68/f8f d15/d18/d40/d8c/f11b 0 2024-06-06T08:43:54.043 INFO:tasks.workunit.client.0.smithi027.stdout:1/827: link dc/d67/d69/d32/d37/d56/df1/d80/fd4 dc/d67/d69/d32/d37/d56/df1/d80/ffe 0 2024-06-06T08:43:54.043 INFO:tasks.workunit.client.0.smithi027.stdout:1/828: write dc/d67/d69/f52 [2212642,106612] 0 2024-06-06T08:43:54.045 INFO:tasks.workunit.client.0.smithi027.stdout:3/812: creat d1/d1f/d4d/dd5/f104 x:0 0 0 2024-06-06T08:43:54.045 INFO:tasks.workunit.client.0.smithi027.stdout:3/813: fdatasync d1/d1f/d45/d5f/f7c 0 2024-06-06T08:43:54.045 INFO:tasks.workunit.client.0.smithi027.stdout:1/829: mknod dc/de/d3d/d4f/cff 0 2024-06-06T08:43:54.047 INFO:tasks.workunit.client.0.smithi027.stdout:1/830: creat dc/d67/d69/d32/d37/d56/df1/d4e/f100 x:0 0 0 2024-06-06T08:43:54.047 INFO:tasks.workunit.client.0.smithi027.stdout:8/760: dwrite d7/dc/d36/d3c/f61 [0,4194304] 0 2024-06-06T08:43:54.048 INFO:tasks.workunit.client.0.smithi027.stdout:1/831: mkdir dc/de/de4/d101 0 2024-06-06T08:43:54.056 INFO:tasks.workunit.client.0.smithi027.stdout:3/814: dread d1/f20 [0,4194304] 0 2024-06-06T08:43:54.056 INFO:tasks.workunit.client.0.smithi027.stdout:7/995: dread d17/d32/d46/d4f/db6/f128 [0,4194304] 0 2024-06-06T08:43:54.057 INFO:tasks.workunit.client.0.smithi027.stdout:3/815: creat d1/d1f/f105 x:0 0 0 2024-06-06T08:43:54.057 INFO:tasks.workunit.client.0.smithi027.stdout:3/816: dread - d1/d8/d59/fd9 zero size 2024-06-06T08:43:54.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/841: dwrite d3/d95/d34/fde [0,4194304] 0 2024-06-06T08:43:54.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/842: chown d3/d95/d23/c99 5681 1 2024-06-06T08:43:54.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/843: stat d3/d76/dd9 0 2024-06-06T08:43:54.069 INFO:tasks.workunit.client.0.smithi027.stdout:9/844: creat d3/d95/d23/da9/f12e x:0 0 0 2024-06-06T08:43:54.070 INFO:tasks.workunit.client.0.smithi027.stdout:0/841: dwrite d15/d18/d23/f27 [0,4194304] 0 2024-06-06T08:43:54.070 INFO:tasks.workunit.client.0.smithi027.stdout:0/842: fdatasync d15/d18/d23/d28/f4e 0 2024-06-06T08:43:54.070 INFO:tasks.workunit.client.0.smithi027.stdout:8/761: dwrite d7/fd0 [0,4194304] 0 2024-06-06T08:43:54.070 INFO:tasks.workunit.client.0.smithi027.stdout:8/762: fdatasync d7/f9 0 2024-06-06T08:43:54.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/843: truncate d15/d18/d23/d28/d2a/d2d/f3a 6805325 0 2024-06-06T08:43:54.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/844: write d15/ffd [1078297,66663] 0 2024-06-06T08:43:54.072 INFO:tasks.workunit.client.0.smithi027.stdout:8/763: mkdir d7/dc/d37/d46/dfb 0 2024-06-06T08:43:54.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/845: truncate d15/d18/d23/d38/d68/d85/fea 1397882 0 2024-06-06T08:43:54.073 INFO:tasks.workunit.client.0.smithi027.stdout:1/832: dwrite dc/d67/d69/d32/d37/d56/df1/d4e/f8c [0,4194304] 0 2024-06-06T08:43:54.074 INFO:tasks.workunit.client.0.smithi027.stdout:1/833: dread - dc/d67/d69/d32/d37/d56/df1/d80/fe6 zero size 2024-06-06T08:43:54.074 INFO:tasks.workunit.client.0.smithi027.stdout:1/834: stat dc/de/d3d/l4b 0 2024-06-06T08:43:54.074 INFO:tasks.workunit.client.0.smithi027.stdout:0/846: chown d15/d18/d40/c63 1225 1 2024-06-06T08:43:54.076 INFO:tasks.workunit.client.0.smithi027.stdout:8/764: symlink d7/dc/d36/lfc 0 2024-06-06T08:43:54.077 INFO:tasks.workunit.client.0.smithi027.stdout:8/765: readlink d7/l63 0 2024-06-06T08:43:54.077 INFO:tasks.workunit.client.0.smithi027.stdout:8/766: stat d7/d13/faa 0 2024-06-06T08:43:54.077 INFO:tasks.workunit.client.0.smithi027.stdout:8/767: dread - d7/dc/d36/d41/d85/fcc zero size 2024-06-06T08:43:54.077 INFO:tasks.workunit.client.0.smithi027.stdout:8/768: truncate d7/dc/d36/d41/d85/fcc 386271 0 2024-06-06T08:43:54.079 INFO:tasks.workunit.client.0.smithi027.stdout:8/769: creat d7/d13/d15/ddc/ffd x:0 0 0 2024-06-06T08:43:54.080 INFO:tasks.workunit.client.0.smithi027.stdout:7/996: dwrite d17/d32/d46/d4f/db6/dc8/de0/f104 [0,4194304] 0 2024-06-06T08:43:54.080 INFO:tasks.workunit.client.0.smithi027.stdout:8/770: mkdir d7/dc/d36/d83/d60/d93/dac/de8/dfe 0 2024-06-06T08:43:54.087 INFO:tasks.workunit.client.0.smithi027.stdout:1/835: dread dc/d67/d69/d32/d71/f74 [0,4194304] 0 2024-06-06T08:43:54.088 INFO:tasks.workunit.client.0.smithi027.stdout:8/771: creat d7/dc/d36/d83/d60/fff x:0 0 0 2024-06-06T08:43:54.088 INFO:tasks.workunit.client.0.smithi027.stdout:9/845: dwrite d3/d95/fba [0,4194304] 0 2024-06-06T08:43:54.093 INFO:tasks.workunit.client.0.smithi027.stdout:8/772: symlink d7/d13/l100 0 2024-06-06T08:43:54.094 INFO:tasks.workunit.client.0.smithi027.stdout:0/847: read d15/d18/f4b [1093849,118042] 0 2024-06-06T08:43:54.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/997: write d17/d32/d46/d4f/d5e/fb0 [5858118,363] 0 2024-06-06T08:43:54.096 INFO:tasks.workunit.client.0.smithi027.stdout:0/848: creat d15/da7/f11c x:0 0 0 2024-06-06T08:43:54.098 INFO:tasks.workunit.client.0.smithi027.stdout:8/773: truncate d7/dc/d36/d83/d60/d93/dac/f7f 666128 0 2024-06-06T08:43:54.102 INFO:tasks.workunit.client.0.smithi027.stdout:7/998: mkdir d17/d32/d46/d4f/db6/d148 0 2024-06-06T08:43:54.104 INFO:tasks.workunit.client.0.smithi027.stdout:8/774: symlink d7/dc/d37/d46/l101 0 2024-06-06T08:43:54.104 INFO:tasks.workunit.client.0.smithi027.stdout:7/999: mknod d17/d32/d46/d4f/db6/d148/c149 0 2024-06-06T08:43:54.104 INFO:tasks.workunit.client.0.smithi027.stdout:8/775: write d7/dc/d36/d83/f1d [1595248,105243] 0 2024-06-06T08:43:54.104 INFO:tasks.workunit.client.0.smithi027.stdout:8/776: write d7/dc/d36/d41/d85/fcc [1024384,103275] 0 2024-06-06T08:43:54.108 INFO:tasks.workunit.client.0.smithi027.stdout:1/836: dread dc/de/d22/f79 [0,4194304] 0 2024-06-06T08:43:54.108 INFO:tasks.workunit.client.0.smithi027.stdout:1/837: chown dc/feb 485856 1 2024-06-06T08:43:54.110 INFO:tasks.workunit.client.0.smithi027.stdout:1/838: mknod dc/d67/d69/d32/d37/d56/df1/d80/da6/db0/c102 0 2024-06-06T08:43:54.111 INFO:tasks.workunit.client.0.smithi027.stdout:1/839: rename dc/d67/d69/d32/d71/la8 to dc/l103 0 2024-06-06T08:43:54.111 INFO:tasks.workunit.client.0.smithi027.stdout:1/840: chown dc/d67/d69/d32/d37/d56/df1/d4e/c6f 1533 1 2024-06-06T08:43:54.113 INFO:tasks.workunit.client.0.smithi027.stdout:9/846: dwrite d3/d4e/d75/dc/f2d [0,4194304] 0 2024-06-06T08:43:54.114 INFO:tasks.workunit.client.0.smithi027.stdout:1/841: unlink dc/d67/d69/d32/d37/d56/df1/d4e/f9b 0 2024-06-06T08:43:54.114 INFO:tasks.workunit.client.0.smithi027.stdout:8/777: read d7/dc/d37/d46/f52 [1724974,40038] 0 2024-06-06T08:43:54.114 INFO:tasks.workunit.client.0.smithi027.stdout:0/849: dwrite d15/da7/fb2 [0,4194304] 0 2024-06-06T08:43:54.118 INFO:tasks.workunit.client.0.smithi027.stdout:0/850: creat d15/d18/d23/d28/d31/dbb/f11d x:0 0 0 2024-06-06T08:43:54.119 INFO:tasks.workunit.client.0.smithi027.stdout:0/851: symlink d15/d18/d23/d28/l11e 0 2024-06-06T08:43:54.130 INFO:tasks.workunit.client.0.smithi027.stdout:1/842: write dc/d67/d69/d32/d37/d56/d78/f83 [2419936,107363] 0 2024-06-06T08:43:54.133 INFO:tasks.workunit.client.0.smithi027.stdout:1/843: unlink dc/d67/d69/d32/d37/d56/f93 0 2024-06-06T08:43:54.134 INFO:tasks.workunit.client.0.smithi027.stdout:9/847: link d3/d95/d23/d36/d3a/c3e d3/d95/d23/d36/d3a/df1/c12f 0 2024-06-06T08:43:54.134 INFO:tasks.workunit.client.0.smithi027.stdout:9/848: dread - d3/fc6 zero size 2024-06-06T08:43:54.134 INFO:tasks.workunit.client.0.smithi027.stdout:1/844: creat dc/d67/d69/d32/d37/d56/df1/dc8/dfc/f104 x:0 0 0 2024-06-06T08:43:54.134 INFO:tasks.workunit.client.0.smithi027.stdout:9/849: chown d3/d95/d34/d41/l59 2 1 2024-06-06T08:43:54.136 INFO:tasks.workunit.client.0.smithi027.stdout:1/845: rmdir dc/da0 39 2024-06-06T08:43:54.137 INFO:tasks.workunit.client.0.smithi027.stdout:0/852: write d15/d18/d23/d38/f51 [7376286,54803] 0 2024-06-06T08:43:54.140 INFO:tasks.workunit.client.0.smithi027.stdout:8/778: dwrite d7/dc/d36/d83/d58/f96 [0,4194304] 0 2024-06-06T08:43:54.140 INFO:tasks.workunit.client.0.smithi027.stdout:8/779: write d7/d2f/fae [5038182,25209] 0 2024-06-06T08:43:54.140 INFO:tasks.workunit.client.0.smithi027.stdout:8/780: fsync d7/dc/d36/d83/d60/d93/dac/fb2 0 2024-06-06T08:43:54.140 INFO:tasks.workunit.client.0.smithi027.stdout:8/781: fsync d7/d13/fa0 0 2024-06-06T08:43:54.140 INFO:tasks.workunit.client.0.smithi027.stdout:1/846: read dc/fbc [163406,21941] 0 2024-06-06T08:43:54.140 INFO:tasks.workunit.client.0.smithi027.stdout:0/853: symlink d15/d18/d23/d28/d2a/d2d/l11f 0 2024-06-06T08:43:54.143 INFO:tasks.workunit.client.0.smithi027.stdout:0/854: mkdir d15/d18/d23/d28/d31/dbb/d120 0 2024-06-06T08:43:54.145 INFO:tasks.workunit.client.0.smithi027.stdout:9/850: rename d3/d95/d34/d41/f81 to d3/d4e/d75/dc/d9b/f130 0 2024-06-06T08:43:54.146 INFO:tasks.workunit.client.0.smithi027.stdout:9/851: chown d3/d4e/d75/dc/d9b/f92 625 1 2024-06-06T08:43:54.147 INFO:tasks.workunit.client.0.smithi027.stdout:0/855: getdents d15/d18/d23/d28/d2a/d2d 0 2024-06-06T08:43:54.148 INFO:tasks.workunit.client.0.smithi027.stdout:1/847: write dc/fc0 [4061980,71056] 0 2024-06-06T08:43:54.148 INFO:tasks.workunit.client.0.smithi027.stdout:1/848: readlink dc/d67/d69/d32/d71/l8e 0 2024-06-06T08:43:54.149 INFO:tasks.workunit.client.0.smithi027.stdout:9/852: getdents d3/d95/d34 0 2024-06-06T08:43:54.150 INFO:tasks.workunit.client.0.smithi027.stdout:8/782: dread d7/dc/f12 [0,4194304] 0 2024-06-06T08:43:54.152 INFO:tasks.workunit.client.0.smithi027.stdout:8/783: symlink d7/d2f/l102 0 2024-06-06T08:43:54.152 INFO:tasks.workunit.client.0.smithi027.stdout:8/784: write d7/f4e [4288344,2665] 0 2024-06-06T08:43:54.152 INFO:tasks.workunit.client.0.smithi027.stdout:8/785: fsync d7/dc/d37/d46/fe6 0 2024-06-06T08:43:54.153 INFO:tasks.workunit.client.0.smithi027.stdout:8/786: dread d7/dc/d36/d41/d85/fce [0,4194304] 0 2024-06-06T08:43:54.159 INFO:tasks.workunit.client.0.smithi027.stdout:0/856: mkdir d15/d18/dbe/d121 0 2024-06-06T08:43:54.162 INFO:tasks.workunit.client.0.smithi027.stdout:1/849: symlink dc/de/d22/l105 0 2024-06-06T08:43:54.163 INFO:tasks.workunit.client.0.smithi027.stdout:8/787: symlink d7/dc/d37/l103 0 2024-06-06T08:43:54.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/857: unlink d15/d18/dbe/d5d/f4a 0 2024-06-06T08:43:54.164 INFO:tasks.workunit.client.0.smithi027.stdout:1/850: creat dc/de/de4/d101/f106 x:0 0 0 2024-06-06T08:43:54.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/858: fdatasync d15/d18/d40/d8c/d9c/fad 0 2024-06-06T08:43:54.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/859: chown d15/d18/d23/d5b/da0/cd7 712553 1 2024-06-06T08:43:54.168 INFO:tasks.workunit.client.0.smithi027.stdout:0/860: symlink d15/l122 0 2024-06-06T08:43:54.172 INFO:tasks.workunit.client.0.smithi027.stdout:0/861: write d15/d18/d23/d28/f55 [1114902,86441] 0 2024-06-06T08:43:54.173 INFO:tasks.workunit.client.0.smithi027.stdout:0/862: creat d15/d18/d23/d6f/dcb/f123 x:0 0 0 2024-06-06T08:43:54.173 INFO:tasks.workunit.client.0.smithi027.stdout:0/863: fsync d15/d18/d40/faf 0 2024-06-06T08:43:54.175 INFO:tasks.workunit.client.0.smithi027.stdout:9/853: dwrite d3/d95/d23/d36/fbd [0,4194304] 0 2024-06-06T08:43:54.178 INFO:tasks.workunit.client.0.smithi027.stdout:9/854: creat d3/d4e/d75/dc/f131 x:0 0 0 2024-06-06T08:43:54.184 INFO:tasks.workunit.client.0.smithi027.stdout:1/851: dwrite dc/de/d3d/f3e [0,4194304] 0 2024-06-06T08:43:54.186 INFO:tasks.workunit.client.0.smithi027.stdout:1/852: mkdir dc/d67/d69/d32/d37/ddf/dfa/d107 0 2024-06-06T08:43:54.187 INFO:tasks.workunit.client.0.smithi027.stdout:1/853: creat dc/de/d3d/f108 x:0 0 0 2024-06-06T08:43:54.197 INFO:tasks.workunit.client.0.smithi027.stdout:1/854: read dc/d67/d69/d32/d37/d56/df1/f15 [3546377,106168] 0 2024-06-06T08:43:54.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/855: truncate dc/d67/d69/d32/d37/d56/df1/d4e/f53 123164 0 2024-06-06T08:43:54.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/856: chown dc/d67/d69/d32/d37/d56/df1/d80/de8 0 1 2024-06-06T08:43:54.200 INFO:tasks.workunit.client.0.smithi027.stdout:1/857: rename dc/de/f75 to dc/d9d/f109 0 2024-06-06T08:43:54.200 INFO:tasks.workunit.client.0.smithi027.stdout:1/858: chown dc/feb 114992 1 2024-06-06T08:43:54.211 INFO:tasks.workunit.client.0.smithi027.stdout:1/859: dread dc/d67/d69/d32/d37/d56/df1/f1a [0,4194304] 0 2024-06-06T08:43:54.211 INFO:tasks.workunit.client.0.smithi027.stdout:1/860: stat dc/de/l65 0 2024-06-06T08:43:54.217 INFO:tasks.workunit.client.0.smithi027.stdout:1/861: dread f5 [0,4194304] 0 2024-06-06T08:43:54.219 INFO:tasks.workunit.client.0.smithi027.stdout:1/862: unlink dc/d67/d69/d32/d37/d56/df1/d4e/f8c 0 2024-06-06T08:43:54.226 INFO:tasks.workunit.client.0.smithi027.stdout:9/855: link d3/d4e/d75/db4/lf9 d3/d95/d23/l132 0 2024-06-06T08:43:54.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/856: write d3/d95/d23/d78/fb3 [405559,73981] 0 2024-06-06T08:43:54.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/857: fsync d3/d95/feb 0 2024-06-06T08:43:54.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/858: dread - d3/d95/feb zero size 2024-06-06T08:43:54.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/859: stat d3/f111 0 2024-06-06T08:43:54.228 INFO:tasks.workunit.client.0.smithi027.stdout:9/860: read d3/d95/d23/d36/d3a/fe4 [219403,118284] 0 2024-06-06T08:43:54.228 INFO:tasks.workunit.client.0.smithi027.stdout:9/861: creat d3/d95/d23/da9/f133 x:0 0 0 2024-06-06T08:43:54.230 INFO:tasks.workunit.client.0.smithi027.stdout:9/862: creat d3/d4e/d75/f134 x:0 0 0 2024-06-06T08:43:54.233 INFO:tasks.workunit.client.0.smithi027.stdout:1/863: dwrite dc/de/fab [0,4194304] 0 2024-06-06T08:43:54.236 INFO:tasks.workunit.client.0.smithi027.stdout:1/864: creat dc/de/d3d/f10a x:0 0 0 2024-06-06T08:43:54.237 INFO:tasks.workunit.client.0.smithi027.stdout:9/863: dread d3/d95/fba [0,4194304] 0 2024-06-06T08:43:54.237 INFO:tasks.workunit.client.0.smithi027.stdout:9/864: stat d3/d95/db6/d117 0 2024-06-06T08:43:54.247 INFO:tasks.workunit.client.0.smithi027.stdout:9/865: dread d3/d95/f2f [0,4194304] 0 2024-06-06T08:43:54.248 INFO:tasks.workunit.client.0.smithi027.stdout:9/866: write d3/d95/f4f [3272990,21782] 0 2024-06-06T08:43:54.248 INFO:tasks.workunit.client.0.smithi027.stdout:9/867: fsync d3/d95/d34/fde 0 2024-06-06T08:43:54.248 INFO:tasks.workunit.client.0.smithi027.stdout:9/868: getdents d3/d95/d23/d36 0 2024-06-06T08:43:54.248 INFO:tasks.workunit.client.0.smithi027.stdout:9/869: write d3/d95/d34/d73/fcc [336395,44010] 0 2024-06-06T08:43:54.250 INFO:tasks.workunit.client.0.smithi027.stdout:9/870: rename d3/d4e/d75/db0 to d3/d95/db6/d117/d135 0 2024-06-06T08:43:54.358 INFO:tasks.workunit.client.0.smithi027.stdout:3/817: sync 2024-06-06T08:43:54.358 INFO:tasks.workunit.client.0.smithi027.stdout:2/867: sync 2024-06-06T08:43:54.358 INFO:tasks.workunit.client.0.smithi027.stdout:3/818: chown d1/d90/d44/l4b 31 1 2024-06-06T08:43:54.359 INFO:tasks.workunit.client.0.smithi027.stdout:2/868: creat d6/d1e/f125 x:0 0 0 2024-06-06T08:43:54.359 INFO:tasks.workunit.client.0.smithi027.stdout:3/819: rmdir d1/d90/d6d/dac 39 2024-06-06T08:43:54.359 INFO:tasks.workunit.client.0.smithi027.stdout:3/820: chown d1/d90/d44/d54/db4/fd3 938 1 2024-06-06T08:43:54.360 INFO:tasks.workunit.client.0.smithi027.stdout:2/869: dread - d6/d1e/d2b/d5a/faf zero size 2024-06-06T08:43:54.360 INFO:tasks.workunit.client.0.smithi027.stdout:2/870: chown d6/d1e 0 1 2024-06-06T08:43:54.360 INFO:tasks.workunit.client.0.smithi027.stdout:2/871: chown d6/d1e/d2b/d5a/fef 26 1 2024-06-06T08:43:54.360 INFO:tasks.workunit.client.0.smithi027.stdout:2/872: fsync d6/dd/dd9/f44 0 2024-06-06T08:43:54.363 INFO:tasks.workunit.client.0.smithi027.stdout:2/873: write d6/dd/dd9/f44 [3847272,91448] 0 2024-06-06T08:43:54.364 INFO:tasks.workunit.client.0.smithi027.stdout:2/874: mknod d6/dd/dd9/d1d/d4d/d8e/d10a/c126 0 2024-06-06T08:43:54.366 INFO:tasks.workunit.client.0.smithi027.stdout:3/821: dread d1/d8/d22/f37 [0,4194304] 0 2024-06-06T08:43:54.366 INFO:tasks.workunit.client.0.smithi027.stdout:3/822: chown d1/d90/d44/d54/db4 173897448 1 2024-06-06T08:43:54.384 INFO:tasks.workunit.client.0.smithi027.stdout:2/875: dwrite d6/dd/dd9/d1d/d4d/ff7 [0,4194304] 0 2024-06-06T08:43:54.385 INFO:tasks.workunit.client.0.smithi027.stdout:3/823: dwrite d1/d1f/d45/fad [0,4194304] 0 2024-06-06T08:43:54.400 INFO:tasks.workunit.client.0.smithi027.stdout:3/824: dwrite d1/d8/d22/f37 [0,4194304] 0 2024-06-06T08:43:54.401 INFO:tasks.workunit.client.0.smithi027.stdout:3/825: chown d1 0 1 2024-06-06T08:43:54.401 INFO:tasks.workunit.client.0.smithi027.stdout:3/826: truncate d1/d90/d44/d54/fc4 232366 0 2024-06-06T08:43:54.401 INFO:tasks.workunit.client.0.smithi027.stdout:2/876: dwrite d6/f8b [0,4194304] 0 2024-06-06T08:43:54.411 INFO:tasks.workunit.client.0.smithi027.stdout:8/788: sync 2024-06-06T08:43:54.411 INFO:tasks.workunit.client.0.smithi027.stdout:2/877: dread d6/dd/dd9/d1d/f86 [0,4194304] 0 2024-06-06T08:43:54.412 INFO:tasks.workunit.client.0.smithi027.stdout:2/878: creat d6/dd/dd9/d3f/db0/d123/d105/f127 x:0 0 0 2024-06-06T08:43:54.414 INFO:tasks.workunit.client.0.smithi027.stdout:2/879: rename d6/dd/d6b/fb3 to d6/dd/dd9/d1d/d4d/d8e/d10a/f128 0 2024-06-06T08:43:54.415 INFO:tasks.workunit.client.0.smithi027.stdout:2/880: symlink d6/dd/dd9/d3f/db0/d108/l129 0 2024-06-06T08:43:54.415 INFO:tasks.workunit.client.0.smithi027.stdout:2/881: chown d6/dd/dd9/d1d/d4d/d8e/d10a/f128 148 1 2024-06-06T08:43:54.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/789: dread d7/dc/d37/d46/f8d [0,4194304] 0 2024-06-06T08:43:54.434 INFO:tasks.workunit.client.0.smithi027.stdout:8/790: write d7/d2f/f3b [3144458,20773] 0 2024-06-06T08:43:54.434 INFO:tasks.workunit.client.0.smithi027.stdout:2/882: dwrite d6/dd/dd9/d3f/db0/d123/fa5 [0,4194304] 0 2024-06-06T08:43:54.434 INFO:tasks.workunit.client.0.smithi027.stdout:2/883: write d6/dd/f23 [1698151,128797] 0 2024-06-06T08:43:54.435 INFO:tasks.workunit.client.0.smithi027.stdout:8/791: creat d7/dc/d37/f104 x:0 0 0 2024-06-06T08:43:54.437 INFO:tasks.workunit.client.0.smithi027.stdout:2/884: symlink d6/d1e/d2b/d5a/l12a 0 2024-06-06T08:43:54.445 INFO:tasks.workunit.client.0.smithi027.stdout:2/885: write d6/dd/dd9/d3f/db0/d123/fa5 [3519476,33603] 0 2024-06-06T08:43:54.450 INFO:tasks.workunit.client.0.smithi027.stdout:2/886: write d6/d1e/d2b/f2f [5055838,45593] 0 2024-06-06T08:43:54.452 INFO:tasks.workunit.client.0.smithi027.stdout:8/792: dwrite d7/dc/d37/d46/f52 [0,4194304] 0 2024-06-06T08:43:54.466 INFO:tasks.workunit.client.0.smithi027.stdout:8/793: dwrite d7/f9 [0,4194304] 0 2024-06-06T08:43:54.467 INFO:tasks.workunit.client.0.smithi027.stdout:2/887: dwrite d6/d1e/d2b/d5a/ff9 [0,4194304] 0 2024-06-06T08:43:54.471 INFO:tasks.workunit.client.0.smithi027.stdout:2/888: link d6/dd/dd9/d1d/d4d/d8e/fc3 d6/dd/dd9/d3f/d77/de5/f12b 0 2024-06-06T08:43:54.471 INFO:tasks.workunit.client.0.smithi027.stdout:2/889: truncate d6/dd/dd9/d3f/d111/de1/fec 526272 0 2024-06-06T08:43:54.471 INFO:tasks.workunit.client.0.smithi027.stdout:8/794: mknod d7/dc/d36/d83/d60/d93/dac/d73/c105 0 2024-06-06T08:43:54.473 INFO:tasks.workunit.client.0.smithi027.stdout:8/795: getdents d7/dc/d36/d83/d60/d93/dac 0 2024-06-06T08:43:54.473 INFO:tasks.workunit.client.0.smithi027.stdout:8/796: read - d7/d13/d15/ddc/ffd zero size 2024-06-06T08:43:54.473 INFO:tasks.workunit.client.0.smithi027.stdout:2/890: mknod d6/dd/dd9/d1d/c12c 0 2024-06-06T08:43:54.476 INFO:tasks.workunit.client.0.smithi027.stdout:2/891: creat d6/d1e/d2b/d5a/db7/d115/f12d x:0 0 0 2024-06-06T08:43:54.493 INFO:tasks.workunit.client.0.smithi027.stdout:2/892: dwrite d6/fe [0,4194304] 0 2024-06-06T08:43:54.495 INFO:tasks.workunit.client.0.smithi027.stdout:8/797: dwrite d7/dc/d36/d41/f34 [0,4194304] 0 2024-06-06T08:43:54.497 INFO:tasks.workunit.client.0.smithi027.stdout:8/798: mkdir d7/dc/d36/d41/d85/db7/d106 0 2024-06-06T08:43:54.497 INFO:tasks.workunit.client.0.smithi027.stdout:8/799: stat d7/dc/d36/d41/dbf/dc3/ce1 0 2024-06-06T08:43:54.499 INFO:tasks.workunit.client.0.smithi027.stdout:8/800: creat d7/d9c/f107 x:0 0 0 2024-06-06T08:43:54.501 INFO:tasks.workunit.client.0.smithi027.stdout:8/801: rmdir d7/dc/d36/d41/d85/db7/dd3 39 2024-06-06T08:43:54.501 INFO:tasks.workunit.client.0.smithi027.stdout:8/802: chown d7/d13/ldd 2327 1 2024-06-06T08:43:54.501 INFO:tasks.workunit.client.0.smithi027.stdout:8/803: chown d7/dc/d37/d46/fe6 0 1 2024-06-06T08:43:54.501 INFO:tasks.workunit.client.0.smithi027.stdout:8/804: fsync d7/dc/d36/d83/d60/d93/dac/fb2 0 2024-06-06T08:43:54.502 INFO:tasks.workunit.client.0.smithi027.stdout:8/805: mknod d7/dc/d36/d83/d60/d93/dac/de8/dfe/c108 0 2024-06-06T08:43:54.503 INFO:tasks.workunit.client.0.smithi027.stdout:0/864: sync 2024-06-06T08:43:54.504 INFO:tasks.workunit.client.0.smithi027.stdout:1/865: sync 2024-06-06T08:43:54.504 INFO:tasks.workunit.client.0.smithi027.stdout:9/871: sync 2024-06-06T08:43:54.505 INFO:tasks.workunit.client.0.smithi027.stdout:9/872: creat d3/d95/d34/d73/f136 x:0 0 0 2024-06-06T08:43:54.506 INFO:tasks.workunit.client.0.smithi027.stdout:9/873: write d3/d95/d23/da9/f104 [341288,33004] 0 2024-06-06T08:43:54.506 INFO:tasks.workunit.client.0.smithi027.stdout:9/874: truncate d3/d4e/f9d 792153 0 2024-06-06T08:43:54.506 INFO:tasks.workunit.client.0.smithi027.stdout:0/865: symlink d15/d18/dbe/l124 0 2024-06-06T08:43:54.507 INFO:tasks.workunit.client.0.smithi027.stdout:0/866: write d15/d18/d23/d28/f4e [1977604,29024] 0 2024-06-06T08:43:54.507 INFO:tasks.workunit.client.0.smithi027.stdout:9/875: truncate d3/d95/d23/d36/f74 3966437 0 2024-06-06T08:43:54.508 INFO:tasks.workunit.client.0.smithi027.stdout:2/893: dwrite d6/dd/dd9/d3f/d48/f62 [0,4194304] 0 2024-06-06T08:43:54.508 INFO:tasks.workunit.client.0.smithi027.stdout:0/867: dread d15/d18/d40/fc2 [0,4194304] 0 2024-06-06T08:43:54.508 INFO:tasks.workunit.client.0.smithi027.stdout:0/868: dread - d15/d18/d23/d38/fdc zero size 2024-06-06T08:43:54.509 INFO:tasks.workunit.client.0.smithi027.stdout:2/894: creat d6/dd/dd9/d3f/d48/d83/de3/de4/f12e x:0 0 0 2024-06-06T08:43:54.510 INFO:tasks.workunit.client.0.smithi027.stdout:0/869: symlink d15/d18/d23/d28/d31/dbb/l125 0 2024-06-06T08:43:54.510 INFO:tasks.workunit.client.0.smithi027.stdout:9/876: symlink d3/d95/d23/d36/d3a/d84/l137 0 2024-06-06T08:43:54.511 INFO:tasks.workunit.client.0.smithi027.stdout:9/877: write d3/d95/d34/f60 [805884,108115] 0 2024-06-06T08:43:54.512 INFO:tasks.workunit.client.0.smithi027.stdout:0/870: creat d15/d18/d23/d5b/f126 x:0 0 0 2024-06-06T08:43:54.512 INFO:tasks.workunit.client.0.smithi027.stdout:0/871: dread - d15/d18/d23/d5b/f126 zero size 2024-06-06T08:43:54.512 INFO:tasks.workunit.client.0.smithi027.stdout:0/872: truncate d15/d18/d23/d38/d68/f9d 322449 0 2024-06-06T08:43:54.516 INFO:tasks.workunit.client.0.smithi027.stdout:9/878: chown d3/d4e/d75/dc/l19 21051 1 2024-06-06T08:43:54.517 INFO:tasks.workunit.client.0.smithi027.stdout:8/806: dwrite d7/dc/d36/d3c/f61 [0,4194304] 0 2024-06-06T08:43:54.518 INFO:tasks.workunit.client.0.smithi027.stdout:8/807: rename d7/d13/d15/de9 to d7/d9c/daf/d109 0 2024-06-06T08:43:54.520 INFO:tasks.workunit.client.0.smithi027.stdout:8/808: truncate d7/dc/f12 3956024 0 2024-06-06T08:43:54.520 INFO:tasks.workunit.client.0.smithi027.stdout:9/879: write d3/d4e/fa8 [30452,43976] 0 2024-06-06T08:43:54.520 INFO:tasks.workunit.client.0.smithi027.stdout:9/880: chown d3/f20 7322 1 2024-06-06T08:43:54.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/809: readlink d7/dc/d36/d41/l2d 0 2024-06-06T08:43:54.523 INFO:tasks.workunit.client.0.smithi027.stdout:8/810: mkdir d7/dc/d36/d41/dbf/d10a 0 2024-06-06T08:43:54.523 INFO:tasks.workunit.client.0.smithi027.stdout:8/811: chown d7/dc/d37 0 1 2024-06-06T08:43:54.525 INFO:tasks.workunit.client.0.smithi027.stdout:2/895: read d6/dd/dd9/d1d/d4d/d8e/d10a/f128 [2005663,9714] 0 2024-06-06T08:43:54.526 INFO:tasks.workunit.client.0.smithi027.stdout:1/866: dwrite dc/d67/d69/d32/d37/f41 [0,4194304] 0 2024-06-06T08:43:54.528 INFO:tasks.workunit.client.0.smithi027.stdout:1/867: truncate dc/d67/d69/f52 3137610 0 2024-06-06T08:43:54.530 INFO:tasks.workunit.client.0.smithi027.stdout:1/868: creat dc/d67/d69/d32/d37/d56/d78/f10b x:0 0 0 2024-06-06T08:43:54.534 INFO:tasks.workunit.client.0.smithi027.stdout:0/873: dwrite d15/d18/d64/f87 [0,4194304] 0 2024-06-06T08:43:54.536 INFO:tasks.workunit.client.0.smithi027.stdout:0/874: creat d15/d18/dbe/d5c/dfa/f127 x:0 0 0 2024-06-06T08:43:54.540 INFO:tasks.workunit.client.0.smithi027.stdout:0/875: rmdir d15/d18/d40 39 2024-06-06T08:43:54.541 INFO:tasks.workunit.client.0.smithi027.stdout:8/812: dwrite d7/d2f/f92 [0,4194304] 0 2024-06-06T08:43:54.543 INFO:tasks.workunit.client.0.smithi027.stdout:8/813: truncate d7/d13/f17 817067 0 2024-06-06T08:43:54.544 INFO:tasks.workunit.client.0.smithi027.stdout:2/896: write d6/d1e/d2b/f2f [4053695,94429] 0 2024-06-06T08:43:54.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/881: dwrite d3/fc6 [0,4194304] 0 2024-06-06T08:43:54.544 INFO:tasks.workunit.client.0.smithi027.stdout:1/869: write dc/de/d3d/f3e [565319,14859] 0 2024-06-06T08:43:54.545 INFO:tasks.workunit.client.0.smithi027.stdout:2/897: symlink d6/dd/dd9/d3f/d48/d83/l12f 0 2024-06-06T08:43:54.547 INFO:tasks.workunit.client.0.smithi027.stdout:8/814: symlink d7/dc/d37/d46/l10b 0 2024-06-06T08:43:54.550 INFO:tasks.workunit.client.0.smithi027.stdout:1/870: rename dc/d67/d69/d32/d37/d56/df1/d80/ca3 to dc/d67/d69/d32/d37/c10c 0 2024-06-06T08:43:54.551 INFO:tasks.workunit.client.0.smithi027.stdout:1/871: chown dc/d67/d69/c29 1629315 1 2024-06-06T08:43:54.552 INFO:tasks.workunit.client.0.smithi027.stdout:2/898: truncate d6/dd/dd9/d1d/f36 1003782 0 2024-06-06T08:43:54.554 INFO:tasks.workunit.client.0.smithi027.stdout:8/815: dread d7/dc/d36/d83/d60/fad [0,4194304] 0 2024-06-06T08:43:54.554 INFO:tasks.workunit.client.0.smithi027.stdout:1/872: creat dc/d67/d69/d32/d71/f10d x:0 0 0 2024-06-06T08:43:54.554 INFO:tasks.workunit.client.0.smithi027.stdout:8/816: fsync d7/dc/d36/d41/ff4 0 2024-06-06T08:43:54.554 INFO:tasks.workunit.client.0.smithi027.stdout:1/873: stat dc/fd 0 2024-06-06T08:43:54.556 INFO:tasks.workunit.client.0.smithi027.stdout:2/899: creat d6/dd/dd9/d3f/f130 x:0 0 0 2024-06-06T08:43:54.557 INFO:tasks.workunit.client.0.smithi027.stdout:2/900: readlink d6/dd/dd9/d3f/db0/lf0 0 2024-06-06T08:43:54.557 INFO:tasks.workunit.client.0.smithi027.stdout:8/817: getdents d7/dc/d37/dc8 0 2024-06-06T08:43:54.558 INFO:tasks.workunit.client.0.smithi027.stdout:8/818: read d7/dc/d36/d41/d85/fce [163728,15109] 0 2024-06-06T08:43:54.558 INFO:tasks.workunit.client.0.smithi027.stdout:8/819: chown d7/dc/d36/f75 1649803774 1 2024-06-06T08:43:54.559 INFO:tasks.workunit.client.0.smithi027.stdout:2/901: readlink d6/dd/l1a 0 2024-06-06T08:43:54.560 INFO:tasks.workunit.client.0.smithi027.stdout:8/820: truncate d7/dc/d36/d83/d58/f96 4059864 0 2024-06-06T08:43:54.562 INFO:tasks.workunit.client.0.smithi027.stdout:2/902: stat d6/dd/dd9/d3f/d77/d55/fb6 0 2024-06-06T08:43:54.564 INFO:tasks.workunit.client.0.smithi027.stdout:2/903: creat d6/dd/dd9/d3f/d48/d83/de3/f131 x:0 0 0 2024-06-06T08:43:54.564 INFO:tasks.workunit.client.0.smithi027.stdout:2/904: chown d6/dd/dd9/d3f/db0/fdb 15 1 2024-06-06T08:43:54.565 INFO:tasks.workunit.client.0.smithi027.stdout:9/882: dwrite d3/d95/d23/d36/f94 [0,4194304] 0 2024-06-06T08:43:54.565 INFO:tasks.workunit.client.0.smithi027.stdout:9/883: fdatasync d3/d95/d23/d36/d3a/fe4 0 2024-06-06T08:43:54.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/905: creat d6/dd/dd9/d3f/d48/d83/de3/d100/f132 x:0 0 0 2024-06-06T08:43:54.568 INFO:tasks.workunit.client.0.smithi027.stdout:2/906: creat d6/dd/dd9/d3f/d77/d55/d69/f133 x:0 0 0 2024-06-06T08:43:54.570 INFO:tasks.workunit.client.0.smithi027.stdout:2/907: mknod d6/dd/dd9/d3f/d77/d55/d120/c134 0 2024-06-06T08:43:54.574 INFO:tasks.workunit.client.0.smithi027.stdout:2/908: rename d6/d1e/d2b/d5a/c97 to d6/d1e/d2b/c135 0 2024-06-06T08:43:54.575 INFO:tasks.workunit.client.0.smithi027.stdout:9/884: dread d3/d95/f2f [0,4194304] 0 2024-06-06T08:43:54.575 INFO:tasks.workunit.client.0.smithi027.stdout:9/885: chown d3/d95/d34/d41/lfc 472361 1 2024-06-06T08:43:54.577 INFO:tasks.workunit.client.0.smithi027.stdout:9/886: mkdir d3/d95/d23/d78/d138 0 2024-06-06T08:43:54.579 INFO:tasks.workunit.client.0.smithi027.stdout:8/821: dwrite d7/dc/d36/d3c/f61 [0,4194304] 0 2024-06-06T08:43:54.580 INFO:tasks.workunit.client.0.smithi027.stdout:1/874: dwrite dc/de/d3d/f50 [4194304,4194304] 0 2024-06-06T08:43:54.581 INFO:tasks.workunit.client.0.smithi027.stdout:1/875: write dc/f36 [2411371,80958] 0 2024-06-06T08:43:54.581 INFO:tasks.workunit.client.0.smithi027.stdout:1/876: stat dc/d67/d69/d32/f94 0 2024-06-06T08:43:54.581 INFO:tasks.workunit.client.0.smithi027.stdout:8/822: creat d7/dc/d36/d41/d85/df6/f10c x:0 0 0 2024-06-06T08:43:54.581 INFO:tasks.workunit.client.0.smithi027.stdout:8/823: fdatasync d7/f9 0 2024-06-06T08:43:54.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/824: symlink d7/dc/d37/d46/l10d 0 2024-06-06T08:43:54.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/825: write d7/dc/d36/d83/d60/f68 [4910987,63999] 0 2024-06-06T08:43:54.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/826: chown d7/dc/d36/d41/dbf/fe5 0 1 2024-06-06T08:43:54.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/827: write d7/dc/d36/d83/f1d [381605,115567] 0 2024-06-06T08:43:54.588 INFO:tasks.workunit.client.0.smithi027.stdout:2/909: write d6/f8b [2276330,1991] 0 2024-06-06T08:43:54.600 INFO:tasks.workunit.client.0.smithi027.stdout:1/877: dwrite dc/d67/d69/d32/fed [0,4194304] 0 2024-06-06T08:43:54.601 INFO:tasks.workunit.client.0.smithi027.stdout:1/878: link dc/d67/d69/d32/d37/d56/df1/f87 dc/de/de4/f10e 0 2024-06-06T08:43:54.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/828: dwrite d7/dc/d36/d3c/f72 [0,4194304] 0 2024-06-06T08:43:54.603 INFO:tasks.workunit.client.0.smithi027.stdout:1/879: unlink dc/d67/d69/ff9 0 2024-06-06T08:43:54.603 INFO:tasks.workunit.client.0.smithi027.stdout:1/880: write dc/d9d/fa2 [419396,120994] 0 2024-06-06T08:43:54.609 INFO:tasks.workunit.client.0.smithi027.stdout:2/910: dwrite d6/f6e [4194304,4194304] 0 2024-06-06T08:43:54.620 INFO:tasks.workunit.client.0.smithi027.stdout:1/881: dwrite dc/d6d/fde [0,4194304] 0 2024-06-06T08:43:54.620 INFO:tasks.workunit.client.0.smithi027.stdout:1/882: read - dc/de/d3d/f10a zero size 2024-06-06T08:43:54.623 INFO:tasks.workunit.client.0.smithi027.stdout:1/883: creat dc/d6d/f10f x:0 0 0 2024-06-06T08:43:54.623 INFO:tasks.workunit.client.0.smithi027.stdout:1/884: write dc/d67/d69/d32/d37/d56/df1/f15 [4748704,88636] 0 2024-06-06T08:43:54.624 INFO:tasks.workunit.client.0.smithi027.stdout:8/829: rename d7/d13/d15/f1c to d7/dc/d36/d41/f10e 0 2024-06-06T08:43:54.625 INFO:tasks.workunit.client.0.smithi027.stdout:8/830: creat d7/dc/d36/d83/d60/d93/f10f x:0 0 0 2024-06-06T08:43:54.625 INFO:tasks.workunit.client.0.smithi027.stdout:8/831: read d7/d13/f17 [134465,18163] 0 2024-06-06T08:43:54.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/885: rename dc/de/d3d/d4f/c8f to dc/de/c110 0 2024-06-06T08:43:54.627 INFO:tasks.workunit.client.0.smithi027.stdout:8/832: readlink d7/dc/d36/d3c/l4c 0 2024-06-06T08:43:54.628 INFO:tasks.workunit.client.0.smithi027.stdout:2/911: dwrite d6/dd/dd9/f29 [0,4194304] 0 2024-06-06T08:43:54.629 INFO:tasks.workunit.client.0.smithi027.stdout:2/912: stat d6/dd/f24 0 2024-06-06T08:43:54.629 INFO:tasks.workunit.client.0.smithi027.stdout:8/833: truncate d7/dc/d36/d83/d60/f7a 2221138 0 2024-06-06T08:43:54.629 INFO:tasks.workunit.client.0.smithi027.stdout:8/834: write d7/d13/fa0 [836825,96394] 0 2024-06-06T08:43:54.630 INFO:tasks.workunit.client.0.smithi027.stdout:8/835: stat d7/dc/d36/d41/ca6 0 2024-06-06T08:43:54.631 INFO:tasks.workunit.client.0.smithi027.stdout:8/836: mkdir d7/dc/d36/d83/d60/d93/dac/d110 0 2024-06-06T08:43:54.633 INFO:tasks.workunit.client.0.smithi027.stdout:8/837: truncate d7/f22 2936713 0 2024-06-06T08:43:54.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/886: dread dc/d67/d69/f52 [0,4194304] 0 2024-06-06T08:43:54.633 INFO:tasks.workunit.client.0.smithi027.stdout:1/887: chown dc/d67/d69/d32/d37/d56/df1/d4e/ld8 903 1 2024-06-06T08:43:54.639 INFO:tasks.workunit.client.0.smithi027.stdout:2/913: write d6/d1e/d2b/d5a/d7b/f7e [2789268,130709] 0 2024-06-06T08:43:54.642 INFO:tasks.workunit.client.0.smithi027.stdout:1/888: dread dc/de/f13 [4194304,4194304] 0 2024-06-06T08:43:54.645 INFO:tasks.workunit.client.0.smithi027.stdout:8/838: write d7/d2f/f92 [5983198,75046] 0 2024-06-06T08:43:54.645 INFO:tasks.workunit.client.0.smithi027.stdout:8/839: fdatasync d7/dc/d36/f75 0 2024-06-06T08:43:54.648 INFO:tasks.workunit.client.0.smithi027.stdout:2/914: write d6/d1e/d2b/f91 [120172,16584] 0 2024-06-06T08:43:54.653 INFO:tasks.workunit.client.0.smithi027.stdout:2/915: write d6/dd/dd9/d1d/f45 [4216536,117035] 0 2024-06-06T08:43:54.654 INFO:tasks.workunit.client.0.smithi027.stdout:1/889: symlink dc/d67/d69/d32/d37/d56/df1/d4e/l111 0 2024-06-06T08:43:54.656 INFO:tasks.workunit.client.0.smithi027.stdout:2/916: creat d6/dd/dd9/d1d/d4d/f136 x:0 0 0 2024-06-06T08:43:54.657 INFO:tasks.workunit.client.0.smithi027.stdout:2/917: truncate d6/dd/dd9/f29 4690737 0 2024-06-06T08:43:54.658 INFO:tasks.workunit.client.0.smithi027.stdout:1/890: symlink dc/d67/d69/d32/d37/d56/df1/l112 0 2024-06-06T08:43:54.659 INFO:tasks.workunit.client.0.smithi027.stdout:2/918: chown d6/dd/dd9/ca0 1316613 1 2024-06-06T08:43:54.661 INFO:tasks.workunit.client.0.smithi027.stdout:1/891: symlink dc/d67/d69/d32/d37/d56/df1/d80/da6/db0/l113 0 2024-06-06T08:43:54.661 INFO:tasks.workunit.client.0.smithi027.stdout:1/892: fdatasync dc/d67/d69/d32/d71/fd5 0 2024-06-06T08:43:54.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/919: mkdir d6/d1e/d2b/d5a/d137 0 2024-06-06T08:43:54.669 INFO:tasks.workunit.client.0.smithi027.stdout:3/827: sync 2024-06-06T08:43:54.669 INFO:tasks.workunit.client.0.smithi027.stdout:3/828: write d1/d1f/fd8 [169040,95525] 0 2024-06-06T08:43:54.669 INFO:tasks.workunit.client.0.smithi027.stdout:3/829: mkdir d1/d8/d22/d27/d79/d106 0 2024-06-06T08:43:54.679 INFO:tasks.workunit.client.0.smithi027.stdout:3/830: dread d1/f29 [0,4194304] 0 2024-06-06T08:43:54.680 INFO:tasks.workunit.client.0.smithi027.stdout:1/893: dwrite dc/de/d3d/d4f/f64 [4194304,4194304] 0 2024-06-06T08:43:54.680 INFO:tasks.workunit.client.0.smithi027.stdout:3/831: truncate d1/d1f/d45/d5f/fc1 875064 0 2024-06-06T08:43:54.683 INFO:tasks.workunit.client.0.smithi027.stdout:1/894: symlink dc/d67/d69/l114 0 2024-06-06T08:43:54.683 INFO:tasks.workunit.client.0.smithi027.stdout:1/895: dread - dc/d67/fdc zero size 2024-06-06T08:43:54.686 INFO:tasks.workunit.client.0.smithi027.stdout:1/896: rename dc/d67/d69/d32/d71/fbb to dc/d67/d69/d32/d37/d56/df1/d80/da6/f115 0 2024-06-06T08:43:54.688 INFO:tasks.workunit.client.0.smithi027.stdout:1/897: rename dc/da0/cb7 to dc/d67/d69/d32/d37/ddf/c116 0 2024-06-06T08:43:54.689 INFO:tasks.workunit.client.0.smithi027.stdout:1/898: chown dc/d9d 1567122 1 2024-06-06T08:43:54.690 INFO:tasks.workunit.client.0.smithi027.stdout:2/920: dwrite d6/dd/f26 [0,4194304] 0 2024-06-06T08:43:54.690 INFO:tasks.workunit.client.0.smithi027.stdout:2/921: chown d6/d3b/d80/cb9 899262 1 2024-06-06T08:43:54.690 INFO:tasks.workunit.client.0.smithi027.stdout:1/899: creat dc/d67/d69/d32/d37/d56/de1/f117 x:0 0 0 2024-06-06T08:43:54.693 INFO:tasks.workunit.client.0.smithi027.stdout:2/922: rename d6/dd/dd9/d3f/d77/d55/f6d to d6/d1e/d2b/d5a/db7/de2/f138 0 2024-06-06T08:43:54.695 INFO:tasks.workunit.client.0.smithi027.stdout:1/900: link dc/d67/d69/d32/d71/l8e dc/d67/d69/d32/d37/d56/df1/dc8/dfc/l118 0 2024-06-06T08:43:54.697 INFO:tasks.workunit.client.0.smithi027.stdout:2/923: creat d6/d1e/d2b/dc5/f139 x:0 0 0 2024-06-06T08:43:54.698 INFO:tasks.workunit.client.0.smithi027.stdout:3/832: dwrite d1/d90/d6d/dac/ddb/f8d [0,4194304] 0 2024-06-06T08:43:54.699 INFO:tasks.workunit.client.0.smithi027.stdout:1/901: creat dc/d67/d69/d32/d37/d56/de1/f119 x:0 0 0 2024-06-06T08:43:54.699 INFO:tasks.workunit.client.0.smithi027.stdout:1/902: truncate dc/fbc 1005337 0 2024-06-06T08:43:54.701 INFO:tasks.workunit.client.0.smithi027.stdout:2/924: creat d6/dd/dd9/d3f/d48/d83/de3/d100/f13a x:0 0 0 2024-06-06T08:43:54.703 INFO:tasks.workunit.client.0.smithi027.stdout:1/903: getdents dc/de/d3d/d4a 0 2024-06-06T08:43:54.703 INFO:tasks.workunit.client.0.smithi027.stdout:1/904: fdatasync dc/de/d3d/d4a/d76/fc3 0 2024-06-06T08:43:54.711 INFO:tasks.workunit.client.0.smithi027.stdout:3/833: rename d1/f20 to d1/d90/d6d/f107 0 2024-06-06T08:43:54.711 INFO:tasks.workunit.client.0.smithi027.stdout:1/905: truncate dc/de/d3d/d4a/f58 1999169 0 2024-06-06T08:43:54.712 INFO:tasks.workunit.client.0.smithi027.stdout:1/906: creat dc/d67/d69/d32/f11a x:0 0 0 2024-06-06T08:43:54.713 INFO:tasks.workunit.client.0.smithi027.stdout:1/907: symlink dc/de/d3d/d4a/l11b 0 2024-06-06T08:43:54.714 INFO:tasks.workunit.client.0.smithi027.stdout:1/908: creat dc/de/d3d/d4a/f11c x:0 0 0 2024-06-06T08:43:54.716 INFO:tasks.workunit.client.0.smithi027.stdout:1/909: symlink dc/de/d3d/l11d 0 2024-06-06T08:43:54.723 INFO:tasks.workunit.client.0.smithi027.stdout:2/925: dwrite d6/dd/dd9/d3f/d77/d55/fa4 [0,4194304] 0 2024-06-06T08:43:54.726 INFO:tasks.workunit.client.0.smithi027.stdout:1/910: truncate dc/de/d3d/d4f/f64 1283464 0 2024-06-06T08:43:54.728 INFO:tasks.workunit.client.0.smithi027.stdout:1/911: link dc/de/d22/l1f dc/de/l11e 0 2024-06-06T08:43:54.730 INFO:tasks.workunit.client.0.smithi027.stdout:1/912: mknod dc/d67/d69/d32/d37/d56/d78/c11f 0 2024-06-06T08:43:54.731 INFO:tasks.workunit.client.0.smithi027.stdout:1/913: creat dc/de/d3d/d4a/d76/f120 x:0 0 0 2024-06-06T08:43:54.731 INFO:tasks.workunit.client.0.smithi027.stdout:1/914: read - dc/d67/d69/fba zero size 2024-06-06T08:43:54.734 INFO:tasks.workunit.client.0.smithi027.stdout:1/915: dread dc/fd [0,4194304] 0 2024-06-06T08:43:54.735 INFO:tasks.workunit.client.0.smithi027.stdout:1/916: link dc/da0/fcd dc/d67/d69/d32/d37/ddf/dfa/d107/f121 0 2024-06-06T08:43:54.739 INFO:tasks.workunit.client.0.smithi027.stdout:2/926: dwrite d6/dd/dd9/d3f/f71 [0,4194304] 0 2024-06-06T08:43:54.754 INFO:tasks.workunit.client.0.smithi027.stdout:1/917: dwrite dc/de/d3d/d4a/ff3 [0,4194304] 0 2024-06-06T08:43:54.754 INFO:tasks.workunit.client.0.smithi027.stdout:1/918: fsync dc/d67/d69/d32/fed 0 2024-06-06T08:43:54.757 INFO:tasks.workunit.client.0.smithi027.stdout:2/927: dwrite d6/d1e/dfb/ffe [0,4194304] 0 2024-06-06T08:43:54.758 INFO:tasks.workunit.client.0.smithi027.stdout:1/919: unlink dc/d67/d69/d32/d37/d56/df1/f1a 0 2024-06-06T08:43:54.759 INFO:tasks.workunit.client.0.smithi027.stdout:2/928: mknod d6/dd/dd9/d3f/db0/d108/c13b 0 2024-06-06T08:43:54.760 INFO:tasks.workunit.client.0.smithi027.stdout:3/834: dwrite d1/d90/d6d/f86 [0,4194304] 0 2024-06-06T08:43:54.764 INFO:tasks.workunit.client.0.smithi027.stdout:3/835: rmdir d1/d1f/dbc 39 2024-06-06T08:43:54.764 INFO:tasks.workunit.client.0.smithi027.stdout:3/836: readlink d1/d90/d6d/dac/lf2 0 2024-06-06T08:43:54.765 INFO:tasks.workunit.client.0.smithi027.stdout:1/920: dread dc/fc0 [0,4194304] 0 2024-06-06T08:43:54.765 INFO:tasks.workunit.client.0.smithi027.stdout:1/921: chown dc/d67/d69/fb9 25324 1 2024-06-06T08:43:54.765 INFO:tasks.workunit.client.0.smithi027.stdout:1/922: truncate dc/de/d3d/d4a/d76/dd7/ff5 833953 0 2024-06-06T08:43:54.769 INFO:tasks.workunit.client.0.smithi027.stdout:1/923: creat dc/d67/d69/d32/d71/f122 x:0 0 0 2024-06-06T08:43:54.769 INFO:tasks.workunit.client.0.smithi027.stdout:1/924: chown dc/d67/d69/d32/f11a 5 1 2024-06-06T08:43:54.771 INFO:tasks.workunit.client.0.smithi027.stdout:1/925: rename dc/d67/d69/d32/d37/l6b to dc/de/d3d/d4f/l123 0 2024-06-06T08:43:54.771 INFO:tasks.workunit.client.0.smithi027.stdout:1/926: write dc/de/d3d/d4a/f11c [572988,79760] 0 2024-06-06T08:43:54.771 INFO:tasks.workunit.client.0.smithi027.stdout:1/927: chown dc/de/d3d/l11d 6230 1 2024-06-06T08:43:54.771 INFO:tasks.workunit.client.0.smithi027.stdout:1/928: write dc/d67/d69/d32/d37/d56/df1/d80/fe6 [42636,36492] 0 2024-06-06T08:43:54.771 INFO:tasks.workunit.client.0.smithi027.stdout:1/929: chown dc/f36 29047 1 2024-06-06T08:43:54.771 INFO:tasks.workunit.client.0.smithi027.stdout:1/930: stat dc/d67/d69/l9c 0 2024-06-06T08:43:54.774 INFO:tasks.workunit.client.0.smithi027.stdout:1/931: getdents dc/d67/d69/d32/d37/d56/df1/d4e 0 2024-06-06T08:43:54.776 INFO:tasks.workunit.client.0.smithi027.stdout:1/932: creat dc/d67/d69/d32/d37/d56/df1/f124 x:0 0 0 2024-06-06T08:43:54.779 INFO:tasks.workunit.client.0.smithi027.stdout:2/929: dwrite d6/dd/dd9/f54 [0,4194304] 0 2024-06-06T08:43:54.781 INFO:tasks.workunit.client.0.smithi027.stdout:2/930: unlink d6/d1e/d2b/c135 0 2024-06-06T08:43:54.782 INFO:tasks.workunit.client.0.smithi027.stdout:2/931: fsync d6/d1e/f1f 0 2024-06-06T08:43:54.782 INFO:tasks.workunit.client.0.smithi027.stdout:2/932: chown f5 0 1 2024-06-06T08:43:54.787 INFO:tasks.workunit.client.0.smithi027.stdout:1/933: write dc/d67/d69/d32/d37/d56/df1/d80/fa4 [1446121,119153] 0 2024-06-06T08:43:54.789 INFO:tasks.workunit.client.0.smithi027.stdout:1/934: mkdir dc/d6d/d125 0 2024-06-06T08:43:54.790 INFO:tasks.workunit.client.0.smithi027.stdout:1/935: rename dc/da0/cc5 to dc/d67/d69/d32/d37/c126 0 2024-06-06T08:43:54.791 INFO:tasks.workunit.client.0.smithi027.stdout:1/936: symlink dc/d67/d69/d32/d37/d56/de1/l127 0 2024-06-06T08:43:54.792 INFO:tasks.workunit.client.0.smithi027.stdout:1/937: mknod dc/d67/d69/d32/d37/c128 0 2024-06-06T08:43:54.798 INFO:tasks.workunit.client.0.smithi027.stdout:2/933: dwrite d6/dd/dd9/d3f/d77/d55/fa4 [0,4194304] 0 2024-06-06T08:43:54.799 INFO:tasks.workunit.client.0.smithi027.stdout:2/934: mkdir d6/dd/dd9/d3f/d48/d83/de3/d13c 0 2024-06-06T08:43:54.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/935: rename f5 to d6/dd/dd9/d3f/f13d 0 2024-06-06T08:43:54.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/936: stat d6/dd/dd9/d1d/d4d 0 2024-06-06T08:43:54.805 INFO:tasks.workunit.client.0.smithi027.stdout:2/937: creat d6/dd/dd9/d1d/d4d/d8e/f13e x:0 0 0 2024-06-06T08:43:54.808 INFO:tasks.workunit.client.0.smithi027.stdout:1/938: dwrite dc/de/d3d/d4a/d76/fec [0,4194304] 0 2024-06-06T08:43:54.809 INFO:tasks.workunit.client.0.smithi027.stdout:2/938: dread d6/dd/dd9/d3f/db0/fed [0,4194304] 0 2024-06-06T08:43:54.810 INFO:tasks.workunit.client.0.smithi027.stdout:2/939: mknod d6/d1e/d2b/d5a/c13f 0 2024-06-06T08:43:54.811 INFO:tasks.workunit.client.0.smithi027.stdout:1/939: unlink dc/de/d3d/d4a/lc4 0 2024-06-06T08:43:54.812 INFO:tasks.workunit.client.0.smithi027.stdout:2/940: unlink d6/d3b/l53 0 2024-06-06T08:43:54.814 INFO:tasks.workunit.client.0.smithi027.stdout:1/940: unlink dc/d67/d69/d32/d37/d56/df1/f87 0 2024-06-06T08:43:54.814 INFO:tasks.workunit.client.0.smithi027.stdout:1/941: chown dc/d67/d69/d32/d37/d56/ff7 3 1 2024-06-06T08:43:54.814 INFO:tasks.workunit.client.0.smithi027.stdout:2/941: creat d6/dd/dd9/d3f/d77/d55/d120/f140 x:0 0 0 2024-06-06T08:43:54.815 INFO:tasks.workunit.client.0.smithi027.stdout:1/942: mkdir dc/d67/d129 0 2024-06-06T08:43:54.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/943: readlink dc/de/d3d/l66 0 2024-06-06T08:43:54.817 INFO:tasks.workunit.client.0.smithi027.stdout:1/944: getdents dc/de/d3d/d4a/d76/dd7 0 2024-06-06T08:43:54.817 INFO:tasks.workunit.client.0.smithi027.stdout:1/945: readlink dc/d67/d69/d32/d37/d98/le0 0 2024-06-06T08:43:54.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/942: rmdir d6/dd/dd9 39 2024-06-06T08:43:54.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/943: stat d6/dd/dd9/d1d/d4d/d8e/fc3 0 2024-06-06T08:43:54.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/944: fsync d6/dd/dd9/f25 0 2024-06-06T08:43:54.825 INFO:tasks.workunit.client.0.smithi027.stdout:1/946: write dc/de/d3d/f50 [1544471,129137] 0 2024-06-06T08:43:54.825 INFO:tasks.workunit.client.0.smithi027.stdout:1/947: dread - dc/d67/d69/d32/f11a zero size 2024-06-06T08:43:54.825 INFO:tasks.workunit.client.0.smithi027.stdout:1/948: write dc/d67/d69/d32/d37/d56/df1/f15 [3735407,128272] 0 2024-06-06T08:43:54.826 INFO:tasks.workunit.client.0.smithi027.stdout:2/945: write d6/dd/dd9/d3f/d77/d55/f74 [2675256,81009] 0 2024-06-06T08:43:54.826 INFO:tasks.workunit.client.0.smithi027.stdout:1/949: read dc/de/d3d/f3f [32155,89031] 0 2024-06-06T08:43:54.828 INFO:tasks.workunit.client.0.smithi027.stdout:2/946: creat d6/dd/dd9/d3f/db0/f141 x:0 0 0 2024-06-06T08:43:54.829 INFO:tasks.workunit.client.0.smithi027.stdout:1/950: rename dc/d67/d69/d32/d37/d56/df1/d80/da6 to dc/d67/d69/d32/d37/d56/df1/dc8/d12a 0 2024-06-06T08:43:54.832 INFO:tasks.workunit.client.0.smithi027.stdout:1/951: write dc/de/d22/f3b [4858350,9766] 0 2024-06-06T08:43:54.832 INFO:tasks.workunit.client.0.smithi027.stdout:1/952: read - dc/d67/d69/d32/d37/d56/df1/f8a zero size 2024-06-06T08:43:54.833 INFO:tasks.workunit.client.0.smithi027.stdout:1/953: unlink dc/de/d3d/d4f/cc2 0 2024-06-06T08:43:54.833 INFO:tasks.workunit.client.0.smithi027.stdout:1/954: chown dc/de 901 1 2024-06-06T08:43:54.841 INFO:tasks.workunit.client.0.smithi027.stdout:1/955: dread dc/de/d3d/d4a/f58 [0,4194304] 0 2024-06-06T08:43:54.841 INFO:tasks.workunit.client.0.smithi027.stdout:1/956: readlink dc/de/d3d/d4f/lcc 0 2024-06-06T08:43:54.841 INFO:tasks.workunit.client.0.smithi027.stdout:1/957: chown dc/de/d3d/d4a/l11b 94525859 1 2024-06-06T08:43:54.858 INFO:tasks.workunit.client.0.smithi027.stdout:1/958: dwrite dc/de/d3d/f3f [0,4194304] 0 2024-06-06T08:43:54.858 INFO:tasks.workunit.client.0.smithi027.stdout:1/959: chown dc/de/d3d/f3f 15 1 2024-06-06T08:43:54.860 INFO:tasks.workunit.client.0.smithi027.stdout:1/960: link dc/d67/d69/d32/d37/d98/cc1 dc/d6d/d125/c12b 0 2024-06-06T08:43:54.864 INFO:tasks.workunit.client.0.smithi027.stdout:2/947: dwrite d6/fc0 [0,4194304] 0 2024-06-06T08:43:54.880 INFO:tasks.workunit.client.0.smithi027.stdout:2/948: dwrite d6/d1e/d2b/d5a/db7/de2/f11b [0,4194304] 0 2024-06-06T08:43:54.890 INFO:tasks.workunit.client.0.smithi027.stdout:2/949: write d6/dd/f26 [2649723,59401] 0 2024-06-06T08:43:54.892 INFO:tasks.workunit.client.0.smithi027.stdout:2/950: rmdir d6/d1e/d2b/d5a/db7/de2 39 2024-06-06T08:43:54.892 INFO:tasks.workunit.client.0.smithi027.stdout:2/951: truncate d6/dd/dd9/d1d/d4d/fff 420842 0 2024-06-06T08:43:54.893 INFO:tasks.workunit.client.0.smithi027.stdout:2/952: truncate d6/f6e 410664 0 2024-06-06T08:43:54.893 INFO:tasks.workunit.client.0.smithi027.stdout:2/953: truncate d6/dd/dd9/d3f/d77/d55/d69/d82/fa3 1110770 0 2024-06-06T08:43:54.894 INFO:tasks.workunit.client.0.smithi027.stdout:2/954: chown d6/dd/l8c 1976 1 2024-06-06T08:43:54.894 INFO:tasks.workunit.client.0.smithi027.stdout:2/955: readlink d6/d1e/d2b/ldd 0 2024-06-06T08:43:54.894 INFO:tasks.workunit.client.0.smithi027.stdout:2/956: stat d6/dd/dd9/d3f/d77/d55/d69/d82/l66 0 2024-06-06T08:43:54.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/957: fdatasync d6/dd/dd9/f12 0 2024-06-06T08:43:54.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/958: creat d6/dd/dd9/d3f/d77/de5/f142 x:0 0 0 2024-06-06T08:43:54.896 INFO:tasks.workunit.client.0.smithi027.stdout:2/959: truncate d6/dd/dd9/d3f/d48/fde 445116 0 2024-06-06T08:43:54.897 INFO:tasks.workunit.client.0.smithi027.stdout:2/960: rename d6/d1e/dfb/ffe to d6/dd/dd9/d3f/d111/f143 0 2024-06-06T08:43:54.906 INFO:tasks.workunit.client.0.smithi027.stdout:2/961: link d6/d3b/d80/ff3 d6/d1e/d2b/d5a/d137/f144 0 2024-06-06T08:43:54.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/962: link d6/dd/dd9/d3f/db0/d123/f70 d6/dd/dd9/d1d/dbc/f145 0 2024-06-06T08:43:54.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/963: fdatasync d6/dd/dd9/d3f/d48/d83/de3/f131 0 2024-06-06T08:43:54.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/964: fsync d6/dd/dd9/f29 0 2024-06-06T08:43:54.921 INFO:tasks.workunit.client.0.smithi027.stdout:2/965: dwrite d6/d1e/d2b/fe0 [0,4194304] 0 2024-06-06T08:43:54.924 INFO:tasks.workunit.client.0.smithi027.stdout:2/966: mknod d6/dd/dd9/d3f/d77/d55/c146 0 2024-06-06T08:43:54.924 INFO:tasks.workunit.client.0.smithi027.stdout:2/967: chown d6/dd/dd9/d1d/f60 6 1 2024-06-06T08:43:54.926 INFO:tasks.workunit.client.0.smithi027.stdout:2/968: dread d6/fd3 [0,4194304] 0 2024-06-06T08:43:54.927 INFO:tasks.workunit.client.0.smithi027.stdout:2/969: creat d6/dd/dd9/d1d/d4d/d8e/f147 x:0 0 0 2024-06-06T08:43:54.942 INFO:tasks.workunit.client.0.smithi027.stdout:2/970: dwrite d6/dd/dd9/d3f/d48/fde [0,4194304] 0 2024-06-06T08:43:54.944 INFO:tasks.workunit.client.0.smithi027.stdout:2/971: mknod d6/dd/dd9/d3f/db0/d123/c148 0 2024-06-06T08:43:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:54 smithi027 ceph-mon[32335]: pgmap v112: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 28 MiB/s wr, 364 op/s 2024-06-06T08:43:54.958 INFO:tasks.workunit.client.0.smithi027.stdout:2/972: dwrite d6/dd/dd9/d1d/d4d/f5f [0,4194304] 0 2024-06-06T08:43:54.959 INFO:tasks.workunit.client.0.smithi027.stdout:2/973: creat d6/dd/f149 x:0 0 0 2024-06-06T08:43:54.960 INFO:tasks.workunit.client.0.smithi027.stdout:2/974: truncate d6/f9d 952969 0 2024-06-06T08:43:54.961 INFO:tasks.workunit.client.0.smithi027.stdout:2/975: link d6/d1e/d2b/d5a/d7b/c122 d6/dd/d6b/c14a 0 2024-06-06T08:43:54.971 INFO:tasks.workunit.client.0.smithi027.stdout:2/976: dread d6/dd/dd9/f22 [0,4194304] 0 2024-06-06T08:43:54.971 INFO:tasks.workunit.client.0.smithi027.stdout:2/977: chown d6/dd/dd9/f40 40292268 1 2024-06-06T08:43:54.972 INFO:tasks.workunit.client.0.smithi027.stdout:2/978: read - d6/d1e/d2b/d5a/fd6 zero size 2024-06-06T08:43:54.973 INFO:tasks.workunit.client.0.smithi027.stdout:2/979: mkdir d6/d1e/d2b/d5a/db7/de2/d14b 0 2024-06-06T08:43:54.973 INFO:tasks.workunit.client.0.smithi027.stdout:2/980: fsync d6/dd/dd9/d3f/d48/d83/fe6 0 2024-06-06T08:43:54.974 INFO:tasks.workunit.client.0.smithi027.stdout:2/981: getdents d6/d1e/d2b 0 2024-06-06T08:43:54.975 INFO:tasks.workunit.client.0.smithi027.stdout:2/982: creat d6/d1e/d2b/d5a/d7b/f14c x:0 0 0 2024-06-06T08:43:54.975 INFO:tasks.workunit.client.0.smithi027.stdout:2/983: creat d6/dd/dd9/d1d/d4d/f14d x:0 0 0 2024-06-06T08:43:54.977 INFO:tasks.workunit.client.0.smithi027.stdout:2/984: mknod d6/dd/dd9/d1d/dbc/c14e 0 2024-06-06T08:43:54.978 INFO:tasks.workunit.client.0.smithi027.stdout:2/985: truncate d6/dd/dd9/d3f/d77/d55/fa4 1108606 0 2024-06-06T08:43:54.979 INFO:tasks.workunit.client.0.smithi027.stdout:2/986: mkdir d6/dd/dd9/d1d/dbc/d14f 0 2024-06-06T08:43:54.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/987: readlink d6/dd/dd9/d3f/l99 0 2024-06-06T08:43:54.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/988: dread - d6/dd/dd9/d3f/d48/d83/de3/d100/f132 zero size 2024-06-06T08:43:54.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/989: chown d6/dd/dd9/d3f/d77/d55/d69/f133 16444849 1 2024-06-06T08:43:54.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/990: write d6/fd3 [954356,54915] 0 2024-06-06T08:43:54.981 INFO:tasks.workunit.client.0.smithi027.stdout:2/991: write d6/f2a [4768972,98663] 0 2024-06-06T08:43:54.982 INFO:tasks.workunit.client.0.smithi027.stdout:2/992: creat d6/dd/dd9/d3f/d48/d83/de3/de4/f150 x:0 0 0 2024-06-06T08:43:54.983 INFO:tasks.workunit.client.0.smithi027.stdout:2/993: creat d6/dd/dd9/d3f/d48/d83/de3/d13c/f151 x:0 0 0 2024-06-06T08:43:54.984 INFO:tasks.workunit.client.0.smithi027.stdout:2/994: dread d6/dd/dd9/d3f/d111/de1/fec [0,4194304] 0 2024-06-06T08:43:54.986 INFO:tasks.workunit.client.0.smithi027.stdout:2/995: mknod d6/c152 0 2024-06-06T08:43:54.987 INFO:tasks.workunit.client.0.smithi027.stdout:2/996: write d6/dd/dd9/d1d/d4d/d8e/f147 [1036620,13168] 0 2024-06-06T08:43:54.987 INFO:tasks.workunit.client.0.smithi027.stdout:2/997: chown d6/dd/dd9/d3f/d77/d55/d69/f133 53289664 1 2024-06-06T08:43:54.989 INFO:tasks.workunit.client.0.smithi027.stdout:2/998: mknod d6/dd/dd9/d1d/d4d/ddc/c153 0 2024-06-06T08:43:54.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:54 smithi203 ceph-mon[36579]: pgmap v112: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 28 MiB/s wr, 364 op/s 2024-06-06T08:43:54.995 INFO:tasks.workunit.client.0.smithi027.stdout:2/999: write d6/dd/dd9/d3f/d77/d55/f7f [1026299,47465] 0 2024-06-06T08:43:55.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:54 smithi176 ceph-mon[36786]: pgmap v112: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 28 MiB/s wr, 364 op/s 2024-06-06T08:43:55.018 INFO:tasks.workunit.client.0.smithi027.stdout:0/876: sync 2024-06-06T08:43:55.018 INFO:tasks.workunit.client.0.smithi027.stdout:0/877: truncate d15/d18/d23/d28/d31/dbb/f11d 492326 0 2024-06-06T08:43:55.032 INFO:tasks.workunit.client.0.smithi027.stdout:0/878: dwrite d15/d18/d40/d8c/d9c/fa6 [0,4194304] 0 2024-06-06T08:43:55.045 INFO:tasks.workunit.client.0.smithi027.stdout:0/879: dwrite d15/d18/d23/d6f/ffb [0,4194304] 0 2024-06-06T08:43:55.045 INFO:tasks.workunit.client.0.smithi027.stdout:0/880: stat d15/d18/d40/cc3 0 2024-06-06T08:43:55.051 INFO:tasks.workunit.client.0.smithi027.stdout:0/881: write d15/d18/dbe/d5c/d83/f86 [2915524,87866] 0 2024-06-06T08:43:55.063 INFO:tasks.workunit.client.0.smithi027.stdout:0/882: dwrite d15/d18/d64/fcf [0,4194304] 0 2024-06-06T08:43:55.063 INFO:tasks.workunit.client.0.smithi027.stdout:0/883: dread - d15/d18/d23/d38/d68/d85/ff3 zero size 2024-06-06T08:43:55.063 INFO:tasks.workunit.client.0.smithi027.stdout:0/884: read - d15/d18/ff6 zero size 2024-06-06T08:43:55.063 INFO:tasks.workunit.client.0.smithi027.stdout:0/885: chown d15/d18/c112 25 1 2024-06-06T08:43:55.076 INFO:tasks.workunit.client.0.smithi027.stdout:0/886: dwrite d15/d18/d23/d6f/dcb/f123 [0,4194304] 0 2024-06-06T08:43:55.078 INFO:tasks.workunit.client.0.smithi027.stdout:0/887: mkdir d15/d18/d23/d28/d128 0 2024-06-06T08:43:55.085 INFO:tasks.workunit.client.0.smithi027.stdout:0/888: dread d15/d18/d23/d6f/dcb/f123 [0,4194304] 0 2024-06-06T08:43:55.088 INFO:tasks.workunit.client.0.smithi027.stdout:0/889: mknod d15/d18/d23/d28/d31/c129 0 2024-06-06T08:43:55.089 INFO:tasks.workunit.client.0.smithi027.stdout:0/890: creat d15/d18/d23/d28/f12a x:0 0 0 2024-06-06T08:43:55.091 INFO:tasks.workunit.client.0.smithi027.stdout:0/891: dread d15/d18/d23/d28/d2a/d2d/f54 [0,4194304] 0 2024-06-06T08:43:55.091 INFO:tasks.workunit.client.0.smithi027.stdout:0/892: chown d15/d18/d23/d38/d68/f8f 0 1 2024-06-06T08:43:55.103 INFO:tasks.workunit.client.0.smithi027.stdout:0/893: dwrite d15/d18/d23/d6f/f94 [0,4194304] 0 2024-06-06T08:43:55.237 INFO:tasks.workunit.client.0.smithi027.stdout:1/961: sync 2024-06-06T08:43:55.237 INFO:tasks.workunit.client.0.smithi027.stdout:3/837: sync 2024-06-06T08:43:55.238 INFO:tasks.workunit.client.0.smithi027.stdout:9/887: sync 2024-06-06T08:43:55.238 INFO:tasks.workunit.client.0.smithi027.stdout:8/840: sync 2024-06-06T08:43:55.238 INFO:tasks.workunit.client.0.smithi027.stdout:1/962: dread - dc/d67/d69/d32/d37/d56/d78/f10b zero size 2024-06-06T08:43:55.238 INFO:tasks.workunit.client.0.smithi027.stdout:1/963: write dc/de/d3d/d4a/d76/f120 [118442,53729] 0 2024-06-06T08:43:55.238 INFO:tasks.workunit.client.0.smithi027.stdout:1/964: readlink dc/d67/d69/d32/d37/d56/df1/dc8/lc9 0 2024-06-06T08:43:55.238 INFO:tasks.workunit.client.0.smithi027.stdout:1/965: chown dc/d67/d69/d32/d37/l97 163336 1 2024-06-06T08:43:55.239 INFO:tasks.workunit.client.0.smithi027.stdout:1/966: chown dc/d67/d69/d32/d37 21 1 2024-06-06T08:43:55.239 INFO:tasks.workunit.client.0.smithi027.stdout:1/967: dread - dc/d67/d69/d32/d37/d56/df1/f124 zero size 2024-06-06T08:43:55.240 INFO:tasks.workunit.client.0.smithi027.stdout:3/838: creat d1/d8/da9/f108 x:0 0 0 2024-06-06T08:43:55.241 INFO:tasks.workunit.client.0.smithi027.stdout:8/841: creat d7/dc/d36/d3c/f111 x:0 0 0 2024-06-06T08:43:55.241 INFO:tasks.workunit.client.0.smithi027.stdout:1/968: mknod dc/d6d/d125/c12c 0 2024-06-06T08:43:55.241 INFO:tasks.workunit.client.0.smithi027.stdout:8/842: stat d7/dc/d36/d41/d85/l9e 0 2024-06-06T08:43:55.241 INFO:tasks.workunit.client.0.smithi027.stdout:3/839: creat d1/d1f/d4d/ddc/f109 x:0 0 0 2024-06-06T08:43:55.241 INFO:tasks.workunit.client.0.smithi027.stdout:3/840: read - d1/d90/d6d/f81 zero size 2024-06-06T08:43:55.241 INFO:tasks.workunit.client.0.smithi027.stdout:1/969: mknod dc/d67/d69/d32/d37/d98/c12d 0 2024-06-06T08:43:55.242 INFO:tasks.workunit.client.0.smithi027.stdout:8/843: creat d7/dc/d36/d83/d60/f112 x:0 0 0 2024-06-06T08:43:55.242 INFO:tasks.workunit.client.0.smithi027.stdout:1/970: link dc/de/d3d/d4f/l5c dc/de/d3d/d4a/l12e 0 2024-06-06T08:43:55.244 INFO:tasks.workunit.client.0.smithi027.stdout:1/971: rename dc/de/d3d/f85 to dc/d67/d69/d32/d37/d56/df1/dc8/d12a/db0/f12f 0 2024-06-06T08:43:55.245 INFO:tasks.workunit.client.0.smithi027.stdout:1/972: getdents dc/d6d 0 2024-06-06T08:43:55.253 INFO:tasks.workunit.client.0.smithi027.stdout:9/888: dwrite d3/d95/d34/f60 [0,4194304] 0 2024-06-06T08:43:55.257 INFO:tasks.workunit.client.0.smithi027.stdout:3/841: dwrite d1/d8/d59/f94 [0,4194304] 0 2024-06-06T08:43:55.257 INFO:tasks.workunit.client.0.smithi027.stdout:9/889: creat d3/d95/f139 x:0 0 0 2024-06-06T08:43:55.259 INFO:tasks.workunit.client.0.smithi027.stdout:3/842: mkdir d1/d8/d22/d27/d79/d10a 0 2024-06-06T08:43:55.259 INFO:tasks.workunit.client.0.smithi027.stdout:1/973: write dc/de/d3d/d4a/ff3 [1967959,111683] 0 2024-06-06T08:43:55.260 INFO:tasks.workunit.client.0.smithi027.stdout:8/844: dwrite d7/f4e [0,4194304] 0 2024-06-06T08:43:55.260 INFO:tasks.workunit.client.0.smithi027.stdout:8/845: write d7/dc/f69 [1029924,95268] 0 2024-06-06T08:43:55.261 INFO:tasks.workunit.client.0.smithi027.stdout:3/843: creat d1/d1f/d45/d5f/dd6/f10b x:0 0 0 2024-06-06T08:43:55.261 INFO:tasks.workunit.client.0.smithi027.stdout:3/844: truncate d1/d1f/d45/ff3 599776 0 2024-06-06T08:43:55.262 INFO:tasks.workunit.client.0.smithi027.stdout:1/974: creat dc/d67/d69/d32/d37/d56/f130 x:0 0 0 2024-06-06T08:43:55.266 INFO:tasks.workunit.client.0.smithi027.stdout:1/975: dread - dc/de/de4/d101/f106 zero size 2024-06-06T08:43:55.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/890: write d3/d95/faa [3832501,29660] 0 2024-06-06T08:43:55.268 INFO:tasks.workunit.client.0.smithi027.stdout:1/976: dread dc/d67/d69/d32/d37/d56/df1/d80/fc6 [0,4194304] 0 2024-06-06T08:43:55.271 INFO:tasks.workunit.client.0.smithi027.stdout:3/845: mkdir d1/d1f/d45/d5f/d10c 0 2024-06-06T08:43:55.272 INFO:tasks.workunit.client.0.smithi027.stdout:3/846: getdents d1/d1f/d45/d5f/dd6/df1 0 2024-06-06T08:43:55.272 INFO:tasks.workunit.client.0.smithi027.stdout:3/847: chown d1/d90/d6d/dac/de0 1044681 1 2024-06-06T08:43:55.273 INFO:tasks.workunit.client.0.smithi027.stdout:3/848: symlink d1/l10d 0 2024-06-06T08:43:55.273 INFO:tasks.workunit.client.0.smithi027.stdout:3/849: fsync d1/d8/da9/f108 0 2024-06-06T08:43:55.273 INFO:tasks.workunit.client.0.smithi027.stdout:3/850: fsync d1/d90/d6d/f9b 0 2024-06-06T08:43:55.273 INFO:tasks.workunit.client.0.smithi027.stdout:3/851: stat d1/d8/d22/d2d/f7f 0 2024-06-06T08:43:55.277 INFO:tasks.workunit.client.0.smithi027.stdout:9/891: dread d3/d95/f2f [0,4194304] 0 2024-06-06T08:43:55.277 INFO:tasks.workunit.client.0.smithi027.stdout:9/892: readlink d3/d95/d34/d73/l12b 0 2024-06-06T08:43:55.295 INFO:tasks.workunit.client.0.smithi027.stdout:1/977: dwrite dc/d67/d69/d32/d37/d56/de1/f119 [0,4194304] 0 2024-06-06T08:43:55.297 INFO:tasks.workunit.client.0.smithi027.stdout:1/978: mknod dc/d67/c131 0 2024-06-06T08:43:55.298 INFO:tasks.workunit.client.0.smithi027.stdout:3/852: dwrite d1/d1f/dbc/fcc [0,4194304] 0 2024-06-06T08:43:55.298 INFO:tasks.workunit.client.0.smithi027.stdout:3/853: chown d1/d8/d22/d27/d79/fb8 3522390 1 2024-06-06T08:43:55.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/893: dwrite d3/d95/d23/f3d [0,4194304] 0 2024-06-06T08:43:55.309 INFO:tasks.workunit.client.0.smithi027.stdout:9/894: mkdir d3/d4e/d75/dc/d13a 0 2024-06-06T08:43:55.316 INFO:tasks.workunit.client.0.smithi027.stdout:3/854: dwrite d1/d90/d44/d54/fce [0,4194304] 0 2024-06-06T08:43:55.317 INFO:tasks.workunit.client.0.smithi027.stdout:1/979: dwrite dc/de/d3d/d4a/f58 [0,4194304] 0 2024-06-06T08:43:55.318 INFO:tasks.workunit.client.0.smithi027.stdout:9/895: dread d3/d95/d23/d36/f37 [0,4194304] 0 2024-06-06T08:43:55.328 INFO:tasks.workunit.client.0.smithi027.stdout:3/855: getdents d1/d90/d6d/dac/de0 0 2024-06-06T08:43:55.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/856: readlink d1/lfb 0 2024-06-06T08:43:55.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/857: write d1/d1f/d45/d5f/dd6/f10b [200483,122802] 0 2024-06-06T08:43:55.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/858: write d1/d8/f61 [521984,81368] 0 2024-06-06T08:43:55.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/859: write d1/d1f/d4d/dd5/f104 [257381,121430] 0 2024-06-06T08:43:55.335 INFO:tasks.workunit.client.0.smithi027.stdout:1/980: creat dc/d67/d69/d32/d37/d56/df1/dc8/d12a/f132 x:0 0 0 2024-06-06T08:43:55.335 INFO:tasks.workunit.client.0.smithi027.stdout:1/981: write dc/f81 [5444023,43367] 0 2024-06-06T08:43:55.335 INFO:tasks.workunit.client.0.smithi027.stdout:1/982: write dc/d67/d69/d32/d37/d56/df1/d80/fc6 [485442,100235] 0 2024-06-06T08:43:55.337 INFO:tasks.workunit.client.0.smithi027.stdout:9/896: unlink d3/d4e/f9d 0 2024-06-06T08:43:55.339 INFO:tasks.workunit.client.0.smithi027.stdout:9/897: mkdir d3/d95/d23/d78/d138/d13b 0 2024-06-06T08:43:55.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/860: truncate d1/d90/d6d/f9b 2484626 0 2024-06-06T08:43:55.353 INFO:tasks.workunit.client.0.smithi027.stdout:1/983: dwrite dc/d67/d69/d32/d37/d56/d78/fea [0,4194304] 0 2024-06-06T08:43:55.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/898: dwrite d3/d95/d34/d41/fd3 [0,4194304] 0 2024-06-06T08:43:55.360 INFO:tasks.workunit.client.0.smithi027.stdout:9/899: symlink d3/d95/d23/d78/ddf/l13c 0 2024-06-06T08:43:55.372 INFO:tasks.workunit.client.0.smithi027.stdout:1/984: dread dc/f81 [0,4194304] 0 2024-06-06T08:43:55.372 INFO:tasks.workunit.client.0.smithi027.stdout:1/985: stat dc/d67/d69/d32/d37/d56 0 2024-06-06T08:43:55.374 INFO:tasks.workunit.client.0.smithi027.stdout:1/986: rmdir dc/d9d 39 2024-06-06T08:43:55.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/861: dwrite d1/d8/d22/d27/d79/dfd/fe4 [0,4194304] 0 2024-06-06T08:43:55.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/862: chown d1/d1f/l35 378608 1 2024-06-06T08:43:55.383 INFO:tasks.workunit.client.0.smithi027.stdout:3/863: link d1/d90/d44/l96 d1/d8/d22/d2d/l10e 0 2024-06-06T08:43:55.383 INFO:tasks.workunit.client.0.smithi027.stdout:3/864: fdatasync d1/d8/d22/d27/d79/dfd/f82 0 2024-06-06T08:43:55.383 INFO:tasks.workunit.client.0.smithi027.stdout:3/865: stat d1/d8/d22/d27/d79/dfd/fbe 0 2024-06-06T08:43:55.385 INFO:tasks.workunit.client.0.smithi027.stdout:1/987: dread dc/de/d3d/d4a/f58 [0,4194304] 0 2024-06-06T08:43:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/866: rmdir d1/d1f/d45/d5f/d10c 0 2024-06-06T08:43:55.388 INFO:tasks.workunit.client.0.smithi027.stdout:0/894: sync 2024-06-06T08:43:55.391 INFO:tasks.workunit.client.0.smithi027.stdout:3/867: mkdir d1/d8/d22/d27/d79/d10a/d10f 0 2024-06-06T08:43:55.395 INFO:tasks.workunit.client.0.smithi027.stdout:3/868: mkdir d1/d8/d59/d110 0 2024-06-06T08:43:55.395 INFO:tasks.workunit.client.0.smithi027.stdout:3/869: chown d1/d90/d44/l96 86904 1 2024-06-06T08:43:55.395 INFO:tasks.workunit.client.0.smithi027.stdout:1/988: dread dc/de/d3d/f3f [0,4194304] 0 2024-06-06T08:43:55.400 INFO:tasks.workunit.client.0.smithi027.stdout:1/989: symlink dc/d67/d69/d32/d37/d56/df1/dc8/dfc/l133 0 2024-06-06T08:43:55.400 INFO:tasks.workunit.client.0.smithi027.stdout:1/990: write dc/d67/d69/d32/d37/d56/df1/f124 [552407,121856] 0 2024-06-06T08:43:55.402 INFO:tasks.workunit.client.0.smithi027.stdout:3/870: rename d1/d8/f16 to d1/d90/d6d/f111 0 2024-06-06T08:43:55.405 INFO:tasks.workunit.client.0.smithi027.stdout:1/991: rename dc/l2d to dc/d6d/l134 0 2024-06-06T08:43:55.406 INFO:tasks.workunit.client.0.smithi027.stdout:1/992: chown dc/d67/d69/d32/d37/c128 15548 1 2024-06-06T08:43:55.409 INFO:tasks.workunit.client.0.smithi027.stdout:3/871: getdents d1/d90/d44/d54 0 2024-06-06T08:43:55.411 INFO:tasks.workunit.client.0.smithi027.stdout:3/872: mkdir d1/d8/da9/d112 0 2024-06-06T08:43:55.411 INFO:tasks.workunit.client.0.smithi027.stdout:3/873: chown d1/d8/d22/d2d/fd0 315 1 2024-06-06T08:43:55.413 INFO:tasks.workunit.client.0.smithi027.stdout:3/874: creat d1/d8/d22/d27/d79/d106/f113 x:0 0 0 2024-06-06T08:43:55.421 INFO:tasks.workunit.client.0.smithi027.stdout:1/993: dwrite dc/de/d3d/d4a/d76/f120 [0,4194304] 0 2024-06-06T08:43:55.423 INFO:tasks.workunit.client.0.smithi027.stdout:3/875: write d1/d90/d6d/f9a [2564868,101191] 0 2024-06-06T08:43:55.423 INFO:tasks.workunit.client.0.smithi027.stdout:3/876: fdatasync d1/d90/d6d/dac/fee 0 2024-06-06T08:43:55.432 INFO:tasks.workunit.client.0.smithi027.stdout:3/877: unlink d1/d8/d22/d27/d79/dfd/f82 0 2024-06-06T08:43:55.434 INFO:tasks.workunit.client.0.smithi027.stdout:1/994: mknod dc/d67/d69/d32/d71/c135 0 2024-06-06T08:43:55.437 INFO:tasks.workunit.client.0.smithi027.stdout:3/878: unlink d1/d90/d6d/dac/ddb/l4a 0 2024-06-06T08:43:55.440 INFO:tasks.workunit.client.0.smithi027.stdout:1/995: creat dc/d67/d69/d32/d37/d56/f136 x:0 0 0 2024-06-06T08:43:55.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/996: mknod dc/d67/d69/d32/d37/d56/df1/d80/de8/c137 0 2024-06-06T08:43:55.446 INFO:tasks.workunit.client.0.smithi027.stdout:1/997: truncate dc/d67/d69/d32/d37/f41 3568239 0 2024-06-06T08:43:55.446 INFO:tasks.workunit.client.0.smithi027.stdout:1/998: chown dc/d67/d69/d32/d71/cad 0 1 2024-06-06T08:43:55.448 INFO:tasks.workunit.client.0.smithi027.stdout:3/879: write d1/d8/d22/d27/f78 [3854168,107508] 0 2024-06-06T08:43:55.453 INFO:tasks.workunit.client.0.smithi027.stdout:3/880: unlink d1/d8/d22/d27/d79/d106/f113 0 2024-06-06T08:43:55.453 INFO:tasks.workunit.client.0.smithi027.stdout:3/881: chown d1/d90/cc 7505 1 2024-06-06T08:43:55.463 INFO:tasks.workunit.client.0.smithi027.stdout:1/999: dwrite dc/de/d3d/fb4 [0,4194304] 0 2024-06-06T08:43:55.467 INFO:tasks.workunit.client.0.smithi027.stdout:9/900: rmdir d3/d95/d23/d36/d3a/d84/db2 39 2024-06-06T08:43:55.468 INFO:tasks.workunit.client.0.smithi027.stdout:9/901: truncate d3/d95/d34/ff5 3609076 0 2024-06-06T08:43:55.469 INFO:tasks.workunit.client.0.smithi027.stdout:9/902: write d3/d4e/fd2 [89524,2256] 0 2024-06-06T08:43:55.469 INFO:tasks.workunit.client.0.smithi027.stdout:3/882: dread d1/d8/d22/d27/d79/dfd/fe4 [0,4194304] 0 2024-06-06T08:43:55.469 INFO:tasks.workunit.client.0.smithi027.stdout:9/903: mknod d3/d95/d23/d78/ddf/de5/c13d 0 2024-06-06T08:43:55.470 INFO:tasks.workunit.client.0.smithi027.stdout:9/904: fsync d3/d95/d34/fde 0 2024-06-06T08:43:55.470 INFO:tasks.workunit.client.0.smithi027.stdout:0/895: unlink d15/d18/c20 0 2024-06-06T08:43:55.470 INFO:tasks.workunit.client.0.smithi027.stdout:0/896: fdatasync d15/d18/d23/d28/d31/dbb/fcc 0 2024-06-06T08:43:55.471 INFO:tasks.workunit.client.0.smithi027.stdout:3/883: mkdir d1/d1f/d45/df0/d114 0 2024-06-06T08:43:55.471 INFO:tasks.workunit.client.0.smithi027.stdout:3/884: chown d1/d90/d44/d54/fb1 3414792 1 2024-06-06T08:43:55.472 INFO:tasks.workunit.client.0.smithi027.stdout:3/885: write d1/d90/fa2 [334442,116699] 0 2024-06-06T08:43:55.472 INFO:tasks.workunit.client.0.smithi027.stdout:3/886: write d1/d1f/d4d/ddc/f109 [625978,125784] 0 2024-06-06T08:43:55.474 INFO:tasks.workunit.client.0.smithi027.stdout:3/887: rename d1/d90/fa2 to d1/d8/d59/d110/f115 0 2024-06-06T08:43:55.476 INFO:tasks.workunit.client.0.smithi027.stdout:3/888: symlink d1/d1f/d45/d102/l116 0 2024-06-06T08:43:55.476 INFO:tasks.workunit.client.0.smithi027.stdout:3/889: truncate d1/d90/d6d/dac/fee 4831476 0 2024-06-06T08:43:55.477 INFO:tasks.workunit.client.0.smithi027.stdout:3/890: readlink d1/d90/d6d/dac/ddb/l58 0 2024-06-06T08:43:55.485 INFO:tasks.workunit.client.0.smithi027.stdout:0/897: creat d15/d18/d64/f12b x:0 0 0 2024-06-06T08:43:55.486 INFO:tasks.workunit.client.0.smithi027.stdout:9/905: dwrite d3/d95/d23/d36/fd6 [0,4194304] 0 2024-06-06T08:43:55.489 INFO:tasks.workunit.client.0.smithi027.stdout:8/846: sync 2024-06-06T08:43:55.489 INFO:tasks.workunit.client.0.smithi027.stdout:0/898: link d15/d18/d23/d28/c90 d15/d18/c12c 0 2024-06-06T08:43:55.490 INFO:tasks.workunit.client.0.smithi027.stdout:9/906: dread d3/f89 [0,4194304] 0 2024-06-06T08:43:55.494 INFO:tasks.workunit.client.0.smithi027.stdout:8/847: symlink d7/dc/d36/l113 0 2024-06-06T08:43:55.508 INFO:tasks.workunit.client.0.smithi027.stdout:9/907: dwrite d3/d95/d23/d36/f94 [4194304,4194304] 0 2024-06-06T08:43:55.508 INFO:tasks.workunit.client.0.smithi027.stdout:3/891: creat d1/d90/d44/d54/db4/deb/f117 x:0 0 0 2024-06-06T08:43:55.508 INFO:tasks.workunit.client.0.smithi027.stdout:3/892: fdatasync d1/d90/d44/d54/fa6 0 2024-06-06T08:43:55.508 INFO:tasks.workunit.client.0.smithi027.stdout:3/893: write d1/d8/da9/f108 [409084,66062] 0 2024-06-06T08:43:55.508 INFO:tasks.workunit.client.0.smithi027.stdout:3/894: fdatasync d1/d8/d22/f37 0 2024-06-06T08:43:55.509 INFO:tasks.workunit.client.0.smithi027.stdout:9/908: mknod d3/d95/d23/d36/d3a/d106/c13e 0 2024-06-06T08:43:55.512 INFO:tasks.workunit.client.0.smithi027.stdout:3/895: mknod d1/d1f/c118 0 2024-06-06T08:43:55.512 INFO:tasks.workunit.client.0.smithi027.stdout:3/896: fdatasync d1/d90/d6d/f111 0 2024-06-06T08:43:55.512 INFO:tasks.workunit.client.0.smithi027.stdout:3/897: truncate d1/d8/f61 1949997 0 2024-06-06T08:43:55.514 INFO:tasks.workunit.client.0.smithi027.stdout:3/898: creat d1/d1f/d4d/f119 x:0 0 0 2024-06-06T08:43:55.525 INFO:tasks.workunit.client.0.smithi027.stdout:8/848: rename d7/d13/c45 to d7/dc/d36/d41/de2/c114 0 2024-06-06T08:43:55.528 INFO:tasks.workunit.client.0.smithi027.stdout:3/899: dwrite d1/d90/d6d/dac/ddb/f7d [0,4194304] 0 2024-06-06T08:43:55.528 INFO:tasks.workunit.client.0.smithi027.stdout:3/900: stat d1/d90/d6d/dac/ddb/f76 0 2024-06-06T08:43:55.531 INFO:tasks.workunit.client.0.smithi027.stdout:3/901: link d1/d1f/d4d/l53 d1/d1f/d4d/dd5/l11a 0 2024-06-06T08:43:55.539 INFO:tasks.workunit.client.0.smithi027.stdout:9/909: rename d3/f4 to d3/d95/d23/d36/d3a/f13f 0 2024-06-06T08:43:55.543 INFO:tasks.workunit.client.0.smithi027.stdout:9/910: unlink d3/d95/db6/lc3 0 2024-06-06T08:43:55.543 INFO:tasks.workunit.client.0.smithi027.stdout:8/849: rename d7/dc/d36/l71 to d7/dc/d36/d83/l115 0 2024-06-06T08:43:55.543 INFO:tasks.workunit.client.0.smithi027.stdout:9/911: chown d3/d4e/d75/f134 1 1 2024-06-06T08:43:55.543 INFO:tasks.workunit.client.0.smithi027.stdout:8/850: chown d7/f4e 703707 1 2024-06-06T08:43:55.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/912: chown d3/d95/d23/da9/fc5 1430699 1 2024-06-06T08:43:55.544 INFO:tasks.workunit.client.0.smithi027.stdout:8/851: rename d7/dc/d36/d83 to d7/dc/d36/d83/d60/d93/dac/de8/d116 22 2024-06-06T08:43:55.544 INFO:tasks.workunit.client.0.smithi027.stdout:8/852: fdatasync d7/f9 0 2024-06-06T08:43:55.544 INFO:tasks.workunit.client.0.smithi027.stdout:8/853: chown d7/dc/d36/d41/d85/l9e 1540493 1 2024-06-06T08:43:55.544 INFO:tasks.workunit.client.0.smithi027.stdout:8/854: stat d7/f4e 0 2024-06-06T08:43:55.544 INFO:tasks.workunit.client.0.smithi027.stdout:8/855: getdents d7/dc/d36/d83/d60/d93/dac 0 2024-06-06T08:43:55.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/913: symlink d3/d95/d23/d36/d3a/d84/df0/l140 0 2024-06-06T08:43:55.545 INFO:tasks.workunit.client.0.smithi027.stdout:8/856: creat d7/d13/d15/ddc/f117 x:0 0 0 2024-06-06T08:43:55.546 INFO:tasks.workunit.client.0.smithi027.stdout:8/857: symlink d7/dc/d36/d83/d60/d93/dac/l118 0 2024-06-06T08:43:55.547 INFO:tasks.workunit.client.0.smithi027.stdout:8/858: mknod d7/dc/d36/d41/dbf/dd7/c119 0 2024-06-06T08:43:55.548 INFO:tasks.workunit.client.0.smithi027.stdout:9/914: dread d3/d95/d23/d78/fb3 [0,4194304] 0 2024-06-06T08:43:55.548 INFO:tasks.workunit.client.0.smithi027.stdout:9/915: chown d3/fc6 157686089 1 2024-06-06T08:43:55.548 INFO:tasks.workunit.client.0.smithi027.stdout:9/916: dread - d3/d95/fec zero size 2024-06-06T08:43:55.548 INFO:tasks.workunit.client.0.smithi027.stdout:9/917: truncate d3/d95/f39 224 0 2024-06-06T08:43:55.548 INFO:tasks.workunit.client.0.smithi027.stdout:3/902: dwrite d1/d8/d59/fc5 [0,4194304] 0 2024-06-06T08:43:55.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/903: chown d1/d8/d22/fba 3045 1 2024-06-06T08:43:55.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/904: fsync d1/d8/d22/d2d/f99 0 2024-06-06T08:43:55.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/918: dread d3/d4e/d75/dc/f8d [0,4194304] 0 2024-06-06T08:43:55.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/905: mknod d1/d90/d6d/dac/c11b 0 2024-06-06T08:43:55.551 INFO:tasks.workunit.client.0.smithi027.stdout:9/919: dread - d3/d95/d23/da9/f12e zero size 2024-06-06T08:43:55.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/906: write d1/d1f/d45/d5f/dd6/f10b [393977,84279] 0 2024-06-06T08:43:55.552 INFO:tasks.workunit.client.0.smithi027.stdout:9/920: mknod d3/d4e/d75/dc/dd1/c141 0 2024-06-06T08:43:55.554 INFO:tasks.workunit.client.0.smithi027.stdout:9/921: rmdir d3/d95/db6/d117/d135 39 2024-06-06T08:43:55.556 INFO:tasks.workunit.client.0.smithi027.stdout:9/922: symlink d3/d95/d23/d36/d3a/d84/db2/l142 0 2024-06-06T08:43:55.557 INFO:tasks.workunit.client.0.smithi027.stdout:8/859: dread d7/dc/d36/d83/d58/f8a [0,4194304] 0 2024-06-06T08:43:55.558 INFO:tasks.workunit.client.0.smithi027.stdout:8/860: mkdir d7/dc/d36/d41/dbf/d11a 0 2024-06-06T08:43:55.558 INFO:tasks.workunit.client.0.smithi027.stdout:8/861: fdatasync d7/f9 0 2024-06-06T08:43:55.559 INFO:tasks.workunit.client.0.smithi027.stdout:8/862: symlink d7/dc/d36/d41/dbf/d10a/l11b 0 2024-06-06T08:43:55.560 INFO:tasks.workunit.client.0.smithi027.stdout:8/863: chown d7/dc/d36/d41/dbf/dc3 233255458 1 2024-06-06T08:43:55.560 INFO:tasks.workunit.client.0.smithi027.stdout:8/864: symlink d7/d9c/l11c 0 2024-06-06T08:43:55.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/923: read d3/fc6 [972566,109505] 0 2024-06-06T08:43:55.569 INFO:tasks.workunit.client.0.smithi027.stdout:9/924: mkdir d3/d95/d34/d73/d143 0 2024-06-06T08:43:55.569 INFO:tasks.workunit.client.0.smithi027.stdout:9/925: chown d3/fae 388498 1 2024-06-06T08:43:55.570 INFO:tasks.workunit.client.0.smithi027.stdout:9/926: stat d3/d4e/d75/l1a 0 2024-06-06T08:43:55.570 INFO:tasks.workunit.client.0.smithi027.stdout:9/927: write d3/d95/d23/da9/fc5 [658086,69092] 0 2024-06-06T08:43:55.570 INFO:tasks.workunit.client.0.smithi027.stdout:9/928: truncate d3/d95/d23/da9/f12e 521898 0 2024-06-06T08:43:55.570 INFO:tasks.workunit.client.0.smithi027.stdout:3/907: dwrite d1/d8/d22/d27/f14 [0,4194304] 0 2024-06-06T08:43:55.571 INFO:tasks.workunit.client.0.smithi027.stdout:9/929: creat d3/d4e/d75/dc/d13a/f144 x:0 0 0 2024-06-06T08:43:55.578 INFO:tasks.workunit.client.0.smithi027.stdout:8/865: dwrite d7/d2f/f98 [0,4194304] 0 2024-06-06T08:43:55.580 INFO:tasks.workunit.client.0.smithi027.stdout:8/866: rmdir d7/dc/d36/d83/d60/d93 39 2024-06-06T08:43:55.581 INFO:tasks.workunit.client.0.smithi027.stdout:8/867: chown d7/dc/fd1 150 1 2024-06-06T08:43:55.582 INFO:tasks.workunit.client.0.smithi027.stdout:8/868: creat d7/dc/d36/d83/d58/f11d x:0 0 0 2024-06-06T08:43:55.584 INFO:tasks.workunit.client.0.smithi027.stdout:8/869: mknod d7/dc/d36/d83/c11e 0 2024-06-06T08:43:55.584 INFO:tasks.workunit.client.0.smithi027.stdout:8/870: dread - d7/dc/d36/d83/d60/f112 zero size 2024-06-06T08:43:55.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/871: creat d7/d2f/f11f x:0 0 0 2024-06-06T08:43:55.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/872: dread - d7/dc/d36/d83/d60/d93/dac/ff0 zero size 2024-06-06T08:43:55.587 INFO:tasks.workunit.client.0.smithi027.stdout:8/873: symlink d7/dc/d36/d41/d85/db7/l120 0 2024-06-06T08:43:55.591 INFO:tasks.workunit.client.0.smithi027.stdout:9/930: dread d3/fae [0,4194304] 0 2024-06-06T08:43:55.592 INFO:tasks.workunit.client.0.smithi027.stdout:3/908: dwrite d1/d8/fab [0,4194304] 0 2024-06-06T08:43:55.592 INFO:tasks.workunit.client.0.smithi027.stdout:3/909: rename d1 to d1/d1f/d4d/ddc/d11c 22 2024-06-06T08:43:55.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/931: symlink d3/d95/d23/d36/d3a/l145 0 2024-06-06T08:43:55.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/932: dread - d3/d4e/d75/faf zero size 2024-06-06T08:43:55.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/933: dread - d3/d4e/d75/dc/d9b/f130 zero size 2024-06-06T08:43:55.595 INFO:tasks.workunit.client.0.smithi027.stdout:9/934: creat d3/d76/f146 x:0 0 0 2024-06-06T08:43:55.598 INFO:tasks.workunit.client.0.smithi027.stdout:9/935: dread d3/d95/d23/da9/f104 [0,4194304] 0 2024-06-06T08:43:55.602 INFO:tasks.workunit.client.0.smithi027.stdout:3/910: write d1/d8/d22/d2d/fbb [6551864,123151] 0 2024-06-06T08:43:55.603 INFO:tasks.workunit.client.0.smithi027.stdout:3/911: mkdir d1/d1f/d45/d102/d11d 0 2024-06-06T08:43:55.604 INFO:tasks.workunit.client.0.smithi027.stdout:3/912: link d1/d90/d44/d54/fc4 d1/d1f/d45/df0/f11e 0 2024-06-06T08:43:55.605 INFO:tasks.workunit.client.0.smithi027.stdout:3/913: chown d1/d90/d44/d54/fb1 2903 1 2024-06-06T08:43:55.605 INFO:tasks.workunit.client.0.smithi027.stdout:3/914: symlink d1/d90/d44/d54/db4/l11f 0 2024-06-06T08:43:55.605 INFO:tasks.workunit.client.0.smithi027.stdout:3/915: dread - d1/d8/d22/d27/d79/fb8 zero size 2024-06-06T08:43:55.612 INFO:tasks.workunit.client.0.smithi027.stdout:9/936: link d3/d95/d23/d36/fd6 d3/d4e/f147 0 2024-06-06T08:43:55.786 INFO:tasks.workunit.client.0.smithi027.stdout:0/899: sync 2024-06-06T08:43:55.854 INFO:tasks.workunit.client.0.smithi027.stdout:9/937: sync 2024-06-06T08:43:55.855 INFO:tasks.workunit.client.0.smithi027.stdout:8/874: sync 2024-06-06T08:43:55.855 INFO:tasks.workunit.client.0.smithi027.stdout:3/916: sync 2024-06-06T08:43:55.855 INFO:tasks.workunit.client.0.smithi027.stdout:9/938: creat d3/d4e/d75/dc/d9b/f148 x:0 0 0 2024-06-06T08:43:55.856 INFO:tasks.workunit.client.0.smithi027.stdout:8/875: unlink d7/dc/d36/d83/d60/d93/dac/fb2 0 2024-06-06T08:43:55.857 INFO:tasks.workunit.client.0.smithi027.stdout:3/917: unlink d1/d1f/d45/d5f/dd6/ce5 0 2024-06-06T08:43:55.858 INFO:tasks.workunit.client.0.smithi027.stdout:8/876: creat d7/d13/d15/f121 x:0 0 0 2024-06-06T08:43:55.859 INFO:tasks.workunit.client.0.smithi027.stdout:3/918: mkdir d1/d8/d59/d110/d120 0 2024-06-06T08:43:55.860 INFO:tasks.workunit.client.0.smithi027.stdout:9/939: mknod d3/c149 0 2024-06-06T08:43:55.860 INFO:tasks.workunit.client.0.smithi027.stdout:9/940: write d3/d95/d23/d78/ddf/fe9 [964392,7830] 0 2024-06-06T08:43:55.860 INFO:tasks.workunit.client.0.smithi027.stdout:9/941: chown d3/d95/d23/d36/d3a/d84/df0 1184 1 2024-06-06T08:43:55.860 INFO:tasks.workunit.client.0.smithi027.stdout:9/942: chown d3/d95/d23/d36/lb8 210894 1 2024-06-06T08:43:55.868 INFO:tasks.workunit.client.0.smithi027.stdout:8/877: write d7/d13/f1e [1252812,130562] 0 2024-06-06T08:43:55.868 INFO:tasks.workunit.client.0.smithi027.stdout:8/878: chown d7/dc/d36/d41/dbf/dc3/led 84069255 1 2024-06-06T08:43:55.876 INFO:tasks.workunit.client.0.smithi027.stdout:9/943: write d3/d95/d23/d78/d11a/f122 [4185254,23500] 0 2024-06-06T08:43:55.876 INFO:tasks.workunit.client.0.smithi027.stdout:3/919: dwrite d1/d90/d6d/dac/ddb/fb6 [0,4194304] 0 2024-06-06T08:43:55.877 INFO:tasks.workunit.client.0.smithi027.stdout:9/944: mknod d3/d95/db6/d117/c14a 0 2024-06-06T08:43:55.877 INFO:tasks.workunit.client.0.smithi027.stdout:9/945: write d3/d95/d23/d78/ddf/fe9 [1671147,121694] 0 2024-06-06T08:43:55.883 INFO:tasks.workunit.client.0.smithi027.stdout:3/920: creat d1/d1f/d45/d5f/f121 x:0 0 0 2024-06-06T08:43:55.883 INFO:tasks.workunit.client.0.smithi027.stdout:3/921: readlink d1/d1f/d4d/dd5/l88 0 2024-06-06T08:43:55.883 INFO:tasks.workunit.client.0.smithi027.stdout:3/922: dread - d1/d8/d22/d27/d79/fb8 zero size 2024-06-06T08:43:55.883 INFO:tasks.workunit.client.0.smithi027.stdout:3/923: chown d1/d8/f55 470 1 2024-06-06T08:43:55.884 INFO:tasks.workunit.client.0.smithi027.stdout:3/924: dread - d1/d90/d44/fb0 zero size 2024-06-06T08:43:55.886 INFO:tasks.workunit.client.0.smithi027.stdout:8/879: dwrite d7/d13/d15/ddc/f117 [0,4194304] 0 2024-06-06T08:43:55.891 INFO:tasks.workunit.client.0.smithi027.stdout:3/925: dread d1/d90/d6d/f9a [0,4194304] 0 2024-06-06T08:43:55.892 INFO:tasks.workunit.client.0.smithi027.stdout:3/926: rmdir d1/d1f 39 2024-06-06T08:43:55.893 INFO:tasks.workunit.client.0.smithi027.stdout:3/927: chown d1/ce 9 1 2024-06-06T08:43:55.893 INFO:tasks.workunit.client.0.smithi027.stdout:3/928: chown d1/d8/fab 5290471 1 2024-06-06T08:43:55.893 INFO:tasks.workunit.client.0.smithi027.stdout:3/929: chown d1/d8/d22 293 1 2024-06-06T08:43:55.893 INFO:tasks.workunit.client.0.smithi027.stdout:3/930: truncate d1/f29 3760201 0 2024-06-06T08:43:55.894 INFO:tasks.workunit.client.0.smithi027.stdout:8/880: write d7/dc/d36/d83/f11 [1298081,97939] 0 2024-06-06T08:43:55.894 INFO:tasks.workunit.client.0.smithi027.stdout:8/881: write d7/dc/d37/f80 [4455476,21132] 0 2024-06-06T08:43:55.897 INFO:tasks.workunit.client.0.smithi027.stdout:9/946: dwrite d3/d4e/d75/dc/f108 [0,4194304] 0 2024-06-06T08:43:55.899 INFO:tasks.workunit.client.0.smithi027.stdout:9/947: readlink d3/d95/d23/d36/d3a/l49 0 2024-06-06T08:43:55.899 INFO:tasks.workunit.client.0.smithi027.stdout:9/948: dread - d3/d95/d34/fe7 zero size 2024-06-06T08:43:55.899 INFO:tasks.workunit.client.0.smithi027.stdout:9/949: write d3/d95/d34/f51 [251864,5279] 0 2024-06-06T08:43:55.899 INFO:tasks.workunit.client.0.smithi027.stdout:9/950: getdents d3/d95/db6 0 2024-06-06T08:43:55.912 INFO:tasks.workunit.client.0.smithi027.stdout:8/882: dread d7/dc/d36/fdf [0,4194304] 0 2024-06-06T08:43:55.914 INFO:tasks.workunit.client.0.smithi027.stdout:8/883: unlink d7/dc/d36/d83/d60/fff 0 2024-06-06T08:43:55.915 INFO:tasks.workunit.client.0.smithi027.stdout:8/884: truncate d7/d13/f24 778401 0 2024-06-06T08:43:55.915 INFO:tasks.workunit.client.0.smithi027.stdout:3/931: dwrite d1/d8/d22/d27/f14 [4194304,4194304] 0 2024-06-06T08:43:55.915 INFO:tasks.workunit.client.0.smithi027.stdout:8/885: write d7/dc/d36/f7b [9229816,110457] 0 2024-06-06T08:43:55.915 INFO:tasks.workunit.client.0.smithi027.stdout:8/886: truncate d7/dc/d36/d41/d85/fce 1243092 0 2024-06-06T08:43:55.920 INFO:tasks.workunit.client.0.smithi027.stdout:9/951: dwrite d3/d95/d23/d36/d3a/fac [0,4194304] 0 2024-06-06T08:43:55.923 INFO:tasks.workunit.client.0.smithi027.stdout:9/952: unlink d3/d4e/d75/dc/d9b/f92 0 2024-06-06T08:43:55.923 INFO:tasks.workunit.client.0.smithi027.stdout:9/953: write d3/d4e/d75/dc/f131 [60522,122511] 0 2024-06-06T08:43:55.923 INFO:tasks.workunit.client.0.smithi027.stdout:9/954: chown d3/d95/d118/f11d 491 1 2024-06-06T08:43:55.923 INFO:tasks.workunit.client.0.smithi027.stdout:9/955: stat d3/d95/d23/da9/f12e 0 2024-06-06T08:43:55.932 INFO:tasks.workunit.client.0.smithi027.stdout:8/887: rename d7/d13/c6b to d7/dc/d36/d41/d85/db7/dd3/c122 0 2024-06-06T08:43:55.939 INFO:tasks.workunit.client.0.smithi027.stdout:3/932: dwrite d1/d8/d22/d27/f78 [0,4194304] 0 2024-06-06T08:43:55.940 INFO:tasks.workunit.client.0.smithi027.stdout:3/933: truncate d1/d8/d22/fba 4224191 0 2024-06-06T08:43:55.940 INFO:tasks.workunit.client.0.smithi027.stdout:3/934: chown d1/d1f/d4d/ddc/fde 21199775 1 2024-06-06T08:43:55.940 INFO:tasks.workunit.client.0.smithi027.stdout:3/935: write d1/d90/d6d/dac/fe1 [1820216,103586] 0 2024-06-06T08:43:55.941 INFO:tasks.workunit.client.0.smithi027.stdout:3/936: mkdir d1/d1f/d45/d122 0 2024-06-06T08:43:55.942 INFO:tasks.workunit.client.0.smithi027.stdout:3/937: symlink d1/d1f/d4d/ddc/l123 0 2024-06-06T08:43:55.943 INFO:tasks.workunit.client.0.smithi027.stdout:9/956: dwrite d3/d95/fba [0,4194304] 0 2024-06-06T08:43:55.946 INFO:tasks.workunit.client.0.smithi027.stdout:9/957: rename d3/d95/d23/da9/f133 to d3/d4e/d75/dc/df6/f14b 0 2024-06-06T08:43:55.946 INFO:tasks.workunit.client.0.smithi027.stdout:3/938: unlink d1/d90/d44/d54/fa6 0 2024-06-06T08:43:55.948 INFO:tasks.workunit.client.0.smithi027.stdout:9/958: rename d3/d95/d34/f60 to d3/d95/db6/d117/d135/f14c 0 2024-06-06T08:43:55.951 INFO:tasks.workunit.client.0.smithi027.stdout:3/939: mknod d1/d1f/d45/d5f/dd6/c124 0 2024-06-06T08:43:55.953 INFO:tasks.workunit.client.0.smithi027.stdout:8/888: dwrite d7/d9c/f107 [0,4194304] 0 2024-06-06T08:43:55.959 INFO:tasks.workunit.client.0.smithi027.stdout:0/900: sync 2024-06-06T08:43:55.961 INFO:tasks.workunit.client.0.smithi027.stdout:8/889: mkdir d7/dc/d37/d123 0 2024-06-06T08:43:55.964 INFO:tasks.workunit.client.0.smithi027.stdout:0/901: rename d15/d18/d23/f27 to d15/d18/d23/d28/d10b/f12d 0 2024-06-06T08:43:55.966 INFO:tasks.workunit.client.0.smithi027.stdout:8/890: link d7/dc/d36/d41/c33 d7/d9c/c124 0 2024-06-06T08:43:55.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/940: dwrite d1/d90/d44/d54/fc4 [0,4194304] 0 2024-06-06T08:43:55.971 INFO:tasks.workunit.client.0.smithi027.stdout:3/941: mknod d1/d90/d6d/c125 0 2024-06-06T08:43:55.977 INFO:tasks.workunit.client.0.smithi027.stdout:0/902: dread d15/d18/d23/d28/d10b/f12d [0,4194304] 0 2024-06-06T08:43:55.977 INFO:tasks.workunit.client.0.smithi027.stdout:0/903: chown d15/d18/d40/d8c/d9c/fa6 255366841 1 2024-06-06T08:43:55.977 INFO:tasks.workunit.client.0.smithi027.stdout:8/891: dread d7/d2f/f3b [0,4194304] 0 2024-06-06T08:43:55.978 INFO:tasks.workunit.client.0.smithi027.stdout:8/892: chown d7/dc/d36/d3c/c53 307 1 2024-06-06T08:43:55.978 INFO:tasks.workunit.client.0.smithi027.stdout:0/904: rmdir d15/d18/d23/d6f 39 2024-06-06T08:43:55.979 INFO:tasks.workunit.client.0.smithi027.stdout:8/893: mkdir d7/dc/d36/d83/d60/d93/dac/d125 0 2024-06-06T08:43:55.979 INFO:tasks.workunit.client.0.smithi027.stdout:8/894: write d7/dc/d37/fda [413688,32679] 0 2024-06-06T08:43:55.986 INFO:tasks.workunit.client.0.smithi027.stdout:3/942: dwrite d1/d8/d22/d2d/fbb [4194304,4194304] 0 2024-06-06T08:43:55.986 INFO:tasks.workunit.client.0.smithi027.stdout:3/943: chown d1/d8/d22/d2d/c101 24 1 2024-06-06T08:43:55.986 INFO:tasks.workunit.client.0.smithi027.stdout:3/944: stat d1/l43 0 2024-06-06T08:43:55.986 INFO:tasks.workunit.client.0.smithi027.stdout:3/945: fdatasync d1/d1f/d45/fc7 0 2024-06-06T08:43:55.986 INFO:tasks.workunit.client.0.smithi027.stdout:3/946: fsync d1/d8/fa7 0 2024-06-06T08:43:55.988 INFO:tasks.workunit.client.0.smithi027.stdout:3/947: rename d1/d90/l11 to d1/d8/d59/l126 0 2024-06-06T08:43:55.989 INFO:tasks.workunit.client.0.smithi027.stdout:3/948: stat d1/d90/d44/c4e 0 2024-06-06T08:43:55.989 INFO:tasks.workunit.client.0.smithi027.stdout:3/949: fsync d1/d90/d44/d54/f63 0 2024-06-06T08:43:55.990 INFO:tasks.workunit.client.0.smithi027.stdout:3/950: symlink d1/d8/da9/l127 0 2024-06-06T08:43:55.991 INFO:tasks.workunit.client.0.smithi027.stdout:8/895: write d7/f42 [568360,22606] 0 2024-06-06T08:43:55.992 INFO:tasks.workunit.client.0.smithi027.stdout:3/951: creat d1/d8/f128 x:0 0 0 2024-06-06T08:43:55.992 INFO:tasks.workunit.client.0.smithi027.stdout:3/952: getdents d1/d1f/d45/df0/d114 0 2024-06-06T08:43:55.992 INFO:tasks.workunit.client.0.smithi027.stdout:3/953: write d1/d90/d44/f74 [167174,75532] 0 2024-06-06T08:43:55.998 INFO:tasks.workunit.client.0.smithi027.stdout:0/905: dwrite d15/d18/dbe/d5d/f98 [0,4194304] 0 2024-06-06T08:43:56.001 INFO:tasks.workunit.client.0.smithi027.stdout:8/896: dread d7/fd0 [0,4194304] 0 2024-06-06T08:43:56.008 INFO:tasks.workunit.client.0.smithi027.stdout:0/906: rename d15/d18/d23/d28/d10b to d15/d18/d23/d5b/da0/d12e 0 2024-06-06T08:43:56.009 INFO:tasks.workunit.client.0.smithi027.stdout:3/954: dwrite d1/d8/d22/fba [0,4194304] 0 2024-06-06T08:43:56.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/955: creat d1/d1f/d45/d5f/dd6/f129 x:0 0 0 2024-06-06T08:43:56.010 INFO:tasks.workunit.client.0.smithi027.stdout:0/907: symlink d15/d18/d23/d28/d2a/d2d/l12f 0 2024-06-06T08:43:56.011 INFO:tasks.workunit.client.0.smithi027.stdout:8/897: rename d7/d9c/daf/ld2 to d7/dc/d36/d83/d60/l126 0 2024-06-06T08:43:56.011 INFO:tasks.workunit.client.0.smithi027.stdout:3/956: symlink d1/d1f/df4/l12a 0 2024-06-06T08:43:56.012 INFO:tasks.workunit.client.0.smithi027.stdout:3/957: dread - d1/d90/d44/d54/db4/deb/fec zero size 2024-06-06T08:43:56.013 INFO:tasks.workunit.client.0.smithi027.stdout:0/908: unlink f9 0 2024-06-06T08:43:56.013 INFO:tasks.workunit.client.0.smithi027.stdout:0/909: chown d15/ffc 9632190 1 2024-06-06T08:43:56.013 INFO:tasks.workunit.client.0.smithi027.stdout:8/898: getdents d7/dc/d36/d83/d60/d93/dac/d73 0 2024-06-06T08:43:56.013 INFO:tasks.workunit.client.0.smithi027.stdout:8/899: chown d7/d2f/f92 901 1 2024-06-06T08:43:56.014 INFO:tasks.workunit.client.0.smithi027.stdout:0/910: symlink d15/d18/dbe/d121/l130 0 2024-06-06T08:43:56.015 INFO:tasks.workunit.client.0.smithi027.stdout:8/900: creat d7/d13/d15/ddc/f127 x:0 0 0 2024-06-06T08:43:56.015 INFO:tasks.workunit.client.0.smithi027.stdout:8/901: fdatasync d7/dc/d36/d83/f1d 0 2024-06-06T08:43:56.015 INFO:tasks.workunit.client.0.smithi027.stdout:8/902: dread - d7/dc/d36/d83/d60/d93/f10f zero size 2024-06-06T08:43:56.015 INFO:tasks.workunit.client.0.smithi027.stdout:8/903: fsync d7/d2f/f77 0 2024-06-06T08:43:56.015 INFO:tasks.workunit.client.0.smithi027.stdout:8/904: dread - d7/d2f/f11f zero size 2024-06-06T08:43:56.017 INFO:tasks.workunit.client.0.smithi027.stdout:8/905: rename d7/dc/d36/d83/cc6 to d7/d13/d15/c128 0 2024-06-06T08:43:56.018 INFO:tasks.workunit.client.0.smithi027.stdout:8/906: creat d7/dc/d36/d83/d60/d93/dac/f129 x:0 0 0 2024-06-06T08:43:56.019 INFO:tasks.workunit.client.0.smithi027.stdout:8/907: mknod d7/dc/d36/d41/d85/db7/c12a 0 2024-06-06T08:43:56.019 INFO:tasks.workunit.client.0.smithi027.stdout:8/908: write d7/dc/d36/d41/dbf/dd7/fd9 [124569,28429] 0 2024-06-06T08:43:56.020 INFO:tasks.workunit.client.0.smithi027.stdout:8/909: truncate d7/dc/d36/d83/d60/f7a 151368 0 2024-06-06T08:43:56.022 INFO:tasks.workunit.client.0.smithi027.stdout:8/910: unlink d7/dc/d36/d83/d60/d93/dac/c66 0 2024-06-06T08:43:56.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/911: write d15/d18/d23/d38/d68/f8f [19709,96451] 0 2024-06-06T08:43:56.026 INFO:tasks.workunit.client.0.smithi027.stdout:3/958: getdents d1/d1f/df4 0 2024-06-06T08:43:56.033 INFO:tasks.workunit.client.0.smithi027.stdout:0/912: write d15/d18/d23/d6f/dcb/f123 [4074747,48872] 0 2024-06-06T08:43:56.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/959: mknod d1/d90/d6d/c12b 0 2024-06-06T08:43:56.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/960: fdatasync d1/d8/d22/d27/d79/dfd/fe4 0 2024-06-06T08:43:56.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/961: truncate d1/d8/d59/fd9 71318 0 2024-06-06T08:43:56.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/962: chown d1/d8/f55 0 1 2024-06-06T08:43:56.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/963: fdatasync d1/d1f/d45/d5f/dd6/f10b 0 2024-06-06T08:43:56.034 INFO:tasks.workunit.client.0.smithi027.stdout:0/913: mkdir d15/d18/d108/d131 0 2024-06-06T08:43:56.035 INFO:tasks.workunit.client.0.smithi027.stdout:3/964: truncate d1/d90/d6d/dac/ddb/f5d 428081 0 2024-06-06T08:43:56.048 INFO:tasks.workunit.client.0.smithi027.stdout:3/965: write d1/d90/d44/d54/fc4 [2471367,80838] 0 2024-06-06T08:43:56.048 INFO:tasks.workunit.client.0.smithi027.stdout:3/966: write d1/d90/d44/d54/fb1 [982251,18981] 0 2024-06-06T08:43:56.048 INFO:tasks.workunit.client.0.smithi027.stdout:0/914: dwrite d15/d18/dbe/d5c/fb8 [0,4194304] 0 2024-06-06T08:43:56.049 INFO:tasks.workunit.client.0.smithi027.stdout:3/967: truncate d1/d1f/d45/d5f/dd6/f10b 223748 0 2024-06-06T08:43:56.050 INFO:tasks.workunit.client.0.smithi027.stdout:0/915: mknod d15/c132 0 2024-06-06T08:43:56.051 INFO:tasks.workunit.client.0.smithi027.stdout:3/968: rmdir d1/d1f/d4d 39 2024-06-06T08:43:56.051 INFO:tasks.workunit.client.0.smithi027.stdout:3/969: truncate d1/d8/d22/d27/fae 949919 0 2024-06-06T08:43:56.051 INFO:tasks.workunit.client.0.smithi027.stdout:3/970: chown d1/d8/d22/d27/l9d 10798 1 2024-06-06T08:43:56.052 INFO:tasks.workunit.client.0.smithi027.stdout:3/971: truncate d1/d1f/d4d/fb9 915468 0 2024-06-06T08:43:56.052 INFO:tasks.workunit.client.0.smithi027.stdout:3/972: chown d1/d1f/d45/d5f/dd6 4489278 1 2024-06-06T08:43:56.057 INFO:tasks.workunit.client.0.smithi027.stdout:0/916: symlink d15/d18/d23/d6f/l133 0 2024-06-06T08:43:56.067 INFO:tasks.workunit.client.0.smithi027.stdout:3/973: dwrite d1/d90/d44/f74 [0,4194304] 0 2024-06-06T08:43:56.067 INFO:tasks.workunit.client.0.smithi027.stdout:3/974: chown d1/d8/d22/fc8 6 1 2024-06-06T08:43:56.069 INFO:tasks.workunit.client.0.smithi027.stdout:3/975: link d1/d90/d6d/dac/le9 d1/d8/d59/d110/l12c 0 2024-06-06T08:43:56.069 INFO:tasks.workunit.client.0.smithi027.stdout:3/976: read - d1/d1f/d4d/f119 zero size 2024-06-06T08:43:56.069 INFO:tasks.workunit.client.0.smithi027.stdout:3/977: dread - d1/d90/d44/fb0 zero size 2024-06-06T08:43:56.074 INFO:tasks.workunit.client.0.smithi027.stdout:0/917: dwrite d15/d18/d23/d28/d2a/d2d/f3a [0,4194304] 0 2024-06-06T08:43:56.087 INFO:tasks.workunit.client.0.smithi027.stdout:3/978: dwrite d1/d90/d6d/dac/ddb/fb6 [0,4194304] 0 2024-06-06T08:43:56.091 INFO:tasks.workunit.client.0.smithi027.stdout:3/979: rename d1/d1f/d4d/dd5/f104 to d1/d8/d22/d27/f12d 0 2024-06-06T08:43:56.094 INFO:tasks.workunit.client.0.smithi027.stdout:0/918: dwrite d15/d18/dbe/d5c/d83/fb6 [4194304,4194304] 0 2024-06-06T08:43:56.096 INFO:tasks.workunit.client.0.smithi027.stdout:0/919: creat d15/d18/f134 x:0 0 0 2024-06-06T08:43:56.096 INFO:tasks.workunit.client.0.smithi027.stdout:0/920: chown d15/d18/dbe/d121 43370 1 2024-06-06T08:43:56.097 INFO:tasks.workunit.client.0.smithi027.stdout:0/921: creat d15/d18/d23/d5b/da0/d12e/f135 x:0 0 0 2024-06-06T08:43:56.108 INFO:tasks.workunit.client.0.smithi027.stdout:3/980: dwrite d1/d90/d6d/dac/fe1 [0,4194304] 0 2024-06-06T08:43:56.111 INFO:tasks.workunit.client.0.smithi027.stdout:3/981: rmdir d1/d90 39 2024-06-06T08:43:56.113 INFO:tasks.workunit.client.0.smithi027.stdout:0/922: rename d15/d18/d23/d28/d31/f5e to d15/d18/dbe/f136 0 2024-06-06T08:43:56.113 INFO:tasks.workunit.client.0.smithi027.stdout:0/923: write d15/d18/dbe/d5c/d83/fd4 [137958,33758] 0 2024-06-06T08:43:56.113 INFO:tasks.workunit.client.0.smithi027.stdout:3/982: creat d1/d8/d22/d27/d79/dfd/f12e x:0 0 0 2024-06-06T08:43:56.113 INFO:tasks.workunit.client.0.smithi027.stdout:3/983: chown d1/d8/d22/d27/l6e 0 1 2024-06-06T08:43:56.115 INFO:tasks.workunit.client.0.smithi027.stdout:0/924: dread d15/d18/d40/fc2 [0,4194304] 0 2024-06-06T08:43:56.115 INFO:tasks.workunit.client.0.smithi027.stdout:3/984: rename d1/d90/d6d/dac/ddb/f3a to d1/d1f/d4d/f12f 0 2024-06-06T08:43:56.115 INFO:tasks.workunit.client.0.smithi027.stdout:0/925: symlink d15/d18/d23/d6f/deb/l137 0 2024-06-06T08:43:56.116 INFO:tasks.workunit.client.0.smithi027.stdout:3/985: creat d1/d1f/f130 x:0 0 0 2024-06-06T08:43:56.118 INFO:tasks.workunit.client.0.smithi027.stdout:3/986: symlink d1/d90/l131 0 2024-06-06T08:43:56.118 INFO:tasks.workunit.client.0.smithi027.stdout:3/987: stat d1/d8/d22/d27/d79/dfd/l7a 0 2024-06-06T08:43:56.120 INFO:tasks.workunit.client.0.smithi027.stdout:3/988: rmdir d1/d8/d59/d110/d120 0 2024-06-06T08:43:56.122 INFO:tasks.workunit.client.0.smithi027.stdout:3/989: rename d1/d90/d6d/f9a to d1/d1f/d4d/f132 0 2024-06-06T08:43:56.122 INFO:tasks.workunit.client.0.smithi027.stdout:0/926: dread d15/d18/f73 [0,4194304] 0 2024-06-06T08:43:56.123 INFO:tasks.workunit.client.0.smithi027.stdout:0/927: symlink d15/d18/d23/d28/d31/dbb/d120/l138 0 2024-06-06T08:43:56.141 INFO:tasks.workunit.client.0.smithi027.stdout:3/990: dwrite d1/f1e [0,4194304] 0 2024-06-06T08:43:56.142 INFO:tasks.workunit.client.0.smithi027.stdout:0/928: dwrite d15/d18/d23/f103 [0,4194304] 0 2024-06-06T08:43:56.142 INFO:tasks.workunit.client.0.smithi027.stdout:0/929: write d15/d18/dbe/d5c/f109 [334344,104464] 0 2024-06-06T08:43:56.143 INFO:tasks.workunit.client.0.smithi027.stdout:0/930: mkdir d15/d18/d23/d139 0 2024-06-06T08:43:56.153 INFO:tasks.workunit.client.0.smithi027.stdout:0/931: dread d15/d18/d40/d8c/f11b [4194304,4194304] 0 2024-06-06T08:43:56.168 INFO:tasks.workunit.client.0.smithi027.stdout:0/932: dwrite d15/d18/d64/ff4 [0,4194304] 0 2024-06-06T08:43:56.170 INFO:tasks.workunit.client.0.smithi027.stdout:0/933: mknod d15/d18/d23/d5b/c13a 0 2024-06-06T08:43:56.185 INFO:tasks.workunit.client.0.smithi027.stdout:0/934: dwrite d15/d18/d23/d28/d31/dbb/fcc [0,4194304] 0 2024-06-06T08:43:56.186 INFO:tasks.workunit.client.0.smithi027.stdout:0/935: stat d15/d18/d23/d38/d3e/c7e 0 2024-06-06T08:43:56.186 INFO:tasks.workunit.client.0.smithi027.stdout:0/936: dread - d15/d18/d23/d38/d3e/fd2 zero size 2024-06-06T08:43:56.187 INFO:tasks.workunit.client.0.smithi027.stdout:0/937: rmdir d15/d18/d23/d28/d2a 39 2024-06-06T08:43:56.226 INFO:tasks.workunit.client.0.smithi027.stdout:9/959: sync 2024-06-06T08:43:56.227 INFO:tasks.workunit.client.0.smithi027.stdout:8/911: sync 2024-06-06T08:43:56.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/960: read - d3/d95/d23/d78/fe1 zero size 2024-06-06T08:43:56.227 INFO:tasks.workunit.client.0.smithi027.stdout:8/912: truncate d7/dc/d36/d83/d60/d93/dac/ff0 204257 0 2024-06-06T08:43:56.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/961: stat d3/d4e/d75/dc/l72 0 2024-06-06T08:43:56.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/962: stat d3/d95/d23/d78/d100/f10f 0 2024-06-06T08:43:56.228 INFO:tasks.workunit.client.0.smithi027.stdout:8/913: unlink d7/d2f/f11f 0 2024-06-06T08:43:56.231 INFO:tasks.workunit.client.0.smithi027.stdout:8/914: unlink d7/dc/d36/d83/d60/fad 0 2024-06-06T08:43:56.231 INFO:tasks.workunit.client.0.smithi027.stdout:9/963: rmdir d3 39 2024-06-06T08:43:56.232 INFO:tasks.workunit.client.0.smithi027.stdout:9/964: symlink d3/d95/d23/d78/ddf/de5/l14d 0 2024-06-06T08:43:56.233 INFO:tasks.workunit.client.0.smithi027.stdout:8/915: dread d7/dc/d36/d83/f1d [0,4194304] 0 2024-06-06T08:43:56.237 INFO:tasks.workunit.client.0.smithi027.stdout:8/916: rename d7/dc/d36/d83/d58/l7c to d7/dc/d37/d46/dfb/l12b 0 2024-06-06T08:43:56.237 INFO:tasks.workunit.client.0.smithi027.stdout:8/917: dread - d7/d13/d15/fef zero size 2024-06-06T08:43:56.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/918: creat d7/dc/d37/dc8/f12c x:0 0 0 2024-06-06T08:43:56.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/919: write d7/dc/d36/d3c/f79 [5034390,127245] 0 2024-06-06T08:43:56.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/965: dwrite d3/d95/d34/f107 [0,4194304] 0 2024-06-06T08:43:56.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/966: fsync d3/d95/d34/ff5 0 2024-06-06T08:43:56.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/967: chown d3/d4e/d75/dc/l101 28361004 1 2024-06-06T08:43:56.251 INFO:tasks.workunit.client.0.smithi027.stdout:8/920: write d7/d9c/f107 [3537566,74648] 0 2024-06-06T08:43:56.251 INFO:tasks.workunit.client.0.smithi027.stdout:8/921: write d7/dc/fd1 [174340,121392] 0 2024-06-06T08:43:56.253 INFO:tasks.workunit.client.0.smithi027.stdout:8/922: creat d7/dc/d37/d46/f12d x:0 0 0 2024-06-06T08:43:56.253 INFO:tasks.workunit.client.0.smithi027.stdout:8/923: fdatasync d7/dc/d36/d41/d85/fcc 0 2024-06-06T08:43:56.253 INFO:tasks.workunit.client.0.smithi027.stdout:8/924: fdatasync d7/dc/d36/d41/d85/df6/f10c 0 2024-06-06T08:43:56.255 INFO:tasks.workunit.client.0.smithi027.stdout:8/925: rename d7/dc/d36/d83/f91 to d7/dc/d36/d83/d60/d93/dac/d125/f12e 0 2024-06-06T08:43:56.261 INFO:tasks.workunit.client.0.smithi027.stdout:9/968: write d3/d95/f2f [1211198,44673] 0 2024-06-06T08:43:56.269 INFO:tasks.workunit.client.0.smithi027.stdout:8/926: dwrite d7/dc/d36/d41/f10e [0,4194304] 0 2024-06-06T08:43:56.278 INFO:tasks.workunit.client.0.smithi027.stdout:9/969: dwrite d3/d4e/fa8 [0,4194304] 0 2024-06-06T08:43:56.281 INFO:tasks.workunit.client.0.smithi027.stdout:9/970: rmdir d3/d4e 39 2024-06-06T08:43:56.287 INFO:tasks.workunit.client.0.smithi027.stdout:8/927: dwrite d7/dc/d37/d46/f8d [0,4194304] 0 2024-06-06T08:43:56.287 INFO:tasks.workunit.client.0.smithi027.stdout:8/928: mknod d7/dc/d36/d41/d85/db7/dd3/dee/c12f 0 2024-06-06T08:43:56.287 INFO:tasks.workunit.client.0.smithi027.stdout:8/929: stat d7/fd0 0 2024-06-06T08:43:56.288 INFO:tasks.workunit.client.0.smithi027.stdout:9/971: write d3/d95/d23/d36/d3a/f13f [3398360,77289] 0 2024-06-06T08:43:56.290 INFO:tasks.workunit.client.0.smithi027.stdout:9/972: mkdir d3/d4e/d75/dc/d13a/d14e 0 2024-06-06T08:43:56.291 INFO:tasks.workunit.client.0.smithi027.stdout:9/973: mkdir d3/d76/d14f 0 2024-06-06T08:43:56.298 INFO:tasks.workunit.client.0.smithi027.stdout:8/930: dread d7/d2f/f98 [0,4194304] 0 2024-06-06T08:43:56.298 INFO:tasks.workunit.client.0.smithi027.stdout:8/931: creat d7/dc/d36/d83/d58/f130 x:0 0 0 2024-06-06T08:43:56.298 INFO:tasks.workunit.client.0.smithi027.stdout:8/932: chown d7/dc/d36/d41/dbf/d10a 974 1 2024-06-06T08:43:56.299 INFO:tasks.workunit.client.0.smithi027.stdout:8/933: read - d7/dc/d36/d83/d58/f130 zero size 2024-06-06T08:43:56.299 INFO:tasks.workunit.client.0.smithi027.stdout:8/934: chown d7/dc/d36/d83/d60/f78 183047 1 2024-06-06T08:43:56.307 INFO:tasks.workunit.client.0.smithi027.stdout:9/974: dwrite d3/d4e/d75/f50 [0,4194304] 0 2024-06-06T08:43:56.307 INFO:tasks.workunit.client.0.smithi027.stdout:9/975: fdatasync d3/d95/fec 0 2024-06-06T08:43:56.307 INFO:tasks.workunit.client.0.smithi027.stdout:8/935: dread d7/dc/d36/fdf [0,4194304] 0 2024-06-06T08:43:56.309 INFO:tasks.workunit.client.0.smithi027.stdout:9/976: symlink d3/d95/d23/d78/d100/l150 0 2024-06-06T08:43:56.317 INFO:tasks.workunit.client.0.smithi027.stdout:9/977: link d3/d95/d23/d36/d3a/l93 d3/d76/l151 0 2024-06-06T08:43:56.317 INFO:tasks.workunit.client.0.smithi027.stdout:9/978: chown d3/d95/f90 787124 1 2024-06-06T08:43:56.317 INFO:tasks.workunit.client.0.smithi027.stdout:9/979: truncate d3/d4e/d75/dc/d9b/f12a 1030576 0 2024-06-06T08:43:56.317 INFO:tasks.workunit.client.0.smithi027.stdout:8/936: read d7/f9 [4173841,116822] 0 2024-06-06T08:43:56.318 INFO:tasks.workunit.client.0.smithi027.stdout:8/937: read d7/dc/d36/d83/d60/d93/dac/ff0 [87665,90935] 0 2024-06-06T08:43:56.318 INFO:tasks.workunit.client.0.smithi027.stdout:8/938: chown d7/dc/d36/d41/dbf/dc3/ce1 1336752814 1 2024-06-06T08:43:56.339 INFO:tasks.workunit.client.0.smithi027.stdout:8/939: dwrite d7/dc/d36/d83/d60/f112 [0,4194304] 0 2024-06-06T08:43:56.339 INFO:tasks.workunit.client.0.smithi027.stdout:8/940: truncate d7/dc/d36/d83/d60/d93/fd5 465157 0 2024-06-06T08:43:56.339 INFO:tasks.workunit.client.0.smithi027.stdout:9/980: dwrite d3/d95/d23/d36/d3a/f13f [0,4194304] 0 2024-06-06T08:43:56.339 INFO:tasks.workunit.client.0.smithi027.stdout:9/981: fdatasync d3/d95/d34/f51 0 2024-06-06T08:43:56.339 INFO:tasks.workunit.client.0.smithi027.stdout:9/982: readlink d3/d95/d23/d36/d3a/d84/df0/l140 0 2024-06-06T08:43:56.339 INFO:tasks.workunit.client.0.smithi027.stdout:8/941: readlink d7/dc/d36/l8b 0 2024-06-06T08:43:56.340 INFO:tasks.workunit.client.0.smithi027.stdout:8/942: symlink d7/dc/d36/d41/d85/db7/dd3/dee/l131 0 2024-06-06T08:43:56.340 INFO:tasks.workunit.client.0.smithi027.stdout:8/943: chown d7/dc/d36/d83/d60/d93/dac/d125 7858420 1 2024-06-06T08:43:56.341 INFO:tasks.workunit.client.0.smithi027.stdout:8/944: readlink d7/dc/d36/l8b 0 2024-06-06T08:43:56.342 INFO:tasks.workunit.client.0.smithi027.stdout:9/983: truncate d3/d95/d23/d36/f5d 2087702 0 2024-06-06T08:43:56.342 INFO:tasks.workunit.client.0.smithi027.stdout:9/984: read - d3/d95/db6/d117/d135/fd8 zero size 2024-06-06T08:43:56.350 INFO:tasks.workunit.client.0.smithi027.stdout:8/945: dread d7/dc/d36/d41/dbf/fe5 [0,4194304] 0 2024-06-06T08:43:56.351 INFO:tasks.workunit.client.0.smithi027.stdout:8/946: symlink d7/dc/d36/d41/d85/db7/dd3/dee/l132 0 2024-06-06T08:43:56.351 INFO:tasks.workunit.client.0.smithi027.stdout:8/947: chown d7/dc/d36/d41/dbf/dc3/led 1 1 2024-06-06T08:43:56.352 INFO:tasks.workunit.client.0.smithi027.stdout:8/948: creat d7/dc/d36/d41/dbf/f133 x:0 0 0 2024-06-06T08:43:56.353 INFO:tasks.workunit.client.0.smithi027.stdout:8/949: creat d7/dc/d36/f134 x:0 0 0 2024-06-06T08:43:56.353 INFO:tasks.workunit.client.0.smithi027.stdout:8/950: dread - d7/dc/d37/ff7 zero size 2024-06-06T08:43:56.354 INFO:tasks.workunit.client.0.smithi027.stdout:9/985: dread d3/d95/d23/d36/fd6 [0,4194304] 0 2024-06-06T08:43:56.354 INFO:tasks.workunit.client.0.smithi027.stdout:9/986: write d3/d95/d23/f128 [403213,51080] 0 2024-06-06T08:43:56.370 INFO:tasks.workunit.client.0.smithi027.stdout:8/951: dread d7/dc/d36/d83/d60/f112 [0,4194304] 0 2024-06-06T08:43:56.371 INFO:tasks.workunit.client.0.smithi027.stdout:8/952: stat d7/dc/d37/d46/c5d 0 2024-06-06T08:43:56.371 INFO:tasks.workunit.client.0.smithi027.stdout:8/953: read d7/d2f/f92 [4459860,75203] 0 2024-06-06T08:43:56.371 INFO:tasks.workunit.client.0.smithi027.stdout:8/954: write d7/d13/d15/fef [948336,11597] 0 2024-06-06T08:43:56.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/955: symlink d7/dc/d36/d83/d60/l135 0 2024-06-06T08:43:56.374 INFO:tasks.workunit.client.0.smithi027.stdout:9/987: dwrite d3/f20 [0,4194304] 0 2024-06-06T08:43:56.375 INFO:tasks.workunit.client.0.smithi027.stdout:8/956: dread d7/dc/d36/d83/d60/f7a [0,4194304] 0 2024-06-06T08:43:56.376 INFO:tasks.workunit.client.0.smithi027.stdout:8/957: creat d7/d2f/f136 x:0 0 0 2024-06-06T08:43:56.377 INFO:tasks.workunit.client.0.smithi027.stdout:8/958: creat d7/dc/d36/d83/d60/d93/dac/de8/f137 x:0 0 0 2024-06-06T08:43:56.380 INFO:tasks.workunit.client.0.smithi027.stdout:8/959: dread d7/d13/fa0 [0,4194304] 0 2024-06-06T08:43:56.384 INFO:tasks.workunit.client.0.smithi027.stdout:8/960: getdents d7/dc/d36/d41/dbf 0 2024-06-06T08:43:56.388 INFO:tasks.workunit.client.0.smithi027.stdout:8/961: dread d7/d13/f24 [0,4194304] 0 2024-06-06T08:43:56.388 INFO:tasks.workunit.client.0.smithi027.stdout:8/962: readlink d7/dc/d37/d46/l101 0 2024-06-06T08:43:56.391 INFO:tasks.workunit.client.0.smithi027.stdout:8/963: fdatasync d7/d13/d15/fef 0 2024-06-06T08:43:56.392 INFO:tasks.workunit.client.0.smithi027.stdout:8/964: readlink d7/d13/d15/l84 0 2024-06-06T08:43:56.394 INFO:tasks.workunit.client.0.smithi027.stdout:9/988: dwrite d3/f46 [0,4194304] 0 2024-06-06T08:43:56.394 INFO:tasks.workunit.client.0.smithi027.stdout:9/989: chown d3/d4e/d75/db4/cf8 3871 1 2024-06-06T08:43:56.394 INFO:tasks.workunit.client.0.smithi027.stdout:8/965: link d7/dc/d36/d41/c2b d7/dc/d36/d41/d85/db7/dd3/dee/c138 0 2024-06-06T08:43:56.395 INFO:tasks.workunit.client.0.smithi027.stdout:8/966: chown d7/dc/d36/d3c/fca 133 1 2024-06-06T08:43:56.395 INFO:tasks.workunit.client.0.smithi027.stdout:9/990: mknod d3/d95/db6/d117/c152 0 2024-06-06T08:43:56.396 INFO:tasks.workunit.client.0.smithi027.stdout:8/967: unlink d7/d2f/f136 0 2024-06-06T08:43:56.398 INFO:tasks.workunit.client.0.smithi027.stdout:8/968: symlink d7/dc/d36/d41/l139 0 2024-06-06T08:43:56.399 INFO:tasks.workunit.client.0.smithi027.stdout:8/969: mknod d7/dc/d37/d123/c13a 0 2024-06-06T08:43:56.405 INFO:tasks.workunit.client.0.smithi027.stdout:9/991: write d3/d95/d34/d41/fd3 [1878538,50488] 0 2024-06-06T08:43:56.406 INFO:tasks.workunit.client.0.smithi027.stdout:8/970: write d7/dc/d36/d83/d60/f78 [49284,23873] 0 2024-06-06T08:43:56.409 INFO:tasks.workunit.client.0.smithi027.stdout:8/971: rename d7/dc/d36/d83/f6e to d7/dc/d37/d46/dfb/f13b 0 2024-06-06T08:43:56.411 INFO:tasks.workunit.client.0.smithi027.stdout:0/938: sync 2024-06-06T08:43:56.411 INFO:tasks.workunit.client.0.smithi027.stdout:3/991: sync 2024-06-06T08:43:56.412 INFO:tasks.workunit.client.0.smithi027.stdout:3/992: fdatasync d1/d1f/d4d/f6b 0 2024-06-06T08:43:56.412 INFO:tasks.workunit.client.0.smithi027.stdout:3/993: fdatasync d1/d8/d22/d27/d79/dfd/f12e 0 2024-06-06T08:43:56.414 INFO:tasks.workunit.client.0.smithi027.stdout:3/994: link d1/f85 d1/d8/da9/f133 0 2024-06-06T08:43:56.414 INFO:tasks.workunit.client.0.smithi027.stdout:3/995: readlink d1/d90/d6d/dac/ddb/l58 0 2024-06-06T08:43:56.414 INFO:tasks.workunit.client.0.smithi027.stdout:3/996: write d1/d8/d22/fba [4259207,116189] 0 2024-06-06T08:43:56.414 INFO:tasks.workunit.client.0.smithi027.stdout:3/997: readlink d1/d8/d59/d110/l12c 0 2024-06-06T08:43:56.415 INFO:tasks.workunit.client.0.smithi027.stdout:3/998: symlink d1/l134 0 2024-06-06T08:43:56.415 INFO:tasks.workunit.client.0.smithi027.stdout:3/999: dread - d1/d8/d22/d27/d79/dfd/fbe zero size 2024-06-06T08:43:56.432 INFO:tasks.workunit.client.0.smithi027.stdout:8/972: dwrite d7/dc/d37/d46/fe6 [0,4194304] 0 2024-06-06T08:43:56.432 INFO:tasks.workunit.client.0.smithi027.stdout:8/973: write d7/dc/d36/d83/d60/d93/dac/ff0 [467555,112472] 0 2024-06-06T08:43:56.432 INFO:tasks.workunit.client.0.smithi027.stdout:8/974: fdatasync d7/dc/d36/d83/fbc 0 2024-06-06T08:43:56.433 INFO:tasks.workunit.client.0.smithi027.stdout:0/939: dwrite d15/d18/d23/d28/d31/f84 [0,4194304] 0 2024-06-06T08:43:56.435 INFO:tasks.workunit.client.0.smithi027.stdout:8/975: rename d7/dc/f69 to d7/d9c/daf/f13c 0 2024-06-06T08:43:56.444 INFO:tasks.workunit.client.0.smithi027.stdout:0/940: write d15/dae/fbf [1431842,126185] 0 2024-06-06T08:43:56.447 INFO:tasks.workunit.client.0.smithi027.stdout:0/941: mkdir d15/d18/dbe/d13b 0 2024-06-06T08:43:56.450 INFO:tasks.workunit.client.0.smithi027.stdout:8/976: dwrite d7/f9 [0,4194304] 0 2024-06-06T08:43:56.450 INFO:tasks.workunit.client.0.smithi027.stdout:8/977: fsync d7/d2f/f77 0 2024-06-06T08:43:56.463 INFO:tasks.workunit.client.0.smithi027.stdout:0/942: dwrite d15/f22 [0,4194304] 0 2024-06-06T08:43:56.464 INFO:tasks.workunit.client.0.smithi027.stdout:0/943: readlink d15/d18/d23/l6c 0 2024-06-06T08:43:56.550 INFO:tasks.workunit.client.0.smithi027.stdout:8/978: rename d7/d13/faa to d7/dc/d36/d41/f13d 0 2024-06-06T08:43:56.550 INFO:tasks.workunit.client.0.smithi027.stdout:8/979: chown d7/dc/d36/f7b 0 1 2024-06-06T08:43:56.552 INFO:tasks.workunit.client.0.smithi027.stdout:8/980: dread d7/dc/d36/d83/d60/f7a [0,4194304] 0 2024-06-06T08:43:56.558 INFO:tasks.workunit.client.0.smithi027.stdout:0/944: rename d15/d18/d23/d28/d31/cd3 to d15/d18/d23/d139/c13c 0 2024-06-06T08:43:56.559 INFO:tasks.workunit.client.0.smithi027.stdout:8/981: dread d7/dc/d36/d83/d60/f68 [0,4194304] 0 2024-06-06T08:43:56.560 INFO:tasks.workunit.client.0.smithi027.stdout:0/945: mkdir d15/d18/d40/d13d 0 2024-06-06T08:43:56.560 INFO:tasks.workunit.client.0.smithi027.stdout:0/946: fdatasync d15/d18/f81 0 2024-06-06T08:43:56.560 INFO:tasks.workunit.client.0.smithi027.stdout:0/947: write d15/fa1 [1029913,34755] 0 2024-06-06T08:43:56.560 INFO:tasks.workunit.client.0.smithi027.stdout:8/982: rmdir d7/dc/d36/d41/d85/db7/dd3/dee 39 2024-06-06T08:43:56.561 INFO:tasks.workunit.client.0.smithi027.stdout:8/983: getdents d7/dc/d36/d3c 0 2024-06-06T08:43:56.565 INFO:tasks.workunit.client.0.smithi027.stdout:0/948: dread d15/d18/d23/d38/d68/d85/fea [0,4194304] 0 2024-06-06T08:43:56.565 INFO:tasks.workunit.client.0.smithi027.stdout:0/949: truncate d15/d18/d23/d3c/fe2 451070 0 2024-06-06T08:43:56.565 INFO:tasks.workunit.client.0.smithi027.stdout:8/984: write d7/dc/d36/d83/f11 [3049632,42971] 0 2024-06-06T08:43:56.566 INFO:tasks.workunit.client.0.smithi027.stdout:0/950: rename d15/d18/dbe/d5c/d83/fd4 to d15/d18/d23/d5b/da0/d12e/f13e 0 2024-06-06T08:43:56.571 INFO:tasks.workunit.client.0.smithi027.stdout:0/951: write d15/d18/d23/d28/d2a/d2d/f56 [1106768,74044] 0 2024-06-06T08:43:56.571 INFO:tasks.workunit.client.0.smithi027.stdout:0/952: write d15/d18/d64/fed [746605,32345] 0 2024-06-06T08:43:56.573 INFO:tasks.workunit.client.0.smithi027.stdout:8/985: dread d7/dc/d36/d83/d60/f68 [0,4194304] 0 2024-06-06T08:43:56.590 INFO:tasks.workunit.client.0.smithi027.stdout:8/986: dwrite d7/dc/d36/fdf [0,4194304] 0 2024-06-06T08:43:56.592 INFO:tasks.workunit.client.0.smithi027.stdout:8/987: write d7/dc/d36/d83/d60/d93/dac/ff0 [408492,49169] 0 2024-06-06T08:43:56.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/988: rename d7/dc/f9f to d7/dc/d36/d41/d85/db7/dd3/dee/f13e 0 2024-06-06T08:43:56.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/989: fsync d7/d2f/f31 0 2024-06-06T08:43:56.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/990: stat d7/dc/d36/d83/l115 0 2024-06-06T08:43:56.603 INFO:tasks.workunit.client.0.smithi027.stdout:8/991: chown d7/d2f/f77 1128422787 1 2024-06-06T08:43:56.604 INFO:tasks.workunit.client.0.smithi027.stdout:8/992: symlink d7/d9c/l13f 0 2024-06-06T08:43:56.604 INFO:tasks.workunit.client.0.smithi027.stdout:8/993: symlink d7/dc/d36/d41/d85/db7/d106/l140 0 2024-06-06T08:43:56.605 INFO:tasks.workunit.client.0.smithi027.stdout:8/994: truncate d7/dc/d36/d83/d60/d93/dac/d125/f12e 214253 0 2024-06-06T08:43:56.605 INFO:tasks.workunit.client.0.smithi027.stdout:8/995: write d7/dc/d36/d3c/f111 [811609,71726] 0 2024-06-06T08:43:56.607 INFO:tasks.workunit.client.0.smithi027.stdout:8/996: rename d7/dc/d36/d83/d60/lb5 to d7/dc/d36/d83/d60/d93/l141 0 2024-06-06T08:43:56.609 INFO:tasks.workunit.client.0.smithi027.stdout:8/997: write d7/dc/d36/d83/fbc [266671,5585] 0 2024-06-06T08:43:56.609 INFO:tasks.workunit.client.0.smithi027.stdout:8/998: chown d7/dc/dde 30144469 1 2024-06-06T08:43:56.611 INFO:tasks.workunit.client.0.smithi027.stdout:8/999: rename d7/dc/d36/d83/f1d to d7/dc/d36/d83/d60/d93/dac/d110/f142 0 2024-06-06T08:43:56.651 INFO:tasks.workunit.client.0.smithi027.stdout:9/992: sync 2024-06-06T08:43:56.655 INFO:tasks.workunit.client.0.smithi027.stdout:9/993: stat d3/d95 0 2024-06-06T08:43:56.665 INFO:tasks.workunit.client.0.smithi027.stdout:9/994: dread d3/d95/d34/f107 [0,4194304] 0 2024-06-06T08:43:56.665 INFO:tasks.workunit.client.0.smithi027.stdout:9/995: fdatasync d3/d4e/d75/dc/d9b/f11c 0 2024-06-06T08:43:56.665 INFO:tasks.workunit.client.0.smithi027.stdout:9/996: write d3/d95/d23/d36/d3a/f96 [169004,12600] 0 2024-06-06T08:43:56.667 INFO:tasks.workunit.client.0.smithi027.stdout:9/997: mkdir d3/d4e/d75/dc/d153 0 2024-06-06T08:43:56.698 INFO:tasks.workunit.client.0.smithi027.stdout:0/953: sync 2024-06-06T08:43:56.700 INFO:tasks.workunit.client.0.smithi027.stdout:0/954: creat d15/d18/f13f x:0 0 0 2024-06-06T08:43:56.701 INFO:tasks.workunit.client.0.smithi027.stdout:0/955: creat d15/d18/d23/d6f/deb/f140 x:0 0 0 2024-06-06T08:43:56.710 INFO:tasks.workunit.client.0.smithi027.stdout:0/956: dread d15/d18/dbe/d5d/f98 [0,4194304] 0 2024-06-06T08:43:56.719 INFO:tasks.workunit.client.0.smithi027.stdout:0/957: write d15/d18/d23/d6f/ffb [302811,89421] 0 2024-06-06T08:43:56.734 INFO:tasks.workunit.client.0.smithi027.stdout:0/958: dwrite d15/d18/d23/f103 [0,4194304] 0 2024-06-06T08:43:56.734 INFO:tasks.workunit.client.0.smithi027.stdout:0/959: dread - d15/d18/d23/d38/d3e/fd2 zero size 2024-06-06T08:43:56.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/960: mknod d15/d18/dbe/d5d/c141 0 2024-06-06T08:43:56.737 INFO:tasks.workunit.client.0.smithi027.stdout:0/961: getdents d15/d18/d23 0 2024-06-06T08:43:56.745 INFO:tasks.workunit.client.0.smithi027.stdout:0/962: dread d15/d18/d40/d8c/f11b [0,4194304] 0 2024-06-06T08:43:56.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/963: dwrite d15/d18/dbe/d5c/d83/fb6 [12582912,4194304] 0 2024-06-06T08:43:56.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/964: fdatasync d15/d18/d64/f6d 0 2024-06-06T08:43:56.762 INFO:tasks.workunit.client.0.smithi027.stdout:0/965: symlink d15/d18/dbe/d5c/d83/l142 0 2024-06-06T08:43:56.763 INFO:tasks.workunit.client.0.smithi027.stdout:0/966: symlink d15/d18/d23/l143 0 2024-06-06T08:43:56.764 INFO:tasks.workunit.client.0.smithi027.stdout:0/967: mknod d15/d18/de0/c144 0 2024-06-06T08:43:56.766 INFO:tasks.workunit.client.0.smithi027.stdout:0/968: mkdir d15/d18/d23/d3c/d145 0 2024-06-06T08:43:56.766 INFO:tasks.workunit.client.0.smithi027.stdout:0/969: getdents d15/d18/d23/d28/d128 0 2024-06-06T08:43:56.766 INFO:tasks.workunit.client.0.smithi027.stdout:0/970: stat d15/d18/d40/f9b 0 2024-06-06T08:43:56.766 INFO:tasks.workunit.client.0.smithi027.stdout:0/971: fdatasync d15/da7/ff8 0 2024-06-06T08:43:56.766 INFO:tasks.workunit.client.0.smithi027.stdout:0/972: fdatasync d15/d18/d23/d28/d31/ff7 0 2024-06-06T08:43:56.774 INFO:tasks.workunit.client.0.smithi027.stdout:0/973: dread d15/d18/d23/d38/f51 [12582912,4194304] 0 2024-06-06T08:43:56.776 INFO:tasks.workunit.client.0.smithi027.stdout:0/974: mknod d15/d18/d23/d6f/c146 0 2024-06-06T08:43:56.778 INFO:tasks.workunit.client.0.smithi027.stdout:0/975: mkdir d15/d18/dbe/d5c/d83/d147 0 2024-06-06T08:43:56.779 INFO:tasks.workunit.client.0.smithi027.stdout:0/976: mknod d15/d18/d40/d8c/c148 0 2024-06-06T08:43:56.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/977: truncate d15/d18/d23/d28/d31/f84 2074548 0 2024-06-06T08:43:56.781 INFO:tasks.workunit.client.0.smithi027.stdout:0/978: rmdir d15/d18/d23/d3c 39 2024-06-06T08:43:56.782 INFO:tasks.workunit.client.0.smithi027.stdout:0/979: fdatasync d15/d18/d23/d38/d68/ff2 0 2024-06-06T08:43:56.788 INFO:tasks.workunit.client.0.smithi027.stdout:9/998: sync 2024-06-06T08:43:56.790 INFO:tasks.workunit.client.0.smithi027.stdout:0/980: write d15/d18/d23/d28/d31/f84 [228184,123049] 0 2024-06-06T08:43:56.792 INFO:tasks.workunit.client.0.smithi027.stdout:0/981: rmdir d15/d18/d23/d38/d68/d85 39 2024-06-06T08:43:56.792 INFO:tasks.workunit.client.0.smithi027.stdout:0/982: fdatasync d15/d18/dbe/d5c/f100 0 2024-06-06T08:43:56.792 INFO:tasks.workunit.client.0.smithi027.stdout:0/983: readlink d15/d18/d23/d28/d2a/d2d/l11f 0 2024-06-06T08:43:56.793 INFO:tasks.workunit.client.0.smithi027.stdout:0/984: readlink d15/d18/d40/l66 0 2024-06-06T08:43:56.794 INFO:tasks.workunit.client.0.smithi027.stdout:0/985: creat d15/d18/d23/d28/d31/f149 x:0 0 0 2024-06-06T08:43:56.795 INFO:tasks.workunit.client.0.smithi027.stdout:0/986: unlink d15/d18/d23/d28/d2a/c2b 0 2024-06-06T08:43:56.796 INFO:tasks.workunit.client.0.smithi027.stdout:0/987: creat d15/d18/d40/d8c/d110/f14a x:0 0 0 2024-06-06T08:43:56.797 INFO:tasks.workunit.client.0.smithi027.stdout:0/988: creat d15/d18/d40/f14b x:0 0 0 2024-06-06T08:43:56.798 INFO:tasks.workunit.client.0.smithi027.stdout:9/999: dread d3/d4e/d75/dc/f2a [0,4194304] 0 2024-06-06T08:43:56.799 INFO:tasks.workunit.client.0.smithi027.stdout:0/989: mkdir d15/d18/d23/d28/d31/d14c 0 2024-06-06T08:43:56.800 INFO:tasks.workunit.client.0.smithi027.stdout:0/990: creat d15/d18/d23/d38/d68/f14d x:0 0 0 2024-06-06T08:43:56.807 INFO:tasks.workunit.client.0.smithi027.stdout:0/991: write d15/d18/dbe/d5d/f98 [2739575,52712] 0 2024-06-06T08:43:56.823 INFO:tasks.workunit.client.0.smithi027.stdout:0/992: dwrite d15/d18/d108/f11a [0,4194304] 0 2024-06-06T08:43:56.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/993: rmdir d15/d18/d40/d8c/d110 39 2024-06-06T08:43:56.825 INFO:tasks.workunit.client.0.smithi027.stdout:0/994: mknod d15/d18/dbe/d5c/dfa/c14e 0 2024-06-06T08:43:56.826 INFO:tasks.workunit.client.0.smithi027.stdout:0/995: truncate d15/d18/d23/d28/d31/dbb/fcc 798682 0 2024-06-06T08:43:56.828 INFO:tasks.workunit.client.0.smithi027.stdout:0/996: mkdir d15/d18/d23/d5b/da0/d14f 0 2024-06-06T08:43:56.934 INFO:tasks.workunit.client.0.smithi027.stdout:0/997: sync 2024-06-06T08:43:56.934 INFO:tasks.workunit.client.0.smithi027.stdout:0/998: write d15/d18/d40/faf [471846,125767] 0 2024-06-06T08:43:56.934 INFO:tasks.workunit.client.0.smithi027.stdout:0/999: mknod d15/d18/d23/d28/d2a/d74/c150 0 2024-06-06T08:43:56.938 INFO:tasks.workunit.client.0.smithi027.stderr:+ rm -rf -- ./tmp.Es9eopsTLb 2024-06-06T08:43:56.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:56 smithi027 ceph-mon[32335]: pgmap v113: 129 pgs: 129 active+clean; 1.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 104 MiB/s wr, 415 op/s 2024-06-06T08:43:56.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:56 smithi203 ceph-mon[36579]: pgmap v113: 129 pgs: 129 active+clean; 1.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 104 MiB/s wr, 415 op/s 2024-06-06T08:43:57.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:56 smithi176 ceph-mon[36786]: pgmap v113: 129 pgs: 129 active+clean; 1.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 104 MiB/s wr, 415 op/s 2024-06-06T08:43:57.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.645+0000 7f2f75871700 1 -- 172.21.15.27:0/2428116080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f701009e0 msgr2=0x7f2f70100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.645+0000 7f2f75871700 1 --2- 172.21.15.27:0/2428116080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f701009e0 0x7f2f70100e60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2f640099c0 tx=0x7f2f64009cd0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.645+0000 7f2f75871700 1 -- 172.21.15.27:0/2428116080 shutdown_connections 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.645+0000 7f2f75871700 1 --2- 172.21.15.27:0/2428116080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f701013a0 0x7f2f7010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.646+0000 7f2f75871700 1 --2- 172.21.15.27:0/2428116080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f70108560 0x7f2f70108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.646+0000 7f2f75871700 1 --2- 172.21.15.27:0/2428116080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f701009e0 0x7f2f70100e60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.646+0000 7f2f75871700 1 -- 172.21.15.27:0/2428116080 >> 172.21.15.27:0/2428116080 conn(0x7f2f70077990 msgr2=0x7f2f700fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.646+0000 7f2f75871700 1 -- 172.21.15.27:0/2428116080 shutdown_connections 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.646+0000 7f2f75871700 1 -- 172.21.15.27:0/2428116080 wait complete. 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.646+0000 7f2f75871700 1 Processor -- start 2024-06-06T08:43:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f75871700 1 -- start start 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f75871700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f701009e0 0x7f2f70101bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f75871700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f701013a0 0x7f2f70102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f75871700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f70108560 0x7f2f70102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f75871700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f701127c0 con 0x7f2f70108560 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f75871700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70112640 con 0x7f2f701009e0 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f75871700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f70112940 con 0x7f2f701013a0 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f701009e0 0x7f2f70101bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f70108560 0x7f2f70102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f701009e0 0x7f2f70101bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:52380/0 (socket says 172.21.15.27:52380) 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6e7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f701013a0 0x7f2f70102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f70108560 0x7f2f70102640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57990/0 (socket says 172.21.15.27:57990) 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6effd700 1 -- 172.21.15.27:0/1382297711 learned_addr learned my addr 172.21.15.27:0/1382297711 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6effd700 1 -- 172.21.15.27:0/1382297711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f701013a0 msgr2=0x7f2f70102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6effd700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f701013a0 0x7f2f70102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6effd700 1 -- 172.21.15.27:0/1382297711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f70108560 msgr2=0x7f2f70102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:57.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6effd700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f70108560 0x7f2f70102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.647+0000 7f2f6effd700 1 -- 172.21.15.27:0/1382297711 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f701a70d0 con 0x7f2f701009e0 2024-06-06T08:43:57.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.648+0000 7f2f6effd700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f701009e0 0x7f2f70101bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e9c0 tx=0x7f2f6000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:57.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.648+0000 7f2f7486f700 1 -- 172.21.15.27:0/1382297711 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6000cbd0 con 0x7f2f701009e0 2024-06-06T08:43:57.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.648+0000 7f2f75871700 1 -- 172.21.15.27:0/1382297711 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f701a73c0 con 0x7f2f701009e0 2024-06-06T08:43:57.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.648+0000 7f2f75871700 1 -- 172.21.15.27:0/1382297711 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f701a78d0 con 0x7f2f701009e0 2024-06-06T08:43:57.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.648+0000 7f2f7486f700 1 -- 172.21.15.27:0/1382297711 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6000ee60 con 0x7f2f701009e0 2024-06-06T08:43:57.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.648+0000 7f2f7486f700 1 -- 172.21.15.27:0/1382297711 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60005630 con 0x7f2f701009e0 2024-06-06T08:43:57.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.650+0000 7f2f7486f700 1 -- 172.21.15.27:0/1382297711 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2f600104b0 con 0x7f2f701009e0 2024-06-06T08:43:57.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.651+0000 7f2f7486f700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f580a2d90 0x7f2f580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.651+0000 7f2f7486f700 1 -- 172.21.15.27:0/1382297711 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f2f600cf9f0 con 0x7f2f701009e0 2024-06-06T08:43:57.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.651+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f580a2d90 0x7f2f580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.652+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f580a2d90 0x7f2f580a5250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f70103600 tx=0x7f2f64005fb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:57.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.652+0000 7f2f800f4b80 1 Processor -- start 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.652+0000 7f2f800f4b80 1 -- start start 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.652+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a4f60 0x557a758a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.652+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a7589f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.652+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a7589f7a0 0x557a7589fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557a75791f60 con 0x557a758a4f60 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557a757f0850 con 0x557a7589f7a0 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557a75758460 con 0x557a758a5880 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a4f60 0x557a758a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a4f60 0x557a758a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58000/0 (socket says 172.21.15.27:58000) 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6effd700 1 -- 172.21.15.27:0/1636157342 learned_addr learned my addr 172.21.15.27:0/1636157342 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6f7fe700 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a7589f7a0 0x557a7589fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a7589f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6effd700 1 -- 172.21.15.27:0/1636157342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 msgr2=0x557a7589f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6effd700 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a7589f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6effd700 1 -- 172.21.15.27:0/1636157342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a7589f7a0 msgr2=0x557a7589fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:57.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6effd700 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a7589f7a0 0x557a7589fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6effd700 1 -- 172.21.15.27:0/1636157342 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7573f8d0 con 0x557a758a4f60 2024-06-06T08:43:57.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6f7fe700 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a7589f7a0 0x557a7589fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:57.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f6effd700 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a4f60 0x557a758a5340 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2f600984c0 tx=0x7f2f60097e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eba5ce29f6a259 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:57.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f55ffb700 1 -- 172.21.15.27:0/1636157342 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6009fea0 con 0x557a758a4f60 2024-06-06T08:43:57.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f55ffb700 1 -- 172.21.15.27:0/1636157342 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f600a4760 con 0x557a758a4f60 2024-06-06T08:43:57.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.653+0000 7f2f55ffb700 1 -- 172.21.15.27:0/1636157342 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f600a42d0 con 0x557a758a4f60 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.654+0000 7f2f800f4b80 1 -- 172.21.15.27:0/1636157342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a4f60 msgr2=0x557a758a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.654+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a4f60 0x557a758a5340 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2f600984c0 tx=0x7f2f60097e80 comp rx=0 tx=0).stop 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.654+0000 7f2f800f4b80 1 -- 172.21.15.27:0/1636157342 shutdown_connections 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.654+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a7589f7a0 0x557a7589fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.654+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a4f60 0x557a758a5340 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.654+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/1636157342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a7589f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.654+0000 7f2f800f4b80 1 -- 172.21.15.27:0/1636157342 >> 172.21.15.27:0/1636157342 conn(0x557a757cc080 msgr2=0x557a7589a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.654+0000 7f2f800f4b80 1 -- 172.21.15.27:0/1636157342 shutdown_connections 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.654+0000 7f2f800f4b80 1 -- 172.21.15.27:0/1636157342 wait complete. 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f800f4b80 1 -- start start 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f800f4b80 1 Processor -- start 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a7589f7a0 0x557a758ab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a5880 0x557a758a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557a75790fc0 con 0x557a758a5880 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557a757ec810 con 0x557a758a4f60 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557a758ac260 con 0x557a7589f7a0 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f800f4b80 10 client.0 _ll_register_callbacks cb 0x557a757865b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f6e7fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f6f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a5880 0x557a758a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.655+0000 7f2f6e7fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:52396/0 (socket says 172.21.15.27:52396) 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f6e7fc700 1 -- 172.21.15.27:0/112675309 learned_addr learned my addr 172.21.15.27:0/112675309 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f6effd700 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a7589f7a0 0x557a758ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f6e7fc700 1 -- 172.21.15.27:0/112675309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a7589f7a0 msgr2=0x557a758ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a7589f7a0 0x557a758ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f6e7fc700 1 -- 172.21.15.27:0/112675309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a5880 msgr2=0x557a758a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a5880 0x557a758a8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f6e7fc700 1 -- 172.21.15.27:0/112675309 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a757b6750 con 0x557a758a4f60 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f6f7fe700 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a5880 0x557a758a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a8970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f64015000 tx=0x7f2f64009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794054a777ebf389 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f4effd700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f4effd700 1 -- 172.21.15.27:0/112675309 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64034060 con 0x557a758a4f60 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f4effd700 1 -- 172.21.15.27:0/112675309 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f2f64024ca0 con 0x557a758a4f60 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.656+0000 7f2f800f4b80 1 -- 172.21.15.27:0/112675309 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f600a4760 con 0x557a758a4f60 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f4effd700 1 -- 172.21.15.27:0/112675309 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f6403b050 con 0x557a758a4f60 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f800f4b80 10 client.25784 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f800f4b80 1 -- 172.21.15.27:0/112675309 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a758b6aa0 con 0x557a758a4f60 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f4effd700 1 -- 172.21.15.27:0/112675309 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f600a4760 con 0x557a758a4f60 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f4effd700 1 -- 172.21.15.27:0/112675309 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2f6402f410 con 0x557a758a4f60 2024-06-06T08:43:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f800f4b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f800f4b80 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f800f4b80 20 client.25784 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a758b9130 0x557a758bb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f800f4b80 4 client.25784 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:43:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f800f4b80 1 -- 172.21.15.27:0/112675309 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a757fdd50 con 0x557a758b9130 2024-06-06T08:43:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.657+0000 7f2f6effd700 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a758b9130 0x557a758bb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.658+0000 7f2f6effd700 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a758b9130 0x557a758bb520 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1668e12a271ccb server_cookie=d96cc63d02e5f425 in_seq=0 out_seq=0 2024-06-06T08:43:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.658+0000 7f2f4effd700 10 client.25784 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.658+0000 7f2f4effd700 1 -- 172.21.15.27:0/112675309 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557a757fdd50 con 0x557a758b9130 2024-06-06T08:43:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.658+0000 7f2f4effd700 10 client.25784 handle_command_reply: tid=0 2024-06-06T08:43:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 client.25784 shutdown 2024-06-06T08:43:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 -- 172.21.15.27:0/112675309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 msgr2=0x557a758a8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a8970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f64015000 tx=0x7f2f64009310 comp rx=0 tx=0).stop 2024-06-06T08:43:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 -- 172.21.15.27:0/112675309 shutdown_connections 2024-06-06T08:43:57.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a758b9130 0x557a758bb520 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a758a5880 0x557a758a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a7589f7a0 0x557a758ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/112675309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a8970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f4effd700 0 client.25784 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:57.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 -- 172.21.15.27:0/112675309 >> 172.21.15.27:0/112675309 conn(0x557a757cc080 msgr2=0x557a75899700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:57.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 -- 172.21.15.27:0/112675309 shutdown_connections 2024-06-06T08:43:57.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 1 -- 172.21.15.27:0/112675309 wait complete. 2024-06-06T08:43:57.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.659+0000 7f2f800f4b80 20 client.25784 trim_cache size 0 max 16384 2024-06-06T08:43:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.680+0000 7f2f800f4b80 1 Processor -- start 2024-06-06T08:43:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.680+0000 7f2f800f4b80 1 -- start start 2024-06-06T08:43:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a758aa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c0116e0 con 0x557a7589f7a0 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c00eeb0 con 0x557a758a4f60 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c00cd00 con 0x557a758a5880 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58018/0 (socket says 172.21.15.27:58018) 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6effd700 1 -- 172.21.15.27:0/796978143 learned_addr learned my addr 172.21.15.27:0/796978143 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6f7fe700 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a758aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6effd700 1 -- 172.21.15.27:0/796978143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 msgr2=0x557a758aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6effd700 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a758aa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6effd700 1 -- 172.21.15.27:0/796978143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 msgr2=0x557a758a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6effd700 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6effd700 1 -- 172.21.15.27:0/796978143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a758a6ee0 con 0x557a7589f7a0 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6f7fe700 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a758aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:43:57.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:43:57.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.681+0000 7f2f6effd700 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758a3050 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009fd0 tx=0x7f2f6009bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c29d8be2e0314e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:57.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.682+0000 7f2f557fa700 1 -- 172.21.15.27:0/796978143 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f600a4760 con 0x557a7589f7a0 2024-06-06T08:43:57.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.682+0000 7f2f557fa700 1 -- 172.21.15.27:0/796978143 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f60099e70 con 0x557a7589f7a0 2024-06-06T08:43:57.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.682+0000 7f2f557fa700 1 -- 172.21.15.27:0/796978143 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6009bd50 con 0x557a7589f7a0 2024-06-06T08:43:57.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.682+0000 7f2f800f4b80 1 -- 172.21.15.27:0/796978143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 msgr2=0x557a758a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:57.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.682+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758a3050 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009fd0 tx=0x7f2f6009bc50 comp rx=0 tx=0).stop 2024-06-06T08:43:57.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 -- 172.21.15.27:0/796978143 shutdown_connections 2024-06-06T08:43:57.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a758aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758a3050 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/796978143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 -- 172.21.15.27:0/796978143 >> 172.21.15.27:0/796978143 conn(0x557a757cc080 msgr2=0x557a758aa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 -- 172.21.15.27:0/796978143 shutdown_connections 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 -- 172.21.15.27:0/796978143 wait complete. 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 -- start start 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 Processor -- start 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.683+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f800f4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a758c0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c00d370 con 0x557a7589f7a0 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c00e7e0 con 0x557a758a4f60 2024-06-06T08:43:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f800f4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c00da40 con 0x557a758a5880 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f800f4b80 10 client.0 _ll_register_callbacks cb 0x557a757865b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58024/0 (socket says 172.21.15.27:58024) 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6effd700 1 -- 172.21.15.27:0/703058940 learned_addr learned my addr 172.21.15.27:0/703058940 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6f7fe700 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a758c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6effd700 1 -- 172.21.15.27:0/703058940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 msgr2=0x557a758c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6effd700 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a758c0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6effd700 1 -- 172.21.15.27:0/703058940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 msgr2=0x557a758b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6effd700 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6effd700 1 -- 172.21.15.27:0/703058940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a758c0c80 con 0x557a7589f7a0 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f6effd700 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758b1810 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2f6009a040 tx=0x7f2f600cce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41705a8b68f91b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:43:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.684+0000 7f2f4effd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f4effd700 1 -- 172.21.15.27:0/703058940 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60081060 con 0x557a7589f7a0 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f4effd700 1 -- 172.21.15.27:0/703058940 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f2f600ba2d0 con 0x557a7589f7a0 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f800f4b80 1 -- 172.21.15.27:0/703058940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f60099e70 con 0x557a7589f7a0 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f4effd700 1 -- 172.21.15.27:0/703058940 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f600ba510 con 0x557a7589f7a0 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f800f4b80 10 client.16293 fetch_fsmap learned FSMap version 34 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f800f4b80 1 -- 172.21.15.27:0/703058940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a75923960 con 0x557a7589f7a0 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f4effd700 1 -- 172.21.15.27:0/703058940 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f60099e70 con 0x557a7589f7a0 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f4effd700 1 -- 172.21.15.27:0/703058940 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2f60016be0 con 0x557a7589f7a0 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f800f4b80 10 client.16293 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f800f4b80 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f800f4b80 20 client.16293 populate_metadata read hostname 'smithi027' 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a75923b80 0x557a75925f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f800f4b80 4 client.16293 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T08:43:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.685+0000 7f2f800f4b80 1 -- 172.21.15.27:0/703058940 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557a757f82d0 con 0x557a75923b80 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.686+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a75923b80 0x557a75925f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.686+0000 7f2f6e7fc700 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a75923b80 0x557a75925f70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef933ced18c0ab63 server_cookie=3fdff9c1138cfb41 in_seq=0 out_seq=0 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.686+0000 7f2f4effd700 10 client.16293 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.686+0000 7f2f4effd700 1 -- 172.21.15.27:0/703058940 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557a757f82d0 con 0x557a75923b80 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.686+0000 7f2f4effd700 10 client.16293 handle_command_reply: tid=0 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.686+0000 7f2f800f4b80 1 client.16293 shutdown 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 -- 172.21.15.27:0/703058940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 msgr2=0x557a758b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758b1810 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2f6009a040 tx=0x7f2f600cce00 comp rx=0 tx=0).stop 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 -- 172.21.15.27:0/703058940 shutdown_connections 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557a758a5880 0x557a758c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557a75923b80 0x557a75925f70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557a7589f7a0 0x557a758b1810 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 --2- 172.21.15.27:0/703058940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a758a4f60 0x557a758b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 -- 172.21.15.27:0/703058940 >> 172.21.15.27:0/703058940 conn(0x557a757cc080 msgr2=0x557a758aa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f4effd700 0 client.16293 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:43:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 -- 172.21.15.27:0/703058940 shutdown_connections 2024-06-06T08:43:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 1 -- 172.21.15.27:0/703058940 wait complete. 2024-06-06T08:43:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.687+0000 7f2f800f4b80 20 client.16293 trim_cache size 0 max 16384 2024-06-06T08:43:57.690 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:43:57.690 INFO:teuthology.orchestra.run.smithi027.stdout: "status": "no active scrubs running", 2024-06-06T08:43:57.690 INFO:teuthology.orchestra.run.smithi027.stdout: "scrubs": {} 2024-06-06T08:43:57.690 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:43:57.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 -- 172.21.15.27:0/1382297711 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f580a2d90 msgr2=0x7f2f580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:57.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f580a2d90 0x7f2f580a5250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f70103600 tx=0x7f2f64005fb0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 -- 172.21.15.27:0/1382297711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f701009e0 msgr2=0x7f2f70101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:43:57.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f701009e0 0x7f2f70101bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e9c0 tx=0x7f2f6000ecd0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 -- 172.21.15.27:0/1382297711 shutdown_connections 2024-06-06T08:43:57.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f70108560 0x7f2f70102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f580a2d90 0x7f2f580a5250 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f701009e0 0x7f2f70101bc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 --2- 172.21.15.27:0/1382297711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f701013a0 0x7f2f70102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:43:57.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 -- 172.21.15.27:0/1382297711 >> 172.21.15.27:0/1382297711 conn(0x7f2f70077990 msgr2=0x7f2f700feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:43:57.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 -- 172.21.15.27:0/1382297711 shutdown_connections 2024-06-06T08:43:57.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:43:57.688+0000 7f2f75871700 1 -- 172.21.15.27:0/1382297711 wait complete. 2024-06-06T08:43:58.472 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T08:43:58.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:43:58.796 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:43:58.919 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:43:58 smithi027 ceph-mon[32335]: pgmap v114: 129 pgs: 129 active+clean; 1.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 104 MiB/s wr, 411 op/s 2024-06-06T08:43:58.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:43:58 smithi203 ceph-mon[36579]: pgmap v114: 129 pgs: 129 active+clean; 1.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 104 MiB/s wr, 411 op/s 2024-06-06T08:43:59.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:43:58 smithi176 ceph-mon[36786]: pgmap v114: 129 pgs: 129 active+clean; 1.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 104 MiB/s wr, 411 op/s 2024-06-06T08:44:00.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.577+0000 7f80f6789700 1 -- 172.21.15.27:0/1369608312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 msgr2=0x7f80f0100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:00.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.577+0000 7f80f6789700 1 --2- 172.21.15.27:0/1369608312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 0x7f80f0100fd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009990 tx=0x7f80d8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.577+0000 7f80f6789700 1 -- 172.21.15.27:0/1369608312 shutdown_connections 2024-06-06T08:44:00.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.577+0000 7f80f6789700 1 --2- 172.21.15.27:0/1369608312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80f0101510 0x7f80f010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.577+0000 7f80f6789700 1 --2- 172.21.15.27:0/1369608312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80f0106b90 0x7f80f0106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.577+0000 7f80f6789700 1 --2- 172.21.15.27:0/1369608312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 0x7f80f0100fd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.577+0000 7f80f6789700 1 -- 172.21.15.27:0/1369608312 >> 172.21.15.27:0/1369608312 conn(0x7f80f0077a00 msgr2=0x7f80f00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.577+0000 7f80f6789700 1 -- 172.21.15.27:0/1369608312 shutdown_connections 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.577+0000 7f80f6789700 1 -- 172.21.15.27:0/1369608312 wait complete. 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f6789700 1 Processor -- start 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f6789700 1 -- start start 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f6789700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 0x7f80f019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f6789700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80f0101510 0x7f80f019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f6789700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80f0106b90 0x7f80f0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f6789700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01116f0 con 0x7f80f0106b90 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f6789700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0111570 con 0x7f80f0100b50 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f6789700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80f0111870 con 0x7f80f0101510 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80effff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 0x7f80f019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f4d26700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80f0106b90 0x7f80f0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:00.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80effff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 0x7f80f019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:60772/0 (socket says 172.21.15.27:60772) 2024-06-06T08:44:00.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80effff700 1 -- 172.21.15.27:0/1407567861 learned_addr learned my addr 172.21.15.27:0/1407567861 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:00.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80f4d26700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80f0106b90 0x7f80f0198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55906/0 (socket says 172.21.15.27:55906) 2024-06-06T08:44:00.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80e7fff700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80f0101510 0x7f80f019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:00.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80effff700 1 -- 172.21.15.27:0/1407567861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80f0101510 msgr2=0x7f80f019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:00.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80effff700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80f0101510 0x7f80f019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80effff700 1 -- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80f0106b90 msgr2=0x7f80f0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:00.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80effff700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80f0106b90 0x7f80f0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.578+0000 7f80effff700 1 -- 172.21.15.27:0/1407567861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f0199260 con 0x7f80f0100b50 2024-06-06T08:44:00.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.579+0000 7f80effff700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 0x7f80f019e8a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80e000e9a0 tx=0x7f80e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.579+0000 7f80edffb700 1 -- 172.21.15.27:0/1407567861 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e000cb50 con 0x7f80f0100b50 2024-06-06T08:44:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.579+0000 7f80edffb700 1 -- 172.21.15.27:0/1407567861 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e000ccf0 con 0x7f80f0100b50 2024-06-06T08:44:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.579+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f0199550 con 0x7f80f0100b50 2024-06-06T08:44:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.579+0000 7f80edffb700 1 -- 172.21.15.27:0/1407567861 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0018680 con 0x7f80f0100b50 2024-06-06T08:44:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.579+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f00fcda0 con 0x7f80f0100b50 2024-06-06T08:44:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.580+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f0069180 con 0x7f80f0100b50 2024-06-06T08:44:00.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.581+0000 7f80edffb700 1 -- 172.21.15.27:0/1407567861 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f80e0010430 con 0x7f80f0100b50 2024-06-06T08:44:00.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.582+0000 7f80edffb700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f80dc0a2e10 0x7f80dc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:00.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.582+0000 7f80e7fff700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f80dc0a2e10 0x7f80dc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:00.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.582+0000 7f80edffb700 1 -- 172.21.15.27:0/1407567861 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f80e00cfd40 con 0x7f80f0100b50 2024-06-06T08:44:00.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.583+0000 7f80e7fff700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f80dc0a2e10 0x7f80dc0a52d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80d8006010 tx=0x7f80d801a040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:00.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.588+0000 7f80edffb700 1 -- 172.21.15.27:0/1407567861 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f80e009c540 con 0x7f80f0100b50 2024-06-06T08:44:00.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.845+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80f0199ff0 con 0x7f80f0100b50 2024-06-06T08:44:00.844 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:00 smithi027 ceph-mon[32335]: pgmap v115: 129 pgs: 129 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 153 MiB/s wr, 535 op/s 2024-06-06T08:44:00.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.846+0000 7f80edffb700 1 -- 172.21.15.27:0/1407567861 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11896 (secure 0 0 0) 0x7f80e009be40 con 0x7f80f0100b50 2024-06-06T08:44:00.845 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:44:00.845 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:43:25.179611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:44:00.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f80dc0a2e10 msgr2=0x7f80dc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:00.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f80dc0a2e10 0x7f80dc0a52d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80d8006010 tx=0x7f80d801a040 comp rx=0 tx=0).stop 2024-06-06T08:44:00.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 msgr2=0x7f80f019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:00.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 0x7f80f019e8a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80e000e9a0 tx=0x7f80e000ecb0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 shutdown_connections 2024-06-06T08:44:00.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80f0106b90 0x7f80f0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f80dc0a2e10 0x7f80dc0a52d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80f0100b50 0x7f80f019e8a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 --2- 172.21.15.27:0/1407567861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80f0101510 0x7f80f019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 >> 172.21.15.27:0/1407567861 conn(0x7f80f0077a00 msgr2=0x7f80f010e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 shutdown_connections 2024-06-06T08:44:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:00.848+0000 7f80f6789700 1 -- 172.21.15.27:0/1407567861 wait complete. 2024-06-06T08:44:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-06-06T08:44:00.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:00 smithi203 ceph-mon[36579]: pgmap v115: 129 pgs: 129 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 153 MiB/s wr, 535 op/s 2024-06-06T08:44:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:00 smithi176 ceph-mon[36786]: pgmap v115: 129 pgs: 129 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 153 MiB/s wr, 535 op/s 2024-06-06T08:44:01.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T08:44:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.743+0000 7f241dc93640 1 -- 172.21.15.27:0/4001553476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2418101540 msgr2=0x7f241810d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.743+0000 7f241dc93640 1 --2- 172.21.15.27:0/4001553476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2418101540 0x7f241810d6b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2410009d00 tx=0x7f241002a910 comp rx=0 tx=0).stop 2024-06-06T08:44:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.743+0000 7f241dc93640 1 -- 172.21.15.27:0/4001553476 shutdown_connections 2024-06-06T08:44:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.743+0000 7f241dc93640 1 --2- 172.21.15.27:0/4001553476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2418101540 0x7f241810d6b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.743+0000 7f241dc93640 1 --2- 172.21.15.27:0/4001553476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418100b80 0x7f2418101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.743+0000 7f241dc93640 1 --2- 172.21.15.27:0/4001553476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24181069c0 0x7f2418106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.743+0000 7f241dc93640 1 -- 172.21.15.27:0/4001553476 >> 172.21.15.27:0/4001553476 conn(0x7f24180775f0 msgr2=0x7f2418077a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.744+0000 7f241dc93640 1 -- 172.21.15.27:0/4001553476 shutdown_connections 2024-06-06T08:44:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.744+0000 7f241dc93640 1 -- 172.21.15.27:0/4001553476 wait complete. 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.744+0000 7f241dc93640 1 Processor -- start 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.744+0000 7f241dc93640 1 -- start start 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f241dc93640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2418100b80 0x7f2418196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f241dc93640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418101540 0x7f2418197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f241dc93640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24181069c0 0x7f24181977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f241dc93640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2418110c90 con 0x7f2418101540 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f241dc93640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2418110b10 con 0x7f2418100b80 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f241dc93640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2418110e10 con 0x7f24181069c0 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2416ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418101540 0x7f2418197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2416ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418101540 0x7f2418197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55920/0 (socket says 172.21.15.27:55920) 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f24177fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2418100b80 0x7f2418196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2416ffd640 1 -- 172.21.15.27:0/2947155327 learned_addr learned my addr 172.21.15.27:0/2947155327 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2417fff640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24181069c0 0x7f24181977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2416ffd640 1 -- 172.21.15.27:0/2947155327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24181069c0 msgr2=0x7f24181977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2416ffd640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24181069c0 0x7f24181977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2416ffd640 1 -- 172.21.15.27:0/2947155327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2418100b80 msgr2=0x7f2418196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2416ffd640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2418100b80 0x7f2418196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2416ffd640 1 -- 172.21.15.27:0/2947155327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24181a65b0 con 0x7f2418101540 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2416ffd640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418101540 0x7f2418197280 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f240c00ea60 tx=0x7f240c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2414ff9640 1 -- 172.21.15.27:0/2947155327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c00cde0 con 0x7f2418101540 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f2414ff9640 1 -- 172.21.15.27:0/2947155327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240c0035f0 con 0x7f2418101540 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.745+0000 7f241dc93640 1 -- 172.21.15.27:0/2947155327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24181a68a0 con 0x7f2418101540 2024-06-06T08:44:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.746+0000 7f2414ff9640 1 -- 172.21.15.27:0/2947155327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c010670 con 0x7f2418101540 2024-06-06T08:44:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.746+0000 7f241dc93640 1 -- 172.21.15.27:0/2947155327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24181a6df0 con 0x7f2418101540 2024-06-06T08:44:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.748+0000 7f242099e740 1 Processor -- start 2024-06-06T08:44:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.748+0000 7f242099e740 1 -- start start 2024-06-06T08:44:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2414ff9640 1 -- 172.21.15.27:0/2947155327 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f240c0037a0 con 0x7f2418101540 2024-06-06T08:44:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f242099e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eeee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f242099e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eef8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f242099e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eef9350 0x557f0eefb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f242099e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557f0ee524f0 con 0x557f0eeee170 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f242099e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557f0edf0870 con 0x557f0eeeea90 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f242099e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557f0ee4e4b0 con 0x557f0eef9350 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2417fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eeee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f24177fe640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eef8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2417fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eeee550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55934/0 (socket says 172.21.15.27:55934) 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2417fff640 1 -- 172.21.15.27:0/3254724922 learned_addr learned my addr 172.21.15.27:0/3254724922 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2416ffd640 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eef9350 0x557f0eefb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2417fff640 1 -- 172.21.15.27:0/3254724922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eef9350 msgr2=0x557f0eefb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2417fff640 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eef9350 0x557f0eefb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2417fff640 1 -- 172.21.15.27:0/3254724922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 msgr2=0x557f0eef8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2417fff640 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eef8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2417fff640 1 -- 172.21.15.27:0/3254724922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0ee0c9b0 con 0x557f0eeee170 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2416ffd640 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eef9350 0x557f0eefb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f24177fe640 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eef8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.749+0000 7f2417fff640 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eeee550 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2410005cc0 tx=0x7f241002f320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35d95b89cae6c30 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.750+0000 7f2414ff9640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24000a2da0 0x7f24000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.750+0000 7f23fdffb640 1 -- 172.21.15.27:0/3254724922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410031b40 con 0x557f0eeee170 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.750+0000 7f23fdffb640 1 -- 172.21.15.27:0/3254724922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2410036040 con 0x557f0eeee170 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.750+0000 7f23fdffb640 1 -- 172.21.15.27:0/3254724922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241002f670 con 0x557f0eeee170 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.750+0000 7f24177fe640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24000a2da0 0x7f24000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.750+0000 7f2414ff9640 1 -- 172.21.15.27:0/2947155327 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f240c0cfb40 con 0x7f2418101540 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f24177fe640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24000a2da0 0x7f24000a5260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x557f0eeef710 tx=0x7f2408005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 -- 172.21.15.27:0/3254724922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 msgr2=0x557f0eeee550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eeee550 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2410005cc0 tx=0x7f241002f320 comp rx=0 tx=0).stop 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 -- 172.21.15.27:0/3254724922 shutdown_connections 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eef9350 0x557f0eefb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eef8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 --2- 172.21.15.27:0/3254724922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eeee550 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 -- 172.21.15.27:0/3254724922 >> 172.21.15.27:0/3254724922 conn(0x557f0eeea680 msgr2=0x557f0eeeaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 -- 172.21.15.27:0/3254724922 shutdown_connections 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 -- 172.21.15.27:0/3254724922 wait complete. 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 -- start start 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 Processor -- start 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eef62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeeea90 0x557f0eef6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 0x557f0eef6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557f0eefbf20 con 0x557f0eeeea90 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557f0eddbae0 con 0x557f0eef9350 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.751+0000 7f242099e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557f0eefc0a0 con 0x557f0eeee170 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f242099e740 10 client.0 _ll_register_callbacks cb 0x557f0ee149b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f2416ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeeea90 0x557f0eef6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f24177fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eef62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f2416ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeeea90 0x557f0eef6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55950/0 (socket says 172.21.15.27:55950) 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f2416ffd640 1 -- 172.21.15.27:0/165782353 learned_addr learned my addr 172.21.15.27:0/165782353 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f2417fff640 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 0x557f0eef6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f24177fe640 1 -- 172.21.15.27:0/165782353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 msgr2=0x557f0eef6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f24177fe640 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 0x557f0eef6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f24177fe640 1 -- 172.21.15.27:0/165782353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeeea90 msgr2=0x557f0eef6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f24177fe640 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeeea90 0x557f0eef6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f24177fe640 1 -- 172.21.15.27:0/165782353 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0eb66210 con 0x557f0eeee170 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f2416ffd640 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeeea90 0x557f0eef6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f24177fe640 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eef62f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f240801e950 tx=0x7f2408004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dafc400561b0c334 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f23f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f23f6ffd640 1 -- 172.21.15.27:0/165782353 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408009c60 con 0x557f0eeee170 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f242099e740 1 -- 172.21.15.27:0/165782353 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2410036040 con 0x557f0eeee170 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.752+0000 7f23f6ffd640 1 -- 172.21.15.27:0/165782353 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f2408018030 con 0x557f0eeee170 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f23f6ffd640 1 -- 172.21.15.27:0/165782353 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2408033050 con 0x557f0eeee170 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f242099e740 10 client.25792 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f242099e740 1 -- 172.21.15.27:0/165782353 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557f0ec32620 con 0x557f0eeee170 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f23f6ffd640 1 -- 172.21.15.27:0/165782353 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2410036040 con 0x557f0eeee170 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f23f6ffd640 1 -- 172.21.15.27:0/165782353 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2408029420 con 0x557f0eeee170 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f242099e740 10 client.25792 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f242099e740 10 client.25792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f242099e740 20 client.25792 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f242099e740 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557f0ef095b0 0x557f0ef0ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f242099e740 4 client.25792 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f242099e740 1 -- 172.21.15.27:0/165782353 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f0ee65f10 con 0x557f0ef095b0 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.753+0000 7f2416ffd640 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557f0ef095b0 0x557f0ef0ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.754+0000 7f2416ffd640 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557f0ef095b0 0x557f0ef0ba20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795dfce62d96d2de server_cookie=4af687a6d2aebd1e in_seq=0 out_seq=0 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.754+0000 7f23f6ffd640 10 client.25792 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f23f6ffd640 1 -- 172.21.15.27:0/165782353 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f0ee65f10 con 0x557f0ef095b0 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f23f6ffd640 10 client.25792 handle_command_reply: tid=0 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 client.25792 shutdown 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 -- 172.21.15.27:0/165782353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 msgr2=0x557f0eef62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eef62f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f240801e950 tx=0x7f2408004730 comp rx=0 tx=0).stop 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 -- 172.21.15.27:0/165782353 shutdown_connections 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557f0ef095b0 0x557f0ef0ba20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 0x557f0eef6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f23f6ffd640 0 client.25792 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeeea90 0x557f0eef6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 --2- 172.21.15.27:0/165782353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eef62f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 -- 172.21.15.27:0/165782353 >> 172.21.15.27:0/165782353 conn(0x557f0eeea680 msgr2=0x557f0eef0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 -- 172.21.15.27:0/165782353 shutdown_connections 2024-06-06T08:44:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 1 -- 172.21.15.27:0/165782353 wait complete. 2024-06-06T08:44:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.756+0000 7f242099e740 20 client.25792 trim_cache size 0 max 16384 2024-06-06T08:44:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.771+0000 7f242099e740 1 Processor -- start 2024-06-06T08:44:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.771+0000 7f242099e740 1 -- start start 2024-06-06T08:44:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f242099e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eef0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f242099e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeeea90 0x557f0eef8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f242099e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 0x557f0ef11330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f242099e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24181b3a10 con 0x557f0eeee170 2024-06-06T08:44:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f242099e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f24181b1220 con 0x557f0eef9350 2024-06-06T08:44:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f242099e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f24181af070 con 0x557f0eeeea90 2024-06-06T08:44:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f24177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eef0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f24177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eef0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55954/0 (socket says 172.21.15.27:55954) 2024-06-06T08:44:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f2417fff640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 0x557f0ef11330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f24177fe640 1 -- 172.21.15.27:0/4264902034 learned_addr learned my addr 172.21.15.27:0/4264902034 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f2416ffd640 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeeea90 0x557f0eef8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f24177fe640 1 -- 172.21.15.27:0/4264902034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeeea90 msgr2=0x557f0eef8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f24177fe640 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeeea90 0x557f0eef8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f24177fe640 1 -- 172.21.15.27:0/4264902034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 msgr2=0x557f0ef11330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f24177fe640 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 0x557f0ef11330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f24177fe640 1 -- 172.21.15.27:0/4264902034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0eef10c0 con 0x557f0eeee170 2024-06-06T08:44:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f2416ffd640 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeeea90 0x557f0eef8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f2417fff640 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 0x557f0ef11330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f24177fe640 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eef0b80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f240801ee90 tx=0x7f2408012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cce1540c87dcde server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.772+0000 7f23fd7fa640 1 -- 172.21.15.27:0/4264902034 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240803c070 con 0x557f0eeee170 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f23fd7fa640 1 -- 172.21.15.27:0/4264902034 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2408029e50 con 0x557f0eeee170 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f23fd7fa640 1 -- 172.21.15.27:0/4264902034 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408003850 con 0x557f0eeee170 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f242099e740 1 -- 172.21.15.27:0/4264902034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 msgr2=0x557f0eef0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f242099e740 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eef0b80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f240801ee90 tx=0x7f2408012d30 comp rx=0 tx=0).stop 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f242099e740 1 -- 172.21.15.27:0/4264902034 shutdown_connections 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f242099e740 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eef9350 0x557f0ef11330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f242099e740 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeeea90 0x557f0eef8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f242099e740 1 --2- 172.21.15.27:0/4264902034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eeee170 0x557f0eef0b80 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f242099e740 1 -- 172.21.15.27:0/4264902034 >> 172.21.15.27:0/4264902034 conn(0x557f0eeea680 msgr2=0x557f0eef7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f242099e740 1 -- 172.21.15.27:0/4264902034 shutdown_connections 2024-06-06T08:44:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.773+0000 7f242099e740 1 -- 172.21.15.27:0/4264902034 wait complete. 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f242099e740 1 -- start start 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f242099e740 1 Processor -- start 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f242099e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eefd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f242099e740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eefd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f242099e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eef9350 0x557f0eefda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f242099e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24181af6e0 con 0x557f0eef9350 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f242099e740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f24181b0b50 con 0x557f0eeeea90 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f242099e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f24181afdb0 con 0x557f0eeee170 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f242099e740 10 client.0 _ll_register_callbacks cb 0x7f24080098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f24177fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eefd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f2417fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eef9350 0x557f0eefda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f2416ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eefd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f2417fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eef9350 0x557f0eefda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55966/0 (socket says 172.21.15.27:55966) 2024-06-06T08:44:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f24177fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eefd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:59844/0 (socket says 172.21.15.27:59844) 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f2416ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eefd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:60820/0 (socket says 172.21.15.27:60820) 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.774+0000 7f2417fff640 1 -- 172.21.15.27:0/202968465 learned_addr learned my addr 172.21.15.27:0/202968465 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f2417fff640 1 -- 172.21.15.27:0/202968465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 msgr2=0x557f0eefd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f2417fff640 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eefd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f2417fff640 1 -- 172.21.15.27:0/202968465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 msgr2=0x557f0eefd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f2417fff640 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eefd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f2417fff640 1 -- 172.21.15.27:0/202968465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0ef73300 con 0x557f0eef9350 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f24177fe640 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eefd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f2416ffd640 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eefd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f2417fff640 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eef9350 0x557f0eefda80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f24100316f0 tx=0x7f2410031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c504436a21be1538 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f23f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f23f6ffd640 1 -- 172.21.15.27:0/202968465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241004a040 con 0x557f0eef9350 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f23f6ffd640 1 -- 172.21.15.27:0/202968465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f2410031790 con 0x557f0eef9350 2024-06-06T08:44:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f242099e740 1 -- 172.21.15.27:0/202968465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2408029e50 con 0x557f0eef9350 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f23f6ffd640 1 -- 172.21.15.27:0/202968465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2410004b40 con 0x557f0eef9350 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f242099e740 10 client.16317 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f242099e740 1 -- 172.21.15.27:0/202968465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557f0ef74550 con 0x557f0eef9350 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.775+0000 7f23f6ffd640 1 -- 172.21.15.27:0/202968465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2408029e50 con 0x557f0eef9350 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f23f6ffd640 1 -- 172.21.15.27:0/202968465 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f241004d030 con 0x557f0eef9350 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f242099e740 10 client.16317 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f242099e740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f242099e740 20 client.16317 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f242099e740 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557f0ef74830 0x557f0ef76c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f242099e740 4 client.16317 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f242099e740 1 -- 172.21.15.27:0/202968465 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f0ee5fc20 con 0x557f0ef74830 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f24177fe640 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557f0ef74830 0x557f0ef76c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f23f6ffd640 10 client.16317 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:01.776+0000 7f24177fe640 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557f0ef74830 0x557f0ef76c20 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8098561420176d3b server_cookie=90362b3ee7cc1cae in_seq=0 out_seq=0 2024-06-06T08:44:01.953 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:44:01.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:01 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1407567861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:44:01.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:01 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1407567861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:44:02.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:01 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1407567861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:44:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f23f6ffd640 1 -- 172.21.15.27:0/202968465 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557f0ee5fc20 con 0x557f0ef74830 2024-06-06T08:44:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f23f6ffd640 10 client.16317 handle_command_reply: tid=0 2024-06-06T08:44:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 client.16317 shutdown 2024-06-06T08:44:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 -- 172.21.15.27:0/202968465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eef9350 msgr2=0x557f0eefda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eef9350 0x557f0eefda80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f24100316f0 tx=0x7f2410031680 comp rx=0 tx=0).stop 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 -- 172.21.15.27:0/202968465 shutdown_connections 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557f0ef74830 0x557f0ef76c20 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f0eef9350 0x557f0eefda80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f0eeeea90 0x557f0eefd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 --2- 172.21.15.27:0/202968465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557f0eeee170 0x557f0eefd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 -- 172.21.15.27:0/202968465 >> 172.21.15.27:0/202968465 conn(0x557f0eeea680 msgr2=0x557f0eee97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f23f6ffd640 0 client.16317 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 -- 172.21.15.27:0/202968465 shutdown_connections 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 1 -- 172.21.15.27:0/202968465 wait complete. 2024-06-06T08:44:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.050+0000 7f242099e740 20 client.16317 trim_cache size 0 max 16384 2024-06-06T08:44:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:44:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:44:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:44:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 -- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24000a2da0 msgr2=0x7f24000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24000a2da0 0x7f24000a5260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x557f0eeef710 tx=0x7f2408005f90 comp rx=0 tx=0).stop 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 -- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418101540 msgr2=0x7f2418197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418101540 0x7f2418197280 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f240c00ea60 tx=0x7f240c00ef20 comp rx=0 tx=0).stop 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 -- 172.21.15.27:0/2947155327 shutdown_connections 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f24000a2da0 0x7f24000a5260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24181069c0 0x7f24181977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418101540 0x7f2418197280 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 --2- 172.21.15.27:0/2947155327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2418100b80 0x7f2418196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 -- 172.21.15.27:0/2947155327 >> 172.21.15.27:0/2947155327 conn(0x7f24180775f0 msgr2=0x7f24180ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 -- 172.21.15.27:0/2947155327 shutdown_connections 2024-06-06T08:44:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:02.051+0000 7f241dc93640 1 -- 172.21.15.27:0/2947155327 wait complete. 2024-06-06T08:44:02.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:02 smithi027 ceph-mon[32335]: pgmap v116: 129 pgs: 129 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 MiB/s rd, 140 MiB/s wr, 363 op/s 2024-06-06T08:44:02.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:02 smithi203 ceph-mon[36579]: pgmap v116: 129 pgs: 129 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 MiB/s rd, 140 MiB/s wr, 363 op/s 2024-06-06T08:44:03.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:02 smithi176 ceph-mon[36786]: pgmap v116: 129 pgs: 129 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 MiB/s rd, 140 MiB/s wr, 363 op/s 2024-06-06T08:44:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.849+0000 7f1317f57700 1 -- 172.21.15.27:0/1926053376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006b3c0 msgr2=0x7f131010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.849+0000 7f1317f57700 1 --2- 172.21.15.27:0/1926053376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006b3c0 0x7f131010f310 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f130800b0a0 tx=0x7f130800b3b0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.849+0000 7f1317f57700 1 -- 172.21.15.27:0/1926053376 shutdown_connections 2024-06-06T08:44:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.849+0000 7f1317f57700 1 --2- 172.21.15.27:0/1926053376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006b3c0 0x7f131010f310 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.849+0000 7f1317f57700 1 --2- 172.21.15.27:0/1926053376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101084f0 0x7f13101088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.849+0000 7f1317f57700 1 --2- 172.21.15.27:0/1926053376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131006aa00 0x7f131006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.849+0000 7f1317f57700 1 -- 172.21.15.27:0/1926053376 >> 172.21.15.27:0/1926053376 conn(0x7f1310078300 msgr2=0x7f1310078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.849+0000 7f1317f57700 1 -- 172.21.15.27:0/1926053376 shutdown_connections 2024-06-06T08:44:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.849+0000 7f1317f57700 1 -- 172.21.15.27:0/1926053376 wait complete. 2024-06-06T08:44:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1317f57700 1 Processor -- start 2024-06-06T08:44:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1317f57700 1 -- start start 2024-06-06T08:44:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1317f57700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006aa00 0x7f131019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1317f57700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131006b3c0 0x7f131019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1317f57700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101084f0 0x7f1310198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1317f57700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1310112750 con 0x7f131006aa00 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1317f57700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f13101125d0 con 0x7f13101084f0 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1317f57700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13101128d0 con 0x7f131006b3c0 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1315cf3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006aa00 0x7f131019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f13154f2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131006b3c0 0x7f131019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1315cf3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006aa00 0x7f131019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55976/0 (socket says 172.21.15.27:55976) 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f13154f2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131006b3c0 0x7f131019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:59876/0 (socket says 172.21.15.27:59876) 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f1315cf3700 1 -- 172.21.15.27:0/1096357772 learned_addr learned my addr 172.21.15.27:0/1096357772 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.850+0000 7f13164f4700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101084f0 0x7f1310198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1315cf3700 1 -- 172.21.15.27:0/1096357772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131006b3c0 msgr2=0x7f131019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1315cf3700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131006b3c0 0x7f131019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1315cf3700 1 -- 172.21.15.27:0/1096357772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101084f0 msgr2=0x7f1310198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1315cf3700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101084f0 0x7f1310198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1315cf3700 1 -- 172.21.15.27:0/1096357772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13101990d0 con 0x7f131006aa00 2024-06-06T08:44:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1315cf3700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006aa00 0x7f131019e7b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d950 tx=0x7f130c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1306ffd700 1 -- 172.21.15.27:0/1096357772 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c016070 con 0x7f131006aa00 2024-06-06T08:44:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1317f57700 1 -- 172.21.15.27:0/1096357772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13101993c0 con 0x7f131006aa00 2024-06-06T08:44:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1306ffd700 1 -- 172.21.15.27:0/1096357772 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c00fd70 con 0x7f131006aa00 2024-06-06T08:44:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1306ffd700 1 -- 172.21.15.27:0/1096357772 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c00e610 con 0x7f131006aa00 2024-06-06T08:44:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.851+0000 7f1317f57700 1 -- 172.21.15.27:0/1096357772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1310101380 con 0x7f131006aa00 2024-06-06T08:44:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.853+0000 7f1306ffd700 1 -- 172.21.15.27:0/1096357772 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f130c019070 con 0x7f131006aa00 2024-06-06T08:44:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.854+0000 7f1306ffd700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f12fc0a2d90 0x7f12fc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.854+0000 7f13154f2700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f12fc0a2d90 0x7f12fc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.854+0000 7f1306ffd700 1 -- 172.21.15.27:0/1096357772 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f130c0cf220 con 0x7f131006aa00 2024-06-06T08:44:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.854+0000 7f13154f2700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f12fc0a2d90 0x7f12fc0a5250 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1310078890 tx=0x7f1300012040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13227dab80 1 Processor -- start 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13227dab80 1 -- start start 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13227dab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddfe0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13227dab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13227dab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfdaf20 0x556eddfdb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13227dab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556eddf2c810 con 0x556eddfe0f60 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13227dab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556edded4720 con 0x556eddfe0640 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13227dab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556eddf287d0 con 0x556eddfdaf20 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13154f2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13154f2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfda9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55992/0 (socket says 172.21.15.27:55992) 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13154f2700 1 -- 172.21.15.27:0/3816289609 learned_addr learned my addr 172.21.15.27:0/3816289609 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f13164f4700 1 --2- 172.21.15.27:0/3816289609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfdaf20 0x556eddfdb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.855+0000 7f1315cf3700 1 --2- 172.21.15.27:0/3816289609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddfe0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.856+0000 7f13154f2700 1 -- 172.21.15.27:0/3816289609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfdaf20 msgr2=0x556eddfdb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.856+0000 7f13154f2700 1 --2- 172.21.15.27:0/3816289609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfdaf20 0x556eddfdb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.856+0000 7f13154f2700 1 -- 172.21.15.27:0/3816289609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 msgr2=0x556eddfe0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.856+0000 7f13154f2700 1 --2- 172.21.15.27:0/3816289609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddfe0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.856+0000 7f13154f2700 1 -- 172.21.15.27:0/3816289609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eddecaf50 con 0x556eddfe0f60 2024-06-06T08:44:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.856+0000 7f13154f2700 1 --2- 172.21.15.27:0/3816289609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfda9e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f130001c520 tx=0x7f1300018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168758f645a63384 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.856+0000 7f12fbfff700 1 -- 172.21.15.27:0/3816289609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300022040 con 0x556eddfe0f60 2024-06-06T08:44:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.856+0000 7f12fbfff700 1 -- 172.21.15.27:0/3816289609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1300004500 con 0x556eddfe0f60 2024-06-06T08:44:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.856+0000 7f12fbfff700 1 -- 172.21.15.27:0/3816289609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300003e50 con 0x556eddfe0f60 2024-06-06T08:44:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 -- 172.21.15.27:0/3816289609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 msgr2=0x556eddfda9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 --2- 172.21.15.27:0/3816289609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfda9e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f130001c520 tx=0x7f1300018000 comp rx=0 tx=0).stop 2024-06-06T08:44:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 -- 172.21.15.27:0/3816289609 shutdown_connections 2024-06-06T08:44:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 --2- 172.21.15.27:0/3816289609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfdaf20 0x556eddfdb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 --2- 172.21.15.27:0/3816289609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddfe0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 --2- 172.21.15.27:0/3816289609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfda9e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 -- 172.21.15.27:0/3816289609 >> 172.21.15.27:0/3816289609 conn(0x556eddf05cc0 msgr2=0x556eddfd5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 -- 172.21.15.27:0/3816289609 shutdown_connections 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 -- 172.21.15.27:0/3816289609 wait complete. 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.857+0000 7f13227dab80 1 -- start start 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13227dab80 1 Processor -- start 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13227dab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddfe6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13227dab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddfe4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13227dab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 0x556eddfe4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13227dab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556eddfe7970 con 0x556eddfdaf20 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13227dab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556eddeccbf0 con 0x556eddfe0640 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13227dab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556eddfe7af0 con 0x556eddfe0f60 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13227dab80 10 client.0 _ll_register_callbacks cb 0x556eddec6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f1315cf3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddfe6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f1315cf3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddfe6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56000/0 (socket says 172.21.15.27:56000) 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f1315cf3700 1 -- 172.21.15.27:0/1091602921 learned_addr learned my addr 172.21.15.27:0/1091602921 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13154f2700 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddfe4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.858+0000 7f13164f4700 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 0x556eddfe4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f13154f2700 1 -- 172.21.15.27:0/1091602921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 msgr2=0x556eddfe4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f13154f2700 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 0x556eddfe4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f13154f2700 1 -- 172.21.15.27:0/1091602921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 msgr2=0x556eddfe6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f13154f2700 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddfe6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f13154f2700 1 -- 172.21.15.27:0/1091602921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556edde7b8d0 con 0x556eddfe0640 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f1315cf3700 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddfe6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f13154f2700 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddfe4030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1300012010 tx=0x7f13000057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba38a74251fa2cee server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f12f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f12f97fa700 1 -- 172.21.15.27:0/1091602921 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300034070 con 0x556eddfe0640 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f13227dab80 1 -- 172.21.15.27:0/1091602921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1300004500 con 0x556eddfe0640 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f12f97fa700 1 -- 172.21.15.27:0/1091602921 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f1300003a90 con 0x556eddfe0640 2024-06-06T08:44:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f12f97fa700 1 -- 172.21.15.27:0/1091602921 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1300022040 con 0x556eddfe0640 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f13227dab80 10 client.25826 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f13227dab80 1 -- 172.21.15.27:0/1091602921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556eddff22d0 con 0x556eddfe0640 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.859+0000 7f12f97fa700 1 -- 172.21.15.27:0/1091602921 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1300004500 con 0x556eddfe0640 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f12f97fa700 1 -- 172.21.15.27:0/1091602921 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f130002f8f0 con 0x556eddfe0640 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f13227dab80 10 client.25826 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f13227dab80 10 client.25826 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f13227dab80 20 client.25826 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f13227dab80 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556eddff4860 0x556eddff6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f13227dab80 4 client.25826 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f13227dab80 1 -- 172.21.15.27:0/1091602921 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eddf39d10 con 0x556eddff4860 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f1315cf3700 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556eddff4860 0x556eddff6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f12f97fa700 10 client.25826 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.860+0000 7f1315cf3700 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556eddff4860 0x556eddff6c50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0c09736e5104184 server_cookie=c5519261c77f8f92 in_seq=0 out_seq=0 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f12f97fa700 1 -- 172.21.15.27:0/1091602921 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556eddf39d10 con 0x556eddff4860 2024-06-06T08:44:03.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f12f97fa700 10 client.25826 handle_command_reply: tid=0 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 client.25826 shutdown 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 -- 172.21.15.27:0/1091602921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 msgr2=0x556eddfe4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddfe4030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1300012010 tx=0x7f13000057a0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 -- 172.21.15.27:0/1091602921 shutdown_connections 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556eddff4860 0x556eddff6c50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 0x556eddfe4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddfe6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 --2- 172.21.15.27:0/1091602921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddfe4030 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f12f97fa700 0 client.25826 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 -- 172.21.15.27:0/1091602921 >> 172.21.15.27:0/1091602921 conn(0x556eddf05cc0 msgr2=0x556eddfd55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 -- 172.21.15.27:0/1091602921 shutdown_connections 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 1 -- 172.21.15.27:0/1091602921 wait complete. 2024-06-06T08:44:03.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.861+0000 7f13227dab80 20 client.25826 trim_cache size 0 max 16384 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.877+0000 7f13227dab80 1 Processor -- start 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.877+0000 7f13227dab80 1 -- start start 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.877+0000 7f13227dab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfdaf20 0x556eddfda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.877+0000 7f13227dab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0640 0x556eddfe1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.877+0000 7f13227dab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfcee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.877+0000 7f13227dab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f13101aed40 con 0x556eddfe0f60 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.877+0000 7f13227dab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f13101ac510 con 0x556eddfdaf20 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.877+0000 7f13227dab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13101aa360 con 0x556eddfe0640 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f1315cf3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfdaf20 0x556eddfda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f13164f4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfcee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f1315cf3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfdaf20 0x556eddfda9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:60848/0 (socket says 172.21.15.27:60848) 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f1315cf3700 1 -- 172.21.15.27:0/162433701 learned_addr learned my addr 172.21.15.27:0/162433701 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:03.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f13154f2700 1 --2- 172.21.15.27:0/162433701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0640 0x556eddfe1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f1315cf3700 1 -- 172.21.15.27:0/162433701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0640 msgr2=0x556eddfe1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f1315cf3700 1 --2- 172.21.15.27:0/162433701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0640 0x556eddfe1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f1315cf3700 1 -- 172.21.15.27:0/162433701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 msgr2=0x556eddfcee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f1315cf3700 1 --2- 172.21.15.27:0/162433701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfcee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f1315cf3700 1 -- 172.21.15.27:0/162433701 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eddfd6500 con 0x556eddfdaf20 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f13164f4700 1 --2- 172.21.15.27:0/162433701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfcee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f1315cf3700 1 --2- 172.21.15.27:0/162433701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfdaf20 0x556eddfda9e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f130c0cc660 tx=0x7f130c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ca17fa78b2d215 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f12fb7fe700 1 -- 172.21.15.27:0/162433701 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c014070 con 0x556eddfdaf20 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f12fb7fe700 1 -- 172.21.15.27:0/162433701 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c09f730 con 0x556eddfdaf20 2024-06-06T08:44:03.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.878+0000 7f12fb7fe700 1 -- 172.21.15.27:0/162433701 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c020040 con 0x556eddfdaf20 2024-06-06T08:44:03.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.879+0000 7f13227dab80 1 -- 172.21.15.27:0/162433701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfdaf20 msgr2=0x556eddfda9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.879+0000 7f13227dab80 1 --2- 172.21.15.27:0/162433701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfdaf20 0x556eddfda9e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f130c0cc660 tx=0x7f130c000bc0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.879+0000 7f13227dab80 1 -- 172.21.15.27:0/162433701 shutdown_connections 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.879+0000 7f13227dab80 1 --2- 172.21.15.27:0/162433701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfe0f60 0x556eddfcee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.879+0000 7f13227dab80 1 --2- 172.21.15.27:0/162433701 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfdaf20 0x556eddfda9e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.879+0000 7f13227dab80 1 --2- 172.21.15.27:0/162433701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0640 0x556eddfe1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.879+0000 7f13227dab80 1 -- 172.21.15.27:0/162433701 >> 172.21.15.27:0/162433701 conn(0x556eddf05cc0 msgr2=0x556eddfe1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.879+0000 7f13227dab80 1 -- 172.21.15.27:0/162433701 shutdown_connections 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.879+0000 7f13227dab80 1 -- 172.21.15.27:0/162433701 wait complete. 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13227dab80 1 -- start start 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13227dab80 1 Processor -- start 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13227dab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddff71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13227dab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddff7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13227dab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 0x556ede05d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13227dab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f13101aa9d0 con 0x556eddfdaf20 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13227dab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f13101abe40 con 0x556eddfe0640 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13227dab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13101ab0a0 con 0x556eddfe0f60 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13227dab80 10 client.0 _ll_register_callbacks cb 0x556eddec6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13154f2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddff7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f1315cf3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddff71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13154f2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddff7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:60858/0 (socket says 172.21.15.27:60858) 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13164f4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 0x556ede05d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.880+0000 7f13154f2700 1 -- 172.21.15.27:0/2483709800 learned_addr learned my addr 172.21.15.27:0/2483709800 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f13154f2700 1 -- 172.21.15.27:0/2483709800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 msgr2=0x556ede05d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f13154f2700 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 0x556ede05d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f13154f2700 1 -- 172.21.15.27:0/2483709800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 msgr2=0x556eddff71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f13154f2700 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddff71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f13154f2700 1 -- 172.21.15.27:0/2483709800 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ede05d700 con 0x556eddfe0640 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f1315cf3700 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddff71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f13154f2700 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddff7710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f130002f040 tx=0x7f13000038e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11933abb6b9d4061 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f12f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f12f97fa700 1 -- 172.21.15.27:0/2483709800 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300034050 con 0x556eddfe0640 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f12f97fa700 1 -- 172.21.15.27:0/2483709800 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f13000052c0 con 0x556eddfe0640 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f13227dab80 1 -- 172.21.15.27:0/2483709800 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f130c09f730 con 0x556eddfe0640 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f12f97fa700 1 -- 172.21.15.27:0/2483709800 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f130001cb10 con 0x556eddfe0640 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f13227dab80 10 client.25832 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f13227dab80 1 -- 172.21.15.27:0/2483709800 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556ede05ea00 con 0x556eddfe0640 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f12f97fa700 1 -- 172.21.15.27:0/2483709800 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c09f730 con 0x556eddfe0640 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.881+0000 7f12f97fa700 1 -- 172.21.15.27:0/2483709800 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1300043030 con 0x556eddfe0640 2024-06-06T08:44:03.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.882+0000 7f13227dab80 10 client.25832 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.882+0000 7f13227dab80 10 client.25832 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.882+0000 7f13227dab80 20 client.25832 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.882+0000 7f13227dab80 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556ede05f100 0x556ede061520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.882+0000 7f13227dab80 4 client.25832 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.882+0000 7f13227dab80 1 -- 172.21.15.27:0/2483709800 --> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556eddf34290 con 0x556ede05f100 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.882+0000 7f1315cf3700 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556ede05f100 0x556ede061520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.882+0000 7f12f97fa700 10 client.25832 ms_handle_connect on v2:172.21.15.176:6836/3697558303 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.882+0000 7f1315cf3700 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556ede05f100 0x556ede061520 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfed64c5153355b7 server_cookie=bcdfc9be1cef9b7f in_seq=0 out_seq=0 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f12f97fa700 1 -- 172.21.15.27:0/2483709800 <== mds.2 v2:172.21.15.176:6836/3697558303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25452 (crc 0 0 0) 0x556eddf34290 con 0x556ede05f100 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f12f97fa700 10 client.25832 handle_command_reply: tid=0 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 client.25832 shutdown 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 -- 172.21.15.27:0/2483709800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 msgr2=0x556eddff7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:03.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddff7710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f130002f040 tx=0x7f13000038e0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 -- 172.21.15.27:0/2483709800 shutdown_connections 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.176:6836/3697558303,v1:172.21.15.176:6837/3697558303] conn(0x556ede05f100 0x556ede061520 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556eddfe0f60 0x556ede05d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556eddfdaf20 0x556eddff71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f12f97fa700 0 client.25832 ms_handle_reset on v2:172.21.15.176:6836/3697558303 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 --2- 172.21.15.27:0/2483709800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556eddfe0640 0x556eddff7710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 -- 172.21.15.27:0/2483709800 >> 172.21.15.27:0/2483709800 conn(0x556eddf05cc0 msgr2=0x556eddfe1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 -- 172.21.15.27:0/2483709800 shutdown_connections 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 1 -- 172.21.15.27:0/2483709800 wait complete. 2024-06-06T08:44:03.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.883+0000 7f13227dab80 20 client.25832 trim_cache size 0 max 16384 2024-06-06T08:44:03.887 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-06-06T08:44:03.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.887 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.887 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 47767623, 2024-06-06T08:44:03.887 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637409, 2024-06-06T08:44:03.887 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/Makefile" 2024-06-06T08:44:03.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.887 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 85027831, 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638694, 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv01.c" 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 133985643, 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637740, 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/Makefile" 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 161656084, 2024-06-06T08:44:03.888 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638218, 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam02.c" 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 162234054, 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637257, 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_wait.te" 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 169270857, 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637307, 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrans.te" 2024-06-06T08:44:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 176095730, 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637256, 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/Makefile" 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 180101207, 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637361, 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create_parent.c" 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 299657047, 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511636876, 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/run_pec_test" 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 305991791, 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638695, 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv04.c" 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.891 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 308115055, 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637318, 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rxdir.te" 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 316212287, 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637995, 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink07.c" 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 373507260, 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637260, 2024-06-06T08:44:03.892 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat" 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 584648919, 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637253, 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_sys.te" 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 588352546, 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637240, 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setnice.te" 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 598029909, 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637739, 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap03.c" 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 617237840, 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638215, 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam03.c" 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.894 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 634432376, 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511636877, 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/pec_listener.c" 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 756978216, 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637252, 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getpgid.te" 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 776544644, 2024-06-06T08:44:03.895 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638397, 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/Makefile" 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 816944814, 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637319, 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_inherit.te" 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 923183281, 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637243, 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ioctl.te" 2024-06-06T08:44:03.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 958623052, 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638724, 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget04.c" 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 987156839, 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637316, 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsid.te" 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.897 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1059699645, 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637997, 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink06.c" 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1097443868, 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511636873, 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/event_generator.c" 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.898 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1142496099, 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637087, 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/testsuite.fns" 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1166445378, 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637311, 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_global.te" 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1217697651, 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637310, 2024-06-06T08:44:03.899 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_exectrace.te" 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1238082066, 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638398, 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind01.c" 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1256288015, 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637250, 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_stat.te" 2024-06-06T08:44:03.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1282029215, 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511636875, 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/README" 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1324215238, 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638144, 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/Makefile" 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.901 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1454934467, 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637300, 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.fc" 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1527888081, 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637313, 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsched.te" 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1621321686, 2024-06-06T08:44:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638396, 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind02.c" 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1656154362, 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637302, 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sigkill.te" 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1736825889, 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637360, 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/Makefile" 2024-06-06T08:44:03.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1743443685, 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637303, 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_open.te" 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1848518863, 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637248, 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_mkdir.te" 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.904 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1922288461, 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637238, 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rename.te" 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1922871121, 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637242, 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ipc.te" 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.905 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1954127575, 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637411, 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_raw.c" 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2055558444, 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637249, 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ptrace.te" 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.906 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2072383775, 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637244, 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setsched.te" 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2072707265, 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637308, 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_transition.te" 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2077373242, 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637088, 2024-06-06T08:44:03.907 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/hotplug.fns" 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2095107112, 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637259, 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_bounds.te" 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2229209791, 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637301, 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_fdreceive.te" 2024-06-06T08:44:03.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2262416955, 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638696, 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/Makefile" 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2264884910, 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638721, 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget01.c" 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.909 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2341245558, 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637255, 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_file.te" 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2514440219, 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638216, 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/Makefile" 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.910 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2562826071, 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637315, 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_net.te" 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2573981135, 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638692, 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv02.c" 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2652371192, 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637304, 2024-06-06T08:44:03.911 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/TODO" 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2663114964, 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638693, 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv05.c" 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2693602133, 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637245, 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_file.te" 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.912 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2694442776, 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637741, 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap02.c" 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 -- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f12fc0a2d90 msgr2=0x7f12fc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f12fc0a2d90 0x7f12fc0a5250 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1310078890 tx=0x7f1300012040 comp rx=0 tx=0).stop 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 -- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006aa00 msgr2=0x7f131019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006aa00 0x7f131019e7b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d950 tx=0x7f130c00dc60 comp rx=0 tx=0).stop 2024-06-06T08:44:03.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 -- 172.21.15.27:0/1096357772 shutdown_connections 2024-06-06T08:44:03.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f12fc0a2d90 0x7f12fc0a5250 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101084f0 0x7f1310198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f131006aa00 0x7f131019e7b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 --2- 172.21.15.27:0/1096357772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131006b3c0 0x7f131019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:03.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 -- 172.21.15.27:0/1096357772 >> 172.21.15.27:0/1096357772 conn(0x7f1310078300 msgr2=0x7f13100ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:03.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 -- 172.21.15.27:0/1096357772 shutdown_connections 2024-06-06T08:44:03.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:03.884+0000 7f1317f57700 1 -- 172.21.15.27:0/1096357772 wait complete. 2024-06-06T08:44:03.914 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.914 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2763601317, 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637239, 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setpgid.te" 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2813421309, 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638722, 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/Makefile" 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2870622398, 2024-06-06T08:44:03.915 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637246, 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_readlink.te" 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2929092279, 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637408, 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_capable_net.sh" 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2941980714, 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638759, 2024-06-06T08:44:03.916 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat02.c" 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2949710311, 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637306, 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sysctl.te" 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2966475604, 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638697, 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv03.c" 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3022706740, 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511636874, 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/Makefile" 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3084360332, 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638217, 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam01.c" 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.918 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3086218559, 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637996, 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink08.c" 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3113118168, 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638723, 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget03.c" 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.919 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3131650730, 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638758, 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/Makefile" 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3140498571, 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637251, 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execute_no_trans.te" 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3145223599, 2024-06-06T08:44:03.920 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637258, 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_entrypoint.te" 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3174636827, 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637312, 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrace.te" 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3273210643, 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638757, 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat03.c" 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3280058410, 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637247, 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.if" 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3287660590, 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637993, 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink05.c" 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.922 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3311918679, 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637359, 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create.sh" 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3401160589, 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637732, 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/Makefile" 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3414252424, 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637742, 2024-06-06T08:44:03.923 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap01.c" 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3446403678, 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637994, 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/Makefile" 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3608266080, 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638756, 2024-06-06T08:44:03.924 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat01.c" 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3609443824, 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637309, 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setattr.te" 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3625324994, 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637254, 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execshare.te" 2024-06-06T08:44:03.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3711337345, 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637317, 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_create.te" 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3729648493, 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637733, 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/asyncio02.c" 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3797379556, 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638698, 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv06.c" 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4191649129, 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637410, 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_bind.c" 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.927 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4200893922, 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638725, 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget02.c" 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4208741105, 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637241, 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_link.te" 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4272616053, 2024-06-06T08:44:03.928 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637305, 2024-06-06T08:44:03.929 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_relabel.te" 2024-06-06T08:44:03.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:03.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:03.929 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:03.929 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4290071542, 2024-06-06T08:44:03.929 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638145, 2024-06-06T08:44:03.929 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/execv01.c" 2024-06-06T08:44:03.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:44:03.929 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-06-06T08:44:04.946 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:04 smithi027 ceph-mon[32335]: pgmap v117: 129 pgs: 129 active+clean; 1.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 MiB/s rd, 140 MiB/s wr, 435 op/s 2024-06-06T08:44:04.948 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T08:44:04.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:04 smithi203 ceph-mon[36579]: pgmap v117: 129 pgs: 129 active+clean; 1.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 MiB/s rd, 140 MiB/s wr, 435 op/s 2024-06-06T08:44:05.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:04 smithi176 ceph-mon[36786]: pgmap v117: 129 pgs: 129 active+clean; 1.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 MiB/s rd, 140 MiB/s wr, 435 op/s 2024-06-06T08:44:05.247 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:44:06.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:06 smithi027 ceph-mon[32335]: pgmap v118: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 126 MiB/s wr, 575 op/s 2024-06-06T08:44:07.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.028+0000 7f99b4976700 1 -- 172.21.15.27:0/1940092199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0107400 msgr2=0x7f99b01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:07.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.028+0000 7f99b4976700 1 --2- 172.21.15.27:0/1940092199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0107400 0x7f99b01077e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009990 tx=0x7f99a0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.028+0000 7f99b4976700 1 -- 172.21.15.27:0/1940092199 shutdown_connections 2024-06-06T08:44:07.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.028+0000 7f99b4976700 1 --2- 172.21.15.27:0/1940092199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99b0101d50 0x7f99b010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.028+0000 7f99b4976700 1 --2- 172.21.15.27:0/1940092199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0107400 0x7f99b01077e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.028+0000 7f99b4976700 1 --2- 172.21.15.27:0/1940092199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99b0101390 0x7f99b0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.028+0000 7f99b4976700 1 -- 172.21.15.27:0/1940092199 >> 172.21.15.27:0/1940092199 conn(0x7f99b0078340 msgr2=0x7f99b0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.028+0000 7f99b4976700 1 -- 172.21.15.27:0/1940092199 shutdown_connections 2024-06-06T08:44:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.028+0000 7f99b4976700 1 -- 172.21.15.27:0/1940092199 wait complete. 2024-06-06T08:44:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99b4976700 1 Processor -- start 2024-06-06T08:44:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99b4976700 1 -- start start 2024-06-06T08:44:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99b4976700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99b0101390 0x7f99b0197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99b4976700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0101d50 0x7f99b0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99b4976700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99b0107400 0x7f99b019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99b4976700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01116a0 con 0x7f99b0101d50 2024-06-06T08:44:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99b4976700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0111520 con 0x7f99b0101390 2024-06-06T08:44:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99b4976700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0111820 con 0x7f99b0107400 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99add9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0101d50 0x7f99b0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99add9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0101d50 0x7f99b0198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56052/0 (socket says 172.21.15.27:56052) 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99add9b700 1 -- 172.21.15.27:0/3344517451 learned_addr learned my addr 172.21.15.27:0/3344517451 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99aed9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99b0107400 0x7f99b019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99ae59c700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99b0101390 0x7f99b0197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.029+0000 7f99add9b700 1 -- 172.21.15.27:0/3344517451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99b0107400 msgr2=0x7f99b019c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f99add9b700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99b0107400 0x7f99b019c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f99add9b700 1 -- 172.21.15.27:0/3344517451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99b0101390 msgr2=0x7f99b0197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f99add9b700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99b0101390 0x7f99b0197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f99add9b700 1 -- 172.21.15.27:0/3344517451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b019cbe0 con 0x7f99b0101d50 2024-06-06T08:44:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f99aed9d700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99b0107400 0x7f99b019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f99ae59c700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99b0101390 0x7f99b0197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f99add9b700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0101d50 0x7f99b0198170 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f99a400e9a0 tx=0x7f99a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f999f7fe700 1 -- 172.21.15.27:0/3344517451 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a400ca10 con 0x7f99b0101d50 2024-06-06T08:44:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f99b4976700 1 -- 172.21.15.27:0/3344517451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b019ced0 con 0x7f99b0101d50 2024-06-06T08:44:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f999f7fe700 1 -- 172.21.15.27:0/3344517451 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a400cbb0 con 0x7f99b0101d50 2024-06-06T08:44:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f999f7fe700 1 -- 172.21.15.27:0/3344517451 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4018680 con 0x7f99b0101d50 2024-06-06T08:44:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.030+0000 7f99b4976700 1 -- 172.21.15.27:0/3344517451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b01a4790 con 0x7f99b0101d50 2024-06-06T08:44:07.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.032+0000 7f999f7fe700 1 -- 172.21.15.27:0/3344517451 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f99a4010430 con 0x7f99b0101d50 2024-06-06T08:44:07.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.033+0000 7f999f7fe700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f99980a2d90 0x7f99980a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.033+0000 7f99ae59c700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f99980a2d90 0x7f99980a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.033+0000 7f999f7fe700 1 -- 172.21.15.27:0/3344517451 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f99a40cfa50 con 0x7f99b0101d50 2024-06-06T08:44:07.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99bf1f9b80 1 Processor -- start 2024-06-06T08:44:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99ae59c700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f99980a2d90 0x7f99980a5250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f99a000b2e0 tx=0x7f99a0005960 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99bf1f9b80 1 -- start start 2024-06-06T08:44:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112640 0x5599a6112a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a610c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a610cf20 0x5599a610d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5599a605e810 con 0x5599a6112f60 2024-06-06T08:44:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5599a6006720 con 0x5599a6112640 2024-06-06T08:44:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599a605a7d0 con 0x5599a610cf20 2024-06-06T08:44:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99add9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a610c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99add9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a610c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56062/0 (socket says 172.21.15.27:56062) 2024-06-06T08:44:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99add9b700 1 -- 172.21.15.27:0/4222319287 learned_addr learned my addr 172.21.15.27:0/4222319287 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99ae59c700 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112640 0x5599a6112a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.034+0000 7f99aed9d700 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a610cf20 0x5599a610d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f99add9b700 1 -- 172.21.15.27:0/4222319287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a610cf20 msgr2=0x5599a610d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f99add9b700 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a610cf20 0x5599a610d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f99add9b700 1 -- 172.21.15.27:0/4222319287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112640 msgr2=0x5599a6112a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f99add9b700 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112640 0x5599a6112a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f99add9b700 1 -- 172.21.15.27:0/4222319287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a5ffcf50 con 0x5599a6112f60 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f99ae59c700 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112640 0x5599a6112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f99add9b700 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a610c9e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f99a4098580 tx=0x7f99a409c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff0712e656f4b32 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f999cff9700 1 -- 172.21.15.27:0/4222319287 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a40a4820 con 0x5599a6112f60 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f999cff9700 1 -- 172.21.15.27:0/4222319287 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a4099eb0 con 0x5599a6112f60 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.035+0000 7f999cff9700 1 -- 172.21.15.27:0/4222319287 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4010980 con 0x5599a6112f60 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4222319287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 msgr2=0x5599a610c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a610c9e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f99a4098580 tx=0x7f99a409c330 comp rx=0 tx=0).stop 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4222319287 shutdown_connections 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a610cf20 0x5599a610d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112640 0x5599a6112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/4222319287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a610c9e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4222319287 >> 172.21.15.27:0/4222319287 conn(0x5599a6037cc0 msgr2=0x5599a6107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4222319287 shutdown_connections 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4222319287 wait complete. 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 -- start start 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.036+0000 7f99bf1f9b80 1 Processor -- start 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a6118de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a61160e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6116620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5599a61199d0 con 0x5599a6112f60 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5599a5ffebf0 con 0x5599a610cf20 2024-06-06T08:44:07.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599a6119b50 con 0x5599a6112640 2024-06-06T08:44:07.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99bf1f9b80 10 client.0 _ll_register_callbacks cb 0x5599a5ff8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:07.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99aed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6116620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99aed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6116620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56070/0 (socket says 172.21.15.27:56070) 2024-06-06T08:44:07.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99aed9d700 1 -- 172.21.15.27:0/470503780 learned_addr learned my addr 172.21.15.27:0/470503780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:07.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99ae59c700 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a6118de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99add9b700 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a61160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99aed9d700 1 -- 172.21.15.27:0/470503780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 msgr2=0x5599a61160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:07.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99aed9d700 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a61160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99aed9d700 1 -- 172.21.15.27:0/470503780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 msgr2=0x5599a6118de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99aed9d700 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a6118de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99aed9d700 1 -- 172.21.15.27:0/470503780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a5fad8d0 con 0x5599a6112f60 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99ae59c700 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a6118de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f99aed9d700 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6116620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f99a8003fd0 tx=0x7f99a8004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c0fbb5369861f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.037+0000 7f9995ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f9995ffb700 1 -- 172.21.15.27:0/470503780 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a80048c0 con 0x5599a6112f60 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f9995ffb700 1 -- 172.21.15.27:0/470503780 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f99a8007500 con 0x5599a6112f60 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/470503780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a4099eb0 con 0x5599a6112f60 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f9995ffb700 1 -- 172.21.15.27:0/470503780 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f99a8007700 con 0x5599a6112f60 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f99bf1f9b80 10 client.16347 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/470503780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599a61244c0 con 0x5599a6112f60 2024-06-06T08:44:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f9995ffb700 1 -- 172.21.15.27:0/470503780 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a4099eb0 con 0x5599a6112f60 2024-06-06T08:44:07.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f9995ffb700 1 -- 172.21.15.27:0/470503780 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f99a801f070 con 0x5599a6112f60 2024-06-06T08:44:07.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f99bf1f9b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:07.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f99bf1f9b80 10 client.16347 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:44:07.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f99bf1f9b80 20 client.16347 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:07.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x5599a6126b50 0x5599a6128fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f99bf1f9b80 4 client.16347 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:44:07.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.038+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/470503780 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599a606bd10 con 0x5599a6126b50 2024-06-06T08:44:07.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.039+0000 7f99ae59c700 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x5599a6126b50 0x5599a6128fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.039+0000 7f99ae59c700 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x5599a6126b50 0x5599a6128fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9928f152395561ff server_cookie=e86ebaffd9b68ce8 in_seq=0 out_seq=0 2024-06-06T08:44:07.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.039+0000 7f9995ffb700 10 client.16347 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:44:07.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f9995ffb700 1 -- 172.21.15.27:0/470503780 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5599a606bd10 con 0x5599a6126b50 2024-06-06T08:44:07.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f9995ffb700 10 client.16347 handle_command_reply: tid=0 2024-06-06T08:44:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f99bf1f9b80 1 client.16347 shutdown 2024-06-06T08:44:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/470503780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 msgr2=0x5599a6116620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6116620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f99a8003fd0 tx=0x7f99a8004320 comp rx=0 tx=0).stop 2024-06-06T08:44:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/470503780 shutdown_connections 2024-06-06T08:44:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6116620 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x5599a6126b50 0x5599a6128fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a6118de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/470503780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a61160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/470503780 >> 172.21.15.27:0/470503780 conn(0x5599a6037cc0 msgr2=0x5599a61075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.041+0000 7f9995ffb700 0 client.16347 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:44:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.042+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/470503780 shutdown_connections 2024-06-06T08:44:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.042+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/470503780 wait complete. 2024-06-06T08:44:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.042+0000 7f99bf1f9b80 20 client.16347 trim_cache size 0 max 16384 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.058+0000 7f99bf1f9b80 1 Processor -- start 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.058+0000 7f99bf1f9b80 1 -- start start 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a610c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a61107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6122ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01aedb0 con 0x5599a6112f60 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01ac580 con 0x5599a610cf20 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01aa3d0 con 0x5599a6112640 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99aed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6122ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99add9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a61107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99ae59c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a610c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99aed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6122ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56086/0 (socket says 172.21.15.27:56086) 2024-06-06T08:44:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99aed9d700 1 -- 172.21.15.27:0/4007160717 learned_addr learned my addr 172.21.15.27:0/4007160717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99ae59c700 1 -- 172.21.15.27:0/4007160717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 msgr2=0x5599a61107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99ae59c700 1 --2- 172.21.15.27:0/4007160717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a61107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99ae59c700 1 -- 172.21.15.27:0/4007160717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 msgr2=0x5599a6122ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99ae59c700 1 --2- 172.21.15.27:0/4007160717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6122ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99ae59c700 1 -- 172.21.15.27:0/4007160717 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a6108500 con 0x5599a610cf20 2024-06-06T08:44:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99aed9d700 1 --2- 172.21.15.27:0/4007160717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6122ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99add9b700 1 --2- 172.21.15.27:0/4007160717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a61107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:07.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.059+0000 7f99ae59c700 1 --2- 172.21.15.27:0/4007160717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a610c9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f99a00059d0 tx=0x7f99a0020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b70cefaa08640e18 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:07.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f9997fff700 1 -- 172.21.15.27:0/4007160717 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a003c670 con 0x5599a610cf20 2024-06-06T08:44:07.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f9997fff700 1 -- 172.21.15.27:0/4007160717 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a003ccf0 con 0x5599a610cf20 2024-06-06T08:44:07.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f9997fff700 1 -- 172.21.15.27:0/4007160717 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a000fc50 con 0x5599a610cf20 2024-06-06T08:44:07.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4007160717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 msgr2=0x5599a610c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:07.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/4007160717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a610c9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f99a00059d0 tx=0x7f99a0020000 comp rx=0 tx=0).stop 2024-06-06T08:44:07.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4007160717 shutdown_connections 2024-06-06T08:44:07.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/4007160717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a6112f60 0x5599a6122ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/4007160717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a610cf20 0x5599a610c9e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/4007160717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a61107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4007160717 >> 172.21.15.27:0/4007160717 conn(0x5599a6037cc0 msgr2=0x5599a61100c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4007160717 shutdown_connections 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.060+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/4007160717 wait complete. 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99bf1f9b80 1 -- start start 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99bf1f9b80 1 Processor -- start 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a610cf20 0x5599a611ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a611f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99bf1f9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112f60 0x5599a612d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01aaa40 con 0x5599a610cf20 2024-06-06T08:44:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01abeb0 con 0x5599a6112f60 2024-06-06T08:44:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99bf1f9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01ab110 con 0x5599a6112640 2024-06-06T08:44:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99bf1f9b80 10 client.0 _ll_register_callbacks cb 0x5599a5ff8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99ae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a610cf20 0x5599a611ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99ae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a610cf20 0x5599a611ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56096/0 (socket says 172.21.15.27:56096) 2024-06-06T08:44:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99add9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a611f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99ae59c700 1 -- 172.21.15.27:0/2838942308 learned_addr learned my addr 172.21.15.27:0/2838942308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.061+0000 7f99aed9d700 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112f60 0x5599a612d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f99ae59c700 1 -- 172.21.15.27:0/2838942308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 msgr2=0x5599a611f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f99ae59c700 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a611f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f99ae59c700 1 -- 172.21.15.27:0/2838942308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112f60 msgr2=0x5599a612d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f99ae59c700 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112f60 0x5599a612d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f99ae59c700 1 -- 172.21.15.27:0/2838942308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a612dea0 con 0x5599a610cf20 2024-06-06T08:44:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f99ae59c700 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a610cf20 0x5599a611ee10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009990 tx=0x7f99a003cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45e0dd304466251 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f9995ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f9995ffb700 1 -- 172.21.15.27:0/2838942308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0040070 con 0x5599a610cf20 2024-06-06T08:44:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f9995ffb700 1 -- 172.21.15.27:0/2838942308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f99a000faf0 con 0x5599a610cf20 2024-06-06T08:44:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/2838942308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a003ccf0 con 0x5599a610cf20 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f9995ffb700 1 -- 172.21.15.27:0/2838942308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f99a004b050 con 0x5599a610cf20 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f99bf1f9b80 10 client.16353 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/2838942308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599a6191080 con 0x5599a610cf20 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.062+0000 7f9995ffb700 1 -- 172.21.15.27:0/2838942308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a003ccf0 con 0x5599a610cf20 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f9995ffb700 1 -- 172.21.15.27:0/2838942308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f99a0005280 con 0x5599a610cf20 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f99bf1f9b80 10 client.16353 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f99bf1f9b80 10 client.16353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f99bf1f9b80 20 client.16353 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x5599a61916d0 0x5599a6193af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f99bf1f9b80 4 client.16353 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:44:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/2838942308 --> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599a6066290 con 0x5599a61916d0 2024-06-06T08:44:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f99add9b700 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x5599a61916d0 0x5599a6193af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f99add9b700 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x5599a61916d0 0x5599a6193af0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cbe93c774b785f5 server_cookie=ebb777a90547d5c6 in_seq=0 out_seq=0 2024-06-06T08:44:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.063+0000 7f9995ffb700 10 client.16353 ms_handle_connect on v2:172.21.15.203:6838/693474346 2024-06-06T08:44:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.065+0000 7f9995ffb700 1 -- 172.21.15.27:0/2838942308 <== mds.1 v2:172.21.15.203:6838/693474346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22188 (crc 0 0 0) 0x5599a6066290 con 0x5599a61916d0 2024-06-06T08:44:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f9995ffb700 10 client.16353 handle_command_reply: tid=0 2024-06-06T08:44:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 client.16353 shutdown 2024-06-06T08:44:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/2838942308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a610cf20 msgr2=0x5599a611ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a610cf20 0x5599a611ee10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009990 tx=0x7f99a003cf10 comp rx=0 tx=0).stop 2024-06-06T08:44:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/2838942308 shutdown_connections 2024-06-06T08:44:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.203:6838/693474346,v1:172.21.15.203:6839/693474346] conn(0x5599a61916d0 0x5599a6193af0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5599a6112f60 0x5599a612d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5599a610cf20 0x5599a611ee10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 --2- 172.21.15.27:0/2838942308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599a6112640 0x5599a611f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/2838942308 >> 172.21.15.27:0/2838942308 conn(0x5599a6037cc0 msgr2=0x5599a6100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f9995ffb700 0 client.16353 ms_handle_reset on v2:172.21.15.203:6838/693474346 2024-06-06T08:44:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/2838942308 shutdown_connections 2024-06-06T08:44:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 1 -- 172.21.15.27:0/2838942308 wait complete. 2024-06-06T08:44:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.066+0000 7f99bf1f9b80 20 client.16353 trim_cache size 0 max 16384 2024-06-06T08:44:07.073 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-06-06T08:44:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 6100260, 2024-06-06T08:44:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638093, 2024-06-06T08:44:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/Makefile" 2024-06-06T08:44:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 51762981, 2024-06-06T08:44:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637291, 2024-06-06T08:44:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_exectrace.te" 2024-06-06T08:44:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 91794680, 2024-06-06T08:44:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638042, 2024-06-06T08:44:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir04.c" 2024-06-06T08:44:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 108932203, 2024-06-06T08:44:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637424, 2024-06-06T08:44:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_source.c" 2024-06-06T08:44:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 134153374, 2024-06-06T08:44:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637698, 2024-06-06T08:44:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd02.c" 2024-06-06T08:44:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 255132717, 2024-06-06T08:44:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637282, 2024-06-06T08:44:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_entrypoint.te" 2024-06-06T08:44:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 304354188, 2024-06-06T08:44:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638278, 2024-06-06T08:44:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit01.c" 2024-06-06T08:44:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 343542681, 2024-06-06T08:44:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637284, 2024-06-06T08:44:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sigkill.te" 2024-06-06T08:44:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 387038399, 2024-06-06T08:44:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637701, 2024-06-06T08:44:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd01.c" 2024-06-06T08:44:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 397174915, 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637261, 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5" 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 470889146, 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637270, 2024-06-06T08:44:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_readlink.te" 2024-06-06T08:44:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 481677629, 2024-06-06T08:44:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638527, 2024-06-06T08:44:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/timer_getoverrun01.c" 2024-06-06T08:44:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 485031711, 2024-06-06T08:44:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638091, 2024-06-06T08:44:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink01.c" 2024-06-06T08:44:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 682596649, 2024-06-06T08:44:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637297, 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_create.te" 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 736955901, 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638470, 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/Makefile" 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 743764364, 2024-06-06T08:44:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637700, 2024-06-06T08:44:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd03.c" 2024-06-06T08:44:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 800084220, 2024-06-06T08:44:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637273, 2024-06-06T08:44:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ptrace.te" 2024-06-06T08:44:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 818006050, 2024-06-06T08:44:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637277, 2024-06-06T08:44:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_sys.te" 2024-06-06T08:44:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 976072596, 2024-06-06T08:44:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637274, 2024-06-06T08:44:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_stat.te" 2024-06-06T08:44:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 991632441, 2024-06-06T08:44:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637267, 2024-06-06T08:44:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ioctl.te" 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1023353442, 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637266, 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ipc.te" 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1129292183, 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637422, 2024-06-06T08:44:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/Makefile" 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1148764158, 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637264, 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setnice.te" 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1170300642, 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638043, 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir05.c" 2024-06-06T08:44:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1236463581, 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637938, 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/exit_group01.c" 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1260324563, 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638041, 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir01.c" 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1304175976, 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637298, 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rxdir.te" 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1366370916, 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637263, 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setpgid.te" 2024-06-06T08:44:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1420783535, 2024-06-06T08:44:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638471, 2024-06-06T08:44:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid02.c" 2024-06-06T08:44:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1494340369, 2024-06-06T08:44:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637939, 2024-06-06T08:44:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/Makefile" 2024-06-06T08:44:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1525505568, 2024-06-06T08:44:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637910, 2024-06-06T08:44:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/Makefile" 2024-06-06T08:44:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1928030243, 2024-06-06T08:44:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638094, 2024-06-06T08:44:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink03.c" 2024-06-06T08:44:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1937982364, 2024-06-06T08:44:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637280, 2024-06-06T08:44:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/Makefile" 2024-06-06T08:44:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1957974430, 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637611, 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_01.c" 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2010698425, 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638277, 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/Makefile" 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2012713932, 2024-06-06T08:44:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637612, 2024-06-06T08:44:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_02.c" 2024-06-06T08:44:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2226241574, 2024-06-06T08:44:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637423, 2024-06-06T08:44:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_target.c" 2024-06-06T08:44:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2306999316, 2024-06-06T08:44:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637289, 2024-06-06T08:44:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_transition.te" 2024-06-06T08:44:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2350822892, 2024-06-06T08:44:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637283, 2024-06-06T08:44:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_fdreceive.te" 2024-06-06T08:44:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2404515727, 2024-06-06T08:44:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637278, 2024-06-06T08:44:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execshare.te" 2024-06-06T08:44:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2420391573, 2024-06-06T08:44:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638038, 2024-06-06T08:44:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir09.c" 2024-06-06T08:44:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2420666087, 2024-06-06T08:44:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637699, 2024-06-06T08:44:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/Makefile" 2024-06-06T08:44:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2622216601, 2024-06-06T08:44:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638092, 2024-06-06T08:44:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink05.c" 2024-06-06T08:44:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2712198141, 2024-06-06T08:44:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637610, 2024-06-06T08:44:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/Makefile" 2024-06-06T08:44:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2796125771, 2024-06-06T08:44:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638045, 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir03.c" 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2875382097, 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637299, 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_inherit.te" 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2979505787, 2024-06-06T08:44:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637279, 2024-06-06T08:44:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_file.te" 2024-06-06T08:44:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 2991166628, 2024-06-06T08:44:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637285, 2024-06-06T08:44:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_open.te" 2024-06-06T08:44:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3110686700, 2024-06-06T08:44:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637276, 2024-06-06T08:44:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getpgid.te" 2024-06-06T08:44:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3245964812, 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637269, 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_file.te" 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3282942770, 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637296, 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsid.te" 2024-06-06T08:44:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3301784038, 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637286, 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_relabel.te" 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3323297250, 2024-06-06T08:44:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638044, 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir02.c" 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3324547206, 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637281, 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_wait.te" 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3329718369, 2024-06-06T08:44:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638040, 2024-06-06T08:44:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/Makefile" 2024-06-06T08:44:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3378001524, 2024-06-06T08:44:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637290, 2024-06-06T08:44:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setattr.te" 2024-06-06T08:44:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3390846238, 2024-06-06T08:44:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638276, 2024-06-06T08:44:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit02.c" 2024-06-06T08:44:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3479105804, 2024-06-06T08:44:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637287, 2024-06-06T08:44:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sysctl.te" 2024-06-06T08:44:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3515124165, 2024-06-06T08:44:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637268, 2024-06-06T08:44:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 -- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f99980a2d90 msgr2=0x7f99980a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f99980a2d90 0x7f99980a5250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f99a000b2e0 tx=0x7f99a0005960 comp rx=0 tx=0).stop 2024-06-06T08:44:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 -- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0101d50 msgr2=0x7f99b0198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0101d50 0x7f99b0198170 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f99a400e9a0 tx=0x7f99a400ecb0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 -- 172.21.15.27:0/3344517451 shutdown_connections 2024-06-06T08:44:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99b0107400 0x7f99b019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f99980a2d90 0x7f99980a5250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f99b0101390 0x7f99b0197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 --2- 172.21.15.27:0/3344517451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99b0101d50 0x7f99b0198170 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 -- 172.21.15.27:0/3344517451 >> 172.21.15.27:0/3344517451 conn(0x7f99b0078340 msgr2=0x7f99b00fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 -- 172.21.15.27:0/3344517451 shutdown_connections 2024-06-06T08:44:07.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:07.067+0000 7f99b4976700 1 -- 172.21.15.27:0/3344517451 wait complete. 2024-06-06T08:44:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setsched.te" 2024-06-06T08:44:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3534768640, 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637835, 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork02.c" 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3566984541, 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638039, 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir08.c" 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3664273536, 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637292, 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_global.te" 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3728510284, 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637836, 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/Makefile" 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3763598819, 2024-06-06T08:44:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637265, 2024-06-06T08:44:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_link.te" 2024-06-06T08:44:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3770265328, 2024-06-06T08:44:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638469, 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid01.c" 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3793537247, 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637837, 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork01.c" 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3798228951, 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637271, 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_policy.if" 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3798660450, 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638090, 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink02.c" 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3821224945, 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637421, 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid.sh" 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3850603430, 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637293, 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrace.te" 2024-06-06T08:44:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3928767463, 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637262, 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rename.te" 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 3987881996, 2024-06-06T08:44:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637275, 2024-06-06T08:44:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execute_no_trans.te" 2024-06-06T08:44:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4016530050, 2024-06-06T08:44:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638526, 2024-06-06T08:44:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/Makefile" 2024-06-06T08:44:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4074932601, 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637272, 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_mkdir.te" 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4177101794, 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637295, 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_net.te" 2024-06-06T08:44:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4201740406, 2024-06-06T08:44:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637294, 2024-06-06T08:44:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsched.te" 2024-06-06T08:44:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4235206440, 2024-06-06T08:44:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637911, 2024-06-06T08:44:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/getsockname01.c" 2024-06-06T08:44:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4242933149, 2024-06-06T08:44:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511637288, 2024-06-06T08:44:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrans.te" 2024-06-06T08:44:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:44:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "damage_type": "backtrace", 2024-06-06T08:44:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4275440355, 2024-06-06T08:44:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 1099511638095, 2024-06-06T08:44:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink04.c" 2024-06-06T08:44:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:44:07.127 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-06-06T08:44:07.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:06 smithi203 ceph-mon[36579]: pgmap v118: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 126 MiB/s wr, 575 op/s 2024-06-06T08:44:07.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:06 smithi176 ceph-mon[36786]: pgmap v118: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 126 MiB/s wr, 575 op/s 2024-06-06T08:44:08.038 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T08:44:08.305 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:44:08.368 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-06T08:44:08.368 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-06T08:44:08.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:08 smithi027 ceph-mon[32335]: pgmap v119: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 50 MiB/s wr, 450 op/s 2024-06-06T08:44:09.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:08 smithi203 ceph-mon[36579]: pgmap v119: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 50 MiB/s wr, 450 op/s 2024-06-06T08:44:09.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:08 smithi176 ceph-mon[36786]: pgmap v119: 129 pgs: 129 active+clean; 1.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 50 MiB/s wr, 450 op/s 2024-06-06T08:44:10.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.144+0000 7f94b74f3700 1 -- 172.21.15.27:0/2003009509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b006a170 msgr2=0x7f94b006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:10.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.144+0000 7f94b74f3700 1 --2- 172.21.15.27:0/2003009509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b006a170 0x7f94b006a5f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009990 tx=0x7f94a4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.144+0000 7f94b74f3700 1 -- 172.21.15.27:0/2003009509 shutdown_connections 2024-06-06T08:44:10.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.144+0000 7f94b74f3700 1 --2- 172.21.15.27:0/2003009509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b006ab30 0x7f94b010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.144+0000 7f94b74f3700 1 --2- 172.21.15.27:0/2003009509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94b0108500 0x7f94b01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.144+0000 7f94b74f3700 1 --2- 172.21.15.27:0/2003009509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b006a170 0x7f94b006a5f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.144+0000 7f94b74f3700 1 -- 172.21.15.27:0/2003009509 >> 172.21.15.27:0/2003009509 conn(0x7f94b0078340 msgr2=0x7f94b0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.144+0000 7f94b74f3700 1 -- 172.21.15.27:0/2003009509 shutdown_connections 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.144+0000 7f94b74f3700 1 -- 172.21.15.27:0/2003009509 wait complete. 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b74f3700 1 Processor -- start 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b74f3700 1 -- start start 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b74f3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94b006a170 0x7f94b019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b74f3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b006ab30 0x7f94b019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b74f3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b0108500 0x7f94b0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b74f3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0112780 con 0x7f94b0108500 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b74f3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0112600 con 0x7f94b006a170 2024-06-06T08:44:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b74f3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0112900 con 0x7f94b006ab30 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b5a90700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b0108500 0x7f94b0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b4a8e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b006ab30 0x7f94b019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b5a90700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b0108500 0x7f94b0198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32980/0 (socket says 172.21.15.27:32980) 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.145+0000 7f94b5a90700 1 -- 172.21.15.27:0/254311579 learned_addr learned my addr 172.21.15.27:0/254311579 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b528f700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94b006a170 0x7f94b019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b4a8e700 1 -- 172.21.15.27:0/254311579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94b006a170 msgr2=0x7f94b019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94b006a170 0x7f94b019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b4a8e700 1 -- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b0108500 msgr2=0x7f94b0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b0108500 0x7f94b0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b4a8e700 1 -- 172.21.15.27:0/254311579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b0199200 con 0x7f94b006ab30 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b5a90700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b0108500 0x7f94b0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b006ab30 0x7f94b019ee20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94a40059d0 tx=0x7f94a400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94ae7fc700 1 -- 172.21.15.27:0/254311579 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4027070 con 0x7f94b006ab30 2024-06-06T08:44:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94ae7fc700 1 -- 172.21.15.27:0/254311579 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a4003d70 con 0x7f94b006ab30 2024-06-06T08:44:10.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94ae7fc700 1 -- 172.21.15.27:0/254311579 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4022e30 con 0x7f94b006ab30 2024-06-06T08:44:10.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b74f3700 1 -- 172.21.15.27:0/254311579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b0199490 con 0x7f94b006ab30 2024-06-06T08:44:10.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.146+0000 7f94b74f3700 1 -- 172.21.15.27:0/254311579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b01a57b0 con 0x7f94b006ab30 2024-06-06T08:44:10.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.152+0000 7f94c1d76b80 1 Processor -- start 2024-06-06T08:44:10.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.152+0000 7f94c1d76b80 1 -- start start 2024-06-06T08:44:10.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.153+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.153+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986fa880 0x558b986f4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.153+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b986f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.153+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558b985e6f60 con 0x558b986fa880 2024-06-06T08:44:10.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.153+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558b98645850 con 0x558b986f47a0 2024-06-06T08:44:10.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.153+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558b985ad460 con 0x558b986f9f60 2024-06-06T08:44:10.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.153+0000 7f94b528f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986fa880 0x558b986f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.153+0000 7f94b528f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986fa880 0x558b986f4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32992/0 (socket says 172.21.15.27:32992) 2024-06-06T08:44:10.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.153+0000 7f94b528f700 1 -- 172.21.15.27:0/2592534684 learned_addr learned my addr 172.21.15.27:0/2592534684 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:10.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.154+0000 7f94b5a90700 1 --2- 172.21.15.27:0/2592534684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.155+0000 7f94ae7fc700 1 -- 172.21.15.27:0/254311579 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f94a402a070 con 0x7f94b006ab30 2024-06-06T08:44:10.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.155+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/2592534684 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b986f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.156+0000 7f94b4a8e700 1 -- 172.21.15.27:0/2592534684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 msgr2=0x558b986fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:10.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.156+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/2592534684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.156+0000 7f94b4a8e700 1 -- 172.21.15.27:0/2592534684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986fa880 msgr2=0x558b986f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:10.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.156+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/2592534684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986fa880 0x558b986f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.156+0000 7f94b4a8e700 1 -- 172.21.15.27:0/2592534684 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b985948d0 con 0x558b986f47a0 2024-06-06T08:44:10.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.157+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/2592534684 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b986f4bc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94a40cd900 tx=0x7f94a40d6980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aee27c26730c578 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:10.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.157+0000 7f94977fe700 1 -- 172.21.15.27:0/2592534684 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a40d0e10 con 0x558b986f47a0 2024-06-06T08:44:10.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.158+0000 7f94977fe700 1 -- 172.21.15.27:0/2592534684 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a40e1030 con 0x558b986f47a0 2024-06-06T08:44:10.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.158+0000 7f94977fe700 1 -- 172.21.15.27:0/2592534684 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a40ddb90 con 0x558b986f47a0 2024-06-06T08:44:10.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.159+0000 7f94c1d76b80 1 -- 172.21.15.27:0/2592534684 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 msgr2=0x558b986f4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:10.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.159+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/2592534684 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b986f4bc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94a40cd900 tx=0x7f94a40d6980 comp rx=0 tx=0).stop 2024-06-06T08:44:10.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.159+0000 7f94c1d76b80 1 -- 172.21.15.27:0/2592534684 shutdown_connections 2024-06-06T08:44:10.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.159+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/2592534684 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b986f4bc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.159+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/2592534684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.159+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/2592534684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986fa880 0x558b986f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.159+0000 7f94c1d76b80 1 -- 172.21.15.27:0/2592534684 >> 172.21.15.27:0/2592534684 conn(0x558b98621080 msgr2=0x558b986ef220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.159+0000 7f94ae7fc700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f94980a2e10 0x7f94980a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.160+0000 7f94ae7fc700 1 -- 172.21.15.27:0/254311579 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f94a40d05d0 con 0x7f94b006ab30 2024-06-06T08:44:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.160+0000 7f94c1d76b80 1 -- 172.21.15.27:0/2592534684 shutdown_connections 2024-06-06T08:44:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.160+0000 7f94c1d76b80 1 -- 172.21.15.27:0/2592534684 wait complete. 2024-06-06T08:44:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.160+0000 7f94c1d76b80 1 -- start start 2024-06-06T08:44:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.160+0000 7f94c1d76b80 1 Processor -- start 2024-06-06T08:44:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.161+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f47a0 0x558b98700690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.161+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.161+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 0x558b986fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.161+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558b985e5fc0 con 0x558b986f47a0 2024-06-06T08:44:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.161+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x558b98641810 con 0x558b986fa880 2024-06-06T08:44:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.161+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558b98701010 con 0x558b986f9f60 2024-06-06T08:44:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.161+0000 7f94c1d76b80 10 client.0 _ll_register_callbacks cb 0x558b985db5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.161+0000 7f94b528f700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f94980a2e10 0x7f94980a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.161+0000 7f94b4a8e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b4a8e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:57046/0 (socket says 172.21.15.27:57046) 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b4a8e700 1 -- 172.21.15.27:0/3638713508 learned_addr learned my addr 172.21.15.27:0/3638713508 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b528f700 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f47a0 0x558b98700690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b5a90700 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 0x558b986fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b4a8e700 1 -- 172.21.15.27:0/3638713508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 msgr2=0x558b986fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 0x558b986fdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b4a8e700 1 -- 172.21.15.27:0/3638713508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f47a0 msgr2=0x558b98700690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f47a0 0x558b98700690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b4a8e700 1 -- 172.21.15.27:0/3638713508 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9860b750 con 0x558b986f9f60 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.162+0000 7f94b528f700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f94980a2e10 0x7f94980a52d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f94a00095f0 tx=0x7f94a00119f0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.163+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fd9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94a4098bc0 tx=0x7f94a40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7567cc4f0d8f3c82 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.163+0000 7f9494ff9700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:44:10.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.163+0000 7f9494ff9700 1 -- 172.21.15.27:0/3638713508 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4098a00 con 0x558b986f9f60 2024-06-06T08:44:10.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.163+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3638713508 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a40e1030 con 0x558b986f9f60 2024-06-06T08:44:10.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.163+0000 7f9494ff9700 1 -- 172.21.15.27:0/3638713508 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f94a409aa60 con 0x558b986f9f60 2024-06-06T08:44:10.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.163+0000 7f9494ff9700 1 -- 172.21.15.27:0/3638713508 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94a409c0c0 con 0x558b986f9f60 2024-06-06T08:44:10.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.163+0000 7f94c1d76b80 10 client.25846 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:10.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.163+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3638713508 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558b98713fa0 con 0x558b986f9f60 2024-06-06T08:44:10.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.164+0000 7f9494ff9700 1 -- 172.21.15.27:0/3638713508 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a40e1030 con 0x558b986f9f60 2024-06-06T08:44:10.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.165+0000 7f9494ff9700 1 -- 172.21.15.27:0/3638713508 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f94a40cdb80 con 0x558b986f9f60 2024-06-06T08:44:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.166+0000 7f94c1d76b80 10 client.25846 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.166+0000 7f94c1d76b80 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.166+0000 7f94c1d76b80 20 client.25846 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.166+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x558b98716470 0x558b98718860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.166+0000 7f94c1d76b80 4 client.25846 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:44:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.166+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3638713508 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b98652d50 con 0x558b98716470 2024-06-06T08:44:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.166+0000 7f94b528f700 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x558b98716470 0x558b98718860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.168+0000 7f94b528f700 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x558b98716470 0x558b98718860 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dcdf8f223ef84d4 server_cookie=ec7c69f90634d1b0 in_seq=0 out_seq=0 2024-06-06T08:44:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.169+0000 7f9494ff9700 10 client.25846 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.171+0000 7f9494ff9700 1 -- 172.21.15.27:0/3638713508 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558b98652d50 con 0x558b98716470 2024-06-06T08:44:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.171+0000 7f9494ff9700 10 client.25846 handle_command_reply: tid=0 2024-06-06T08:44:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.171+0000 7f94c1d76b80 1 client.25846 shutdown 2024-06-06T08:44:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3638713508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 msgr2=0x558b986fd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fd9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94a4098bc0 tx=0x7f94a40047c0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3638713508 shutdown_connections 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 0x558b986fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x558b98716470 0x558b98718860 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f47a0 0x558b98700690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3638713508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f9f60 0x558b986fd9e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f9494ff9700 0 client.25846 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3638713508 >> 172.21.15.27:0/3638713508 conn(0x558b98621080 msgr2=0x558b986ee700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3638713508 shutdown_connections 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3638713508 wait complete. 2024-06-06T08:44:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.172+0000 7f94c1d76b80 20 client.25846 trim_cache size 0 max 16384 2024-06-06T08:44:10.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.191+0000 7f94c1d76b80 1 Processor -- start 2024-06-06T08:44:10.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.191+0000 7f94c1d76b80 1 -- start start 2024-06-06T08:44:10.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f47a0 0x558b986f4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b986f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 0x558b986fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01aead0 con 0x558b986f9f60 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01ac2a0 con 0x558b986fa880 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01aa1a0 con 0x558b986f47a0 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b4a8e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b986f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b5a90700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 0x558b986fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b4a8e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b986f8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33012/0 (socket says 172.21.15.27:33012) 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b4a8e700 1 -- 172.21.15.27:0/3217090411 learned_addr learned my addr 172.21.15.27:0/3217090411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b528f700 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f47a0 0x558b986f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b4a8e700 1 -- 172.21.15.27:0/3217090411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f47a0 msgr2=0x558b986f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f47a0 0x558b986f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b4a8e700 1 -- 172.21.15.27:0/3217090411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 msgr2=0x558b986fb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 0x558b986fb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b4a8e700 1 -- 172.21.15.27:0/3217090411 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b9870a0f0 con 0x558b986f9f60 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b528f700 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f47a0 0x558b986f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.192+0000 7f94b5a90700 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 0x558b986fb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.193+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b986f8120 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f94a409c6f0 tx=0x7f94a4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c6e87f7279d504 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.193+0000 7f9496ffd700 1 -- 172.21.15.27:0/3217090411 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a409c2a0 con 0x558b986f9f60 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.193+0000 7f9496ffd700 1 -- 172.21.15.27:0/3217090411 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a40e1030 con 0x558b986f9f60 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.193+0000 7f9496ffd700 1 -- 172.21.15.27:0/3217090411 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a409c530 con 0x558b986f9f60 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.193+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3217090411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 msgr2=0x558b986f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.193+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b986f8120 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f94a409c6f0 tx=0x7f94a4009ca0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3217090411 shutdown_connections 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986fa880 0x558b986fb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986f47a0 0x558b986f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/3217090411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b986f8120 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3217090411 >> 172.21.15.27:0/3217090411 conn(0x558b98621080 msgr2=0x558b986fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3217090411 shutdown_connections 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 -- 172.21.15.27:0/3217090411 wait complete. 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 -- start start 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 Processor -- start 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b98706790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.194+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b9871d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94c1d76b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986fa880 0x558b9871dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01aa760 con 0x558b986f9f60 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01abbd0 con 0x558b986f47a0 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94c1d76b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01aae30 con 0x558b986fa880 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94c1d76b80 10 client.0 _ll_register_callbacks cb 0x558b985db5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b4a8e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b9871d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b528f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b98706790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b4a8e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b9871d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33014/0 (socket says 172.21.15.27:33014) 2024-06-06T08:44:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b4a8e700 1 -- 172.21.15.27:0/1050901810 learned_addr learned my addr 172.21.15.27:0/1050901810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b5a90700 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986fa880 0x558b9871dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b528f700 1 -- 172.21.15.27:0/1050901810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986fa880 msgr2=0x558b9871dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b528f700 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986fa880 0x558b9871dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b528f700 1 -- 172.21.15.27:0/1050901810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 msgr2=0x558b9871d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b528f700 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b9871d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b528f700 1 -- 172.21.15.27:0/1050901810 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9871e0a0 con 0x558b986f47a0 2024-06-06T08:44:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b9871d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:10.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94b528f700 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b98706790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x558b98701160 tx=0x7f94a0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105ef96cd51732f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:10.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f9494ff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:44:10.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f9494ff9700 1 -- 172.21.15.27:0/1050901810 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a003a060 con 0x558b986f47a0 2024-06-06T08:44:10.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f94c1d76b80 1 -- 172.21.15.27:0/1050901810 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a40e1030 con 0x558b986f47a0 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.195+0000 7f9494ff9700 1 -- 172.21.15.27:0/1050901810 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f94a0021c60 con 0x558b986f47a0 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f9494ff9700 1 -- 172.21.15.27:0/1050901810 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94a0041050 con 0x558b986f47a0 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f94c1d76b80 10 client.25868 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f94c1d76b80 1 -- 172.21.15.27:0/1050901810 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558b98780cd0 con 0x558b986f47a0 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f9494ff9700 1 -- 172.21.15.27:0/1050901810 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a40e1030 con 0x558b986f47a0 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f9494ff9700 1 -- 172.21.15.27:0/1050901810 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f94a0035630 con 0x558b986f47a0 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f94c1d76b80 10 client.25868 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f94c1d76b80 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f94c1d76b80 20 client.25868 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x558b987813c0 0x558b987837e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f94c1d76b80 4 client.25868 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f94c1d76b80 1 -- 172.21.15.27:0/1050901810 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b9864d2d0 con 0x558b987813c0 2024-06-06T08:44:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.196+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x558b987813c0 0x558b987837e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.197+0000 7f94b4a8e700 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x558b987813c0 0x558b987837e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=addd3013718c9fab server_cookie=f44aff6afe6562a6 in_seq=0 out_seq=0 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.197+0000 7f9494ff9700 10 client.25868 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f9494ff9700 1 -- 172.21.15.27:0/1050901810 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b9864d2d0 con 0x558b987813c0 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f9494ff9700 10 client.25868 handle_command_reply: tid=0 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 client.25868 shutdown 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 -- 172.21.15.27:0/1050901810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 msgr2=0x558b98706790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b98706790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x558b98701160 tx=0x7f94a0004750 comp rx=0 tx=0).stop 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 -- 172.21.15.27:0/1050901810 shutdown_connections 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b986fa880 0x558b9871dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558b986f47a0 0x558b98706790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x558b987813c0 0x558b987837e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 --2- 172.21.15.27:0/1050901810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558b986f9f60 0x558b9871d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f9494ff9700 0 client.25868 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 -- 172.21.15.27:0/1050901810 >> 172.21.15.27:0/1050901810 conn(0x558b98621080 msgr2=0x558b986f93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:10.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 -- 172.21.15.27:0/1050901810 shutdown_connections 2024-06-06T08:44:10.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 1 -- 172.21.15.27:0/1050901810 wait complete. 2024-06-06T08:44:10.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.198+0000 7f94c1d76b80 20 client.25868 trim_cache size 0 max 16384 2024-06-06T08:44:10.202 INFO:teuthology.orchestra.run.smithi027.stdout:[] 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 -- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f94980a2e10 msgr2=0x7f94980a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f94980a2e10 0x7f94980a52d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f94a00095f0 tx=0x7f94a00119f0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 -- 172.21.15.27:0/254311579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b006ab30 msgr2=0x7f94b019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b006ab30 0x7f94b019ee20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94a40059d0 tx=0x7f94a400bab0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 -- 172.21.15.27:0/254311579 shutdown_connections 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94b0108500 0x7f94b0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f94980a2e10 0x7f94980a52d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94b006a170 0x7f94b019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 --2- 172.21.15.27:0/254311579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94b006ab30 0x7f94b019ee20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 -- 172.21.15.27:0/254311579 >> 172.21.15.27:0/254311579 conn(0x7f94b0078340 msgr2=0x7f94b01000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 -- 172.21.15.27:0/254311579 shutdown_connections 2024-06-06T08:44:10.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:10.199+0000 7f94b74f3700 1 -- 172.21.15.27:0/254311579 wait complete. 2024-06-06T08:44:10.923 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 47767623, 'ino': 1099511637409, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/Makefile'}, {'damage_type': 'backtrace', 'id': 85027831, 'ino': 1099511638694, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv01.c'}, {'damage_type': 'backtrace', 'id': 133985643, 'ino': 1099511637740, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/Makefile'}, {'damage_type': 'backtrace', 'id': 161656084, 'ino': 1099511638218, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam02.c'}, {'damage_type': 'backtrace', 'id': 162234054, 'ino': 1099511637257, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_wait.te'}, {'damage_type': 'backtrace', 'id': 169270857, 'ino': 1099511637307, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrans.te'}, {'damage_type': 'backtrace', 'id': 176095730, 'ino': 1099511637256, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/Makefile'}, {'damage_type': 'backtrace', 'id': 180101207, 'ino': 1099511637361, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create_parent.c'}, {'damage_type': 'backtrace', 'id': 299657047, 'ino': 1099511636876, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/run_pec_test'}, {'damage_type': 'backtrace', 'id': 305991791, 'ino': 1099511638695, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv04.c'}, {'damage_type': 'backtrace', 'id': 308115055, 'ino': 1099511637318, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rxdir.te'}, {'damage_type': 'backtrace', 'id': 316212287, 'ino': 1099511637995, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink07.c'}, {'damage_type': 'backtrace', 'id': 373507260, 'ino': 1099511637260, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat'}, {'damage_type': 'backtrace', 'id': 584648919, 'ino': 1099511637253, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_sys.te'}, {'damage_type': 'backtrace', 'id': 588352546, 'ino': 1099511637240, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setnice.te'}, {'damage_type': 'backtrace', 'id': 598029909, 'ino': 1099511637739, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap03.c'}, {'damage_type': 'backtrace', 'id': 617237840, 'ino': 1099511638215, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam03.c'}, {'damage_type': 'backtrace', 'id': 634432376, 'ino': 1099511636877, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/pec_listener.c'}, {'damage_type': 'backtrace', 'id': 756978216, 'ino': 1099511637252, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getpgid.te'}, {'damage_type': 'backtrace', 'id': 776544644, 'ino': 1099511638397, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/Makefile'}, {'damage_type': 'backtrace', 'id': 816944814, 'ino': 1099511637319, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_inherit.te'}, {'damage_type': 'backtrace', 'id': 923183281, 'ino': 1099511637243, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ioctl.te'}, {'damage_type': 'backtrace', 'id': 958623052, 'ino': 1099511638724, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget04.c'}, {'damage_type': 'backtrace', 'id': 987156839, 'ino': 1099511637316, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsid.te'}, {'damage_type': 'backtrace', 'id': 1059699645, 'ino': 1099511637997, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink06.c'}, {'damage_type': 'backtrace', 'id': 1097443868, 'ino': 1099511636873, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/event_generator.c'}, {'damage_type': 'backtrace', 'id': 1142496099, 'ino': 1099511637087, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/testsuite.fns'}, {'damage_type': 'backtrace', 'id': 1166445378, 'ino': 1099511637311, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_global.te'}, {'damage_type': 'backtrace', 'id': 1217697651, 'ino': 1099511637310, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_exectrace.te'}, {'damage_type': 'backtrace', 'id': 1238082066, 'ino': 1099511638398, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind01.c'}, {'damage_type': 'backtrace', 'id': 1256288015, 'ino': 1099511637250, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_stat.te'}, {'damage_type': 'backtrace', 'id': 1282029215, 'ino': 1099511636875, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/README'}, {'damage_type': 'backtrace', 'id': 1324215238, 'ino': 1099511638144, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/Makefile'}, {'damage_type': 'backtrace', 'id': 1454934467, 'ino': 1099511637300, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.fc'}, {'damage_type': 'backtrace', 'id': 1527888081, 'ino': 1099511637313, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_getsched.te'}, {'damage_type': 'backtrace', 'id': 1621321686, 'ino': 1099511638396, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/bind/bind02.c'}, {'damage_type': 'backtrace', 'id': 1656154362, 'ino': 1099511637302, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sigkill.te'}, {'damage_type': 'backtrace', 'id': 1736825889, 'ino': 1099511637360, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/Makefile'}, {'damage_type': 'backtrace', 'id': 1743443685, 'ino': 1099511637303, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_open.te'}, {'damage_type': 'backtrace', 'id': 1848518863, 'ino': 1099511637248, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_mkdir.te'}, {'damage_type': 'backtrace', 'id': 1922288461, 'ino': 1099511637238, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_rename.te'}, {'damage_type': 'backtrace', 'id': 1922871121, 'ino': 1099511637242, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ipc.te'}, {'damage_type': 'backtrace', 'id': 1954127575, 'ino': 1099511637411, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_raw.c'}, {'damage_type': 'backtrace', 'id': 2055558444, 'ino': 1099511637249, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_ptrace.te'}, {'damage_type': 'backtrace', 'id': 2072383775, 'ino': 1099511637244, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setsched.te'}, {'damage_type': 'backtrace', 'id': 2072707265, 'ino': 1099511637308, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_transition.te'}, {'damage_type': 'backtrace', 'id': 2077373242, 'ino': 1099511637088, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/hotplug/cpu_hotplug/include/hotplug.fns'}, {'damage_type': 'backtrace', 'id': 2095107112, 'ino': 1099511637259, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_bounds.te'}, {'damage_type': 'backtrace', 'id': 2229209791, 'ino': 1099511637301, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_fdreceive.te'}, {'damage_type': 'backtrace', 'id': 2262416955, 'ino': 1099511638696, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/Makefile'}, {'damage_type': 'backtrace', 'id': 2264884910, 'ino': 1099511638721, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget01.c'}, {'damage_type': 'backtrace', 'id': 2341245558, 'ino': 1099511637255, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_file.te'}, {'damage_type': 'backtrace', 'id': 2514440219, 'ino': 1099511638216, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/Makefile'}, {'damage_type': 'backtrace', 'id': 2562826071, 'ino': 1099511637315, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_net.te'}, {'damage_type': 'backtrace', 'id': 2573981135, 'ino': 1099511638692, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv02.c'}, {'damage_type': 'backtrace', 'id': 2652371192, 'ino': 1099511637304, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/TODO'}, {'damage_type': 'backtrace', 'id': 2663114964, 'ino': 1099511638693, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv05.c'}, {'damage_type': 'backtrace', 'id': 2693602133, 'ino': 1099511637245, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_capable_file.te'}, {'damage_type': 'backtrace', 'id': 2694442776, 'ino': 1099511637741, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap02.c'}, {'damage_type': 'backtrace', 'id': 2763601317, 'ino': 1099511637239, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_setpgid.te'}, {'damage_type': 'backtrace', 'id': 2813421309, 'ino': 1099511638722, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/Makefile'}, {'damage_type': 'backtrace', 'id': 2870622398, 'ino': 1099511637246, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_readlink.te'}, {'damage_type': 'backtrace', 'id': 2929092279, 'ino': 1099511637408, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_capable_net.sh'}, {'damage_type': 'backtrace', 'id': 2941980714, 'ino': 1099511638759, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat02.c'}, {'damage_type': 'backtrace', 'id': 2949710311, 'ino': 1099511637306, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_sysctl.te'}, {'damage_type': 'backtrace', 'id': 2966475604, 'ino': 1099511638697, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv03.c'}, {'damage_type': 'backtrace', 'id': 3022706740, 'ino': 1099511636874, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/Makefile'}, {'damage_type': 'backtrace', 'id': 3084360332, 'ino': 1099511638217, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/sched_getparam/sched_getparam01.c'}, {'damage_type': 'backtrace', 'id': 3086218559, 'ino': 1099511637996, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink08.c'}, {'damage_type': 'backtrace', 'id': 3113118168, 'ino': 1099511638723, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget03.c'}, {'damage_type': 'backtrace', 'id': 3131650730, 'ino': 1099511638758, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/Makefile'}, {'damage_type': 'backtrace', 'id': 3140498571, 'ino': 1099511637251, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execute_no_trans.te'}, {'damage_type': 'backtrace', 'id': 3145223599, 'ino': 1099511637258, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_entrypoint.te'}, {'damage_type': 'backtrace', 'id': 3174636827, 'ino': 1099511637312, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_dyntrace.te'}, {'damage_type': 'backtrace', 'id': 3273210643, 'ino': 1099511638757, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat03.c'}, {'damage_type': 'backtrace', 'id': 3280058410, 'ino': 1099511637247, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_policy.if'}, {'damage_type': 'backtrace', 'id': 3287660590, 'ino': 1099511637993, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/unlink05.c'}, {'damage_type': 'backtrace', 'id': 3311918679, 'ino': 1099511637359, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_create/selinux_task_create.sh'}, {'damage_type': 'backtrace', 'id': 3401160589, 'ino': 1099511637732, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/Makefile'}, {'damage_type': 'backtrace', 'id': 3414252424, 'ino': 1099511637742, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/munmap/munmap01.c'}, {'damage_type': 'backtrace', 'id': 3446403678, 'ino': 1099511637994, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/unlink/Makefile'}, {'damage_type': 'backtrace', 'id': 3608266080, 'ino': 1099511638756, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/shmat/shmat01.c'}, {'damage_type': 'backtrace', 'id': 3609443824, 'ino': 1099511637309, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_setattr.te'}, {'damage_type': 'backtrace', 'id': 3625324994, 'ino': 1099511637254, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_execshare.te'}, {'damage_type': 'backtrace', 'id': 3711337345, 'ino': 1099511637317, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_task_create.te'}, {'damage_type': 'backtrace', 'id': 3729648493, 'ino': 1099511637733, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/asyncio/asyncio02.c'}, {'damage_type': 'backtrace', 'id': 3797379556, 'ino': 1099511638698, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgrcv/msgrcv06.c'}, {'damage_type': 'backtrace', 'id': 4191649129, 'ino': 1099511637410, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/capable_net/selinux_bind.c'}, {'damage_type': 'backtrace', 'id': 4200893922, 'ino': 1099511638725, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/ipc/msgget/msgget02.c'}, {'damage_type': 'backtrace', 'id': 4208741105, 'ino': 1099511637241, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_link.te'}, {'damage_type': 'backtrace', 'id': 4272616053, 'ino': 1099511637305, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/test_relabel.te'}, {'damage_type': 'backtrace', 'id': 4290071542, 'ino': 1099511638145, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/execv/execv01.c'}] 2024-06-06T08:44:10.924 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 6100260, 'ino': 1099511638093, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/Makefile'}, {'damage_type': 'backtrace', 'id': 51762981, 'ino': 1099511637291, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_exectrace.te'}, {'damage_type': 'backtrace', 'id': 91794680, 'ino': 1099511638042, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir04.c'}, {'damage_type': 'backtrace', 'id': 108932203, 'ino': 1099511637424, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_source.c'}, {'damage_type': 'backtrace', 'id': 134153374, 'ino': 1099511637698, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd02.c'}, {'damage_type': 'backtrace', 'id': 255132717, 'ino': 1099511637282, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_entrypoint.te'}, {'damage_type': 'backtrace', 'id': 304354188, 'ino': 1099511638278, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit01.c'}, {'damage_type': 'backtrace', 'id': 343542681, 'ino': 1099511637284, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sigkill.te'}, {'damage_type': 'backtrace', 'id': 387038399, 'ino': 1099511637701, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd01.c'}, {'damage_type': 'backtrace', 'id': 397174915, 'ino': 1099511637261, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5'}, {'damage_type': 'backtrace', 'id': 470889146, 'ino': 1099511637270, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_readlink.te'}, {'damage_type': 'backtrace', 'id': 481677629, 'ino': 1099511638527, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/timer_getoverrun01.c'}, {'damage_type': 'backtrace', 'id': 485031711, 'ino': 1099511638091, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink01.c'}, {'damage_type': 'backtrace', 'id': 682596649, 'ino': 1099511637297, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_create.te'}, {'damage_type': 'backtrace', 'id': 736955901, 'ino': 1099511638470, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/Makefile'}, {'damage_type': 'backtrace', 'id': 743764364, 'ino': 1099511637700, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/timerfd03.c'}, {'damage_type': 'backtrace', 'id': 800084220, 'ino': 1099511637273, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ptrace.te'}, {'damage_type': 'backtrace', 'id': 818006050, 'ino': 1099511637277, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_sys.te'}, {'damage_type': 'backtrace', 'id': 976072596, 'ino': 1099511637274, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_stat.te'}, {'damage_type': 'backtrace', 'id': 991632441, 'ino': 1099511637267, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ioctl.te'}, {'damage_type': 'backtrace', 'id': 1023353442, 'ino': 1099511637266, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_ipc.te'}, {'damage_type': 'backtrace', 'id': 1129292183, 'ino': 1099511637422, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/Makefile'}, {'damage_type': 'backtrace', 'id': 1148764158, 'ino': 1099511637264, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setnice.te'}, {'damage_type': 'backtrace', 'id': 1170300642, 'ino': 1099511638043, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir05.c'}, {'damage_type': 'backtrace', 'id': 1236463581, 'ino': 1099511637938, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/exit_group01.c'}, {'damage_type': 'backtrace', 'id': 1260324563, 'ino': 1099511638041, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir01.c'}, {'damage_type': 'backtrace', 'id': 1304175976, 'ino': 1099511637298, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rxdir.te'}, {'damage_type': 'backtrace', 'id': 1366370916, 'ino': 1099511637263, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setpgid.te'}, {'damage_type': 'backtrace', 'id': 1420783535, 'ino': 1099511638471, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid02.c'}, {'damage_type': 'backtrace', 'id': 1494340369, 'ino': 1099511637939, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit_group/Makefile'}, {'damage_type': 'backtrace', 'id': 1525505568, 'ino': 1099511637910, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/Makefile'}, {'damage_type': 'backtrace', 'id': 1928030243, 'ino': 1099511638094, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink03.c'}, {'damage_type': 'backtrace', 'id': 1937982364, 'ino': 1099511637280, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/Makefile'}, {'damage_type': 'backtrace', 'id': 1957974430, 'ino': 1099511637611, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_01.c'}, {'damage_type': 'backtrace', 'id': 2010698425, 'ino': 1099511638277, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/Makefile'}, {'damage_type': 'backtrace', 'id': 2012713932, 'ino': 1099511637612, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/signalfd4_02.c'}, {'damage_type': 'backtrace', 'id': 2226241574, 'ino': 1099511637423, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid_target.c'}, {'damage_type': 'backtrace', 'id': 2306999316, 'ino': 1099511637289, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_transition.te'}, {'damage_type': 'backtrace', 'id': 2350822892, 'ino': 1099511637283, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_fdreceive.te'}, {'damage_type': 'backtrace', 'id': 2404515727, 'ino': 1099511637278, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execshare.te'}, {'damage_type': 'backtrace', 'id': 2420391573, 'ino': 1099511638038, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir09.c'}, {'damage_type': 'backtrace', 'id': 2420666087, 'ino': 1099511637699, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timerfd/Makefile'}, {'damage_type': 'backtrace', 'id': 2622216601, 'ino': 1099511638092, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink05.c'}, {'damage_type': 'backtrace', 'id': 2712198141, 'ino': 1099511637610, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/signalfd4/Makefile'}, {'damage_type': 'backtrace', 'id': 2796125771, 'ino': 1099511638045, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir03.c'}, {'damage_type': 'backtrace', 'id': 2875382097, 'ino': 1099511637299, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_inherit.te'}, {'damage_type': 'backtrace', 'id': 2979505787, 'ino': 1099511637279, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_file.te'}, {'damage_type': 'backtrace', 'id': 2991166628, 'ino': 1099511637285, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_open.te'}, {'damage_type': 'backtrace', 'id': 3110686700, 'ino': 1099511637276, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getpgid.te'}, {'damage_type': 'backtrace', 'id': 3245964812, 'ino': 1099511637269, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_file.te'}, {'damage_type': 'backtrace', 'id': 3282942770, 'ino': 1099511637296, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsid.te'}, {'damage_type': 'backtrace', 'id': 3301784038, 'ino': 1099511637286, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_relabel.te'}, {'damage_type': 'backtrace', 'id': 3323297250, 'ino': 1099511638044, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir02.c'}, {'damage_type': 'backtrace', 'id': 3324547206, 'ino': 1099511637281, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_wait.te'}, {'damage_type': 'backtrace', 'id': 3329718369, 'ino': 1099511638040, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/Makefile'}, {'damage_type': 'backtrace', 'id': 3378001524, 'ino': 1099511637290, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_setattr.te'}, {'damage_type': 'backtrace', 'id': 3390846238, 'ino': 1099511638276, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/exit/exit02.c'}, {'damage_type': 'backtrace', 'id': 3479105804, 'ino': 1099511637287, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_sysctl.te'}, {'damage_type': 'backtrace', 'id': 3515124165, 'ino': 1099511637268, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_setsched.te'}, {'damage_type': 'backtrace', 'id': 3534768640, 'ino': 1099511637835, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork02.c'}, {'damage_type': 'backtrace', 'id': 3566984541, 'ino': 1099511638039, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/mkdir/mkdir08.c'}, {'damage_type': 'backtrace', 'id': 3664273536, 'ino': 1099511637292, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_global.te'}, {'damage_type': 'backtrace', 'id': 3728510284, 'ino': 1099511637836, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/Makefile'}, {'damage_type': 'backtrace', 'id': 3763598819, 'ino': 1099511637265, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_link.te'}, {'damage_type': 'backtrace', 'id': 3770265328, 'ino': 1099511638469, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsid/getsid01.c'}, {'damage_type': 'backtrace', 'id': 3793537247, 'ino': 1099511637837, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/vfork/vfork01.c'}, {'damage_type': 'backtrace', 'id': 3798228951, 'ino': 1099511637271, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_policy.if'}, {'damage_type': 'backtrace', 'id': 3798660450, 'ino': 1099511638090, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink02.c'}, {'damage_type': 'backtrace', 'id': 3821224945, 'ino': 1099511637421, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/task_getpgid/selinux_task_getpgid.sh'}, {'damage_type': 'backtrace', 'id': 3850603430, 'ino': 1099511637293, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrace.te'}, {'damage_type': 'backtrace', 'id': 3928767463, 'ino': 1099511637262, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_rename.te'}, {'damage_type': 'backtrace', 'id': 3987881996, 'ino': 1099511637275, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_execute_no_trans.te'}, {'damage_type': 'backtrace', 'id': 4016530050, 'ino': 1099511638526, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/timer_getoverrun/Makefile'}, {'damage_type': 'backtrace', 'id': 4074932601, 'ino': 1099511637272, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_mkdir.te'}, {'damage_type': 'backtrace', 'id': 4177101794, 'ino': 1099511637295, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_capable_net.te'}, {'damage_type': 'backtrace', 'id': 4201740406, 'ino': 1099511637294, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_task_getsched.te'}, {'damage_type': 'backtrace', 'id': 4235206440, 'ino': 1099511637911, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/getsockname/getsockname01.c'}, {'damage_type': 'backtrace', 'id': 4242933149, 'ino': 1099511637288, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/refpolicy/redhat/5/test_dyntrans.te'}, {'damage_type': 'backtrace', 'id': 4275440355, 'ino': 1099511638095, 'path': '/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/syscalls/symlink/symlink04.c'}] 2024-06-06T08:44:10.924 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-06-06T08:44:11.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:10 smithi027 ceph-mon[32335]: pgmap v120: 129 pgs: 129 active+clean; 1.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 50 MiB/s wr, 583 op/s 2024-06-06T08:44:11.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:10 smithi203 ceph-mon[36579]: pgmap v120: 129 pgs: 129 active+clean; 1.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 50 MiB/s wr, 583 op/s 2024-06-06T08:44:11.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:10 smithi176 ceph-mon[36786]: pgmap v120: 129 pgs: 129 active+clean; 1.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 50 MiB/s wr, 583 op/s 2024-06-06T08:44:13.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:12 smithi203 ceph-mon[36579]: pgmap v121: 129 pgs: 129 active+clean; 1.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 458 op/s 2024-06-06T08:44:13.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:12 smithi176 ceph-mon[36786]: pgmap v121: 129 pgs: 129 active+clean; 1.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 458 op/s 2024-06-06T08:44:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:12 smithi027 ceph-mon[32335]: pgmap v121: 129 pgs: 129 active+clean; 1.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 458 op/s 2024-06-06T08:44:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.144+0000 7f58f6122640 1 -- 172.21.15.27:0/3348192782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 msgr2=0x7f58f0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.144+0000 7f58f6122640 1 --2- 172.21.15.27:0/3348192782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 0x7f58f0100d50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009930 tx=0x7f58e402a420 comp rx=0 tx=0).stop 2024-06-06T08:44:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.144+0000 7f58f6122640 1 -- 172.21.15.27:0/3348192782 shutdown_connections 2024-06-06T08:44:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.144+0000 7f58f6122640 1 --2- 172.21.15.27:0/3348192782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0101290 0x7f58f010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.144+0000 7f58f6122640 1 --2- 172.21.15.27:0/3348192782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 0x7f58f0100d50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.144+0000 7f58f6122640 1 --2- 172.21.15.27:0/3348192782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0106890 0x7f58f0106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.144+0000 7f58f6122640 1 -- 172.21.15.27:0/3348192782 >> 172.21.15.27:0/3348192782 conn(0x7f58f0077a20 msgr2=0x7f58f0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.144+0000 7f58f6122640 1 -- 172.21.15.27:0/3348192782 shutdown_connections 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.144+0000 7f58f6122640 1 -- 172.21.15.27:0/3348192782 wait complete. 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58f6122640 1 Processor -- start 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58f6122640 1 -- start start 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58f6122640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 0x7f58f0197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58f6122640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0101290 0x7f58f01976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58f6122640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106890 0x7f58f019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58f6122640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0110ca0 con 0x7f58f01008d0 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58f6122640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0110b20 con 0x7f58f0106890 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58f6122640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0110e20 con 0x7f58f0101290 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 0x7f58f0197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 0x7f58f0197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33040/0 (socket says 172.21.15.27:33040) 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58ef7fe640 1 -- 172.21.15.27:0/3139262400 learned_addr learned my addr 172.21.15.27:0/3139262400 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58effff640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106890 0x7f58f019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58eeffd640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0101290 0x7f58f01976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58ef7fe640 1 -- 172.21.15.27:0/3139262400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0101290 msgr2=0x7f58f01976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0101290 0x7f58f01976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58ef7fe640 1 -- 172.21.15.27:0/3139262400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106890 msgr2=0x7f58f019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106890 0x7f58f019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.145+0000 7f58ef7fe640 1 -- 172.21.15.27:0/3139262400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f019c0d0 con 0x7f58f01008d0 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.146+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 0x7f58f0197180 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f58e000eb90 tx=0x7f58e000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.146+0000 7f58ecff9640 1 -- 172.21.15.27:0/3139262400 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0004d20 con 0x7f58f01008d0 2024-06-06T08:44:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.146+0000 7f58f6122640 1 -- 172.21.15.27:0/3139262400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f019c3c0 con 0x7f58f01008d0 2024-06-06T08:44:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.146+0000 7f58ecff9640 1 -- 172.21.15.27:0/3139262400 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e000ee90 con 0x7f58f01008d0 2024-06-06T08:44:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.146+0000 7f58ecff9640 1 -- 172.21.15.27:0/3139262400 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0010830 con 0x7f58f01008d0 2024-06-06T08:44:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.146+0000 7f58f6122640 1 -- 172.21.15.27:0/3139262400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01a3c50 con 0x7f58f01008d0 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.149+0000 7f58ecff9640 1 -- 172.21.15.27:0/3139262400 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f58e0005020 con 0x7f58f01008d0 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.149+0000 7f58f8e33740 1 Processor -- start 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.149+0000 7f58f8e33740 1 -- start start 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.149+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb1f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.149+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.149+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb2c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.149+0000 7f58f8e33740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d15ca814f0 con 0x55d15cb1fca0 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.149+0000 7f58f8e33740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d15ca1f870 con 0x55d15cb2a560 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.149+0000 7f58f8e33740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d15ca7d4b0 con 0x55d15cb1f380 2024-06-06T08:44:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb1f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb1f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:57092/0 (socket says 172.21.15.27:57092) 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 -- 172.21.15.27:0/3688890407 learned_addr learned my addr 172.21.15.27:0/3688890407 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/3688890407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33042/0 (socket says 172.21.15.27:33042) 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58eeffd640 1 --2- 172.21.15.27:0/3688890407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb2c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58ecff9640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f58d80a2da0 0x7f58d80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 -- 172.21.15.27:0/3688890407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 msgr2=0x55d15cb2c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 --2- 172.21.15.27:0/3688890407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb2c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 -- 172.21.15.27:0/3688890407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 msgr2=0x55d15cb2a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 --2- 172.21.15.27:0/3688890407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 -- 172.21.15.27:0/3688890407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15ca3b9b0 con 0x55d15cb1f380 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58ecff9640 1 -- 172.21.15.27:0/3139262400 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f58e00cfdd0 con 0x7f58f01008d0 2024-06-06T08:44:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f58d80a2da0 0x7f58d80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 --2- 172.21.15.27:0/3688890407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb1f760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009e20 tx=0x7f58e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c0ffd3f0b3838b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58d5ffb640 1 -- 172.21.15.27:0/3688890407 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8017050 con 0x55d15cb1f380 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58effff640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f58d80a2da0 0x7f58d80a5260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f58e8023000 tx=0x7f58e8024000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.150+0000 7f58d5ffb640 1 -- 172.21.15.27:0/3688890407 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e8003c40 con 0x55d15cb1f380 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.151+0000 7f58d5ffb640 1 -- 172.21.15.27:0/3688890407 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e800f600 con 0x55d15cb1f380 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.151+0000 7f58f8e33740 1 -- 172.21.15.27:0/3688890407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 msgr2=0x55d15cb1f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.151+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3688890407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb1f760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009e20 tx=0x7f58e8011810 comp rx=0 tx=0).stop 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.151+0000 7f58f8e33740 1 -- 172.21.15.27:0/3688890407 shutdown_connections 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.151+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3688890407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb2c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.151+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3688890407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.151+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3688890407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb1f760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.151+0000 7f58f8e33740 1 -- 172.21.15.27:0/3688890407 >> 172.21.15.27:0/3688890407 conn(0x55d15cb1b890 msgr2=0x55d15cb1bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 -- 172.21.15.27:0/3688890407 shutdown_connections 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 -- 172.21.15.27:0/3688890407 wait complete. 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 -- start start 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 Processor -- start 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1f380 0x55d15cb27560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1fca0 0x55d15cb27aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb27fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d15cb2d190 con 0x55d15cb1f380 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d15ca0aae0 con 0x55d15cb2a560 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.152+0000 7f58f8e33740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d15cb2d310 con 0x55d15cb1fca0 2024-06-06T08:44:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58f8e33740 10 client.0 _ll_register_callbacks cb 0x55d15ca439b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1f380 0x55d15cb27560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1f380 0x55d15cb27560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33046/0 (socket says 172.21.15.27:33046) 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ef7fe640 1 -- 172.21.15.27:0/2710412100 learned_addr learned my addr 172.21.15.27:0/2710412100 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58eeffd640 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1fca0 0x55d15cb27aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58effff640 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb27fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ef7fe640 1 -- 172.21.15.27:0/2710412100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1fca0 msgr2=0x55d15cb27aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1fca0 0x55d15cb27aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ef7fe640 1 -- 172.21.15.27:0/2710412100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 msgr2=0x55d15cb27fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb27fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ef7fe640 1 -- 172.21.15.27:0/2710412100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d15c795210 con 0x55d15cb1f380 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1f380 0x55d15cb27560 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f58e00cd760 tx=0x7f58e0004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c8146d6c4d4e98 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ceffd640 1 -- 172.21.15.27:0/2710412100 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e009ba70 con 0x55d15cb1f380 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58f8e33740 1 -- 172.21.15.27:0/2710412100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e8003c40 con 0x55d15cb1f380 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ceffd640 1 -- 172.21.15.27:0/2710412100 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f58e00985a0 con 0x55d15cb1f380 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58ceffd640 1 -- 172.21.15.27:0/2710412100 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f58e009b890 con 0x55d15cb1f380 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58f8e33740 10 client.16383 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.153+0000 7f58f8e33740 1 -- 172.21.15.27:0/2710412100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d15c861620 con 0x55d15cb1f380 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.154+0000 7f58ceffd640 1 -- 172.21.15.27:0/2710412100 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e8003c40 con 0x55d15cb1f380 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.154+0000 7f58ceffd640 1 -- 172.21.15.27:0/2710412100 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f58e00a4710 con 0x55d15cb1f380 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.154+0000 7f58f8e33740 10 client.16383 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.154+0000 7f58f8e33740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.154+0000 7f58f8e33740 20 client.16383 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.154+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d15cb3a7e0 0x55d15cb3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.154+0000 7f58f8e33740 4 client.16383 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:44:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.154+0000 7f58f8e33740 1 -- 172.21.15.27:0/2710412100 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d15ca94f10 con 0x55d15cb3a7e0 2024-06-06T08:44:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.154+0000 7f58eeffd640 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d15cb3a7e0 0x55d15cb3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.155+0000 7f58ceffd640 10 client.16383 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.155+0000 7f58eeffd640 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d15cb3a7e0 0x55d15cb3cbf0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5933a4cd36442e00 server_cookie=f9346906dae8c2c8 in_seq=0 out_seq=0 2024-06-06T08:44:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58ceffd640 1 -- 172.21.15.27:0/2710412100 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d15ca94f10 con 0x55d15cb3a7e0 2024-06-06T08:44:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58ceffd640 10 client.16383 handle_command_reply: tid=0 2024-06-06T08:44:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58f8e33740 1 client.16383 shutdown 2024-06-06T08:44:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58f8e33740 1 -- 172.21.15.27:0/2710412100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1f380 msgr2=0x55d15cb27560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1f380 0x55d15cb27560 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f58e00cd760 tx=0x7f58e0004ec0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58f8e33740 1 -- 172.21.15.27:0/2710412100 shutdown_connections 2024-06-06T08:44:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d15cb3a7e0 0x55d15cb3cbf0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb27fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1fca0 0x55d15cb27aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2710412100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1f380 0x55d15cb27560 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58f8e33740 1 -- 172.21.15.27:0/2710412100 >> 172.21.15.27:0/2710412100 conn(0x55d15cb1b890 msgr2=0x55d15cb187e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.159+0000 7f58ceffd640 0 client.16383 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.160+0000 7f58f8e33740 1 -- 172.21.15.27:0/2710412100 shutdown_connections 2024-06-06T08:44:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.160+0000 7f58f8e33740 1 -- 172.21.15.27:0/2710412100 wait complete. 2024-06-06T08:44:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.160+0000 7f58f8e33740 20 client.16383 trim_cache size 0 max 16384 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58f8e33740 1 Processor -- start 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58f8e33740 1 -- start start 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb1f380 0x55d15cb42500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb42a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb2a560 0x55d15cb20d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58f8e33740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01af5d0 con 0x55d15cb1fca0 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58f8e33740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01acde0 con 0x55d15cb1f380 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58f8e33740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01aac30 con 0x55d15cb2a560 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb42a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb42a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33060/0 (socket says 172.21.15.27:33060) 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58eeffd640 1 -- 172.21.15.27:0/2939850926 learned_addr learned my addr 172.21.15.27:0/2939850926 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58effff640 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb2a560 0x55d15cb20d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb1f380 0x55d15cb42500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58eeffd640 1 -- 172.21.15.27:0/2939850926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb2a560 msgr2=0x55d15cb20d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58eeffd640 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb2a560 0x55d15cb20d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58eeffd640 1 -- 172.21.15.27:0/2939850926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb1f380 msgr2=0x55d15cb42500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58eeffd640 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb1f380 0x55d15cb42500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58eeffd640 1 -- 172.21.15.27:0/2939850926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15cb14f70 con 0x55d15cb1fca0 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.177+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb1f380 0x55d15cb42500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.178+0000 7f58eeffd640 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb42a40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f58e402a940 tx=0x7f58e4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3134a17727d7472b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.178+0000 7f58d57fa640 1 -- 172.21.15.27:0/2939850926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4035040 con 0x55d15cb1fca0 2024-06-06T08:44:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.178+0000 7f58d57fa640 1 -- 172.21.15.27:0/2939850926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e4031a70 con 0x55d15cb1fca0 2024-06-06T08:44:15.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.178+0000 7f58d57fa640 1 -- 172.21.15.27:0/2939850926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4004a90 con 0x55d15cb1fca0 2024-06-06T08:44:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 -- 172.21.15.27:0/2939850926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 msgr2=0x55d15cb42a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb42a40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f58e402a940 tx=0x7f58e4009260 comp rx=0 tx=0).stop 2024-06-06T08:44:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 -- 172.21.15.27:0/2939850926 shutdown_connections 2024-06-06T08:44:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb2a560 0x55d15cb20d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb42a40 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 --2- 172.21.15.27:0/2939850926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb1f380 0x55d15cb42500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 -- 172.21.15.27:0/2939850926 >> 172.21.15.27:0/2939850926 conn(0x55d15cb1b890 msgr2=0x55d15cb22840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 -- 172.21.15.27:0/2939850926 shutdown_connections 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 -- 172.21.15.27:0/2939850926 wait complete. 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 -- start start 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 Processor -- start 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.180+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb32a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:57124/0 (socket says 172.21.15.27:57124) 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33076/0 (socket says 172.21.15.27:33076) 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58f8e33740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb2e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58eeffd640 1 -- 172.21.15.27:0/3262579288 learned_addr learned my addr 172.21.15.27:0/3262579288 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01ab2a0 con 0x55d15cb1fca0 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01ac710 con 0x55d15cb2a560 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01ab970 con 0x55d15cb1f380 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58effff640 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb2e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58eeffd640 1 -- 172.21.15.27:0/3262579288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 msgr2=0x55d15cb32a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58eeffd640 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb32a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58eeffd640 1 -- 172.21.15.27:0/3262579288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 msgr2=0x55d15cb2e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58eeffd640 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb2e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.181+0000 7f58eeffd640 1 -- 172.21.15.27:0/3262579288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d15cb2e980 con 0x55d15cb1fca0 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.182+0000 7f58f8e33740 10 client.0 _ll_register_callbacks cb 0x7f58e0005860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.182+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e4031a70 con 0x55d15cb1fca0 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.182+0000 7f58ceffd640 10 client.16389 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.182+0000 7f58eeffd640 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2ded0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009230 tx=0x7f58e40054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac00a2428b39ac58 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.182+0000 7f58ceffd640 1 -- 172.21.15.27:0/3262579288 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4035040 con 0x55d15cb1fca0 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.183+0000 7f58ceffd640 1 -- 172.21.15.27:0/3262579288 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f58e40055c0 con 0x55d15cb1fca0 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.183+0000 7f58ceffd640 1 -- 172.21.15.27:0/3262579288 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f58e4004770 con 0x55d15cb1fca0 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.183+0000 7f58f8e33740 10 client.16389 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.183+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55d15cb19a30 con 0x55d15cb1fca0 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.183+0000 7f58ceffd640 1 -- 172.21.15.27:0/3262579288 <== mon.0 v2:172.21.15.27:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f58e4069020 con 0x55d15cb1fca0 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.184+0000 7f58f8e33740 10 client.16389 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.184+0000 7f58f8e33740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.184+0000 7f58f8e33740 20 client.16389 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.184+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d15cb1a090 0x55d15cb1a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.184+0000 7f58f8e33740 4 client.16389 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:44:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.184+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d15ca8ec20 con 0x55d15cb1a090 2024-06-06T08:44:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.184+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d15cb1a090 0x55d15cb1a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.184+0000 7f58ceffd640 10 client.16389 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.184+0000 7f58ef7fe640 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d15cb1a090 0x55d15cb1a540 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c47298c56d6afd6 server_cookie=b5de6af9630b605d in_seq=0 out_seq=0 2024-06-06T08:44:15.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:14 smithi203 ceph-mon[36579]: pgmap v122: 129 pgs: 129 active+clean; 1.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 488 op/s 2024-06-06T08:44:15.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:14 smithi176 ceph-mon[36786]: pgmap v122: 129 pgs: 129 active+clean; 1.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 488 op/s 2024-06-06T08:44:15.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:14 smithi027 ceph-mon[32335]: pgmap v122: 129 pgs: 129 active+clean; 1.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 488 op/s 2024-06-06T08:44:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.630+0000 7f58ceffd640 1 -- 172.21.15.27:0/3262579288 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d15ca8ec20 con 0x55d15cb1a090 2024-06-06T08:44:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.630+0000 7f58ceffd640 10 client.16389 handle_command_reply: tid=0 2024-06-06T08:44:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.630+0000 7f58f8e33740 1 client.16389 shutdown 2024-06-06T08:44:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 msgr2=0x55d15cb2ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2ded0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009230 tx=0x7f58e40054a0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 shutdown_connections 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d15cb1a090 0x55d15cb1a540 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d15cb2a560 0x55d15cb2e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d15cb1fca0 0x55d15cb2ded0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58ceffd640 0 client.16389 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 --2- 172.21.15.27:0/3262579288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d15cb1f380 0x55d15cb32a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 >> 172.21.15.27:0/3262579288 conn(0x55d15cb1b890 msgr2=0x55d15cb20d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 shutdown_connections 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 1 -- 172.21.15.27:0/3262579288 wait complete. 2024-06-06T08:44:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f8e33740 20 client.16389 trim_cache size 0 max 16384 2024-06-06T08:44:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:44:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:44:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:44:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f6122640 1 -- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f58d80a2da0 msgr2=0x7f58d80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f6122640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f58d80a2da0 0x7f58d80a5260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f58e8023000 tx=0x7f58e8024000 comp rx=0 tx=0).stop 2024-06-06T08:44:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f6122640 1 -- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 msgr2=0x7f58f0197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.631+0000 7f58f6122640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 0x7f58f0197180 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f58e000eb90 tx=0x7f58e000c6a0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.632+0000 7f58f6122640 1 -- 172.21.15.27:0/3139262400 shutdown_connections 2024-06-06T08:44:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.632+0000 7f58f6122640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f58d80a2da0 0x7f58d80a5260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.632+0000 7f58f6122640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58f0106890 0x7f58f019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.632+0000 7f58f6122640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f0101290 0x7f58f01976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.632+0000 7f58f6122640 1 --2- 172.21.15.27:0/3139262400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58f01008d0 0x7f58f0197180 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.632+0000 7f58f6122640 1 -- 172.21.15.27:0/3139262400 >> 172.21.15.27:0/3139262400 conn(0x7f58f0077a20 msgr2=0x7f58f01033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.632+0000 7f58f6122640 1 -- 172.21.15.27:0/3139262400 shutdown_connections 2024-06-06T08:44:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:15.632+0000 7f58f6122640 1 -- 172.21.15.27:0/3139262400 wait complete. 2024-06-06T08:44:17.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:16 smithi203 ceph-mon[36579]: pgmap v123: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 572 op/s 2024-06-06T08:44:17.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:16 smithi176 ceph-mon[36786]: pgmap v123: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 572 op/s 2024-06-06T08:44:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:16 smithi027 ceph-mon[32335]: pgmap v123: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 572 op/s 2024-06-06T08:44:19.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:18 smithi203 ceph-mon[36579]: pgmap v124: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2024-06-06T08:44:19.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:18 smithi176 ceph-mon[36786]: pgmap v124: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2024-06-06T08:44:19.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:18 smithi027 ceph-mon[32335]: pgmap v124: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2024-06-06T08:44:21.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:20 smithi203 ceph-mon[36579]: pgmap v125: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 735 op/s 2024-06-06T08:44:21.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:20 smithi176 ceph-mon[36786]: pgmap v125: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 735 op/s 2024-06-06T08:44:21.258 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:20 smithi027 ceph-mon[32335]: pgmap v125: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 735 op/s 2024-06-06T08:44:22.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:22 smithi027 ceph-mon[32335]: pgmap v126: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 602 op/s 2024-06-06T08:44:22.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:22 smithi203 ceph-mon[36579]: pgmap v126: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 602 op/s 2024-06-06T08:44:22.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:22 smithi176 ceph-mon[36786]: pgmap v126: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 602 op/s 2024-06-06T08:44:24.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:24 smithi027 ceph-mon[32335]: pgmap v127: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 646 op/s 2024-06-06T08:44:24.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:24 smithi203 ceph-mon[36579]: pgmap v127: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 646 op/s 2024-06-06T08:44:24.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:24 smithi176 ceph-mon[36786]: pgmap v127: 129 pgs: 129 active+clean; 1.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 646 op/s 2024-06-06T08:44:26.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:26 smithi027 ceph-mon[32335]: pgmap v128: 129 pgs: 129 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 783 op/s 2024-06-06T08:44:26.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:26 smithi203 ceph-mon[36579]: pgmap v128: 129 pgs: 129 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 783 op/s 2024-06-06T08:44:26.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:26 smithi176 ceph-mon[36786]: pgmap v128: 129 pgs: 129 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 783 op/s 2024-06-06T08:44:28.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:28 smithi027 ceph-mon[32335]: pgmap v129: 129 pgs: 129 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 627 op/s 2024-06-06T08:44:28.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:28 smithi203 ceph-mon[36579]: pgmap v129: 129 pgs: 129 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 627 op/s 2024-06-06T08:44:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.743+0000 7f3f69431640 1 -- 172.21.15.27:0/651802840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f64102370 msgr2=0x7f3f6410e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.743+0000 7f3f69431640 1 --2- 172.21.15.27:0/651802840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f64102370 0x7f3f6410e880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009ca0 tx=0x7f3f5c02a8c0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.749+0000 7f3f69431640 1 -- 172.21.15.27:0/651802840 shutdown_connections 2024-06-06T08:44:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.749+0000 7f3f69431640 1 --2- 172.21.15.27:0/651802840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f64102370 0x7f3f6410e880 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.749+0000 7f3f69431640 1 --2- 172.21.15.27:0/651802840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f641019b0 0x7f3f64101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.749+0000 7f3f69431640 1 --2- 172.21.15.27:0/651802840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f641079c0 0x7f3f64107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.749+0000 7f3f69431640 1 -- 172.21.15.27:0/651802840 >> 172.21.15.27:0/651802840 conn(0x7f3f64078e30 msgr2=0x7f3f64079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.751+0000 7f3f69431640 1 -- 172.21.15.27:0/651802840 shutdown_connections 2024-06-06T08:44:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.751+0000 7f3f69431640 1 -- 172.21.15.27:0/651802840 wait complete. 2024-06-06T08:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.751+0000 7f3f69431640 1 Processor -- start 2024-06-06T08:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.752+0000 7f3f69431640 1 -- start start 2024-06-06T08:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.752+0000 7f3f69431640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f641019b0 0x7f3f64110930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.752+0000 7f3f69431640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f64102370 0x7f3f64110e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.752+0000 7f3f69431640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f641079c0 0x7f3f641113b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.752+0000 7f3f69431640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f64075140 con 0x7f3f64102370 2024-06-06T08:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.752+0000 7f3f69431640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3f64074fc0 con 0x7f3f641019b0 2024-06-06T08:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.752+0000 7f3f69431640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f640752c0 con 0x7f3f641079c0 2024-06-06T08:44:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.753+0000 7f3f627fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f64102370 0x7f3f64110e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.753+0000 7f3f627fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f64102370 0x7f3f64110e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33502/0 (socket says 172.21.15.27:33502) 2024-06-06T08:44:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.753+0000 7f3f627fc640 1 -- 172.21.15.27:0/3562722031 learned_addr learned my addr 172.21.15.27:0/3562722031 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.753+0000 7f3f627fc640 1 -- 172.21.15.27:0/3562722031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f641079c0 msgr2=0x7f3f641113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.753+0000 7f3f627fc640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f641079c0 0x7f3f641113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.753+0000 7f3f627fc640 1 -- 172.21.15.27:0/3562722031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f641019b0 msgr2=0x7f3f64110930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T08:44:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.753+0000 7f3f627fc640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f641019b0 0x7f3f64110930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.753+0000 7f3f627fc640 1 -- 172.21.15.27:0/3562722031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6419a180 con 0x7f3f64102370 2024-06-06T08:44:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.754+0000 7f3f627fc640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f64102370 0x7f3f64110e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00b4a0 tx=0x7f3f4c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.754+0000 7f3f5bfff640 1 -- 172.21.15.27:0/3562722031 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c014070 con 0x7f3f64102370 2024-06-06T08:44:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.755+0000 7f3f5bfff640 1 -- 172.21.15.27:0/3562722031 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f4c010af0 con 0x7f3f64102370 2024-06-06T08:44:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.755+0000 7f3f5bfff640 1 -- 172.21.15.27:0/3562722031 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c017c50 con 0x7f3f64102370 2024-06-06T08:44:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.755+0000 7f3f69431640 1 -- 172.21.15.27:0/3562722031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6419a470 con 0x7f3f64102370 2024-06-06T08:44:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.755+0000 7f3f69431640 1 -- 172.21.15.27:0/3562722031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6419a9c0 con 0x7f3f64102370 2024-06-06T08:44:28.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:28 smithi176 ceph-mon[36786]: pgmap v129: 129 pgs: 129 active+clean; 1.5 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 627 op/s 2024-06-06T08:44:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.761+0000 7f3f5bfff640 1 -- 172.21.15.27:0/3562722031 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3f4c017070 con 0x7f3f64102370 2024-06-06T08:44:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.764+0000 7f3f5bfff640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3f500a2e70 0x7f3f500a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.764+0000 7f3f5bfff640 1 -- 172.21.15.27:0/3562722031 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f3f4c0cebb0 con 0x7f3f64102370 2024-06-06T08:44:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.765+0000 7f3f6c219740 1 Processor -- start 2024-06-06T08:44:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.765+0000 7f3f62ffd640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3f500a2e70 0x7f3f500a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.766+0000 7f3f6c219740 1 -- start start 2024-06-06T08:44:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.766+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a72682550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.766+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.766+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a7268f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.766+0000 7f3f6c219740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556a725e64f0 con 0x556a72682170 2024-06-06T08:44:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.766+0000 7f3f6c219740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556a72584870 con 0x556a72682a90 2024-06-06T08:44:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.766+0000 7f3f6c219740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556a725e24b0 con 0x556a7268d350 2024-06-06T08:44:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.767+0000 7f3f637fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a7268f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.767+0000 7f3f637fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a7268f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:38176/0 (socket says 172.21.15.27:38176) 2024-06-06T08:44:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.767+0000 7f3f637fe640 1 -- 172.21.15.27:0/2146587129 learned_addr learned my addr 172.21.15.27:0/2146587129 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.767+0000 7f3f627fc640 1 --2- 172.21.15.27:0/2146587129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.768+0000 7f3f637fe640 1 -- 172.21.15.27:0/2146587129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 msgr2=0x556a7268ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.768+0000 7f3f637fe640 1 --2- 172.21.15.27:0/2146587129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.768+0000 7f3f637fe640 1 -- 172.21.15.27:0/2146587129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 msgr2=0x556a72682550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.768+0000 7f3f637fe640 1 --2- 172.21.15.27:0/2146587129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a72682550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.768+0000 7f3f637fe640 1 -- 172.21.15.27:0/2146587129 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a725a09b0 con 0x556a7268d350 2024-06-06T08:44:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.768+0000 7f3f62ffd640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3f500a2e70 0x7f3f500a5330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009a60 tx=0x7f3f54005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.768+0000 7f3f637fe640 1 --2- 172.21.15.27:0/2146587129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a7268f740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c005e30 tx=0x7f3f5c009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c58e09eeca30c29 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.769+0000 7f3f597fa640 1 -- 172.21.15.27:0/2146587129 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c031da0 con 0x556a7268d350 2024-06-06T08:44:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.769+0000 7f3f597fa640 1 -- 172.21.15.27:0/2146587129 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f5c03f070 con 0x556a7268d350 2024-06-06T08:44:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.769+0000 7f3f597fa640 1 -- 172.21.15.27:0/2146587129 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0042a0 con 0x556a7268d350 2024-06-06T08:44:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.770+0000 7f3f6c219740 1 -- 172.21.15.27:0/2146587129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 msgr2=0x556a7268f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.770+0000 7f3f6c219740 1 --2- 172.21.15.27:0/2146587129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a7268f740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c005e30 tx=0x7f3f5c009800 comp rx=0 tx=0).stop 2024-06-06T08:44:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.771+0000 7f3f6c219740 1 -- 172.21.15.27:0/2146587129 shutdown_connections 2024-06-06T08:44:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.771+0000 7f3f6c219740 1 --2- 172.21.15.27:0/2146587129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a7268f740 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.771+0000 7f3f6c219740 1 --2- 172.21.15.27:0/2146587129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.771+0000 7f3f6c219740 1 --2- 172.21.15.27:0/2146587129 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a72682550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.771+0000 7f3f6c219740 1 -- 172.21.15.27:0/2146587129 >> 172.21.15.27:0/2146587129 conn(0x556a7267e680 msgr2=0x556a7267ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.774+0000 7f3f6c219740 1 -- 172.21.15.27:0/2146587129 shutdown_connections 2024-06-06T08:44:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.774+0000 7f3f6c219740 1 -- 172.21.15.27:0/2146587129 wait complete. 2024-06-06T08:44:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.775+0000 7f3f6c219740 1 -- start start 2024-06-06T08:44:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.775+0000 7f3f6c219740 1 Processor -- start 2024-06-06T08:44:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a7268a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a7268a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a7268d350 0x556a7268ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f6c219740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556a7268ff20 con 0x556a72682170 2024-06-06T08:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f6c219740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556a7256fae0 con 0x556a7268d350 2024-06-06T08:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f6c219740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556a726900a0 con 0x556a72682a90 2024-06-06T08:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f6c219740 10 client.0 _ll_register_callbacks cb 0x556a725a89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f627fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a7268a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f627fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a7268a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:38184/0 (socket says 172.21.15.27:38184) 2024-06-06T08:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f627fc640 1 -- 172.21.15.27:0/3978168481 learned_addr learned my addr 172.21.15.27:0/3978168481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f637fe640 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a7268d350 0x556a7268ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f627fc640 1 -- 172.21.15.27:0/3978168481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a7268d350 msgr2=0x556a7268ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f627fc640 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a7268d350 0x556a7268ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f627fc640 1 -- 172.21.15.27:0/3978168481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 msgr2=0x556a7268a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f627fc640 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a7268a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.777+0000 7f3f627fc640 1 -- 172.21.15.27:0/3978168481 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a722fa210 con 0x556a72682a90 2024-06-06T08:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.778+0000 7f3f627fc640 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a7268a8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c09adb0 tx=0x7f3f4c09a860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb9822479389dd5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.778+0000 7f3f4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:44:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.778+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/3978168481 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c01e040 con 0x556a72682a90 2024-06-06T08:44:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.778+0000 7f3f6c219740 1 -- 172.21.15.27:0/3978168481 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f5c03f070 con 0x556a72682a90 2024-06-06T08:44:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.778+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/3978168481 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f3f4c0b96c0 con 0x556a72682a90 2024-06-06T08:44:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.780+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/3978168481 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3f4c0a3a90 con 0x556a72682a90 2024-06-06T08:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.780+0000 7f3f6c219740 10 client.25876 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.780+0000 7f3f6c219740 1 -- 172.21.15.27:0/3978168481 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a723c6620 con 0x556a72682a90 2024-06-06T08:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.780+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/3978168481 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f5c03f070 con 0x556a72682a90 2024-06-06T08:44:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.781+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/3978168481 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3f4c0b92a0 con 0x556a72682a90 2024-06-06T08:44:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.781+0000 7f3f6c219740 10 client.25876 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.781+0000 7f3f6c219740 10 client.25876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.781+0000 7f3f6c219740 20 client.25876 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.781+0000 7f3f6c219740 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a7269d7b0 0x556a7269fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.781+0000 7f3f6c219740 4 client.25876 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:44:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.781+0000 7f3f6c219740 1 -- 172.21.15.27:0/3978168481 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a725f9f10 con 0x556a7269d7b0 2024-06-06T08:44:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.782+0000 7f3f62ffd640 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a7269d7b0 0x556a7269fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.783+0000 7f3f62ffd640 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a7269d7b0 0x556a7269fc30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da2aef730904ea1 server_cookie=cad7d7ed95e653b5 in_seq=0 out_seq=0 2024-06-06T08:44:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.783+0000 7f3f4a7fc640 10 client.25876 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.784+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/3978168481 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556a725f9f10 con 0x556a7269d7b0 2024-06-06T08:44:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.784+0000 7f3f4a7fc640 10 client.25876 handle_command_reply: tid=0 2024-06-06T08:44:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.784+0000 7f3f6c219740 1 client.25876 shutdown 2024-06-06T08:44:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.785+0000 7f3f6c219740 1 -- 172.21.15.27:0/3978168481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 msgr2=0x556a7268a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.785+0000 7f3f6c219740 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a7268a8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c09adb0 tx=0x7f3f4c09a860 comp rx=0 tx=0).stop 2024-06-06T08:44:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f6c219740 1 -- 172.21.15.27:0/3978168481 shutdown_connections 2024-06-06T08:44:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f6c219740 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a7269d7b0 0x556a7269fc30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f6c219740 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a7268d350 0x556a7268ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f6c219740 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a7268a8c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f6c219740 1 --2- 172.21.15.27:0/3978168481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a7268a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f6c219740 1 -- 172.21.15.27:0/3978168481 >> 172.21.15.27:0/3978168481 conn(0x556a7267e680 msgr2=0x556a72684400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f4a7fc640 0 client.25876 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f6c219740 1 -- 172.21.15.27:0/3978168481 shutdown_connections 2024-06-06T08:44:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f6c219740 1 -- 172.21.15.27:0/3978168481 wait complete. 2024-06-06T08:44:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.786+0000 7f3f6c219740 20 client.25876 trim_cache size 0 max 16384 2024-06-06T08:44:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.816+0000 7f3f6c219740 1 Processor -- start 2024-06-06T08:44:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.816+0000 7f3f6c219740 1 -- start start 2024-06-06T08:44:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a72684b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a72698880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f6c219740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641c0660 con 0x556a72682170 2024-06-06T08:44:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f6c219740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641bde70 con 0x556a72682a90 2024-06-06T08:44:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f6c219740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641030f0 con 0x556a7268d350 2024-06-06T08:44:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f627fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f627fc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:37156/0 (socket says 172.21.15.27:37156) 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f627fc640 1 -- 172.21.15.27:0/1738838200 learned_addr learned my addr 172.21.15.27:0/1738838200 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.817+0000 7f3f637fe640 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a72698880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.818+0000 7f3f62ffd640 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a72684b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.818+0000 7f3f627fc640 1 -- 172.21.15.27:0/1738838200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 msgr2=0x556a72698880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.818+0000 7f3f627fc640 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a72698880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.818+0000 7f3f627fc640 1 -- 172.21.15.27:0/1738838200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 msgr2=0x556a72684b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.818+0000 7f3f627fc640 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a72684b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.818+0000 7f3f627fc640 1 -- 172.21.15.27:0/1738838200 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a726850c0 con 0x556a72682a90 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.818+0000 7f3f62ffd640 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a72684b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:44:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.818+0000 7f3f627fc640 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ac20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c09a8f0 tx=0x7f3f4c017670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e109aedb017007a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.819+0000 7f3f58ff9640 1 -- 172.21.15.27:0/1738838200 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c01e070 con 0x556a72682a90 2024-06-06T08:44:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.819+0000 7f3f58ff9640 1 -- 172.21.15.27:0/1738838200 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f4c0cc100 con 0x556a72682a90 2024-06-06T08:44:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.819+0000 7f3f58ff9640 1 -- 172.21.15.27:0/1738838200 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c099ce0 con 0x556a72682a90 2024-06-06T08:44:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.820+0000 7f3f6c219740 1 -- 172.21.15.27:0/1738838200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 msgr2=0x556a7268ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.820+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ac20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c09a8f0 tx=0x7f3f4c017670 comp rx=0 tx=0).stop 2024-06-06T08:44:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.821+0000 7f3f6c219740 1 -- 172.21.15.27:0/1738838200 shutdown_connections 2024-06-06T08:44:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.821+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a7268d350 0x556a72698880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.821+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682a90 0x556a7268ac20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.821+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1738838200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a72682170 0x556a72684b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.821+0000 7f3f6c219740 1 -- 172.21.15.27:0/1738838200 >> 172.21.15.27:0/1738838200 conn(0x556a7267e680 msgr2=0x556a72698170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.821+0000 7f3f6c219740 1 -- 172.21.15.27:0/1738838200 shutdown_connections 2024-06-06T08:44:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.821+0000 7f3f6c219740 1 -- 172.21.15.27:0/1738838200 wait complete. 2024-06-06T08:44:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.821+0000 7f3f6c219740 1 -- start start 2024-06-06T08:44:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.821+0000 7f3f6c219740 1 Processor -- start 2024-06-06T08:44:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682170 0x556a726952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a72695830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f6c219740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a7268d350 0x556a72686a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f6c219740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641bc360 con 0x556a7268d350 2024-06-06T08:44:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f6c219740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641bd7a0 con 0x556a72682170 2024-06-06T08:44:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f6c219740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f641bca00 con 0x556a72682a90 2024-06-06T08:44:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f6c219740 10 client.0 _ll_register_callbacks cb 0x7f3f4c09abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f62ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682170 0x556a726952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f62ffd640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682170 0x556a726952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:37166/0 (socket says 172.21.15.27:37166) 2024-06-06T08:44:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f62ffd640 1 -- 172.21.15.27:0/1717940553 learned_addr learned my addr 172.21.15.27:0/1717940553 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f627fc640 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a72695830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.822+0000 7f3f637fe640 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a7268d350 0x556a72686a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.824+0000 7f3f627fc640 1 -- 172.21.15.27:0/1717940553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682170 msgr2=0x556a726952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.824+0000 7f3f627fc640 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682170 0x556a726952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.825+0000 7f3f627fc640 1 -- 172.21.15.27:0/1717940553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a7268d350 msgr2=0x556a72686a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.825+0000 7f3f627fc640 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a7268d350 0x556a72686a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.825+0000 7f3f627fc640 1 -- 172.21.15.27:0/1717940553 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a72686fb0 con 0x556a72682a90 2024-06-06T08:44:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.825+0000 7f3f627fc640 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a72695830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x556a7269d690 tx=0x7f3f4c006d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2fcc82d9787819 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.825+0000 7f3f4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-06-06T08:44:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.825+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/1717940553 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c01e050 con 0x556a72682a90 2024-06-06T08:44:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.825+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/1717940553 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f3f4c099ce0 con 0x556a72682a90 2024-06-06T08:44:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.826+0000 7f3f6c219740 1 -- 172.21.15.27:0/1717940553 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f4c0cc100 con 0x556a72682a90 2024-06-06T08:44:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.826+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/1717940553 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3f4c0af0a0 con 0x556a72682a90 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.826+0000 7f3f6c219740 10 client.25882 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.826+0000 7f3f6c219740 1 -- 172.21.15.27:0/1717940553 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a72686fb0 con 0x556a72682a90 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.827+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/1717940553 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f4c0a8070 con 0x556a72682a90 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.827+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/1717940553 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3f4c0a3ce0 con 0x556a72682a90 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.827+0000 7f3f6c219740 10 client.25882 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.827+0000 7f3f6c219740 10 client.25882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.827+0000 7f3f6c219740 20 client.25882 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.827+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a727080c0 0x556a7270a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.827+0000 7f3f6c219740 4 client.25882 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:44:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.827+0000 7f3f6c219740 1 -- 172.21.15.27:0/1717940553 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3f4c0cc100 con 0x556a727080c0 2024-06-06T08:44:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.828+0000 7f3f62ffd640 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a727080c0 0x556a7270a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.830+0000 7f3f62ffd640 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a727080c0 0x556a7270a4e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ee7b2029f19212 server_cookie=9dab6f014d5e4835 in_seq=0 out_seq=0 2024-06-06T08:44:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:28.830+0000 7f3f4a7fc640 10 client.25882 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f4a7fc640 1 -- 172.21.15.27:0/1717940553 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f3f4c0cc100 con 0x556a727080c0 2024-06-06T08:44:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f4a7fc640 10 client.25882 handle_command_reply: tid=0 2024-06-06T08:44:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 client.25882 shutdown 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 -- 172.21.15.27:0/1717940553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 msgr2=0x556a72695830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a72695830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x556a7269d690 tx=0x7f3f4c006d60 comp rx=0 tx=0).stop 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 -- 172.21.15.27:0/1717940553 shutdown_connections 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556a727080c0 0x556a7270a4e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556a7268d350 0x556a72686a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f4a7fc640 0 client.25882 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556a72682a90 0x556a72695830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 --2- 172.21.15.27:0/1717940553 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556a72682170 0x556a726952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 -- 172.21.15.27:0/1717940553 >> 172.21.15.27:0/1717940553 conn(0x556a7267e680 msgr2=0x556a72698a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 -- 172.21.15.27:0/1717940553 shutdown_connections 2024-06-06T08:44:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 1 -- 172.21.15.27:0/1717940553 wait complete. 2024-06-06T08:44:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.179+0000 7f3f6c219740 20 client.25882 trim_cache size 0 max 16384 2024-06-06T08:44:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:44:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:44:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 -- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3f500a2e70 msgr2=0x7f3f500a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3f500a2e70 0x7f3f500a5330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009a60 tx=0x7f3f54005f90 comp rx=0 tx=0).stop 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 -- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f64102370 msgr2=0x7f3f64110e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f64102370 0x7f3f64110e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00b4a0 tx=0x7f3f4c00b960 comp rx=0 tx=0).stop 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 -- 172.21.15.27:0/3562722031 shutdown_connections 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3f500a2e70 0x7f3f500a5330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f641079c0 0x7f3f641113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f64102370 0x7f3f64110e70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 --2- 172.21.15.27:0/3562722031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f641019b0 0x7f3f64110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 -- 172.21.15.27:0/3562722031 >> 172.21.15.27:0/3562722031 conn(0x7f3f64078e30 msgr2=0x7f3f640fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 -- 172.21.15.27:0/3562722031 shutdown_connections 2024-06-06T08:44:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:29.180+0000 7f3f69431640 1 -- 172.21.15.27:0/3562722031 wait complete. 2024-06-06T08:44:30.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:30 smithi027 ceph-mon[32335]: pgmap v130: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 833 op/s 2024-06-06T08:44:30.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:30 smithi203 ceph-mon[36579]: pgmap v130: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 833 op/s 2024-06-06T08:44:30.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:30 smithi176 ceph-mon[36786]: pgmap v130: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 833 op/s 2024-06-06T08:44:32.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:32 smithi027 ceph-mon[32335]: pgmap v131: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-06-06T08:44:32.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:32 smithi203 ceph-mon[36579]: pgmap v131: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-06-06T08:44:32.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:32 smithi176 ceph-mon[36786]: pgmap v131: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-06-06T08:44:34.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:34 smithi027 ceph-mon[32335]: pgmap v132: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 503 op/s 2024-06-06T08:44:34.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:34 smithi203 ceph-mon[36579]: pgmap v132: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 503 op/s 2024-06-06T08:44:34.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:34 smithi176 ceph-mon[36786]: pgmap v132: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 503 op/s 2024-06-06T08:44:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:35 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:44:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:35 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:44:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:35 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:44:35.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:35 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:44:35.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:35 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:44:35.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:35 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:44:35.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:35 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:44:35.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:35 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:44:35.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:35 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:44:35.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:35 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:44:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:35 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:44:35.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:35 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:44:36.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:36 smithi027 ceph-mon[32335]: pgmap v133: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 675 op/s 2024-06-06T08:44:36.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:36 smithi203 ceph-mon[36579]: pgmap v133: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 675 op/s 2024-06-06T08:44:36.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:36 smithi176 ceph-mon[36786]: pgmap v133: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 675 op/s 2024-06-06T08:44:37.971 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-06T08:44:37.972 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-06-06T08:44:37.972 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:44:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:44:38.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:38 smithi027 ceph-mon[32335]: pgmap v134: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 508 op/s 2024-06-06T08:44:38.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:38 smithi203 ceph-mon[36579]: pgmap v134: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 508 op/s 2024-06-06T08:44:38.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:38 smithi176 ceph-mon[36786]: pgmap v134: 129 pgs: 129 active+clean; 1.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 508 op/s 2024-06-06T08:44:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.233+0000 7fb453e24700 1 -- 172.21.15.27:0/2715378807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb44c06e8f0 msgr2=0x7fb44c11b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.233+0000 7fb453e24700 1 --2- 172.21.15.27:0/2715378807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb44c06e8f0 0x7fb44c11b600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb448009990 tx=0x7fb448009ca0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.235+0000 7fb453e24700 1 -- 172.21.15.27:0/2715378807 shutdown_connections 2024-06-06T08:44:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.235+0000 7fb453e24700 1 --2- 172.21.15.27:0/2715378807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44c11bb40 0x7fb44c11df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.235+0000 7fb453e24700 1 --2- 172.21.15.27:0/2715378807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44c06dfd0 0x7fb44c06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.235+0000 7fb453e24700 1 --2- 172.21.15.27:0/2715378807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb44c06e8f0 0x7fb44c11b600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.235+0000 7fb453e24700 1 -- 172.21.15.27:0/2715378807 >> 172.21.15.27:0/2715378807 conn(0x7fb44c06d6b0 msgr2=0x7fb44c10b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:40.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.236+0000 7fb453e24700 1 -- 172.21.15.27:0/2715378807 shutdown_connections 2024-06-06T08:44:40.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.236+0000 7fb453e24700 1 -- 172.21.15.27:0/2715378807 wait complete. 2024-06-06T08:44:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.237+0000 7fb453e24700 1 Processor -- start 2024-06-06T08:44:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.237+0000 7fb453e24700 1 -- start start 2024-06-06T08:44:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.237+0000 7fb453e24700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44c06dfd0 0x7fb44c1ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.237+0000 7fb453e24700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb44c06e8f0 0x7fb44c1adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.237+0000 7fb453e24700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44c11bb40 0x7fb44c1a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.237+0000 7fb453e24700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c1216b0 con 0x7fb44c11bb40 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.237+0000 7fb453e24700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c121530 con 0x7fb44c06e8f0 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.237+0000 7fb453e24700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c121830 con 0x7fb44c06dfd0 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb451bc0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44c06dfd0 0x7fb44c1ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb4523c1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44c11bb40 0x7fb44c1a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb451bc0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44c06dfd0 0x7fb44c1ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:39650/0 (socket says 172.21.15.27:39650) 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb451bc0700 1 -- 172.21.15.27:0/3793154990 learned_addr learned my addr 172.21.15.27:0/3793154990 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb4513bf700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb44c06e8f0 0x7fb44c1adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb451bc0700 1 -- 172.21.15.27:0/3793154990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb44c06e8f0 msgr2=0x7fb44c1adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb451bc0700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb44c06e8f0 0x7fb44c1adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb451bc0700 1 -- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44c11bb40 msgr2=0x7fb44c1a78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb451bc0700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44c11bb40 0x7fb44c1a78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb451bc0700 1 -- 172.21.15.27:0/3793154990 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c1a8010 con 0x7fb44c06dfd0 2024-06-06T08:44:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb4523c1700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44c11bb40 0x7fb44c1a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.238+0000 7fb4513bf700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb44c06e8f0 0x7fb44c1adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:44:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.239+0000 7fb451bc0700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44c06dfd0 0x7fb44c1ad6f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb44000e9a0 tx=0x7fb44000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.239+0000 7fb43effd700 1 -- 172.21.15.27:0/3793154990 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44000cbd0 con 0x7fb44c06dfd0 2024-06-06T08:44:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.239+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c1a8300 con 0x7fb44c06dfd0 2024-06-06T08:44:40.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.239+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c10edc0 con 0x7fb44c06dfd0 2024-06-06T08:44:40.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.239+0000 7fb43effd700 1 -- 172.21.15.27:0/3793154990 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb44000cd70 con 0x7fb44c06dfd0 2024-06-06T08:44:40.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.239+0000 7fb43effd700 1 -- 172.21.15.27:0/3793154990 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440018750 con 0x7fb44c06dfd0 2024-06-06T08:44:40.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.242+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb430005330 con 0x7fb44c06dfd0 2024-06-06T08:44:40.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.245+0000 7fb43effd700 1 -- 172.21.15.27:0/3793154990 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb4400188f0 con 0x7fb44c06dfd0 2024-06-06T08:44:40.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.246+0000 7fb43effd700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb4380a2ed0 0x7fb4380a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:40.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.246+0000 7fb43effd700 1 -- 172.21.15.27:0/3793154990 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fb4400cfa30 con 0x7fb44c06dfd0 2024-06-06T08:44:40.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.246+0000 7fb4513bf700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb4380a2ed0 0x7fb4380a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:40.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.247+0000 7fb4513bf700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb4380a2ed0 0x7fb4380a5390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb44800b820 tx=0x7fb44800b610 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:40.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.252+0000 7fb43effd700 1 -- 172.21.15.27:0/3793154990 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb44009c230 con 0x7fb44c06dfd0 2024-06-06T08:44:40.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.567+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4300058f0 con 0x7fb44c06dfd0 2024-06-06T08:44:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.568+0000 7fb43effd700 1 -- 172.21.15.27:0/3793154990 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11896 (secure 0 0 0) 0x7fb44009bb30 con 0x7fb44c06dfd0 2024-06-06T08:44:40.567 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:44:40.567 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:43:25.179611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:44:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb4380a2ed0 msgr2=0x7fb4380a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb4380a2ed0 0x7fb4380a5390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb44800b820 tx=0x7fb44800b610 comp rx=0 tx=0).stop 2024-06-06T08:44:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44c06dfd0 msgr2=0x7fb44c1ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44c06dfd0 0x7fb44c1ad6f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb44000e9a0 tx=0x7fb44000ecb0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 shutdown_connections 2024-06-06T08:44:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb44c11bb40 0x7fb44c1a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44c06dfd0 0x7fb44c1ad6f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb4380a2ed0 0x7fb4380a5390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 --2- 172.21.15.27:0/3793154990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb44c06e8f0 0x7fb44c1adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.570+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 >> 172.21.15.27:0/3793154990 conn(0x7fb44c06d6b0 msgr2=0x7fb44c11ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.571+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 shutdown_connections 2024-06-06T08:44:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:40.571+0000 7fb453e24700 1 -- 172.21.15.27:0/3793154990 wait complete. 2024-06-06T08:44:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-06-06T08:44:40.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:40 smithi027 ceph-mon[32335]: pgmap v135: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 723 op/s 2024-06-06T08:44:40.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:40 smithi203 ceph-mon[36579]: pgmap v135: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 723 op/s 2024-06-06T08:44:40.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:40 smithi176 ceph-mon[36786]: pgmap v135: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 723 op/s 2024-06-06T08:44:41.363 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-06-06T08:44:41.363 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-06-06T08:44:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:44:41.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:41 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3793154990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:44:41.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:41 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3793154990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:44:41.758 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:41 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3793154990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:44:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.285+0000 7f47547f4640 1 -- 172.21.15.27:0/3045069273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c105380 msgr2=0x7f474c10cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.285+0000 7f47547f4640 1 --2- 172.21.15.27:0/3045069273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c105380 0x7f474c10cf50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f474800da00 tx=0x7f474802a290 comp rx=0 tx=0).stop 2024-06-06T08:44:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.285+0000 7f47547f4640 1 -- 172.21.15.27:0/3045069273 shutdown_connections 2024-06-06T08:44:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.285+0000 7f47547f4640 1 --2- 172.21.15.27:0/3045069273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c105380 0x7f474c10cf50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.285+0000 7f47547f4640 1 --2- 172.21.15.27:0/3045069273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f474c100e90 0x7f474c104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.285+0000 7f47547f4640 1 --2- 172.21.15.27:0/3045069273 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c1004e0 0x7f474c1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.285+0000 7f47547f4640 1 -- 172.21.15.27:0/3045069273 >> 172.21.15.27:0/3045069273 conn(0x7f474c075c20 msgr2=0x7f474c076030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.288+0000 7f47547f4640 1 -- 172.21.15.27:0/3045069273 shutdown_connections 2024-06-06T08:44:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.288+0000 7f47547f4640 1 -- 172.21.15.27:0/3045069273 wait complete. 2024-06-06T08:44:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.289+0000 7f47547f4640 1 Processor -- start 2024-06-06T08:44:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.289+0000 7f47547f4640 1 -- start start 2024-06-06T08:44:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.289+0000 7f47547f4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f474c1004e0 0x7f474c116430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.289+0000 7f47547f4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c105380 0x7f474c116970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.289+0000 7f47547f4640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c10f500 0x7f474c10f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.289+0000 7f47547f4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f474c118420 con 0x7f474c1004e0 2024-06-06T08:44:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.289+0000 7f47547f4640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1182a0 con 0x7f474c10f500 2024-06-06T08:44:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.289+0000 7f47547f4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1185a0 con 0x7f474c105380 2024-06-06T08:44:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.290+0000 7f4752d91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c10f500 0x7f474c10f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.290+0000 7f4752d91640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c10f500 0x7f474c10f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:42132/0 (socket says 172.21.15.27:42132) 2024-06-06T08:44:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.290+0000 7f4752d91640 1 -- 172.21.15.27:0/3859631045 learned_addr learned my addr 172.21.15.27:0/3859631045 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.291+0000 7f4752d91640 1 -- 172.21.15.27:0/3859631045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c105380 msgr2=0x7f474c116970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T08:44:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.291+0000 7f4751d8f640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c105380 0x7f474c116970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.291+0000 7f4752d91640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c105380 0x7f474c116970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.291+0000 7f4752d91640 1 -- 172.21.15.27:0/3859631045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f474c1004e0 msgr2=0x7f474c116430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.291+0000 7f4752d91640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f474c1004e0 0x7f474c116430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.291+0000 7f4752d91640 1 -- 172.21.15.27:0/3859631045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c110270 con 0x7f474c10f500 2024-06-06T08:44:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.292+0000 7f4752d91640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c10f500 0x7f474c10f9b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f474800d9d0 tx=0x7f4748004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.293+0000 7f47437fe640 1 -- 172.21.15.27:0/3859631045 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748040020 con 0x7f474c10f500 2024-06-06T08:44:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.293+0000 7f47437fe640 1 -- 172.21.15.27:0/3859631045 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47480044d0 con 0x7f474c10f500 2024-06-06T08:44:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.293+0000 7f47437fe640 1 -- 172.21.15.27:0/3859631045 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748003850 con 0x7f474c10f500 2024-06-06T08:44:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.293+0000 7f47547f4640 1 -- 172.21.15.27:0/3859631045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c113fb0 con 0x7f474c10f500 2024-06-06T08:44:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.293+0000 7f47547f4640 1 -- 172.21.15.27:0/3859631045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c114450 con 0x7f474c10f500 2024-06-06T08:44:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.299+0000 7f47575d6740 1 Processor -- start 2024-06-06T08:44:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.299+0000 7f47437fe640 1 -- 172.21.15.27:0/3859631045 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4748043020 con 0x7f474c10f500 2024-06-06T08:44:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.299+0000 7f47575d6740 1 -- start start 2024-06-06T08:44:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f47575d6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac7041550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f47575d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041a90 0x55bac704be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f47575d6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac704e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f47575d6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bac6fa54f0 con 0x55bac704c350 2024-06-06T08:44:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f47575d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bac6f43870 con 0x55bac7041a90 2024-06-06T08:44:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f47575d6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bac6fa14b0 con 0x55bac7041170 2024-06-06T08:44:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f4751d8f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac7041550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f4751d8f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac7041550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:39676/0 (socket says 172.21.15.27:39676) 2024-06-06T08:44:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f4751d8f640 1 -- 172.21.15.27:0/2094243849 learned_addr learned my addr 172.21.15.27:0/2094243849 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f4751d8f640 1 --2- 172.21.15.27:0/2094243849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac704e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f4751d8f640 1 -- 172.21.15.27:0/2094243849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041a90 msgr2=0x55bac704be10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f4751d8f640 1 --2- 172.21.15.27:0/2094243849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041a90 0x55bac704be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f4751d8f640 1 -- 172.21.15.27:0/2094243849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 msgr2=0x55bac704e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.300+0000 7f4751d8f640 1 --2- 172.21.15.27:0/2094243849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac704e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.301+0000 7f4751d8f640 1 -- 172.21.15.27:0/2094243849 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac6f5f9b0 con 0x55bac7041170 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.301+0000 7f47437fe640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f47380a2f50 0x7f47380a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.301+0000 7f47437fe640 1 -- 172.21.15.27:0/3859631045 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f47480f0170 con 0x7f474c10f500 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.301+0000 7f4751d8f640 1 --2- 172.21.15.27:0/2094243849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac7041550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47440148d0 tx=0x7f4744001760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523fcf466d84174 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.301+0000 7f4740ff9640 1 -- 172.21.15.27:0/2094243849 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474401d070 con 0x55bac7041170 2024-06-06T08:44:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.301+0000 7f4740ff9640 1 -- 172.21.15.27:0/2094243849 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474400de30 con 0x55bac7041170 2024-06-06T08:44:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.301+0000 7f4740ff9640 1 -- 172.21.15.27:0/2094243849 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744018bb0 con 0x55bac7041170 2024-06-06T08:44:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.302+0000 7f4752590640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f47380a2f50 0x7f47380a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.302+0000 7f47575d6740 1 -- 172.21.15.27:0/2094243849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 msgr2=0x55bac7041550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.302+0000 7f47575d6740 1 --2- 172.21.15.27:0/2094243849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac7041550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47440148d0 tx=0x7f4744001760 comp rx=0 tx=0).stop 2024-06-06T08:44:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.303+0000 7f4752590640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f47380a2f50 0x7f47380a5410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55bac7042710 tx=0x7f473c00f040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.303+0000 7f47575d6740 1 -- 172.21.15.27:0/2094243849 shutdown_connections 2024-06-06T08:44:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.303+0000 7f47575d6740 1 --2- 172.21.15.27:0/2094243849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac704e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.303+0000 7f47575d6740 1 --2- 172.21.15.27:0/2094243849 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041a90 0x55bac704be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.303+0000 7f47575d6740 1 --2- 172.21.15.27:0/2094243849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac7041550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.303+0000 7f47575d6740 1 -- 172.21.15.27:0/2094243849 >> 172.21.15.27:0/2094243849 conn(0x55bac703d680 msgr2=0x55bac703da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.303+0000 7f47575d6740 1 -- 172.21.15.27:0/2094243849 shutdown_connections 2024-06-06T08:44:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.303+0000 7f47575d6740 1 -- 172.21.15.27:0/2094243849 wait complete. 2024-06-06T08:44:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f47575d6740 1 -- start start 2024-06-06T08:44:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f47575d6740 1 Processor -- start 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f47575d6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac70493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f47575d6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041a90 0x55bac7049900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f47575d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 0x55bac7049e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f47575d6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bac704ef80 con 0x55bac7041a90 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f47575d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bac6f2eae0 con 0x55bac704c350 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f47575d6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bac704f100 con 0x55bac7041170 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f47575d6740 10 client.0 _ll_register_callbacks cb 0x55bac6f679b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f4751d8f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041a90 0x55bac7049900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f4751d8f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041a90 0x55bac7049900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52244/0 (socket says 172.21.15.27:52244) 2024-06-06T08:44:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.304+0000 7f4751d8f640 1 -- 172.21.15.27:0/4175850002 learned_addr learned my addr 172.21.15.27:0/4175850002 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4752590640 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac70493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4752d91640 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 0x55bac7049e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4751d8f640 1 -- 172.21.15.27:0/4175850002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 msgr2=0x55bac70493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4751d8f640 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac70493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4751d8f640 1 -- 172.21.15.27:0/4175850002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 msgr2=0x55bac7049e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4751d8f640 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 0x55bac7049e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4751d8f640 1 -- 172.21.15.27:0/4175850002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac6cb9210 con 0x55bac7041a90 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4751d8f640 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041a90 0x55bac7049900 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4744007030 tx=0x7f47440187f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7857438f70cdb9aa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4735ffb640 1 -- 172.21.15.27:0/4175850002 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744027070 con 0x55bac7041a90 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4735ffb640 1 -- 172.21.15.27:0/4175850002 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f47440188f0 con 0x55bac7041a90 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f47575d6740 1 -- 172.21.15.27:0/4175850002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474400de30 con 0x55bac7041a90 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f4735ffb640 1 -- 172.21.15.27:0/4175850002 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4744031050 con 0x55bac7041a90 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f47575d6740 10 client.16413 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.305+0000 7f47575d6740 1 -- 172.21.15.27:0/4175850002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bac6d85620 con 0x55bac7041a90 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.306+0000 7f4735ffb640 1 -- 172.21.15.27:0/4175850002 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f474400de30 con 0x55bac7041a90 2024-06-06T08:44:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.306+0000 7f4735ffb640 1 -- 172.21.15.27:0/4175850002 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f474402c740 con 0x55bac7041a90 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.306+0000 7f47575d6740 10 client.16413 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.306+0000 7f47575d6740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.306+0000 7f47575d6740 20 client.16413 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.306+0000 7f47575d6740 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bac705c4a0 0x55bac705e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.306+0000 7f47575d6740 4 client.16413 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.306+0000 7f47575d6740 1 -- 172.21.15.27:0/4175850002 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bac6fb8f10 con 0x55bac705c4a0 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.306+0000 7f4752590640 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bac705c4a0 0x55bac705e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.307+0000 7f4752590640 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bac705c4a0 0x55bac705e890 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a0b1aedd62ae01 server_cookie=4560442ed0b8ceb3 in_seq=0 out_seq=0 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.307+0000 7f4735ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.307+0000 7f4735ffb640 1 -- 172.21.15.27:0/4175850002 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bac6fb8f10 con 0x55bac705c4a0 2024-06-06T08:44:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.307+0000 7f4735ffb640 10 client.16413 handle_command_reply: tid=0 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.307+0000 7f47575d6740 1 client.16413 shutdown 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.307+0000 7f47575d6740 1 -- 172.21.15.27:0/4175850002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041a90 msgr2=0x55bac7049900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.307+0000 7f47575d6740 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041a90 0x55bac7049900 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4744007030 tx=0x7f47440187f0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f47575d6740 1 -- 172.21.15.27:0/4175850002 shutdown_connections 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f47575d6740 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bac705c4a0 0x55bac705e890 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f47575d6740 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 0x55bac7049e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f4735ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f47575d6740 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041a90 0x55bac7049900 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f47575d6740 1 --2- 172.21.15.27:0/4175850002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041170 0x55bac70493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f47575d6740 1 -- 172.21.15.27:0/4175850002 >> 172.21.15.27:0/4175850002 conn(0x55bac703d680 msgr2=0x55bac70433b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f47575d6740 1 -- 172.21.15.27:0/4175850002 shutdown_connections 2024-06-06T08:44:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f47575d6740 1 -- 172.21.15.27:0/4175850002 wait complete. 2024-06-06T08:44:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.308+0000 7f47575d6740 20 client.16413 trim_cache size 0 max 16384 2024-06-06T08:44:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f47575d6740 1 Processor -- start 2024-06-06T08:44:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f47575d6740 1 -- start start 2024-06-06T08:44:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f47575d6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041170 0x55bac703cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f47575d6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 0x55bac7043b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f47575d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 0x55bac7063f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f47575d6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1c8d90 con 0x55bac7041170 2024-06-06T08:44:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f47575d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1c65a0 con 0x55bac704c350 2024-06-06T08:44:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f47575d6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f474c072680 con 0x55bac7041a90 2024-06-06T08:44:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f4752590640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041170 0x55bac703cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f4752590640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041170 0x55bac703cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52260/0 (socket says 172.21.15.27:52260) 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f4752590640 1 -- 172.21.15.27:0/1858132375 learned_addr learned my addr 172.21.15.27:0/1858132375 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.323+0000 7f4752d91640 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 0x55bac7063f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4751d8f640 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 0x55bac7043b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4752590640 1 -- 172.21.15.27:0/1858132375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 msgr2=0x55bac7043b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4752590640 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 0x55bac7043b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4752590640 1 -- 172.21.15.27:0/1858132375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 msgr2=0x55bac7063f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4752590640 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 0x55bac7063f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4752590640 1 -- 172.21.15.27:0/1858132375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac7042080 con 0x55bac7041170 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4751d8f640 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 0x55bac7043b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4752590640 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041170 0x55bac703cf60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f473c00dc50 tx=0x7f473c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bf98ff16b1fb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4737fff640 1 -- 172.21.15.27:0/1858132375 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c014070 con 0x55bac7041170 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4737fff640 1 -- 172.21.15.27:0/1858132375 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473c01ddb0 con 0x55bac7041170 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.324+0000 7f4737fff640 1 -- 172.21.15.27:0/1858132375 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c01be20 con 0x55bac7041170 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 -- 172.21.15.27:0/1858132375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041170 msgr2=0x55bac703cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041170 0x55bac703cf60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f473c00dc50 tx=0x7f473c011000 comp rx=0 tx=0).stop 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 -- 172.21.15.27:0/1858132375 shutdown_connections 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac704c350 0x55bac7063f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 0x55bac7043b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 --2- 172.21.15.27:0/1858132375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac7041170 0x55bac703cf60 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 -- 172.21.15.27:0/1858132375 >> 172.21.15.27:0/1858132375 conn(0x55bac703d680 msgr2=0x55bac704b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 -- 172.21.15.27:0/1858132375 shutdown_connections 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 -- 172.21.15.27:0/1858132375 wait complete. 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 -- start start 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 Processor -- start 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041170 0x55bac7054a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 0x55bac704fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac7050330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f474c072cf0 con 0x55bac704c350 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1c5ed0 con 0x55bac7041170 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f474c0733c0 con 0x55bac7041a90 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.325+0000 7f47575d6740 10 client.0 _ll_register_callbacks cb 0x7f474401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752d91640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac7050330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752590640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041170 0x55bac7054a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752d91640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac7050330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52264/0 (socket says 172.21.15.27:52264) 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752d91640 1 -- 172.21.15.27:0/819665866 learned_addr learned my addr 172.21.15.27:0/819665866 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4751d8f640 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 0x55bac704fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752d91640 1 -- 172.21.15.27:0/819665866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 msgr2=0x55bac704fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752d91640 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 0x55bac704fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752d91640 1 -- 172.21.15.27:0/819665866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041170 msgr2=0x55bac7054a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752d91640 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041170 0x55bac7054a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752d91640 1 -- 172.21.15.27:0/819665866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac7042540 con 0x55bac704c350 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752590640 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041170 0x55bac7054a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4752d91640 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac7050330 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f47480bc370 tx=0x7f47480f0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1ad474f07a9c75 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4735ffb640 1 -- 172.21.15.27:0/819665866 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47480bc3e0 con 0x55bac704c350 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f47575d6740 1 -- 172.21.15.27:0/819665866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473c01ddb0 con 0x55bac704c350 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4735ffb640 1 -- 172.21.15.27:0/819665866 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f47480d52b0 con 0x55bac704c350 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4735ffb640 1 -- 172.21.15.27:0/819665866 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f47480c0790 con 0x55bac704c350 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f47575d6740 10 client.16419 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f47575d6740 1 -- 172.21.15.27:0/819665866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bac70c6d30 con 0x55bac704c350 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.326+0000 7f4735ffb640 1 -- 172.21.15.27:0/819665866 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473c01ddb0 con 0x55bac704c350 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f4735ffb640 1 -- 172.21.15.27:0/819665866 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f474800b040 con 0x55bac704c350 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f47575d6740 10 client.16419 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f47575d6740 10 client.16419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f47575d6740 20 client.16419 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f47575d6740 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bac70c7410 0x55bac70c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f47575d6740 4 client.16419 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:44:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f47575d6740 1 -- 172.21.15.27:0/819665866 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bac6fb2c20 con 0x55bac70c7410 2024-06-06T08:44:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f4752590640 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bac70c7410 0x55bac70c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f4735ffb640 10 client.16419 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.327+0000 7f4752590640 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bac70c7410 0x55bac70c9830 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d2fedbfb6b3aa server_cookie=df81a38af22cdcee in_seq=0 out_seq=0 2024-06-06T08:44:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f4735ffb640 1 -- 172.21.15.27:0/819665866 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bac6fb2c20 con 0x55bac70c7410 2024-06-06T08:44:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f4735ffb640 10 client.16419 handle_command_reply: tid=0 2024-06-06T08:44:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 client.16419 shutdown 2024-06-06T08:44:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 -- 172.21.15.27:0/819665866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 msgr2=0x55bac7050330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac7050330 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f47480bc370 tx=0x7f47480f0040 comp rx=0 tx=0).stop 2024-06-06T08:44:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 -- 172.21.15.27:0/819665866 shutdown_connections 2024-06-06T08:44:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bac70c7410 0x55bac70c9830 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bac704c350 0x55bac7050330 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bac7041a90 0x55bac704fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 --2- 172.21.15.27:0/819665866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bac7041170 0x55bac7054a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 -- 172.21.15.27:0/819665866 >> 172.21.15.27:0/819665866 conn(0x55bac703d680 msgr2=0x55bac70571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f4735ffb640 0 client.16419 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 -- 172.21.15.27:0/819665866 shutdown_connections 2024-06-06T08:44:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 1 -- 172.21.15.27:0/819665866 wait complete. 2024-06-06T08:44:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.558+0000 7f47575d6740 20 client.16419 trim_cache size 0 max 16384 2024-06-06T08:44:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:44:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:44:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:44:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 -- 172.21.15.27:0/3859631045 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f47380a2f50 msgr2=0x7f47380a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f47380a2f50 0x7f47380a5410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55bac7042710 tx=0x7f473c00f040 comp rx=0 tx=0).stop 2024-06-06T08:44:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 -- 172.21.15.27:0/3859631045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c10f500 msgr2=0x7f474c10f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c10f500 0x7f474c10f9b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f474800d9d0 tx=0x7f4748004ef0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 -- 172.21.15.27:0/3859631045 shutdown_connections 2024-06-06T08:44:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f47380a2f50 0x7f47380a5410 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f474c10f500 0x7f474c10f9b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c105380 0x7f474c116970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 --2- 172.21.15.27:0/3859631045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f474c1004e0 0x7f474c116430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 -- 172.21.15.27:0/3859631045 >> 172.21.15.27:0/3859631045 conn(0x7f474c075c20 msgr2=0x7f474c0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 -- 172.21.15.27:0/3859631045 shutdown_connections 2024-06-06T08:44:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:42.559+0000 7f47547f4640 1 -- 172.21.15.27:0/3859631045 wait complete. 2024-06-06T08:44:42.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:42 smithi027 ceph-mon[32335]: pgmap v136: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 516 op/s 2024-06-06T08:44:42.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:42 smithi203 ceph-mon[36579]: pgmap v136: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 516 op/s 2024-06-06T08:44:42.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:42 smithi176 ceph-mon[36786]: pgmap v136: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 516 op/s 2024-06-06T08:44:43.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.464+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2058342992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 msgr2=0x7fbe28107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:43.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.464+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2058342992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 0x7fbe28107830 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009990 tx=0x7fbe24009ca0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.464+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2058342992 shutdown_connections 2024-06-06T08:44:43.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.464+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2058342992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe28101440 0x7fbe2810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.464+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2058342992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 0x7fbe28107830 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.464+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2058342992 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe28100a80 0x7fbe28100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.464+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2058342992 >> 172.21.15.27:0/2058342992 conn(0x7fbe28077a30 msgr2=0x7fbe280fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.464+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2058342992 shutdown_connections 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.464+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2058342992 wait complete. 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2f59d700 1 Processor -- start 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2f59d700 1 -- start start 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2f59d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe28100a80 0x7fbe2819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2f59d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe28101440 0x7fbe2819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2f59d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 0x7fbe281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2f59d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28112800 con 0x7fbe28107450 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2f59d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28112680 con 0x7fbe28101440 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2f59d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28112980 con 0x7fbe28100a80 2024-06-06T08:44:43.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2db3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 0x7fbe281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2d339700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe28100a80 0x7fbe2819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2db3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 0x7fbe281989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52282/0 (socket says 172.21.15.27:52282) 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2db3a700 1 -- 172.21.15.27:0/2939500205 learned_addr learned my addr 172.21.15.27:0/2939500205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.465+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe28101440 0x7fbe2819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe2d339700 1 -- 172.21.15.27:0/2939500205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe28101440 msgr2=0x7fbe2819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe2d339700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe28101440 0x7fbe2819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe2d339700 1 -- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 msgr2=0x7fbe281989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe2d339700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 0x7fbe281989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe2d339700 1 -- 172.21.15.27:0/2939500205 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe28199100 con 0x7fbe28100a80 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe2db3a700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 0x7fbe281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe2d339700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe28100a80 0x7fbe2819e7e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe240059d0 tx=0x7fbe2400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe1e7fc700 1 -- 172.21.15.27:0/2939500205 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24027070 con 0x7fbe28100a80 2024-06-06T08:44:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe1e7fc700 1 -- 172.21.15.27:0/2939500205 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe24003d70 con 0x7fbe28100a80 2024-06-06T08:44:43.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe1e7fc700 1 -- 172.21.15.27:0/2939500205 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24022e30 con 0x7fbe28100a80 2024-06-06T08:44:43.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2939500205 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe28199390 con 0x7fbe28100a80 2024-06-06T08:44:43.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.466+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2939500205 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe281a56b0 con 0x7fbe28100a80 2024-06-06T08:44:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.469+0000 7fbe1e7fc700 1 -- 172.21.15.27:0/2939500205 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbe2402a070 con 0x7fbe28100a80 2024-06-06T08:44:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.469+0000 7fbe1e7fc700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbe140a2e30 0x7fbe140a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.469+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbe140a2e30 0x7fbe140a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.469+0000 7fbe1e7fc700 1 -- 172.21.15.27:0/2939500205 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fbe240d0520 con 0x7fbe28100a80 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.470+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbe140a2e30 0x7fbe140a52f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe28106ae0 tx=0x7fbe18009260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.470+0000 7fbe39e20b80 1 Processor -- start 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.470+0000 7fbe39e20b80 1 -- start start 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.470+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 0x55c8c3161d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.470+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c315bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.470+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c315c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.470+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c8c304a100 con 0x55c8c315c100 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.470+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c8c30a89f0 con 0x55c8c31622d0 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.470+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c8c3010600 con 0x55c8c31619b0 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2cb38700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c315bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2d339700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 0x55c8c3161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2cb38700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c315bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:42208/0 (socket says 172.21.15.27:42208) 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2cb38700 1 -- 172.21.15.27:0/3559058780 learned_addr learned my addr 172.21.15.27:0/3559058780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2db3a700 1 --2- 172.21.15.27:0/3559058780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c315c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2cb38700 1 -- 172.21.15.27:0/3559058780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 msgr2=0x55c8c3161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/3559058780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 0x55c8c3161d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2cb38700 1 -- 172.21.15.27:0/3559058780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 msgr2=0x55c8c315c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/3559058780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c315c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2cb38700 1 -- 172.21.15.27:0/3559058780 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c2ff7a70 con 0x55c8c31622d0 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2d339700 1 --2- 172.21.15.27:0/3559058780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 0x55c8c3161d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/3559058780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c315bbc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe18011d50 tx=0x7fbe18014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25d81fec88cfe9fb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.471+0000 7fbe137fe700 1 -- 172.21.15.27:0/3559058780 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800eb20 con 0x55c8c31622d0 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe137fe700 1 -- 172.21.15.27:0/3559058780 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1800ecc0 con 0x55c8c31622d0 2024-06-06T08:44:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe137fe700 1 -- 172.21.15.27:0/3559058780 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1801d730 con 0x55c8c31622d0 2024-06-06T08:44:43.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe39e20b80 1 -- 172.21.15.27:0/3559058780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 msgr2=0x55c8c315bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:43.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/3559058780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c315bbc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe18011d50 tx=0x7fbe18014000 comp rx=0 tx=0).stop 2024-06-06T08:44:43.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe39e20b80 1 -- 172.21.15.27:0/3559058780 shutdown_connections 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/3559058780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c315c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/3559058780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 0x55c8c3161d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/3559058780 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c315bbc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe39e20b80 1 -- 172.21.15.27:0/3559058780 >> 172.21.15.27:0/3559058780 conn(0x55c8c3084220 msgr2=0x55c8c3152aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe39e20b80 1 -- 172.21.15.27:0/3559058780 shutdown_connections 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.472+0000 7fbe39e20b80 1 -- 172.21.15.27:0/3559058780 wait complete. 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.473+0000 7fbe39e20b80 1 -- start start 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.473+0000 7fbe39e20b80 1 Processor -- start 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.473+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3167fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 0x55c8c31652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c3165810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c8c3049160 con 0x55c8c315c100 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2d339700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3167fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55c8c30a49b0 con 0x55c8c31622d0 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c8c3168d00 con 0x55c8c31619b0 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2d339700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3167fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52302/0 (socket says 172.21.15.27:52302) 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2d339700 1 -- 172.21.15.27:0/2422617767 learned_addr learned my addr 172.21.15.27:0/2422617767 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe39e20b80 10 client.0 _ll_register_callbacks cb 0x55c8c303e750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2db3a700 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c3165810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 0x55c8c31652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2d339700 1 -- 172.21.15.27:0/2422617767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 msgr2=0x55c8c31652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2d339700 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 0x55c8c31652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2d339700 1 -- 172.21.15.27:0/2422617767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 msgr2=0x55c8c3165810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2d339700 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c3165810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2d339700 1 -- 172.21.15.27:0/2422617767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c306e8f0 con 0x55c8c315c100 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe2d339700 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3167fd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbe2409c450 tx=0x7fbe2409c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb3d568a6333c28 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe10ff9700 1 -- 172.21.15.27:0/2422617767 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2409ce50 con 0x55c8c315c100 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe39e20b80 1 -- 172.21.15.27:0/2422617767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1800ecc0 con 0x55c8c315c100 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.474+0000 7fbe10ff9700 1 -- 172.21.15.27:0/2422617767 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fbe240d01b0 con 0x55c8c315c100 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe10ff9700 1 -- 172.21.15.27:0/2422617767 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbe2409c750 con 0x55c8c315c100 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe39e20b80 10 client.16431 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe39e20b80 1 -- 172.21.15.27:0/2422617767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c8c31577b0 con 0x55c8c315c100 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe10ff9700 1 -- 172.21.15.27:0/2422617767 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1800ecc0 con 0x55c8c315c100 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe10ff9700 1 -- 172.21.15.27:0/2422617767 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbe240a4ea0 con 0x55c8c315c100 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe39e20b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe39e20b80 10 client.16431 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe39e20b80 20 client.16431 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c8c3159d40 0x55c8c315a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe39e20b80 4 client.16431 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe39e20b80 1 -- 172.21.15.27:0/2422617767 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8c30b5ef0 con 0x55c8c3159d40 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.475+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c8c3159d40 0x55c8c315a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.476+0000 7fbe10ff9700 10 client.16431 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.476+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c8c3159d40 0x55c8c315a1b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b374352e5751ee server_cookie=a39a58e1461b890b in_seq=0 out_seq=0 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.476+0000 7fbe10ff9700 1 -- 172.21.15.27:0/2422617767 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c8c30b5ef0 con 0x55c8c3159d40 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.476+0000 7fbe10ff9700 10 client.16431 handle_command_reply: tid=0 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.476+0000 7fbe39e20b80 1 client.16431 shutdown 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.476+0000 7fbe39e20b80 1 -- 172.21.15.27:0/2422617767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 msgr2=0x55c8c3167fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.476+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3167fd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbe2409c450 tx=0x7fbe2409c0b0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe39e20b80 1 -- 172.21.15.27:0/2422617767 shutdown_connections 2024-06-06T08:44:43.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c3165810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c8c3159d40 0x55c8c315a1b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3167fd0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/2422617767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31619b0 0x55c8c31652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe10ff9700 0 client.16431 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:43.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe39e20b80 1 -- 172.21.15.27:0/2422617767 >> 172.21.15.27:0/2422617767 conn(0x55c8c3084220 msgr2=0x55c8c31567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:43.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe39e20b80 1 -- 172.21.15.27:0/2422617767 shutdown_connections 2024-06-06T08:44:43.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe39e20b80 1 -- 172.21.15.27:0/2422617767 wait complete. 2024-06-06T08:44:43.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.477+0000 7fbe39e20b80 20 client.16431 trim_cache size 0 max 16384 2024-06-06T08:44:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.499+0000 7fbe39e20b80 1 Processor -- start 2024-06-06T08:44:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.499+0000 7fbe39e20b80 1 -- start start 2024-06-06T08:44:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.499+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c315c100 0x55c8c3151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.499+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c31619b0 0x55c8c3167b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.499+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c3164b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.499+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281aeb60 con 0x55c8c31619b0 2024-06-06T08:44:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.499+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281ac330 con 0x55c8c31622d0 2024-06-06T08:44:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.499+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281aa180 con 0x55c8c315c100 2024-06-06T08:44:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2cb38700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c31619b0 0x55c8c3167b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2cb38700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c31619b0 0x55c8c3167b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52304/0 (socket says 172.21.15.27:52304) 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2cb38700 1 -- 172.21.15.27:0/1227258850 learned_addr learned my addr 172.21.15.27:0/1227258850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2db3a700 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c3164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2d339700 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c315c100 0x55c8c3151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2cb38700 1 -- 172.21.15.27:0/1227258850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c315c100 msgr2=0x55c8c3151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c315c100 0x55c8c3151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2cb38700 1 -- 172.21.15.27:0/1227258850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 msgr2=0x55c8c3164b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c3164b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2cb38700 1 -- 172.21.15.27:0/1227258850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c315a830 con 0x55c8c31619b0 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2db3a700 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c3164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:44:43.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2d339700 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c315c100 0x55c8c3151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:44:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c31619b0 0x55c8c3167b10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbe18011d50 tx=0x7fbe1801ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b882e8c65312112 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe12ffd700 1 -- 172.21.15.27:0/1227258850 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18034070 con 0x55c8c31619b0 2024-06-06T08:44:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe12ffd700 1 -- 172.21.15.27:0/1227258850 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe18005520 con 0x55c8c31619b0 2024-06-06T08:44:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.500+0000 7fbe12ffd700 1 -- 172.21.15.27:0/1227258850 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18003a00 con 0x55c8c31619b0 2024-06-06T08:44:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.501+0000 7fbe39e20b80 1 -- 172.21.15.27:0/1227258850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c31619b0 msgr2=0x55c8c3167b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.501+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c31619b0 0x55c8c3167b10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbe18011d50 tx=0x7fbe1801ce80 comp rx=0 tx=0).stop 2024-06-06T08:44:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.501+0000 7fbe39e20b80 1 -- 172.21.15.27:0/1227258850 shutdown_connections 2024-06-06T08:44:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.501+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31622d0 0x55c8c3164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.501+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c315c100 0x55c8c3151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.501+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/1227258850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c31619b0 0x55c8c3167b10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.501+0000 7fbe39e20b80 1 -- 172.21.15.27:0/1227258850 >> 172.21.15.27:0/1227258850 conn(0x55c8c3084220 msgr2=0x55c8c31513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.501+0000 7fbe39e20b80 1 -- 172.21.15.27:0/1227258850 shutdown_connections 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.501+0000 7fbe39e20b80 1 -- 172.21.15.27:0/1227258850 wait complete. 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe39e20b80 1 -- start start 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe39e20b80 1 Processor -- start 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3178a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31619b0 0x55c8c3175e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe39e20b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31622d0 0x55c8c3176390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281aa7f0 con 0x55c8c315c100 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281abc60 con 0x55c8c31619b0 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe39e20b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281aaec0 con 0x55c8c31622d0 2024-06-06T08:44:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe39e20b80 10 client.0 _ll_register_callbacks cb 0x55c8c315a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.502+0000 7fbe2d339700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3178a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2d339700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3178a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52316/0 (socket says 172.21.15.27:52316) 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2d339700 1 -- 172.21.15.27:0/4231633831 learned_addr learned my addr 172.21.15.27:0/4231633831 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31619b0 0x55c8c3175e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2db3a700 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31622d0 0x55c8c3176390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2d339700 1 -- 172.21.15.27:0/4231633831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31622d0 msgr2=0x55c8c3176390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2d339700 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31622d0 0x55c8c3176390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2d339700 1 -- 172.21.15.27:0/4231633831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31619b0 msgr2=0x55c8c3175e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2d339700 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31619b0 0x55c8c3175e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2d339700 1 -- 172.21.15.27:0/4231633831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c3176bd0 con 0x55c8c315c100 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe2d339700 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3178a10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009990 tx=0x7fbe24005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634201baf7f8df1d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe10ff9700 1 -- 172.21.15.27:0/4231633831 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24003a90 con 0x55c8c315c100 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe10ff9700 1 -- 172.21.15.27:0/4231633831 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fbe240984a0 con 0x55c8c315c100 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe39e20b80 1 -- 172.21.15.27:0/4231633831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe18005520 con 0x55c8c315c100 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe10ff9700 1 -- 172.21.15.27:0/4231633831 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbe240b0be0 con 0x55c8c315c100 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.503+0000 7fbe39e20b80 10 client.16437 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe39e20b80 1 -- 172.21.15.27:0/4231633831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c8c3176bd0 con 0x55c8c315c100 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe10ff9700 1 -- 172.21.15.27:0/4231633831 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe18005520 con 0x55c8c315c100 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe10ff9700 1 -- 172.21.15.27:0/4231633831 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbe240986a0 con 0x55c8c315c100 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe39e20b80 10 client.16437 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe39e20b80 10 client.16437 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe39e20b80 20 client.16437 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c8c31dd890 0x55c8c31dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe39e20b80 4 client.16437 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-06-06T08:44:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe39e20b80 1 -- 172.21.15.27:0/4231633831 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x55c8c30b0470 con 0x55c8c31dd890 2024-06-06T08:44:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.504+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c8c31dd890 0x55c8c31dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.505+0000 7fbe2cb38700 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c8c31dd890 0x55c8c31dfcb0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e88c36fcf8c35f1 server_cookie=c8e0bc5b115c61d8 in_seq=0 out_seq=0 2024-06-06T08:44:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:43.505+0000 7fbe10ff9700 10 client.16437 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:44.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:44 smithi027 ceph-mon[32335]: pgmap v137: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 589 op/s 2024-06-06T08:44:44.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:44 smithi203 ceph-mon[36579]: pgmap v137: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 589 op/s 2024-06-06T08:44:44.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:44 smithi176 ceph-mon[36786]: pgmap v137: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 589 op/s 2024-06-06T08:44:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe10ff9700 1 -- 172.21.15.27:0/4231633831 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c8c30b0470 con 0x55c8c31dd890 2024-06-06T08:44:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe10ff9700 10 client.16437 handle_command_reply: tid=0 2024-06-06T08:44:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 client.16437 shutdown 2024-06-06T08:44:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 -- 172.21.15.27:0/4231633831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 msgr2=0x55c8c3178a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3178a10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009990 tx=0x7fbe24005fb0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 -- 172.21.15.27:0/4231633831 shutdown_connections 2024-06-06T08:44:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c31622d0 0x55c8c3176390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55c8c31dd890 0x55c8c31dfcb0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c8c315c100 0x55c8c3178a10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe10ff9700 0 client.16437 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 --2- 172.21.15.27:0/4231633831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c8c31619b0 0x55c8c3175e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 -- 172.21.15.27:0/4231633831 >> 172.21.15.27:0/4231633831 conn(0x55c8c3084220 msgr2=0x55c8c3157030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 -- 172.21.15.27:0/4231633831 shutdown_connections 2024-06-06T08:44:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.412+0000 7fbe39e20b80 1 -- 172.21.15.27:0/4231633831 wait complete. 2024-06-06T08:44:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.413+0000 7fbe39e20b80 20 client.16437 trim_cache size 0 max 16384 2024-06-06T08:44:45.414 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:44:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-06-06T08:44:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24482, 2024-06-06T08:44:45.414 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 10, 2024-06-06T08:44:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-06-06T08:44:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "6bc9d658": { 2024-06-06T08:44:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 10, 2024-06-06T08:44:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-06-06T08:44:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:44:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-06-06T08:44:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:44:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:44:45.415 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-06-06T08:44:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 145.2, 2024-06-06T08:44:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-06-06T08:44:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-06-06T08:44:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-06-06T08:44:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:44:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-06-06T08:44:45.416 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:44:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:44:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:44:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:44:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:44:45.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:44:45.417 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:44:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbe140a2e30 msgr2=0x7fbe140a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbe140a2e30 0x7fbe140a52f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe28106ae0 tx=0x7fbe18009260 comp rx=0 tx=0).stop 2024-06-06T08:44:45.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2939500205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe28100a80 msgr2=0x7fbe2819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe28100a80 0x7fbe2819e7e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe240059d0 tx=0x7fbe2400bab0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2939500205 shutdown_connections 2024-06-06T08:44:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28107450 0x7fbe281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe28100a80 0x7fbe2819e7e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fbe140a2e30 0x7fbe140a52f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 --2- 172.21.15.27:0/2939500205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe28101440 0x7fbe2819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2939500205 >> 172.21.15.27:0/2939500205 conn(0x7fbe28077a30 msgr2=0x7fbe28105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2939500205 shutdown_connections 2024-06-06T08:44:45.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:44:45.414+0000 7fbe2f59d700 1 -- 172.21.15.27:0/2939500205 wait complete. 2024-06-06T08:44:46.186 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6bc9d658', quiesce took 8.2 seconds. Will release after: 17.05 2024-06-06T08:44:46.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:46 smithi027 ceph-mon[32335]: pgmap v138: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 695 op/s 2024-06-06T08:44:46.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:46 smithi203 ceph-mon[36579]: pgmap v138: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 695 op/s 2024-06-06T08:44:46.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:46 smithi176 ceph-mon[36786]: pgmap v138: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 695 op/s 2024-06-06T08:44:48.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:48 smithi027 ceph-mon[32335]: pgmap v139: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 480 op/s 2024-06-06T08:44:48.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:48 smithi203 ceph-mon[36579]: pgmap v139: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 480 op/s 2024-06-06T08:44:48.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:48 smithi176 ceph-mon[36786]: pgmap v139: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 480 op/s 2024-06-06T08:44:50.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:50 smithi027 ceph-mon[32335]: pgmap v140: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 519 op/s 2024-06-06T08:44:50.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:50 smithi203 ceph-mon[36579]: pgmap v140: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 519 op/s 2024-06-06T08:44:50.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:50 smithi176 ceph-mon[36786]: pgmap v140: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 519 op/s 2024-06-06T08:44:52.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:52 smithi027 ceph-mon[32335]: pgmap v141: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 734 KiB/s wr, 303 op/s 2024-06-06T08:44:52.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:52 smithi203 ceph-mon[36579]: pgmap v141: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 734 KiB/s wr, 303 op/s 2024-06-06T08:44:52.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:52 smithi176 ceph-mon[36786]: pgmap v141: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 734 KiB/s wr, 303 op/s 2024-06-06T08:44:54.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:54 smithi203 ceph-mon[36579]: pgmap v142: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 734 KiB/s wr, 306 op/s 2024-06-06T08:44:54.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:54 smithi176 ceph-mon[36786]: pgmap v142: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 734 KiB/s wr, 306 op/s 2024-06-06T08:44:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:54 smithi027 ceph-mon[32335]: pgmap v142: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 734 KiB/s wr, 306 op/s 2024-06-06T08:44:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 -- 172.21.15.27:0/4003044113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a810c020 msgr2=0x7fb1a810e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 --2- 172.21.15.27:0/4003044113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a810c020 0x7fb1a810e450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009ca0 tx=0x7fb1a002a950 comp rx=0 tx=0).stop 2024-06-06T08:44:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 -- 172.21.15.27:0/4003044113 shutdown_connections 2024-06-06T08:44:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 --2- 172.21.15.27:0/4003044113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a810c020 0x7fb1a810e450 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 --2- 172.21.15.27:0/4003044113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a8103570 0x7fb1a810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 --2- 172.21.15.27:0/4003044113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a8102c50 0x7fb1a8103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 -- 172.21.15.27:0/4003044113 >> 172.21.15.27:0/4003044113 conn(0x7fb1a8076c20 msgr2=0x7fb1a8077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 -- 172.21.15.27:0/4003044113 shutdown_connections 2024-06-06T08:44:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 -- 172.21.15.27:0/4003044113 wait complete. 2024-06-06T08:44:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 Processor -- start 2024-06-06T08:44:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.661+0000 7fb1add6a640 1 -- start start 2024-06-06T08:44:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb1add6a640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a8102c50 0x7fb1a81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb1add6a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a8103570 0x7fb1a8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb1add6a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a810c020 0x7fb1a819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb1add6a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a8111db0 con 0x7fb1a8103570 2024-06-06T08:44:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb1add6a640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a8111c30 con 0x7fb1a8102c50 2024-06-06T08:44:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb1add6a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a8111f30 con 0x7fb1a810c020 2024-06-06T08:44:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a8103570 0x7fb1a8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a8103570 0x7fb1a8198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35186/0 (socket says 172.21.15.27:35186) 2024-06-06T08:44:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb19effd640 1 -- 172.21.15.27:0/3215813891 learned_addr learned my addr 172.21.15.27:0/3215813891 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb1a77fe640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a8102c50 0x7fb1a81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a810c020 0x7fb1a819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb19effd640 1 -- 172.21.15.27:0/3215813891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a810c020 msgr2=0x7fb1a819c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb19effd640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a810c020 0x7fb1a819c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb19effd640 1 -- 172.21.15.27:0/3215813891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a8102c50 msgr2=0x7fb1a81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb19effd640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a8102c50 0x7fb1a81980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.662+0000 7fb19effd640 1 -- 172.21.15.27:0/3215813891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a819d0c0 con 0x7fb1a8103570 2024-06-06T08:44:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.663+0000 7fb19effd640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a8103570 0x7fb1a8198610 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e9f0 tx=0x7fb19000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.663+0000 7fb1a57fa640 1 -- 172.21.15.27:0/3215813891 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19000cd80 con 0x7fb1a8103570 2024-06-06T08:44:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.663+0000 7fb1a57fa640 1 -- 172.21.15.27:0/3215813891 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb190010910 con 0x7fb1a8103570 2024-06-06T08:44:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.663+0000 7fb1add6a640 1 -- 172.21.15.27:0/3215813891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a819d3b0 con 0x7fb1a8103570 2024-06-06T08:44:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.663+0000 7fb1a57fa640 1 -- 172.21.15.27:0/3215813891 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190018990 con 0x7fb1a8103570 2024-06-06T08:44:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.663+0000 7fb1add6a640 1 -- 172.21.15.27:0/3215813891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a81a4c40 con 0x7fb1a8103570 2024-06-06T08:44:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.665+0000 7fb1a57fa640 1 -- 172.21.15.27:0/3215813891 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb190010ac0 con 0x7fb1a8103570 2024-06-06T08:44:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.665+0000 7fb1b0a70740 1 Processor -- start 2024-06-06T08:44:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1b0a70740 1 -- start start 2024-06-06T08:44:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a161c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 0x5575a1626e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a1627350 0x5575a1629740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5575a15804f0 con 0x5575a1627350 2024-06-06T08:44:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5575a151e870 con 0x5575a161ca90 2024-06-06T08:44:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5575a157c4b0 con 0x5575a161c170 2024-06-06T08:44:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a1627350 0x5575a1629740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a1627350 0x5575a1629740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35192/0 (socket says 172.21.15.27:35192) 2024-06-06T08:44:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb19effd640 1 -- 172.21.15.27:0/1179013287 learned_addr learned my addr 172.21.15.27:0/1179013287 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1a77fe640 1 --2- 172.21.15.27:0/1179013287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 0x5575a1626e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/1179013287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a161c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.666+0000 7fb1a57fa640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1940a2e20 0x7fb1940a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb19effd640 1 -- 172.21.15.27:0/1179013287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 msgr2=0x5575a161c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb19effd640 1 --2- 172.21.15.27:0/1179013287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a161c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb19effd640 1 -- 172.21.15.27:0/1179013287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 msgr2=0x5575a1626e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb19effd640 1 --2- 172.21.15.27:0/1179013287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 0x5575a1626e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1940a2e20 0x7fb1940a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb19effd640 1 -- 172.21.15.27:0/1179013287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a153a9b0 con 0x5575a1627350 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb1a57fa640 1 -- 172.21.15.27:0/3215813891 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fb1900cfb60 con 0x7fb1a8103570 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb19effd640 1 --2- 172.21.15.27:0/1179013287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a1627350 0x5575a1629740 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb190098100 tx=0x7fb190024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e41857426426e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1940a2e20 0x7fb1940a52e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb1a002ae70 tx=0x7fb1a002a950 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb19dffb640 1 -- 172.21.15.27:0/1179013287 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190020bf0 con 0x5575a1627350 2024-06-06T08:44:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb19dffb640 1 -- 172.21.15.27:0/1179013287 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1900d4040 con 0x5575a1627350 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.667+0000 7fb19dffb640 1 -- 172.21.15.27:0/1179013287 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190097f40 con 0x5575a1627350 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 -- 172.21.15.27:0/1179013287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a1627350 msgr2=0x5575a1629740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/1179013287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a1627350 0x5575a1629740 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb190098100 tx=0x7fb190024000 comp rx=0 tx=0).stop 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 -- 172.21.15.27:0/1179013287 shutdown_connections 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/1179013287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a1627350 0x5575a1629740 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/1179013287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 0x5575a1626e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/1179013287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a161c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 -- 172.21.15.27:0/1179013287 >> 172.21.15.27:0/1179013287 conn(0x5575a1618680 msgr2=0x5575a1618a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 -- 172.21.15.27:0/1179013287 shutdown_connections 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 -- 172.21.15.27:0/1179013287 wait complete. 2024-06-06T08:44:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 -- start start 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.668+0000 7fb1b0a70740 1 Processor -- start 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161c170 0x5575a1624270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 0x5575a16247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a1627350 0x5575a1624cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5575a1629f20 con 0x5575a161c170 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5575a1509ae0 con 0x5575a161ca90 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5575a162a0a0 con 0x5575a1627350 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1b0a70740 10 client.0 _ll_register_callbacks cb 0x5575a15429b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161c170 0x5575a1624270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161c170 0x5575a1624270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35198/0 (socket says 172.21.15.27:35198) 2024-06-06T08:44:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a77fe640 1 -- 172.21.15.27:0/610318371 learned_addr learned my addr 172.21.15.27:0/610318371 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a1627350 0x5575a1624cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb19effd640 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 0x5575a16247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a77fe640 1 -- 172.21.15.27:0/610318371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a1627350 msgr2=0x5575a1624cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a77fe640 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a1627350 0x5575a1624cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a77fe640 1 -- 172.21.15.27:0/610318371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 msgr2=0x5575a16247b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a77fe640 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 0x5575a16247b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a77fe640 1 -- 172.21.15.27:0/610318371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a1294210 con 0x5575a161c170 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a1627350 0x5575a1624cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb1a77fe640 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161c170 0x5575a1624270 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb19800cef0 tx=0x7fb1980039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb2353f5ac44b9d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.669+0000 7fb186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb186ffd640 1 -- 172.21.15.27:0/610318371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1980119f0 con 0x5575a161c170 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb186ffd640 1 -- 172.21.15.27:0/610318371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fb198011b90 con 0x5575a161c170 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb1b0a70740 1 -- 172.21.15.27:0/610318371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1900d4040 con 0x5575a161c170 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb186ffd640 1 -- 172.21.15.27:0/610318371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb198011d90 con 0x5575a161c170 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb1b0a70740 10 client.16449 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb1b0a70740 1 -- 172.21.15.27:0/610318371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5575a1360620 con 0x5575a161c170 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb186ffd640 1 -- 172.21.15.27:0/610318371 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1900d4040 con 0x5575a161c170 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb186ffd640 1 -- 172.21.15.27:0/610318371 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb198018070 con 0x5575a161c170 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb1b0a70740 10 client.16449 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb1b0a70740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb1b0a70740 20 client.16449 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5575a16375c0 0x5575a1639a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb1b0a70740 4 client.16449 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.670+0000 7fb1b0a70740 1 -- 172.21.15.27:0/610318371 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575a1593f10 con 0x5575a16375c0 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.671+0000 7fb19effd640 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5575a16375c0 0x5575a1639a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.671+0000 7fb19effd640 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5575a16375c0 0x5575a1639a10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d348edf964fb56d server_cookie=4578fb367d86cf1d in_seq=0 out_seq=0 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.671+0000 7fb186ffd640 10 client.16449 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb186ffd640 1 -- 172.21.15.27:0/610318371 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5575a1593f10 con 0x5575a16375c0 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb186ffd640 10 client.16449 handle_command_reply: tid=0 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 client.16449 shutdown 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 -- 172.21.15.27:0/610318371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161c170 msgr2=0x5575a1624270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161c170 0x5575a1624270 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb19800cef0 tx=0x7fb1980039c0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 -- 172.21.15.27:0/610318371 shutdown_connections 2024-06-06T08:44:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5575a16375c0 0x5575a1639a10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a1627350 0x5575a1624cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a161ca90 0x5575a16247b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/610318371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161c170 0x5575a1624270 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb186ffd640 0 client.16449 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 -- 172.21.15.27:0/610318371 >> 172.21.15.27:0/610318371 conn(0x5575a1618680 msgr2=0x5575a161e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 -- 172.21.15.27:0/610318371 shutdown_connections 2024-06-06T08:44:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 1 -- 172.21.15.27:0/610318371 wait complete. 2024-06-06T08:44:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.672+0000 7fb1b0a70740 20 client.16449 trim_cache size 0 max 16384 2024-06-06T08:44:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb1b0a70740 1 Processor -- start 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb1b0a70740 1 -- start start 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a161eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a163f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a163f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81b1750 con 0x5575a161ca90 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81aef60 con 0x5575a1627350 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81acdb0 con 0x5575a161c170 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a163f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a163f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35214/0 (socket says 172.21.15.27:35214) 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.688+0000 7fb19effd640 1 -- 172.21.15.27:0/3399415545 learned_addr learned my addr 172.21.15.27:0/3399415545 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb1a77fe640 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a161eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a163f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb19effd640 1 -- 172.21.15.27:0/3399415545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 msgr2=0x5575a161eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb19effd640 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a161eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb19effd640 1 -- 172.21.15.27:0/3399415545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 msgr2=0x5575a163f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb19effd640 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a163f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb19effd640 1 -- 172.21.15.27:0/3399415545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a161cfd0 con 0x5575a161ca90 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a163f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb19effd640 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a163f320 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb190099f50 tx=0x7fb190000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd75b5a71ffe5a83 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb19d7fa640 1 -- 172.21.15.27:0/3399415545 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19009c3a0 con 0x5575a161ca90 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb19d7fa640 1 -- 172.21.15.27:0/3399415545 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1900a0050 con 0x5575a161ca90 2024-06-06T08:44:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.689+0000 7fb19d7fa640 1 -- 172.21.15.27:0/3399415545 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900d3400 con 0x5575a161ca90 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3399415545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 msgr2=0x5575a163f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a163f320 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb190099f50 tx=0x7fb190000bc0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3399415545 shutdown_connections 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a163f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a163f320 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3399415545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a161eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3399415545 >> 172.21.15.27:0/3399415545 conn(0x5575a1618680 msgr2=0x5575a161f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3399415545 shutdown_connections 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3399415545 wait complete. 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 -- start start 2024-06-06T08:44:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.690+0000 7fb1b0a70740 1 Processor -- start 2024-06-06T08:44:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a162f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a162fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1b0a70740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a162a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81ad420 con 0x5575a161ca90 2024-06-06T08:44:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81ae890 con 0x5575a1627350 2024-06-06T08:44:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1b0a70740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81adaf0 con 0x5575a161c170 2024-06-06T08:44:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1b0a70740 10 client.0 _ll_register_callbacks cb 0x7fb198004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a162fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a162fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35230/0 (socket says 172.21.15.27:35230) 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb19effd640 1 -- 172.21.15.27:0/3909385847 learned_addr learned my addr 172.21.15.27:0/3909385847 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1a77fe640 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a162f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a162a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb19effd640 1 -- 172.21.15.27:0/3909385847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 msgr2=0x5575a162f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb19effd640 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a162f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb19effd640 1 -- 172.21.15.27:0/3909385847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 msgr2=0x5575a162a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb19effd640 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a162a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb19effd640 1 -- 172.21.15.27:0/3909385847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a162b0b0 con 0x5575a161ca90 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb1a7fff640 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a162a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb19effd640 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a162fa10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb190098100 tx=0x7fb1900ba7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfaf26ca097da7d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb186ffd640 1 -- 172.21.15.27:0/3909385847 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190020bf0 con 0x5575a161ca90 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.691+0000 7fb186ffd640 1 -- 172.21.15.27:0/3909385847 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fb1900d33e0 con 0x5575a161ca90 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3909385847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1900a0050 con 0x5575a161ca90 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb186ffd640 1 -- 172.21.15.27:0/3909385847 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb19009c3a0 con 0x5575a161ca90 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1b0a70740 10 client.16455 fetch_fsmap learned FSMap version 34 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3909385847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5575a16a1d80 con 0x5575a161ca90 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb186ffd640 1 -- 172.21.15.27:0/3909385847 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1900a0050 con 0x5575a161ca90 2024-06-06T08:44:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb186ffd640 1 -- 172.21.15.27:0/3909385847 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb190098310 con 0x5575a161ca90 2024-06-06T08:44:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1b0a70740 10 client.16455 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:44:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1b0a70740 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:44:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1b0a70740 20 client.16455 populate_metadata read hostname 'smithi027' 2024-06-06T08:44:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5575a16a2000 0x5575a16a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:44:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1b0a70740 4 client.16455 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:44:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3909385847 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575a158dc20 con 0x5575a16a2000 2024-06-06T08:44:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.692+0000 7fb1a77fe640 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5575a16a2000 0x5575a16a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:44:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.693+0000 7fb186ffd640 10 client.16455 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.693+0000 7fb1a77fe640 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5575a16a2000 0x5575a16a43f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9299020f2c4aa740 server_cookie=a638d611a1fb8d40 in_seq=0 out_seq=0 2024-06-06T08:44:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb186ffd640 1 -- 172.21.15.27:0/3909385847 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5575a158dc20 con 0x5575a16a2000 2024-06-06T08:44:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb186ffd640 10 client.16455 handle_command_reply: tid=0 2024-06-06T08:44:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 client.16455 shutdown 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3909385847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 msgr2=0x5575a162fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a162fa10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb190098100 tx=0x7fb1900ba7d0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3909385847 shutdown_connections 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5575a16a2000 0x5575a16a43f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5575a1627350 0x5575a162a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5575a161ca90 0x5575a162fa10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb186ffd640 0 client.16455 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 --2- 172.21.15.27:0/3909385847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575a161c170 0x5575a162f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3909385847 >> 172.21.15.27:0/3909385847 conn(0x5575a1618680 msgr2=0x5575a1616f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3909385847 shutdown_connections 2024-06-06T08:44:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 1 -- 172.21.15.27:0/3909385847 wait complete. 2024-06-06T08:44:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.745+0000 7fb1b0a70740 20 client.16455 trim_cache size 0 max 16384 2024-06-06T08:44:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:44:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:44:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:44:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 -- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1940a2e20 msgr2=0x7fb1940a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1940a2e20 0x7fb1940a52e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb1a002ae70 tx=0x7fb1a002a950 comp rx=0 tx=0).stop 2024-06-06T08:44:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 -- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a8103570 msgr2=0x7fb1a8198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:44:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a8103570 0x7fb1a8198610 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e9f0 tx=0x7fb19000eeb0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 -- 172.21.15.27:0/3215813891 shutdown_connections 2024-06-06T08:44:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb1940a2e20 0x7fb1940a52e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a810c020 0x7fb1a819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a8103570 0x7fb1a8198610 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 --2- 172.21.15.27:0/3215813891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a8102c50 0x7fb1a81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:44:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 -- 172.21.15.27:0/3215813891 >> 172.21.15.27:0/3215813891 conn(0x7fb1a8076c20 msgr2=0x7fb1a80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:44:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 -- 172.21.15.27:0/3215813891 shutdown_connections 2024-06-06T08:44:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:44:55.746+0000 7fb1add6a640 1 -- 172.21.15.27:0/3215813891 wait complete. 2024-06-06T08:44:56.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:56 smithi203 ceph-mon[36579]: pgmap v143: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 411 KiB/s wr, 243 op/s 2024-06-06T08:44:56.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:56 smithi176 ceph-mon[36786]: pgmap v143: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 411 KiB/s wr, 243 op/s 2024-06-06T08:44:56.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:56 smithi027 ceph-mon[32335]: pgmap v143: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 411 KiB/s wr, 243 op/s 2024-06-06T08:44:58.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:44:58 smithi203 ceph-mon[36579]: pgmap v144: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-06-06T08:44:58.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:44:58 smithi176 ceph-mon[36786]: pgmap v144: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-06-06T08:44:58.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:44:58 smithi027 ceph-mon[32335]: pgmap v144: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-06-06T08:45:00.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:00 smithi203 ceph-mon[36579]: pgmap v145: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 90 KiB/s wr, 79 op/s 2024-06-06T08:45:00.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:00 smithi176 ceph-mon[36786]: pgmap v145: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 90 KiB/s wr, 79 op/s 2024-06-06T08:45:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:00 smithi027 ceph-mon[32335]: pgmap v145: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 90 KiB/s wr, 79 op/s 2024-06-06T08:45:02.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:02 smithi027 ceph-mon[32335]: pgmap v146: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 53 KiB/s wr, 40 op/s 2024-06-06T08:45:02.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:02 smithi203 ceph-mon[36579]: pgmap v146: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 53 KiB/s wr, 40 op/s 2024-06-06T08:45:02.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:02 smithi176 ceph-mon[36786]: pgmap v146: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 53 KiB/s wr, 40 op/s 2024-06-06T08:45:03.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:45:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:45:04.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:04 smithi203 ceph-mon[36579]: pgmap v147: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 KiB/s wr, 41 op/s 2024-06-06T08:45:04.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:04 smithi176 ceph-mon[36786]: pgmap v147: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 KiB/s wr, 41 op/s 2024-06-06T08:45:04.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:04 smithi027 ceph-mon[32335]: pgmap v147: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 KiB/s wr, 41 op/s 2024-06-06T08:45:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.646+0000 7fde7122a700 1 -- 172.21.15.27:0/1177786395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101fc0 msgr2=0x7fde6c106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.646+0000 7fde7122a700 1 --2- 172.21.15.27:0/1177786395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101fc0 0x7fde6c106010 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fde600099c0 tx=0x7fde60009cd0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.646+0000 7fde7122a700 1 -- 172.21.15.27:0/1177786395 shutdown_connections 2024-06-06T08:45:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.646+0000 7fde7122a700 1 --2- 172.21.15.27:0/1177786395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde6c1066f0 0x7fde6c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.646+0000 7fde7122a700 1 --2- 172.21.15.27:0/1177786395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c101610 0x7fde6c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.646+0000 7fde7122a700 1 --2- 172.21.15.27:0/1177786395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101fc0 0x7fde6c106010 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.646+0000 7fde7122a700 1 -- 172.21.15.27:0/1177786395 >> 172.21.15.27:0/1177786395 conn(0x7fde6c078430 msgr2=0x7fde6c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.646+0000 7fde7122a700 1 -- 172.21.15.27:0/1177786395 shutdown_connections 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.646+0000 7fde7122a700 1 -- 172.21.15.27:0/1177786395 wait complete. 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.647+0000 7fde7122a700 1 Processor -- start 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.647+0000 7fde7122a700 1 -- start start 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.647+0000 7fde7122a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101610 0x7fde6c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.647+0000 7fde7122a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde6c101fc0 0x7fde6c199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.647+0000 7fde7122a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c1066f0 0x7fde6c1a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.647+0000 7fde7122a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1116b0 con 0x7fde6c101610 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.647+0000 7fde7122a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c111530 con 0x7fde6c101fc0 2024-06-06T08:45:05.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.647+0000 7fde7122a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c111830 con 0x7fde6c1066f0 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.647+0000 7fde6ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101610 0x7fde6c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101610 0x7fde6c199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49424/0 (socket says 172.21.15.27:49424) 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6ad9d700 1 -- 172.21.15.27:0/2537605025 learned_addr learned my addr 172.21.15.27:0/2537605025 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6b59e700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c1066f0 0x7fde6c1a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6a59c700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde6c101fc0 0x7fde6c199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6ad9d700 1 -- 172.21.15.27:0/2537605025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c1066f0 msgr2=0x7fde6c1a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6ad9d700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c1066f0 0x7fde6c1a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6ad9d700 1 -- 172.21.15.27:0/2537605025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde6c101fc0 msgr2=0x7fde6c199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6ad9d700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde6c101fc0 0x7fde6c199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6ad9d700 1 -- 172.21.15.27:0/2537605025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c1a4ff0 con 0x7fde6c101610 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6a59c700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde6c101fc0 0x7fde6c199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde6ad9d700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101610 0x7fde6c199a30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00c9b0 tx=0x7fde5c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:05.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde53fff700 1 -- 172.21.15.27:0/2537605025 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c007b00 con 0x7fde6c101610 2024-06-06T08:45:05.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6c1a5220 con 0x7fde6c101610 2024-06-06T08:45:05.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde53fff700 1 -- 172.21.15.27:0/2537605025 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5c007ca0 con 0x7fde6c101610 2024-06-06T08:45:05.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde53fff700 1 -- 172.21.15.27:0/2537605025 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c018730 con 0x7fde6c101610 2024-06-06T08:45:05.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.648+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6c1a5730 con 0x7fde6c101610 2024-06-06T08:45:05.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.649+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde6c069180 con 0x7fde6c101610 2024-06-06T08:45:05.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.651+0000 7fde53fff700 1 -- 172.21.15.27:0/2537605025 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fde5c0188d0 con 0x7fde6c101610 2024-06-06T08:45:05.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.651+0000 7fde53fff700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fde540a2d90 0x7fde540a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:05.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.652+0000 7fde6a59c700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fde540a2d90 0x7fde540a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:05.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.652+0000 7fde53fff700 1 -- 172.21.15.27:0/2537605025 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fde5c0cfab0 con 0x7fde6c101610 2024-06-06T08:45:05.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.652+0000 7fde6a59c700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fde540a2d90 0x7fde540a5250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fde60000c00 tx=0x7fde60005ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:05.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.655+0000 7fde53fff700 1 -- 172.21.15.27:0/2537605025 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fde5c014030 con 0x7fde6c101610 2024-06-06T08:45:05.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.908+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde6c1a59f0 con 0x7fde6c101610 2024-06-06T08:45:05.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.909+0000 7fde53fff700 1 -- 172.21.15.27:0/2537605025 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11896 (secure 0 0 0) 0x7fde5c09c3a0 con 0x7fde6c101610 2024-06-06T08:45:05.907 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:45:05.908 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:43:25.179611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:45:05.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fde540a2d90 msgr2=0x7fde540a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:05.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fde540a2d90 0x7fde540a5250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fde60000c00 tx=0x7fde60005ab0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101610 msgr2=0x7fde6c199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:05.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101610 0x7fde6c199a30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00c9b0 tx=0x7fde5c00ccc0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 shutdown_connections 2024-06-06T08:45:05.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c1066f0 0x7fde6c1a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde6c101610 0x7fde6c199a30 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fde540a2d90 0x7fde540a5250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 --2- 172.21.15.27:0/2537605025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde6c101fc0 0x7fde6c199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:05.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 >> 172.21.15.27:0/2537605025 conn(0x7fde6c078430 msgr2=0x7fde6c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:05.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 shutdown_connections 2024-06-06T08:45:05.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:05.911+0000 7fde7122a700 1 -- 172.21.15.27:0/2537605025 wait complete. 2024-06-06T08:45:05.911 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-06-06T08:45:06.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:06 smithi027 ceph-mon[32335]: pgmap v148: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 75 KiB/s wr, 55 op/s 2024-06-06T08:45:06.643 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:06 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2537605025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:06.678 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id 6bc9d658 --release --await' 2024-06-06T08:45:06.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id 6bc9d658 --release --await 2024-06-06T08:45:06.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:06 smithi203 ceph-mon[36579]: pgmap v148: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 75 KiB/s wr, 55 op/s 2024-06-06T08:45:06.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:06 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2537605025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:06.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:06 smithi176 ceph-mon[36786]: pgmap v148: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 75 KiB/s wr, 55 op/s 2024-06-06T08:45:06.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:06 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2537605025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:06.937 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:45:08.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 -- 172.21.15.27:0/2222894097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 msgr2=0x7f06c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/2222894097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 0x7f06c4074aa0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009990 tx=0x7f06c0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 -- 172.21.15.27:0/2222894097 shutdown_connections 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/2222894097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06c4113660 0x7f06c4115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/2222894097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 0x7f06c4074aa0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/2222894097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c4074fe0 0x7f06c4113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 -- 172.21.15.27:0/2222894097 >> 172.21.15.27:0/2222894097 conn(0x7f06c40795f0 msgr2=0x7f06c4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 -- 172.21.15.27:0/2222894097 shutdown_connections 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 -- 172.21.15.27:0/2222894097 wait complete. 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 Processor -- start 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.647+0000 7f06cbe9e700 1 -- start start 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06cbe9e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 0x7f06c419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06cbe9e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c4074fe0 0x7f06c419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06cbe9e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06c4113660 0x7f06c41a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06cbe9e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4118db0 con 0x7f06c40746c0 2024-06-06T08:45:08.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06cbe9e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4118c30 con 0x7f06c4113660 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06cbe9e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4118f30 con 0x7f06c4074fe0 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9c3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 0x7f06c419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9c3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 0x7f06c419f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49438/0 (socket says 172.21.15.27:49438) 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9c3a700 1 -- 172.21.15.27:0/1842351562 learned_addr learned my addr 172.21.15.27:0/1842351562 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9439700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c4074fe0 0x7f06c419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06ca43b700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06c4113660 0x7f06c41a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9c3a700 1 -- 172.21.15.27:0/1842351562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c4074fe0 msgr2=0x7f06c419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c4074fe0 0x7f06c419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9c3a700 1 -- 172.21.15.27:0/1842351562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06c4113660 msgr2=0x7f06c41a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06c4113660 0x7f06c41a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9c3a700 1 -- 172.21.15.27:0/1842351562 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c41a42b0 con 0x7f06c40746c0 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06ca43b700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06c4113660 0x7f06c41a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:08.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 0x7f06c419f2a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000c00 tx=0x7f06c00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.648+0000 7f06baffd700 1 -- 172.21.15.27:0/1842351562 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0027070 con 0x7f06c40746c0 2024-06-06T08:45:08.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.649+0000 7f06baffd700 1 -- 172.21.15.27:0/1842351562 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c0003780 con 0x7f06c40746c0 2024-06-06T08:45:08.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.649+0000 7f06cbe9e700 1 -- 172.21.15.27:0/1842351562 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c41a4540 con 0x7f06c40746c0 2024-06-06T08:45:08.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.649+0000 7f06baffd700 1 -- 172.21.15.27:0/1842351562 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0022dd0 con 0x7f06c40746c0 2024-06-06T08:45:08.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.649+0000 7f06cbe9e700 1 -- 172.21.15.27:0/1842351562 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c41abe20 con 0x7f06c40746c0 2024-06-06T08:45:08.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.651+0000 7f06baffd700 1 -- 172.21.15.27:0/1842351562 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f06c002a070 con 0x7f06c40746c0 2024-06-06T08:45:08.651 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:08 smithi027 ceph-mon[32335]: pgmap v149: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 45 op/s 2024-06-06T08:45:08.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.651+0000 7f06baffd700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06b00a2d90 0x7f06b00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.652+0000 7f06c9439700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06b00a2d90 0x7f06b00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.652+0000 7f06baffd700 1 -- 172.21.15.27:0/1842351562 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f06c00d04f0 con 0x7f06c40746c0 2024-06-06T08:45:08.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.652+0000 7f06c9439700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06b00a2d90 0x7f06b00a5250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06c410ffc0 tx=0x7f06b400b3c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06d6721b80 1 Processor -- start 2024-06-06T08:45:08.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06d6721b80 1 -- start start 2024-06-06T08:45:08.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x56060551d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 0x560605517710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 0x560605518070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06d6721b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56060540a060 con 0x56060551d500 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06d6721b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560605468950 con 0x56060551de20 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06d6721b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5606053d0560 con 0x560605517c50 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06c9c3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x56060551d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06c9c3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x56060551d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49452/0 (socket says 172.21.15.27:49452) 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06c9c3a700 1 -- 172.21.15.27:0/3389708858 learned_addr learned my addr 172.21.15.27:0/3389708858 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06c9439700 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 0x560605517710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.653+0000 7f06ca43b700 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 0x560605518070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06c9c3a700 1 -- 172.21.15.27:0/3389708858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 msgr2=0x560605518070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 0x560605518070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06c9c3a700 1 -- 172.21.15.27:0/3389708858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 msgr2=0x560605517710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 0x560605517710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06c9c3a700 1 -- 172.21.15.27:0/3389708858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606053b79d0 con 0x56060551d500 2024-06-06T08:45:08.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06c9439700 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 0x560605517710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:08.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x56060551d8e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f06c00a4e20 tx=0x7f06c00a30d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce18d00a45b591a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06affff700 1 -- 172.21.15.27:0/3389708858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00a0b20 con 0x56060551d500 2024-06-06T08:45:08.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06affff700 1 -- 172.21.15.27:0/3389708858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00a5350 con 0x56060551d500 2024-06-06T08:45:08.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.654+0000 7f06affff700 1 -- 172.21.15.27:0/3389708858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c009cc30 con 0x56060551d500 2024-06-06T08:45:08.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.655+0000 7f06d6721b80 1 -- 172.21.15.27:0/3389708858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 msgr2=0x56060551d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.655+0000 7f06d6721b80 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x56060551d8e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f06c00a4e20 tx=0x7f06c00a30d0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.655+0000 7f06d6721b80 1 -- 172.21.15.27:0/3389708858 shutdown_connections 2024-06-06T08:45:08.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.655+0000 7f06d6721b80 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 0x560605518070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.655+0000 7f06d6721b80 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x56060551d8e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.655+0000 7f06d6721b80 1 --2- 172.21.15.27:0/3389708858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 0x560605517710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.655+0000 7f06d6721b80 1 -- 172.21.15.27:0/3389708858 >> 172.21.15.27:0/3389708858 conn(0x560605444180 msgr2=0x560605512a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.655+0000 7f06d6721b80 1 -- 172.21.15.27:0/3389708858 shutdown_connections 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.655+0000 7f06d6721b80 1 -- 172.21.15.27:0/3389708858 wait complete. 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06d6721b80 1 -- start start 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06d6721b80 1 Processor -- start 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 0x560605523bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x560605520ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 0x560605521410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06d6721b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5606054090c0 con 0x56060551d500 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06d6721b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x560605464910 con 0x56060551de20 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06d6721b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5606055245b0 con 0x560605517c50 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06d6721b80 10 client.0 _ll_register_callbacks cb 0x5606053fe6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.656+0000 7f06c9439700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x560605520ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06c9439700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x560605520ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49468/0 (socket says 172.21.15.27:49468) 2024-06-06T08:45:08.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06c9439700 1 -- 172.21.15.27:0/1246714409 learned_addr learned my addr 172.21.15.27:0/1246714409 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 0x560605523bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06ca43b700 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 0x560605521410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06c9439700 1 -- 172.21.15.27:0/1246714409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 msgr2=0x560605523bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06c9439700 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 0x560605523bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06c9439700 1 -- 172.21.15.27:0/1246714409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 msgr2=0x560605521410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06c9439700 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 0x560605521410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06c9439700 1 -- 172.21.15.27:0/1246714409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56060542e850 con 0x56060551d500 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06c9439700 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x560605520ed0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f06b4003e70 tx=0x7f06b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2131ce2794e47344 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06ad7fa700 1 -- 172.21.15.27:0/1246714409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4003ea0 con 0x56060551d500 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06ad7fa700 1 -- 172.21.15.27:0/1246714409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f06b4028470 con 0x56060551d500 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.657+0000 7f06d6721b80 1 -- 172.21.15.27:0/1246714409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c00a5350 con 0x56060551d500 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06ad7fa700 1 -- 172.21.15.27:0/1246714409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f06b4028670 con 0x56060551d500 2024-06-06T08:45:08.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06d6721b80 10 client.16473 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06d6721b80 1 -- 172.21.15.27:0/1246714409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56060552ec90 con 0x56060551d500 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06ad7fa700 1 -- 172.21.15.27:0/1246714409 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00a5350 con 0x56060551d500 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06ad7fa700 1 -- 172.21.15.27:0/1246714409 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f06b401fab0 con 0x56060551d500 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06d6721b80 10 client.16473 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06d6721b80 10 client.16473 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06d6721b80 20 client.16473 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06d6721b80 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560605531320 0x560605533710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06d6721b80 4 client.16473 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.658+0000 7f06d6721b80 1 -- 172.21.15.27:0/1246714409 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560605475e50 con 0x560605531320 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.659+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560605531320 0x560605533710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.659+0000 7f06ad7fa700 10 client.16473 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.659+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560605531320 0x560605533710 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472ebfc4ffa68d8f server_cookie=3df1f0e33634cf1a in_seq=0 out_seq=0 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06ad7fa700 1 -- 172.21.15.27:0/1246714409 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560605475e50 con 0x560605531320 2024-06-06T08:45:08.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06ad7fa700 10 client.16473 handle_command_reply: tid=0 2024-06-06T08:45:08.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 client.16473 shutdown 2024-06-06T08:45:08.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 -- 172.21.15.27:0/1246714409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 msgr2=0x560605520ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x560605520ed0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f06b4003e70 tx=0x7f06b40047c0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 -- 172.21.15.27:0/1246714409 shutdown_connections 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551de20 0x560605521410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560605517c50 0x560605523bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x560605531320 0x560605533710 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 --2- 172.21.15.27:0/1246714409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56060551d500 0x560605520ed0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06ad7fa700 0 client.16473 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 -- 172.21.15.27:0/1246714409 >> 172.21.15.27:0/1246714409 conn(0x560605444180 msgr2=0x560605511320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 -- 172.21.15.27:0/1246714409 shutdown_connections 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 1 -- 172.21.15.27:0/1246714409 wait complete. 2024-06-06T08:45:08.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.660+0000 7f06d6721b80 20 client.16473 trim_cache size 0 max 16384 2024-06-06T08:45:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06d6721b80 1 Processor -- start 2024-06-06T08:45:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06d6721b80 1 -- start start 2024-06-06T08:45:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605517710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x56060551e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605538fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06d6721b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41b7500 con 0x560605517c50 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06d6721b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41b4cd0 con 0x56060551d500 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06d6721b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41b2b20 con 0x56060551de20 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9c3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605517710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9c3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605517710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49470/0 (socket says 172.21.15.27:49470) 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9c3a700 1 -- 172.21.15.27:0/780729718 learned_addr learned my addr 172.21.15.27:0/780729718 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9439700 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x56060551e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06ca43b700 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605538fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9c3a700 1 -- 172.21.15.27:0/780729718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 msgr2=0x560605538fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605538fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9c3a700 1 -- 172.21.15.27:0/780729718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 msgr2=0x56060551e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x56060551e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9c3a700 1 -- 172.21.15.27:0/780729718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560605533d70 con 0x560605517c50 2024-06-06T08:45:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06ca43b700 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605538fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:45:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9439700 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x56060551e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.681+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605517710 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f06c0099140 tx=0x7f06c0087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e155345dbc7a737 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06af7fe700 1 -- 172.21.15.27:0/780729718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00a5350 con 0x560605517c50 2024-06-06T08:45:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06af7fe700 1 -- 172.21.15.27:0/780729718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00d0120 con 0x560605517c50 2024-06-06T08:45:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06af7fe700 1 -- 172.21.15.27:0/780729718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c009cc30 con 0x560605517c50 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06d6721b80 1 -- 172.21.15.27:0/780729718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 msgr2=0x560605517710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06d6721b80 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605517710 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f06c0099140 tx=0x7f06c0087040 comp rx=0 tx=0).stop 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06d6721b80 1 -- 172.21.15.27:0/780729718 shutdown_connections 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06d6721b80 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605538fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06d6721b80 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605517710 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06d6721b80 1 --2- 172.21.15.27:0/780729718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x56060551e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06d6721b80 1 -- 172.21.15.27:0/780729718 >> 172.21.15.27:0/780729718 conn(0x560605444180 msgr2=0x560605522f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06d6721b80 1 -- 172.21.15.27:0/780729718 shutdown_connections 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.682+0000 7f06d6721b80 1 -- 172.21.15.27:0/780729718 wait complete. 2024-06-06T08:45:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06d6721b80 1 -- start start 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06d6721b80 1 Processor -- start 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605529730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x560605529c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06d6721b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605526700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06d6721b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41b3190 con 0x560605517c50 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06d6721b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41b4600 con 0x56060551d500 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06d6721b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41b3860 con 0x56060551de20 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06d6721b80 10 client.0 _ll_register_callbacks cb 0x5606053fe6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06c9c3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605529730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06c9c3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605529730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49472/0 (socket says 172.21.15.27:49472) 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.683+0000 7f06c9c3a700 1 -- 172.21.15.27:0/630841032 learned_addr learned my addr 172.21.15.27:0/630841032 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06c9439700 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x560605529c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06ca43b700 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605526700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06c9c3a700 1 -- 172.21.15.27:0/630841032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 msgr2=0x560605526700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605526700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06c9c3a700 1 -- 172.21.15.27:0/630841032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 msgr2=0x560605529c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x560605529c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06c9c3a700 1 -- 172.21.15.27:0/630841032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56060550a4a0 con 0x560605517c50 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06ca43b700 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605526700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06c9439700 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x560605529c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06c9c3a700 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605529730 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f06c0098450 tx=0x7f06c0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed582c54ba6a4b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06ad7fa700 1 -- 172.21.15.27:0/630841032 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0022470 con 0x560605517c50 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06ad7fa700 1 -- 172.21.15.27:0/630841032 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f06c0022610 con 0x560605517c50 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06d6721b80 1 -- 172.21.15.27:0/630841032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c00d0120 con 0x560605517c50 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06ad7fa700 1 -- 172.21.15.27:0/630841032 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f06c009c9d0 con 0x560605517c50 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06d6721b80 10 client.16479 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06d6721b80 1 -- 172.21.15.27:0/630841032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56060559c590 con 0x560605517c50 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.684+0000 7f06ad7fa700 1 -- 172.21.15.27:0/630841032 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c00d0120 con 0x560605517c50 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06ad7fa700 1 -- 172.21.15.27:0/630841032 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f06c009bcc0 con 0x560605517c50 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06d6721b80 10 client.16479 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06d6721b80 10 client.16479 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06d6721b80 20 client.16479 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06d6721b80 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56060559c860 0x56060559ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06d6721b80 4 client.16479 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6bc9d658", "release": true, "await": true}] 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06d6721b80 1 -- 172.21.15.27:0/630841032 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6bc9d658", "release": true, "await": true}) -- 0x5606054703d0 con 0x56060559c860 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06c9439700 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56060559c860 0x56060559ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06c9439700 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56060559c860 0x56060559ec80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c372f48655af6b server_cookie=62148fe5df95b5b0 in_seq=0 out_seq=0 2024-06-06T08:45:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.685+0000 7f06ad7fa700 10 client.16479 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:08.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:08 smithi203 ceph-mon[36579]: pgmap v149: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 45 op/s 2024-06-06T08:45:08.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:08 smithi176 ceph-mon[36786]: pgmap v149: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 45 op/s 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.838+0000 7fa99ece1640 1 -- 172.21.15.27:0/1651325403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 msgr2=0x7fa998106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.838+0000 7fa99ece1640 1 --2- 172.21.15.27:0/1651325403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 0x7fa998106c70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa988009930 tx=0x7fa98802a420 comp rx=0 tx=0).stop 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.838+0000 7fa99ece1640 1 -- 172.21.15.27:0/1651325403 shutdown_connections 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.838+0000 7fa99ece1640 1 --2- 172.21.15.27:0/1651325403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa998101290 0x7fa99810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.838+0000 7fa99ece1640 1 --2- 172.21.15.27:0/1651325403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9981008d0 0x7fa998100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.838+0000 7fa99ece1640 1 --2- 172.21.15.27:0/1651325403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 0x7fa998106c70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.838+0000 7fa99ece1640 1 -- 172.21.15.27:0/1651325403 >> 172.21.15.27:0/1651325403 conn(0x7fa998077a20 msgr2=0x7fa998077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.838+0000 7fa99ece1640 1 -- 172.21.15.27:0/1651325403 shutdown_connections 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.838+0000 7fa99ece1640 1 -- 172.21.15.27:0/1651325403 wait complete. 2024-06-06T08:45:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99ece1640 1 Processor -- start 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99ece1640 1 -- start start 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99ece1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9981008d0 0x7fa9981982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99ece1640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa998101290 0x7fa998198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99ece1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 0x7fa99819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99ece1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa998110ca0 con 0x7fa998106890 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99ece1640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa998110b20 con 0x7fa998101290 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99ece1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa998110e20 con 0x7fa9981008d0 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99d27e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 0x7fa99819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99d27e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 0x7fa99819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49496/0 (socket says 172.21.15.27:49496) 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99d27e640 1 -- 172.21.15.27:0/4114239634 learned_addr learned my addr 172.21.15.27:0/4114239634 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa997fff640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa998101290 0x7fa998198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9981008d0 0x7fa9981982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99d27e640 1 -- 172.21.15.27:0/4114239634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9981008d0 msgr2=0x7fa9981982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99d27e640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9981008d0 0x7fa9981982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.839+0000 7fa99d27e640 1 -- 172.21.15.27:0/4114239634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa998101290 msgr2=0x7fa998198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.840+0000 7fa99d27e640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa998101290 0x7fa998198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.840+0000 7fa99d27e640 1 -- 172.21.15.27:0/4114239634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99819d2e0 con 0x7fa998106890 2024-06-06T08:45:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.840+0000 7fa99d27e640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 0x7fa99819cbb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa99000eb10 tx=0x7fa99000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.840+0000 7fa995ffb640 1 -- 172.21.15.27:0/4114239634 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990004b20 con 0x7fa998106890 2024-06-06T08:45:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.840+0000 7fa995ffb640 1 -- 172.21.15.27:0/4114239634 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa990004cc0 con 0x7fa998106890 2024-06-06T08:45:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.840+0000 7fa995ffb640 1 -- 172.21.15.27:0/4114239634 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990010760 con 0x7fa998106890 2024-06-06T08:45:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.840+0000 7fa99ece1640 1 -- 172.21.15.27:0/4114239634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99819d5d0 con 0x7fa998106890 2024-06-06T08:45:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.840+0000 7fa99ece1640 1 -- 172.21.15.27:0/4114239634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9981a4e60 con 0x7fa998106890 2024-06-06T08:45:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.842+0000 7fa995ffb640 1 -- 172.21.15.27:0/4114239634 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa990005020 con 0x7fa998106890 2024-06-06T08:45:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa9a19f5740 1 Processor -- start 2024-06-06T08:45:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa9a19f5740 1 -- start start 2024-06-06T08:45:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 0x55d6f4ce7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa995ffb640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa9800a2e20 0x7fa9800a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4cf2350 0x55d6f4cf4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d6f4c4b4f0 con 0x55d6f4ce7a90 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d6f4be9870 con 0x55d6f4ce7170 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d6f4c474b0 con 0x55d6f4cf2350 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa9800a2e20 0x7fa9800a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa997fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa997fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49500/0 (socket says 172.21.15.27:49500) 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa997fff640 1 -- 172.21.15.27:0/887673475 learned_addr learned my addr 172.21.15.27:0/887673475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa995ffb640 1 -- 172.21.15.27:0/4114239634 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fa9900cfd50 con 0x7fa998106890 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa997fff640 1 --2- 172.21.15.27:0/887673475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 0x55d6f4ce7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa99d27e640 1 --2- 172.21.15.27:0/887673475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4cf2350 0x55d6f4cf4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa997fff640 1 -- 172.21.15.27:0/887673475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4cf2350 msgr2=0x55d6f4cf4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa997fff640 1 --2- 172.21.15.27:0/887673475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4cf2350 0x55d6f4cf4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.843+0000 7fa997fff640 1 -- 172.21.15.27:0/887673475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 msgr2=0x55d6f4ce7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.844+0000 7fa997fff640 1 --2- 172.21.15.27:0/887673475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 0x55d6f4ce7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.844+0000 7fa997fff640 1 -- 172.21.15.27:0/887673475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6f4c059b0 con 0x55d6f4ce7a90 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.844+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa9800a2e20 0x7fa9800a52e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa988000c00 tx=0x7fa988005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.844+0000 7fa997fff640 1 --2- 172.21.15.27:0/887673475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf1e10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0127a0 tx=0x7fa98c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54f9317214810d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.844+0000 7fa97effd640 1 -- 172.21.15.27:0/887673475 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c012c60 con 0x55d6f4ce7a90 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.844+0000 7fa97effd640 1 -- 172.21.15.27:0/887673475 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa98c012e00 con 0x55d6f4ce7a90 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.844+0000 7fa97effd640 1 -- 172.21.15.27:0/887673475 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c004a50 con 0x55d6f4ce7a90 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 -- 172.21.15.27:0/887673475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 msgr2=0x55d6f4cf1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/887673475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf1e10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0127a0 tx=0x7fa98c00b7c0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 -- 172.21.15.27:0/887673475 shutdown_connections 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/887673475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4cf2350 0x55d6f4cf4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/887673475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf1e10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/887673475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 0x55d6f4ce7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 -- 172.21.15.27:0/887673475 >> 172.21.15.27:0/887673475 conn(0x55d6f4ce3680 msgr2=0x55d6f4ce3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 -- 172.21.15.27:0/887673475 shutdown_connections 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 -- 172.21.15.27:0/887673475 wait complete. 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 -- start start 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.845+0000 7fa9a19f5740 1 Processor -- start 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7170 0x55d6f4cef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 0x55d6f4cefdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d6f4cf4f80 con 0x55d6f4ce7170 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55d6f4bd4ae0 con 0x55d6f4cf2350 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d6f4cf5100 con 0x55d6f4ce7a90 2024-06-06T08:45:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa9a19f5740 10 client.0 _ll_register_callbacks cb 0x55d6f4c0d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99ca7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7170 0x55d6f4cef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99ca7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7170 0x55d6f4cef330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49514/0 (socket says 172.21.15.27:49514) 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99ca7d640 1 -- 172.21.15.27:0/3227421530 learned_addr learned my addr 172.21.15.27:0/3227421530 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99d27e640 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 0x55d6f4cefdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa997fff640 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99ca7d640 1 -- 172.21.15.27:0/3227421530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 msgr2=0x55d6f4cef870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cef870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99ca7d640 1 -- 172.21.15.27:0/3227421530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 msgr2=0x55d6f4cefdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 0x55d6f4cefdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99ca7d640 1 -- 172.21.15.27:0/3227421530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6f495f210 con 0x55d6f4ce7170 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7170 0x55d6f4cef330 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa98802a940 tx=0x7fa988009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac365d411971f5a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa977fff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa977fff640 1 -- 172.21.15.27:0/3227421530 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98804b020 con 0x55d6f4ce7170 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa977fff640 1 -- 172.21.15.27:0/3227421530 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fa988005260 con 0x55d6f4ce7170 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3227421530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa98c012e00 con 0x55d6f4ce7170 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa977fff640 1 -- 172.21.15.27:0/3227421530 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa988056050 con 0x55d6f4ce7170 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.846+0000 7fa9a19f5740 10 client.16491 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3227421530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d6f4a2b620 con 0x55d6f4ce7170 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa977fff640 1 -- 172.21.15.27:0/3227421530 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa98c012e00 con 0x55d6f4ce7170 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa977fff640 1 -- 172.21.15.27:0/3227421530 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa988003680 con 0x55d6f4ce7170 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa9a19f5740 10 client.16491 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa9a19f5740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa9a19f5740 20 client.16491 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6f4d025e0 0x55d6f4d04a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa9a19f5740 4 client.16491 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3227421530 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6f4c5ef10 con 0x55d6f4d025e0 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa997fff640 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6f4d025e0 0x55d6f4d04a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa977fff640 10 client.16491 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.847+0000 7fa997fff640 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6f4d025e0 0x55d6f4d04a70 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e954b33c4a3744d1 server_cookie=442caf1e928886d8 in_seq=0 out_seq=0 2024-06-06T08:45:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06ad7fa700 1 -- 172.21.15.27:0/630841032 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5606054703d0 con 0x56060559c860 2024-06-06T08:45:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06ad7fa700 10 client.16479 handle_command_reply: tid=0 2024-06-06T08:45:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 client.16479 shutdown 2024-06-06T08:45:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa977fff640 1 -- 172.21.15.27:0/3227421530 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d6f4c5ef10 con 0x55d6f4d025e0 2024-06-06T08:45:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa977fff640 10 client.16491 handle_command_reply: tid=0 2024-06-06T08:45:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 -- 172.21.15.27:0/630841032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 msgr2=0x560605529730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605529730 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f06c0098450 tx=0x7f06c0009310 comp rx=0 tx=0).stop 2024-06-06T08:45:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 client.16491 shutdown 2024-06-06T08:45:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 -- 172.21.15.27:0/630841032 shutdown_connections 2024-06-06T08:45:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56060551de20 0x560605526700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x56060559c860 0x56060559ec80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560605517c50 0x560605529730 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 --2- 172.21.15.27:0/630841032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060551d500 0x560605529c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06ad7fa700 0 client.16479 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 -- 172.21.15.27:0/630841032 >> 172.21.15.27:0/630841032 conn(0x560605444180 msgr2=0x560605511bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 -- 172.21.15.27:0/630841032 shutdown_connections 2024-06-06T08:45:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 1 -- 172.21.15.27:0/630841032 wait complete. 2024-06-06T08:45:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3227421530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7170 msgr2=0x55d6f4cef330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7170 0x55d6f4cef330 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa98802a940 tx=0x7fa988009260 comp rx=0 tx=0).stop 2024-06-06T08:45:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3227421530 shutdown_connections 2024-06-06T08:45:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6f4d025e0 0x55d6f4d04a70 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa977fff640 0 client.16491 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 0x55d6f4cefdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3227421530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7170 0x55d6f4cef330 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3227421530 >> 172.21.15.27:0/3227421530 conn(0x55d6f4ce3680 msgr2=0x55d6f4ce9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.871+0000 7f06d6721b80 20 client.16479 trim_cache size 0 max 16384 2024-06-06T08:45:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3227421530 shutdown_connections 2024-06-06T08:45:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.871+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3227421530 wait complete. 2024-06-06T08:45:08.873 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24482, 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 12, 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "6bc9d658": { 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 12, 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-06-06T08:45:08.874 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 145.2, 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:08.875 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:45:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.872+0000 7fa9a19f5740 20 client.16491 trim_cache size 0 max 16384 2024-06-06T08:45:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 -- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06b00a2d90 msgr2=0x7f06b00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06b00a2d90 0x7f06b00a5250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06c410ffc0 tx=0x7f06b400b3c0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 -- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 msgr2=0x7f06c419f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 0x7f06c419f2a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000c00 tx=0x7f06c00047b0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 -- 172.21.15.27:0/1842351562 shutdown_connections 2024-06-06T08:45:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f06b00a2d90 0x7f06b00a5250 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06c4113660 0x7f06c41a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40746c0 0x7f06c419f2a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 --2- 172.21.15.27:0/1842351562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06c4074fe0 0x7f06c419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 -- 172.21.15.27:0/1842351562 >> 172.21.15.27:0/1842351562 conn(0x7f06c40795f0 msgr2=0x7f06c41124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 -- 172.21.15.27:0/1842351562 shutdown_connections 2024-06-06T08:45:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:08.872+0000 7f06cbe9e700 1 -- 172.21.15.27:0/1842351562 wait complete. 2024-06-06T08:45:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.889+0000 7fa9a19f5740 1 Processor -- start 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.889+0000 7fa9a19f5740 1 -- start start 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.889+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 0x55d6f4cee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.889+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 0x55d6f4d0a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.889+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4cf2350 0x55d6f4d0a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.889+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981b1780 con 0x55d6f4cf2350 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.889+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981aef90 con 0x55d6f4ce7170 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.889+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981acde0 con 0x55d6f4ce7a90 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.889+0000 7fa99d27e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4cf2350 0x55d6f4d0a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa99d27e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4cf2350 0x55d6f4d0a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49528/0 (socket says 172.21.15.27:49528) 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa99d27e640 1 -- 172.21.15.27:0/3748647125 learned_addr learned my addr 172.21.15.27:0/3748647125 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/3748647125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 0x55d6f4cee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa997fff640 1 --2- 172.21.15.27:0/3748647125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 0x55d6f4d0a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa99d27e640 1 -- 172.21.15.27:0/3748647125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 msgr2=0x55d6f4d0a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa99d27e640 1 --2- 172.21.15.27:0/3748647125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 0x55d6f4d0a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa99d27e640 1 -- 172.21.15.27:0/3748647125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 msgr2=0x55d6f4cee310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa99d27e640 1 --2- 172.21.15.27:0/3748647125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 0x55d6f4cee310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa99d27e640 1 -- 172.21.15.27:0/3748647125 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6f4ceaf00 con 0x55d6f4cf2350 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa99d27e640 1 --2- 172.21.15.27:0/3748647125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4cf2350 0x55d6f4d0a880 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa99009be40 tx=0x7fa9900bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b487e845bb2ad07 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa97e7fc640 1 -- 172.21.15.27:0/3748647125 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99009b810 con 0x55d6f4cf2350 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa97e7fc640 1 -- 172.21.15.27:0/3748647125 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99009c5b0 con 0x55d6f4cf2350 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.890+0000 7fa97e7fc640 1 -- 172.21.15.27:0/3748647125 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9900ae9f0 con 0x55d6f4cf2350 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3748647125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4cf2350 msgr2=0x55d6f4d0a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3748647125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4cf2350 0x55d6f4d0a880 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa99009be40 tx=0x7fa9900bc040 comp rx=0 tx=0).stop 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3748647125 shutdown_connections 2024-06-06T08:45:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3748647125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4cf2350 0x55d6f4d0a880 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3748647125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7a90 0x55d6f4d0a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3748647125 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4ce7170 0x55d6f4cee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3748647125 >> 172.21.15.27:0/3748647125 conn(0x55d6f4ce3680 msgr2=0x55d6f4cedc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3748647125 shutdown_connections 2024-06-06T08:45:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3748647125 wait complete. 2024-06-06T08:45:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.891+0000 7fa9a19f5740 1 -- start start 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa9a19f5740 1 Processor -- start 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7170 0x55d6f4cf56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa9a19f5740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 0x55d6f4d6b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981ad450 con 0x55d6f4ce7a90 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981ae8c0 con 0x55d6f4cf2350 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa9a19f5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa9981adb20 con 0x55d6f4ce7170 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa9a19f5740 10 client.0 _ll_register_callbacks cb 0x7fa98803f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa997fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa997fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49542/0 (socket says 172.21.15.27:49542) 2024-06-06T08:45:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa997fff640 1 -- 172.21.15.27:0/3465708180 learned_addr learned my addr 172.21.15.27:0/3465708180 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa99d27e640 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 0x55d6f4d6b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.892+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7170 0x55d6f4cf56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa997fff640 1 -- 172.21.15.27:0/3465708180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7170 msgr2=0x55d6f4cf56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa997fff640 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7170 0x55d6f4cf56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa997fff640 1 -- 172.21.15.27:0/3465708180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 msgr2=0x55d6f4d6b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa997fff640 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 0x55d6f4d6b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa997fff640 1 -- 172.21.15.27:0/3465708180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6f4d6b840 con 0x55d6f4ce7a90 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa99d27e640 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 0x55d6f4d6b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa997fff640 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf5c20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa98c000c00 tx=0x7fa98c005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55514a6742cf96a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa977fff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa977fff640 1 -- 172.21.15.27:0/3465708180 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c005570 con 0x55d6f4ce7a90 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3465708180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa99009c5b0 con 0x55d6f4ce7a90 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa977fff640 1 -- 172.21.15.27:0/3465708180 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fa98c0050c0 con 0x55d6f4ce7a90 2024-06-06T08:45:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa977fff640 1 -- 172.21.15.27:0/3465708180 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa98c033050 con 0x55d6f4ce7a90 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa9a19f5740 10 client.16497 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3465708180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d6f4d6ca10 con 0x55d6f4ce7a90 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.893+0000 7fa977fff640 1 -- 172.21.15.27:0/3465708180 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99009c5b0 con 0x55d6f4ce7a90 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa977fff640 1 -- 172.21.15.27:0/3465708180 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa98c0037b0 con 0x55d6f4ce7a90 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa9a19f5740 10 client.16497 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa9a19f5740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa9a19f5740 20 client.16497 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6f4d025e0 0x55d6f4d6efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa9a19f5740 4 client.16497 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3465708180 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6f4c58c20 con 0x55d6f4d025e0 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6f4d025e0 0x55d6f4d6efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa99ca7d640 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6f4d025e0 0x55d6f4d6efb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b738bd2fff390901 server_cookie=5d12056b1ffcf9a4 in_seq=0 out_seq=0 2024-06-06T08:45:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.894+0000 7fa977fff640 10 client.16497 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.947+0000 7fa977fff640 1 -- 172.21.15.27:0/3465708180 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d6f4c58c20 con 0x55d6f4d025e0 2024-06-06T08:45:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.947+0000 7fa977fff640 10 client.16497 handle_command_reply: tid=0 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.947+0000 7fa9a19f5740 1 client.16497 shutdown 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.947+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3465708180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 msgr2=0x55d6f4cf5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.947+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf5c20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa98c000c00 tx=0x7fa98c005ab0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3465708180 shutdown_connections 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55d6f4d025e0 0x55d6f4d6efb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d6f4cf2350 0x55d6f4d6b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d6f4ce7a90 0x55d6f4cf5c20 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa977fff640 0 client.16497 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa9a19f5740 1 --2- 172.21.15.27:0/3465708180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6f4ce7170 0x55d6f4cf56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3465708180 >> 172.21.15.27:0/3465708180 conn(0x55d6f4ce3680 msgr2=0x55d6f4cf0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3465708180 shutdown_connections 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa9a19f5740 1 -- 172.21.15.27:0/3465708180 wait complete. 2024-06-06T08:45:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa9a19f5740 20 client.16497 trim_cache size 0 max 16384 2024-06-06T08:45:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:45:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:45:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa99ece1640 1 -- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa9800a2e20 msgr2=0x7fa9800a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa99ece1640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa9800a2e20 0x7fa9800a52e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa988000c00 tx=0x7fa988005a20 comp rx=0 tx=0).stop 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa99ece1640 1 -- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 msgr2=0x7fa99819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa99ece1640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 0x7fa99819cbb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa99000eb10 tx=0x7fa99000c2d0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa99ece1640 1 -- 172.21.15.27:0/4114239634 shutdown_connections 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa99ece1640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fa9800a2e20 0x7fa9800a52e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa99ece1640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa998106890 0x7fa99819cbb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa99ece1640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa998101290 0x7fa998198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.948+0000 7fa99ece1640 1 --2- 172.21.15.27:0/4114239634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9981008d0 0x7fa9981982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.949+0000 7fa99ece1640 1 -- 172.21.15.27:0/4114239634 >> 172.21.15.27:0/4114239634 conn(0x7fa998077a20 msgr2=0x7fa998069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.949+0000 7fa99ece1640 1 -- 172.21.15.27:0/4114239634 shutdown_connections 2024-06-06T08:45:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:08.949+0000 7fa99ece1640 1 -- 172.21.15.27:0/4114239634 wait complete. 2024-06-06T08:45:09.667 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6bc9d658', total seconds elapsed: 31.7 2024-06-06T08:45:09.668 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.25 2024-06-06T08:45:09.668 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:45:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:45:10.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:10 smithi203 ceph-mon[36579]: pgmap v150: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 67 KiB/s wr, 48 op/s 2024-06-06T08:45:10.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:10 smithi176 ceph-mon[36786]: pgmap v150: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 67 KiB/s wr, 48 op/s 2024-06-06T08:45:10.848 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:10 smithi027 ceph-mon[32335]: pgmap v150: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 67 KiB/s wr, 48 op/s 2024-06-06T08:45:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 -- 172.21.15.27:0/725240342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 msgr2=0x7f6cf8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/725240342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 0x7f6cf8101810 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009990 tx=0x7f6cec009ca0 comp rx=0 tx=0).stop 2024-06-06T08:45:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 -- 172.21.15.27:0/725240342 shutdown_connections 2024-06-06T08:45:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/725240342 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cf8101d50 0x7f6cf810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/725240342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8107400 0x7f6cf81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/725240342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 0x7f6cf8101810 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 -- 172.21.15.27:0/725240342 >> 172.21.15.27:0/725240342 conn(0x7f6cf8078340 msgr2=0x7f6cf8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 -- 172.21.15.27:0/725240342 shutdown_connections 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 -- 172.21.15.27:0/725240342 wait complete. 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.857+0000 7f6cfcddd700 1 Processor -- start 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cfcddd700 1 -- start start 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cfcddd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 0x7f6cf8103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cfcddd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cf8101d50 0x7f6cf81035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cfcddd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8107400 0x7f6cf8103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cfcddd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf8111670 con 0x7f6cf8101390 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cfcddd700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf81114f0 con 0x7f6cf8101d50 2024-06-06T08:45:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cfcddd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf81117f0 con 0x7f6cf8107400 2024-06-06T08:45:11.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 0x7f6cf8103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:11.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 0x7f6cf8103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46936/0 (socket says 172.21.15.27:46936) 2024-06-06T08:45:11.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf659c700 1 -- 172.21.15.27:0/1567003512 learned_addr learned my addr 172.21.15.27:0/1567003512 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:11.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf5d9b700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cf8101d50 0x7f6cf81035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:11.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf6d9d700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8107400 0x7f6cf8103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf659c700 1 -- 172.21.15.27:0/1567003512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8107400 msgr2=0x7f6cf8103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf659c700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8107400 0x7f6cf8103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf659c700 1 -- 172.21.15.27:0/1567003512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cf8101d50 msgr2=0x7f6cf81035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf659c700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cf8101d50 0x7f6cf81035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf659c700 1 -- 172.21.15.27:0/1567003512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf81a7120 con 0x7f6cf8101390 2024-06-06T08:45:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.858+0000 7f6cf659c700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 0x7f6cf8103070 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800e9a0 tx=0x7f6ce800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:11.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.859+0000 7f6ce77fe700 1 -- 172.21.15.27:0/1567003512 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800ca10 con 0x7f6cf8101390 2024-06-06T08:45:11.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.859+0000 7f6ce77fe700 1 -- 172.21.15.27:0/1567003512 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce800cbb0 con 0x7f6cf8101390 2024-06-06T08:45:11.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.859+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf81a7410 con 0x7f6cf8101390 2024-06-06T08:45:11.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.859+0000 7f6ce77fe700 1 -- 172.21.15.27:0/1567003512 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8018680 con 0x7f6cf8101390 2024-06-06T08:45:11.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.859+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf81a7920 con 0x7f6cf8101390 2024-06-06T08:45:11.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.860+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf8102ab0 con 0x7f6cf8101390 2024-06-06T08:45:11.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.861+0000 7f6ce77fe700 1 -- 172.21.15.27:0/1567003512 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6ce8010430 con 0x7f6cf8101390 2024-06-06T08:45:11.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.862+0000 7f6ce77fe700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ce00a2d90 0x7f6ce00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:11.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.862+0000 7f6cf5d9b700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ce00a2d90 0x7f6ce00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:11.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.862+0000 7f6ce77fe700 1 -- 172.21.15.27:0/1567003512 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f6ce80cfe60 con 0x7f6cf8101390 2024-06-06T08:45:11.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.863+0000 7f6cf5d9b700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ce00a2d90 0x7f6ce00a5250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00b2e0 tx=0x7f6cec005960 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:11.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:11.866+0000 7f6ce77fe700 1 -- 172.21.15.27:0/1567003512 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ce809c6c0 con 0x7f6cf8101390 2024-06-06T08:45:12.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.193+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cf8069180 con 0x7f6cf8101390 2024-06-06T08:45:12.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.194+0000 7f6ce77fe700 1 -- 172.21.15.27:0/1567003512 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11896 (secure 0 0 0) 0x7f6ce809bfc0 con 0x7f6cf8101390 2024-06-06T08:45:12.192 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:45:12.193 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:43:25.179611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ce00a2d90 msgr2=0x7f6ce00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ce00a2d90 0x7f6ce00a5250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00b2e0 tx=0x7f6cec005960 comp rx=0 tx=0).stop 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 msgr2=0x7f6cf8103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 0x7f6cf8103070 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800e9a0 tx=0x7f6ce800ecb0 comp rx=0 tx=0).stop 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 shutdown_connections 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8107400 0x7f6cf8103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cf8101390 0x7f6cf8103070 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f6ce00a2d90 0x7f6ce00a5250 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 --2- 172.21.15.27:0/1567003512 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cf8101d50 0x7f6cf81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 >> 172.21.15.27:0/1567003512 conn(0x7f6cf8078340 msgr2=0x7f6cf81005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 shutdown_connections 2024-06-06T08:45:12.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:12.196+0000 7f6cfcddd700 1 -- 172.21.15.27:0/1567003512 wait complete. 2024-06-06T08:45:12.196 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-06-06T08:45:12.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:12 smithi203 ceph-mon[36579]: pgmap v151: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-06-06T08:45:12.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:12 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1567003512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:12.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:12 smithi176 ceph-mon[36786]: pgmap v151: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-06-06T08:45:12.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:12 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1567003512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:12.939 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:12 smithi027 ceph-mon[32335]: pgmap v151: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-06-06T08:45:12.939 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:12 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1567003512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:12.988 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0' 2024-06-06T08:45:12.988 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db / --timeout 90 --expiration 145.25 --await-for 45.0 2024-06-06T08:45:13.401 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:45:14.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:14 smithi203 ceph-mon[36579]: pgmap v152: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 54 op/s 2024-06-06T08:45:14.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:14 smithi176 ceph-mon[36786]: pgmap v152: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 54 op/s 2024-06-06T08:45:14.833 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:14 smithi027 ceph-mon[32335]: pgmap v152: 129 pgs: 129 active+clean; 1.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 54 op/s 2024-06-06T08:45:15.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.160+0000 7f952ec72700 1 -- 172.21.15.27:0/1337209564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528101770 msgr2=0x7f9528101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:15.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.160+0000 7f952ec72700 1 --2- 172.21.15.27:0/1337209564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528101770 0x7f9528101bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9514009990 tx=0x7f9514009ca0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.160+0000 7f952ec72700 1 -- 172.21.15.27:0/1337209564 shutdown_connections 2024-06-06T08:45:15.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.160+0000 7f952ec72700 1 --2- 172.21.15.27:0/1337209564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9528102130 0x7f952806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.160+0000 7f952ec72700 1 --2- 172.21.15.27:0/1337209564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95281074b0 0x7f9528107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.160+0000 7f952ec72700 1 --2- 172.21.15.27:0/1337209564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528101770 0x7f9528101bf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.160+0000 7f952ec72700 1 -- 172.21.15.27:0/1337209564 >> 172.21.15.27:0/1337209564 conn(0x7f9528078340 msgr2=0x7f9528078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:15.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.160+0000 7f952ec72700 1 -- 172.21.15.27:0/1337209564 shutdown_connections 2024-06-06T08:45:15.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.160+0000 7f952ec72700 1 -- 172.21.15.27:0/1337209564 wait complete. 2024-06-06T08:45:15.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.161+0000 7f952ec72700 1 Processor -- start 2024-06-06T08:45:15.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.161+0000 7f952ec72700 1 -- start start 2024-06-06T08:45:15.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ec72700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528101770 0x7f952819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ec72700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528102130 0x7f952819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ec72700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95281074b0 0x7f952819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ec72700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9528114b30 con 0x7f95281074b0 2024-06-06T08:45:15.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ec72700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f95281149b0 con 0x7f9528102130 2024-06-06T08:45:15.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ec72700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9528114cb0 con 0x7f9528101770 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ca0e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528101770 0x7f952819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952d20f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95281074b0 0x7f952819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f9527fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528102130 0x7f952819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ca0e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528101770 0x7f952819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.27:45588/0 (socket says 172.21.15.27:45588) 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f9527fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528102130 0x7f952819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:38848/0 (socket says 172.21.15.27:38848) 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ca0e700 1 -- 172.21.15.27:0/1916514664 learned_addr learned my addr 172.21.15.27:0/1916514664 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ca0e700 1 -- 172.21.15.27:0/1916514664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528102130 msgr2=0x7f952819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ca0e700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528102130 0x7f952819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ca0e700 1 -- 172.21.15.27:0/1916514664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95281074b0 msgr2=0x7f952819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ca0e700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95281074b0 0x7f952819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ca0e700 1 -- 172.21.15.27:0/1916514664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952819fde0 con 0x7f9528101770 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f9527fff700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528102130 0x7f952819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:45:15.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.162+0000 7f952ca0e700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528101770 0x7f952819ae20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f951c00e9a0 tx=0x7f951c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:15.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.163+0000 7f9525ffb700 1 -- 172.21.15.27:0/1916514664 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c00ca10 con 0x7f9528101770 2024-06-06T08:45:15.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.163+0000 7f952ec72700 1 -- 172.21.15.27:0/1916514664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95281a00d0 con 0x7f9528101770 2024-06-06T08:45:15.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.163+0000 7f9525ffb700 1 -- 172.21.15.27:0/1916514664 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f951c00cbb0 con 0x7f9528101770 2024-06-06T08:45:15.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.163+0000 7f9525ffb700 1 -- 172.21.15.27:0/1916514664 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c018680 con 0x7f9528101770 2024-06-06T08:45:15.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.163+0000 7f952ec72700 1 -- 172.21.15.27:0/1916514664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95281a7950 con 0x7f9528101770 2024-06-06T08:45:15.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.165+0000 7f9525ffb700 1 -- 172.21.15.27:0/1916514664 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f951c010430 con 0x7f9528101770 2024-06-06T08:45:15.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.166+0000 7f9525ffb700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f95180a2e80 0x7f95180a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.166+0000 7f9527fff700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f95180a2e80 0x7f95180a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.166+0000 7f9525ffb700 1 -- 172.21.15.27:0/1916514664 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f951c0cf9c0 con 0x7f9528101770 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.166+0000 7f95394f5b80 1 Processor -- start 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f9527fff700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f95180a2e80 0x7f95180a5340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9528101e20 tx=0x7f9514005fb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f95394f5b80 1 -- start start 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x55762334f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334fec0 0x5576233497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x55762334a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f95394f5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55762323c100 con 0x55762334f5a0 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f95394f5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55762329a9f0 con 0x557623349cf0 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f95394f5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557623202600 con 0x55762334fec0 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952d20f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x55762334a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952ca0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x55762334f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952ca0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x55762334f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46952/0 (socket says 172.21.15.27:46952) 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952d20f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x55762334a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:38864/0 (socket says 172.21.15.27:38864) 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952ca0e700 1 -- 172.21.15.27:0/3583378416 learned_addr learned my addr 172.21.15.27:0/3583378416 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f9527fff700 1 --2- 172.21.15.27:0/3583378416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334fec0 0x5576233497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952d20f700 1 -- 172.21.15.27:0/3583378416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334fec0 msgr2=0x5576233497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952d20f700 1 --2- 172.21.15.27:0/3583378416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334fec0 0x5576233497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952d20f700 1 -- 172.21.15.27:0/3583378416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 msgr2=0x55762334f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952d20f700 1 --2- 172.21.15.27:0/3583378416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x55762334f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952d20f700 1 -- 172.21.15.27:0/3583378416 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576231e9a70 con 0x557623349cf0 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.167+0000 7f952ca0e700 1 --2- 172.21.15.27:0/3583378416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x55762334f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.168+0000 7f952d20f700 1 --2- 172.21.15.27:0/3583378416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x55762334a110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9520003f10 tx=0x7f9520009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4ce370bc9cef52 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.168+0000 7f9512ffd700 1 -- 172.21.15.27:0/3583378416 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520004bb0 con 0x557623349cf0 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.168+0000 7f9512ffd700 1 -- 172.21.15.27:0/3583378416 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520004d50 con 0x557623349cf0 2024-06-06T08:45:15.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.168+0000 7f9512ffd700 1 -- 172.21.15.27:0/3583378416 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520007e30 con 0x557623349cf0 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 -- 172.21.15.27:0/3583378416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 msgr2=0x55762334a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 --2- 172.21.15.27:0/3583378416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x55762334a110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9520003f10 tx=0x7f9520009c20 comp rx=0 tx=0).stop 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 -- 172.21.15.27:0/3583378416 shutdown_connections 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 --2- 172.21.15.27:0/3583378416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x55762334a110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 --2- 172.21.15.27:0/3583378416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x55762334f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 --2- 172.21.15.27:0/3583378416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334fec0 0x5576233497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 -- 172.21.15.27:0/3583378416 >> 172.21.15.27:0/3583378416 conn(0x557623276220 msgr2=0x557623344aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 -- 172.21.15.27:0/3583378416 shutdown_connections 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 -- 172.21.15.27:0/3583378416 wait complete. 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 -- start start 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.169+0000 7f95394f5b80 1 Processor -- start 2024-06-06T08:45:15.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x557623355c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 0x557623352f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334fec0 0x557623353470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f95394f5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55762323b160 con 0x55762334fec0 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f95394f5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x5576232969b0 con 0x557623349cf0 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f95394f5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5576233568a0 con 0x55762334f5a0 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f952ca0e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x557623355c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f95394f5b80 10 client.0 _ll_register_callbacks cb 0x557623230750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f952ca0e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x557623355c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:38876/0 (socket says 172.21.15.27:38876) 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f952ca0e700 1 -- 172.21.15.27:0/74680637 learned_addr learned my addr 172.21.15.27:0/74680637 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:15.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f952d20f700 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334fec0 0x557623353470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f9527fff700 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 0x557623352f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f952ca0e700 1 -- 172.21.15.27:0/74680637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 msgr2=0x557623352f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f952ca0e700 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 0x557623352f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f952ca0e700 1 -- 172.21.15.27:0/74680637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334fec0 msgr2=0x557623353470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f952ca0e700 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334fec0 0x557623353470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.170+0000 7f952ca0e700 1 -- 172.21.15.27:0/74680637 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576232608f0 con 0x557623349cf0 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f952ca0e700 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x557623355c30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f951c000c00 tx=0x7f951c09c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=190a34d4f5fc9fe8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f9507fff700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f9507fff700 1 -- 172.21.15.27:0/74680637 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0a4500 con 0x557623349cf0 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f95394f5b80 1 -- 172.21.15.27:0/74680637 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9520004d50 con 0x557623349cf0 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f9507fff700 1 -- 172.21.15.27:0/74680637 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f951c010720 con 0x557623349cf0 2024-06-06T08:45:15.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f9507fff700 1 -- 172.21.15.27:0/74680637 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f951c0a4790 con 0x557623349cf0 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f95394f5b80 10 client.26003 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f95394f5b80 1 -- 172.21.15.27:0/74680637 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557623361000 con 0x557623349cf0 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f9507fff700 1 -- 172.21.15.27:0/74680637 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520004d50 con 0x557623349cf0 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f9507fff700 1 -- 172.21.15.27:0/74680637 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f951c09a9e0 con 0x557623349cf0 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f95394f5b80 10 client.26003 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f95394f5b80 10 client.26003 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f95394f5b80 20 client.26003 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.171+0000 7f95394f5b80 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557623363690 0x557623365a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.172+0000 7f95394f5b80 4 client.26003 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.172+0000 7f95394f5b80 1 -- 172.21.15.27:0/74680637 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576232a7ef0 con 0x557623363690 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.172+0000 7f9527fff700 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557623363690 0x557623365a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.172+0000 7f9507fff700 10 client.26003 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:15.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.172+0000 7f9527fff700 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557623363690 0x557623365a80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=618c1216dfc4017 server_cookie=b785bd507c8bb435 in_seq=0 out_seq=0 2024-06-06T08:45:15.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f9507fff700 1 -- 172.21.15.27:0/74680637 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5576232a7ef0 con 0x557623363690 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f9507fff700 10 client.26003 handle_command_reply: tid=0 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 client.26003 shutdown 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 -- 172.21.15.27:0/74680637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 msgr2=0x557623355c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x557623355c30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f951c000c00 tx=0x7f951c09c360 comp rx=0 tx=0).stop 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 -- 172.21.15.27:0/74680637 shutdown_connections 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557623363690 0x557623365a80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334fec0 0x557623353470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557623349cf0 0x557623355c30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 --2- 172.21.15.27:0/74680637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 0x557623352f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 -- 172.21.15.27:0/74680637 >> 172.21.15.27:0/74680637 conn(0x557623276220 msgr2=0x557623343c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f9507fff700 0 client.26003 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.175+0000 7f95394f5b80 1 -- 172.21.15.27:0/74680637 shutdown_connections 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.176+0000 7f95394f5b80 1 -- 172.21.15.27:0/74680637 wait complete. 2024-06-06T08:45:15.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.176+0000 7f95394f5b80 20 client.26003 trim_cache size 0 max 16384 2024-06-06T08:45:15.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f95394f5b80 1 Processor -- start 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f95394f5b80 1 -- start start 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557623349cf0 0x5576233497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x557623350910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x5576233443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f95394f5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95281b3140 con 0x55762334f5a0 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f95394f5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f95281b0910 con 0x55762334fec0 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f95394f5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f95281ae760 con 0x557623349cf0 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f952d20f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x5576233443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f952d20f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x5576233443d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.27:38892/0 (socket says 172.21.15.27:38892) 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f952d20f700 1 -- 172.21.15.27:0/4207449039 learned_addr learned my addr 172.21.15.27:0/4207449039 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f952ca0e700 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557623349cf0 0x5576233497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f9527fff700 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x557623350910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f9527fff700 1 -- 172.21.15.27:0/4207449039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557623349cf0 msgr2=0x5576233497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f9527fff700 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557623349cf0 0x5576233497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f9527fff700 1 -- 172.21.15.27:0/4207449039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 msgr2=0x5576233443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f9527fff700 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x5576233443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f9527fff700 1 -- 172.21.15.27:0/4207449039 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576233602a0 con 0x55762334f5a0 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f952ca0e700 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557623349cf0 0x5576233497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.197+0000 7f952d20f700 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x5576233443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.198+0000 7f9527fff700 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x557623350910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f951400b2e0 tx=0x7f9514020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ef94618430297b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.198+0000 7f95127fc700 1 -- 172.21.15.27:0/4207449039 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514005040 con 0x55762334f5a0 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.198+0000 7f95127fc700 1 -- 172.21.15.27:0/4207449039 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95140056c0 con 0x55762334f5a0 2024-06-06T08:45:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.198+0000 7f95127fc700 1 -- 172.21.15.27:0/4207449039 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951400f460 con 0x55762334f5a0 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 -- 172.21.15.27:0/4207449039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 msgr2=0x557623350910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x557623350910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f951400b2e0 tx=0x7f9514020000 comp rx=0 tx=0).stop 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 -- 172.21.15.27:0/4207449039 shutdown_connections 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x5576233443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557623349cf0 0x5576233497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 --2- 172.21.15.27:0/4207449039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55762334f5a0 0x557623350910 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 -- 172.21.15.27:0/4207449039 >> 172.21.15.27:0/4207449039 conn(0x557623276220 msgr2=0x557623354ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 -- 172.21.15.27:0/4207449039 shutdown_connections 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 -- 172.21.15.27:0/4207449039 wait complete. 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 -- start start 2024-06-06T08:45:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 Processor -- start 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.199+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557623349cf0 0x55762335bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 0x55762335c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952ca0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557623349cf0 0x55762335bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f95394f5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x557623358b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952ca0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557623349cf0 0x55762335bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46978/0 (socket says 172.21.15.27:46978) 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952ca0e700 1 -- 172.21.15.27:0/1433036006 learned_addr learned my addr 172.21.15.27:0/1433036006 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f95281aedd0 con 0x557623349cf0 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f95281b0240 con 0x55762334fec0 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f95281af4a0 con 0x55762334f5a0 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f95394f5b80 10 client.0 _ll_register_callbacks cb 0x557623230750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952d20f700 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x557623358b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f9527fff700 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 0x55762335c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952ca0e700 1 -- 172.21.15.27:0/1433036006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 msgr2=0x55762335c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952ca0e700 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 0x55762335c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952ca0e700 1 -- 172.21.15.27:0/1433036006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 msgr2=0x557623358b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952ca0e700 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x557623358b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952ca0e700 1 -- 172.21.15.27:0/1433036006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576233591f0 con 0x557623349cf0 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952d20f700 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x557623358b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f952ca0e700 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557623349cf0 0x55762335bd40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f951c09bab0 tx=0x7f951c0a1080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e09f47e46fdc888 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f9507fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f9507fff700 1 -- 172.21.15.27:0/1433036006 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0a4500 con 0x557623349cf0 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f9507fff700 1 -- 172.21.15.27:0/1433036006 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f951c010720 con 0x557623349cf0 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576232a2470 con 0x557623349cf0 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f9507fff700 1 -- 172.21.15.27:0/1433036006 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f951c0a1400 con 0x557623349cf0 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f95394f5b80 10 client.16521 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.200+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5576233d7480 con 0x557623349cf0 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f9507fff700 1 -- 172.21.15.27:0/1433036006 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f951c09ce50 con 0x557623349cf0 2024-06-06T08:45:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f9507fff700 1 -- 172.21.15.27:0/1433036006 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f951c0ba120 con 0x557623349cf0 2024-06-06T08:45:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f95394f5b80 10 client.16521 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f95394f5b80 10 client.16521 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:45:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f95394f5b80 20 client.16521 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f95394f5b80 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5576233d7c80 0x5576233da0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f95394f5b80 4 client.16521 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}] 2024-06-06T08:45:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.25, "await_for": 45.0}) -- 0x5576232a2470 con 0x5576233d7c80 2024-06-06T08:45:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f9527fff700 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5576233d7c80 0x5576233da0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.201+0000 7f9507fff700 10 client.16521 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:15.202+0000 7f9527fff700 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5576233d7c80 0x5576233da0a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75a00e26e7ae1d3 server_cookie=be255b2d9477b588 in_seq=0 out_seq=0 2024-06-06T08:45:16.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:16 smithi203 ceph-mon[36579]: pgmap v153: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 663 KiB/s wr, 192 op/s 2024-06-06T08:45:16.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:16 smithi176 ceph-mon[36786]: pgmap v153: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 663 KiB/s wr, 192 op/s 2024-06-06T08:45:16.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:16 smithi027 ceph-mon[32335]: pgmap v153: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 663 KiB/s wr, 192 op/s 2024-06-06T08:45:18.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:18 smithi203 ceph-mon[36579]: pgmap v154: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 641 KiB/s wr, 175 op/s 2024-06-06T08:45:18.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:18 smithi176 ceph-mon[36786]: pgmap v154: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 641 KiB/s wr, 175 op/s 2024-06-06T08:45:18.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:18 smithi027 ceph-mon[32335]: pgmap v154: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 641 KiB/s wr, 175 op/s 2024-06-06T08:45:20.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f9507fff700 1 -- 172.21.15.27:0/1433036006 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5576232a2470 con 0x5576233d7c80 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f9507fff700 10 client.16521 handle_command_reply: tid=0 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 client.16521 shutdown 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557623349cf0 msgr2=0x55762335bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557623349cf0 0x55762335bd40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f951c09bab0 tx=0x7f951c0a1080 comp rx=0 tx=0).stop 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 shutdown_connections 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55762334fec0 0x557623358b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x5576233d7c80 0x5576233da0a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557623349cf0 0x55762335bd40 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 --2- 172.21.15.27:0/1433036006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55762334f5a0 0x55762335c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f9507fff700 0 client.16521 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 >> 172.21.15.27:0/1433036006 conn(0x557623276220 msgr2=0x557623352f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:20.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 shutdown_connections 2024-06-06T08:45:20.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 1 -- 172.21.15.27:0/1433036006 wait complete. 2024-06-06T08:45:20.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.399+0000 7f95394f5b80 20 client.16521 trim_cache size 0 max 16384 2024-06-06T08:45:20.400 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:45:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-06-06T08:45:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24482, 2024-06-06T08:45:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 14, 2024-06-06T08:45:20.400 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "db363435": { 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 14, 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 145.2, 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:45:20.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-06-06T08:45:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:45:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:20.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:20.402 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.400+0000 7f952ec72700 1 -- 172.21.15.27:0/1916514664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f95180a2e80 msgr2=0x7f95180a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.400+0000 7f952ec72700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f95180a2e80 0x7f95180a5340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9528101e20 tx=0x7f9514005fb0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.400+0000 7f952ec72700 1 -- 172.21.15.27:0/1916514664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528101770 msgr2=0x7f952819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.400+0000 7f952ec72700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528101770 0x7f952819ae20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f951c00e9a0 tx=0x7f951c00ecb0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.401+0000 7f952ec72700 1 -- 172.21.15.27:0/1916514664 shutdown_connections 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.401+0000 7f952ec72700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95281074b0 0x7f952819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.401+0000 7f952ec72700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528101770 0x7f952819ae20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.401+0000 7f952ec72700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f95180a2e80 0x7f95180a5340 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.401+0000 7f952ec72700 1 --2- 172.21.15.27:0/1916514664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528102130 0x7f952819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.401+0000 7f952ec72700 1 -- 172.21.15.27:0/1916514664 >> 172.21.15.27:0/1916514664 conn(0x7f9528078340 msgr2=0x7f952810c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.401+0000 7f952ec72700 1 -- 172.21.15.27:0/1916514664 shutdown_connections 2024-06-06T08:45:20.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:20.401+0000 7f952ec72700 1 -- 172.21.15.27:0/1916514664 wait complete. 2024-06-06T08:45:20.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:20 smithi027 ceph-mon[32335]: pgmap v155: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 871 KiB/s wr, 266 op/s 2024-06-06T08:45:20.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:20 smithi176 ceph-mon[36786]: pgmap v155: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 871 KiB/s wr, 266 op/s 2024-06-06T08:45:20.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:20 smithi203 ceph-mon[36579]: pgmap v155: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 871 KiB/s wr, 266 op/s 2024-06-06T08:45:21.235 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'db363435', quiesce took 11.6 seconds. Will release after: 13.65 2024-06-06T08:45:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 -- 172.21.15.27:0/198265954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 msgr2=0x7ff4f010d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/198265954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 0x7ff4f010d790 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009cd0 tx=0x7ff4e802a8e0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 -- 172.21.15.27:0/198265954 shutdown_connections 2024-06-06T08:45:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/198265954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 0x7ff4f010d790 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/198265954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4f01008c0 0x7ff4f0100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/198265954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f01068d0 0x7ff4f0106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 -- 172.21.15.27:0/198265954 >> 172.21.15.27:0/198265954 conn(0x7ff4f0076c20 msgr2=0x7ff4f0077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 -- 172.21.15.27:0/198265954 shutdown_connections 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 -- 172.21.15.27:0/198265954 wait complete. 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 Processor -- start 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.051+0000 7ff4f6bed640 1 -- start start 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4f6bed640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4f01008c0 0x7ff4f0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4f6bed640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 0x7ff4f01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4f6bed640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f01068d0 0x7ff4f019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4f6bed640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f0110ce0 con 0x7ff4f0101280 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4f6bed640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f0110b60 con 0x7ff4f01008c0 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4f6bed640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f0110e60 con 0x7ff4f01068d0 2024-06-06T08:45:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 0x7ff4f01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 0x7ff4f01986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44846/0 (socket says 172.21.15.27:44846) 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4effff640 1 -- 172.21.15.27:0/3690226903 learned_addr learned my addr 172.21.15.27:0/3690226903 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4f4989640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4f01008c0 0x7ff4f0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4f518a640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f01068d0 0x7ff4f019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4effff640 1 -- 172.21.15.27:0/3690226903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f01068d0 msgr2=0x7ff4f019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4effff640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f01068d0 0x7ff4f019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4effff640 1 -- 172.21.15.27:0/3690226903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4f01008c0 msgr2=0x7ff4f0198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4effff640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4f01008c0 0x7ff4f0198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4effff640 1 -- 172.21.15.27:0/3690226903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f019d160 con 0x7ff4f0101280 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.052+0000 7ff4effff640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 0x7ff4f01986b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c890 tx=0x7ff4dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.053+0000 7ff4edffb640 1 -- 172.21.15.27:0/3690226903 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc004220 con 0x7ff4f0101280 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.053+0000 7ff4edffb640 1 -- 172.21.15.27:0/3690226903 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc0043c0 con 0x7ff4f0101280 2024-06-06T08:45:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.053+0000 7ff4f6bed640 1 -- 172.21.15.27:0/3690226903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f019d450 con 0x7ff4f0101280 2024-06-06T08:45:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.053+0000 7ff4edffb640 1 -- 172.21.15.27:0/3690226903 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0049e0 con 0x7ff4f0101280 2024-06-06T08:45:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.053+0000 7ff4f6bed640 1 -- 172.21.15.27:0/3690226903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f01a4ce0 con 0x7ff4f0101280 2024-06-06T08:45:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.055+0000 7ff4edffb640 1 -- 172.21.15.27:0/3690226903 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff4dc0079b0 con 0x7ff4f0101280 2024-06-06T08:45:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f99ce740 1 Processor -- start 2024-06-06T08:45:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f99ce740 1 -- start start 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4edffb640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff4e00a2e20 0x7ff4e00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 0x55e894245550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245a90 0x55e89424fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e894252740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e8941a94f0 con 0x55e894245a90 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e894147870 con 0x55e894250350 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e8941a54b0 con 0x55e894245170 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f518a640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff4e00a2e20 0x7ff4e00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4edffb640 1 -- 172.21.15.27:0/3690226903 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7ff4dc0cf740 con 0x7ff4f0101280 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245a90 0x55e89424fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245a90 0x55e89424fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44848/0 (socket says 172.21.15.27:44848) 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4effff640 1 -- 172.21.15.27:0/2880362040 learned_addr learned my addr 172.21.15.27:0/2880362040 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f4989640 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 0x55e894245550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f518a640 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e894252740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4effff640 1 -- 172.21.15.27:0/2880362040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 msgr2=0x55e894245550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4effff640 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 0x55e894245550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4effff640 1 -- 172.21.15.27:0/2880362040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 msgr2=0x55e894252740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4effff640 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e894252740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4effff640 1 -- 172.21.15.27:0/2880362040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8941639b0 con 0x55e894245a90 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.056+0000 7ff4f518a640 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e894252740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.057+0000 7ff4effff640 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245a90 0x55e89424fe10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0a3fe0 tx=0x7ff4dc09b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d859f6a5133b291b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.057+0000 7ff4f518a640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff4e00a2e20 0x7ff4e00a52e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8004ab0 tx=0x7ff4e8009800 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.057+0000 7ff4daffd640 1 -- 172.21.15.27:0/2880362040 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc01d040 con 0x55e894245a90 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.057+0000 7ff4daffd640 1 -- 172.21.15.27:0/2880362040 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc0a4510 con 0x55e894245a90 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.057+0000 7ff4daffd640 1 -- 172.21.15.27:0/2880362040 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0cf360 con 0x55e894245a90 2024-06-06T08:45:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.057+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2880362040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245a90 msgr2=0x55e89424fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.057+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245a90 0x55e89424fe10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0a3fe0 tx=0x7ff4dc09b5c0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2880362040 shutdown_connections 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e894252740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245a90 0x55e89424fe10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2880362040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 0x55e894245550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2880362040 >> 172.21.15.27:0/2880362040 conn(0x55e894241680 msgr2=0x55e894241a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2880362040 shutdown_connections 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2880362040 wait complete. 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 -- start start 2024-06-06T08:45:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 Processor -- start 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.058+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e89424d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245a90 0x55e89424d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e89424ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e894257390 con 0x55e894245170 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55e894132ae0 con 0x55e894250350 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e894257510 con 0x55e894245a90 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f99ce740 10 client.0 _ll_register_callbacks cb 0x55e89416b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f4989640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e89424d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f4989640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e89424d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44860/0 (socket says 172.21.15.27:44860) 2024-06-06T08:45:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f4989640 1 -- 172.21.15.27:0/2875349044 learned_addr learned my addr 172.21.15.27:0/2875349044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4effff640 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245a90 0x55e89424d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f518a640 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e89424ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f4989640 1 -- 172.21.15.27:0/2875349044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245a90 msgr2=0x55e89424d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f4989640 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245a90 0x55e89424d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f4989640 1 -- 172.21.15.27:0/2875349044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 msgr2=0x55e89424ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f4989640 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e89424ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f4989640 1 -- 172.21.15.27:0/2875349044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e893ebd210 con 0x55e894245170 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f518a640 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e89424ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f4989640 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e89424d370 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4012740 tx=0x7ff4e4012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d0776fe7644022 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4cffff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4cffff640 1 -- 172.21.15.27:0/2875349044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4004030 con 0x55e894245170 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4cffff640 1 -- 172.21.15.27:0/2875349044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7ff4e40041d0 con 0x55e894245170 2024-06-06T08:45:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.059+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2875349044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4dc0a4510 con 0x55e894245170 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4cffff640 1 -- 172.21.15.27:0/2875349044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff4e4004410 con 0x55e894245170 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4f99ce740 10 client.16533 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2875349044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e893f89620 con 0x55e894245170 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4cffff640 1 -- 172.21.15.27:0/2875349044 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc0a4510 con 0x55e894245170 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4cffff640 1 -- 172.21.15.27:0/2875349044 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff4e4003bc0 con 0x55e894245170 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4f99ce740 10 client.16533 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4f99ce740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4f99ce740 20 client.16533 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e894264a20 0x55e894266e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4f99ce740 4 client.16533 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.060+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2875349044 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8941bcf10 con 0x55e894264a20 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.061+0000 7ff4effff640 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e894264a20 0x55e894266e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.061+0000 7ff4effff640 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e894264a20 0x55e894266e30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e04a3d4cd358d5 server_cookie=c75ac3ef12834d10 in_seq=0 out_seq=0 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.061+0000 7ff4cffff640 10 client.16533 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4cffff640 1 -- 172.21.15.27:0/2875349044 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e8941bcf10 con 0x55e894264a20 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4cffff640 10 client.16533 handle_command_reply: tid=0 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 client.16533 shutdown 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2875349044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 msgr2=0x55e89424d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e89424d370 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4012740 tx=0x7ff4e4012c00 comp rx=0 tx=0).stop 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2875349044 shutdown_connections 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e894264a20 0x55e894266e30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894250350 0x55e89424ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245a90 0x55e89424d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/2875349044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e89424d370 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4cffff640 0 client.16533 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2875349044 >> 172.21.15.27:0/2875349044 conn(0x55e894241680 msgr2=0x55e894247400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2875349044 shutdown_connections 2024-06-06T08:45:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 1 -- 172.21.15.27:0/2875349044 wait complete. 2024-06-06T08:45:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.062+0000 7ff4f99ce740 20 client.16533 trim_cache size 0 max 16384 2024-06-06T08:45:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.077+0000 7ff4f99ce740 1 Processor -- start 2024-06-06T08:45:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.077+0000 7ff4f99ce740 1 -- start start 2024-06-06T08:45:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e894247b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 0x55e89426c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894250350 0x55e89426cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01af600 con 0x55e894245170 2024-06-06T08:45:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01ace10 con 0x55e894245a90 2024-06-06T08:45:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01aac60 con 0x55e894250350 2024-06-06T08:45:22.077 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f4989640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e894247b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f4989640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e894247b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44870/0 (socket says 172.21.15.27:44870) 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f4989640 1 -- 172.21.15.27:0/555656140 learned_addr learned my addr 172.21.15.27:0/555656140 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f4989640 1 -- 172.21.15.27:0/555656140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894250350 msgr2=0x55e89426cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f518a640 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894250350 0x55e89426cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4effff640 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 0x55e89426c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f4989640 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894250350 0x55e89426cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f4989640 1 -- 172.21.15.27:0/555656140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 msgr2=0x55e89426c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f4989640 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 0x55e89426c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f4989640 1 -- 172.21.15.27:0/555656140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e894261790 con 0x55e894245170 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f518a640 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894250350 0x55e89426cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:45:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4f4989640 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e894247b30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff4e401c460 tx=0x7ff4e4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e8293af835c536 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4da7fc640 1 -- 172.21.15.27:0/555656140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e401c710 con 0x55e894245170 2024-06-06T08:45:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4da7fc640 1 -- 172.21.15.27:0/555656140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e4012e50 con 0x55e894245170 2024-06-06T08:45:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.078+0000 7ff4da7fc640 1 -- 172.21.15.27:0/555656140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400fe70 con 0x55e894245170 2024-06-06T08:45:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 -- 172.21.15.27:0/555656140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 msgr2=0x55e894247b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e894247b30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff4e401c460 tx=0x7ff4e4009f80 comp rx=0 tx=0).stop 2024-06-06T08:45:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 -- 172.21.15.27:0/555656140 shutdown_connections 2024-06-06T08:45:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894250350 0x55e89426cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 0x55e89426c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/555656140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894245170 0x55e894247b30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 -- 172.21.15.27:0/555656140 >> 172.21.15.27:0/555656140 conn(0x55e894241680 msgr2=0x55e8942485d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 -- 172.21.15.27:0/555656140 shutdown_connections 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 -- 172.21.15.27:0/555656140 wait complete. 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 -- start start 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.079+0000 7ff4f99ce740 1 Processor -- start 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 0x55e89425cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 0x55e8942580c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f99ce740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894250350 0x55e894258600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01ab2d0 con 0x55e894250350 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01ac740 con 0x55e894245a90 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f99ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f01ab9a0 con 0x55e894245170 2024-06-06T08:45:22.083 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f99ce740 10 client.0 _ll_register_callbacks cb 0x7ff4e4009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f518a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894250350 0x55e894258600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f518a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894250350 0x55e894258600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44882/0 (socket says 172.21.15.27:44882) 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f518a640 1 -- 172.21.15.27:0/1919636903 learned_addr learned my addr 172.21.15.27:0/1919636903 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f4989640 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 0x55e89425cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4effff640 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 0x55e8942580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f518a640 1 -- 172.21.15.27:0/1919636903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 msgr2=0x55e89425cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f518a640 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 0x55e89425cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f518a640 1 -- 172.21.15.27:0/1919636903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 msgr2=0x55e8942580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f518a640 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 0x55e8942580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4f518a640 1 -- 172.21.15.27:0/1919636903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e894258c70 con 0x55e894250350 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.080+0000 7ff4effff640 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 0x55e8942580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f518a640 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894250350 0x55e894258600 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff4e802ae00 tx=0x7ff4e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87e025dcfa8a77 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4cffff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4cffff640 1 -- 172.21.15.27:0/1919636903 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80307a0 con 0x55e894250350 2024-06-06T08:45:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4cffff640 1 -- 172.21.15.27:0/1919636903 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7ff4e8030940 con 0x55e894250350 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f99ce740 1 -- 172.21.15.27:0/1919636903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e4012e50 con 0x55e894250350 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4cffff640 1 -- 172.21.15.27:0/1919636903 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff4e805c050 con 0x55e894250350 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f99ce740 10 client.16539 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f99ce740 1 -- 172.21.15.27:0/1919636903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e8942cf520 con 0x55e894250350 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4cffff640 1 -- 172.21.15.27:0/1919636903 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e4012e50 con 0x55e894250350 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4cffff640 1 -- 172.21.15.27:0/1919636903 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff4e8030070 con 0x55e894250350 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f99ce740 10 client.16539 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f99ce740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f99ce740 20 client.16539 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e8942cfd00 0x55e8942d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f99ce740 4 client.16539 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:45:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.081+0000 7ff4f99ce740 1 -- 172.21.15.27:0/1919636903 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8941b6c20 con 0x55e8942cfd00 2024-06-06T08:45:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.082+0000 7ff4f4989640 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e8942cfd00 0x55e8942d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.082+0000 7ff4f4989640 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e8942cfd00 0x55e8942d2120 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a46588797a49283 server_cookie=a1a62dbf4374a936 in_seq=0 out_seq=0 2024-06-06T08:45:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.082+0000 7ff4cffff640 10 client.16539 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.225+0000 7ff4cffff640 1 -- 172.21.15.27:0/1919636903 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e8941b6c20 con 0x55e8942cfd00 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.225+0000 7ff4cffff640 10 client.16539 handle_command_reply: tid=0 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.225+0000 7ff4f99ce740 1 client.16539 shutdown 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.225+0000 7ff4f99ce740 1 -- 172.21.15.27:0/1919636903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894250350 msgr2=0x55e894258600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.225+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894250350 0x55e894258600 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff4e802ae00 tx=0x7ff4e8004a40 comp rx=0 tx=0).stop 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.225+0000 7ff4f99ce740 1 -- 172.21.15.27:0/1919636903 shutdown_connections 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.226+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55e8942cfd00 0x55e8942d2120 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.226+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e894250350 0x55e894258600 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.226+0000 7ff4cffff640 0 client.16539 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.226+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e894245a90 0x55e8942580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.226+0000 7ff4f99ce740 1 --2- 172.21.15.27:0/1919636903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e894245170 0x55e89425cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.226+0000 7ff4f99ce740 1 -- 172.21.15.27:0/1919636903 >> 172.21.15.27:0/1919636903 conn(0x55e894241680 msgr2=0x55e89424eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.226+0000 7ff4f99ce740 1 -- 172.21.15.27:0/1919636903 shutdown_connections 2024-06-06T08:45:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.226+0000 7ff4f99ce740 1 -- 172.21.15.27:0/1919636903 wait complete. 2024-06-06T08:45:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.226+0000 7ff4f99ce740 20 client.16539 trim_cache size 0 max 16384 2024-06-06T08:45:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:45:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:45:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:45:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 -- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff4e00a2e20 msgr2=0x7ff4e00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff4e00a2e20 0x7ff4e00a52e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8004ab0 tx=0x7ff4e8009800 comp rx=0 tx=0).stop 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 -- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 msgr2=0x7ff4f01986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 0x7ff4f01986b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c890 tx=0x7ff4dc00cd50 comp rx=0 tx=0).stop 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 -- 172.21.15.27:0/3690226903 shutdown_connections 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7ff4e00a2e20 0x7ff4e00a52e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f01068d0 0x7ff4f019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f0101280 0x7ff4f01986b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 --2- 172.21.15.27:0/3690226903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4f01008c0 0x7ff4f0198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 -- 172.21.15.27:0/3690226903 >> 172.21.15.27:0/3690226903 conn(0x7ff4f0076c20 msgr2=0x7ff4f0104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 -- 172.21.15.27:0/3690226903 shutdown_connections 2024-06-06T08:45:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:22.227+0000 7ff4f6bed640 1 -- 172.21.15.27:0/3690226903 wait complete. 2024-06-06T08:45:22.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:22 smithi027 ceph-mon[32335]: pgmap v156: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 865 KiB/s wr, 262 op/s 2024-06-06T08:45:22.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:22 smithi176 ceph-mon[36786]: pgmap v156: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 865 KiB/s wr, 262 op/s 2024-06-06T08:45:22.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:22 smithi203 ceph-mon[36579]: pgmap v156: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 865 KiB/s wr, 262 op/s 2024-06-06T08:45:24.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:24 smithi176 ceph-mon[36786]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 910 KiB/s wr, 301 op/s 2024-06-06T08:45:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:24 smithi027 ceph-mon[32335]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 910 KiB/s wr, 301 op/s 2024-06-06T08:45:24.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:24 smithi203 ceph-mon[36579]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 910 KiB/s wr, 301 op/s 2024-06-06T08:45:26.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:26 smithi176 ceph-mon[36786]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 997 KiB/s wr, 351 op/s 2024-06-06T08:45:26.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:26 smithi027 ceph-mon[32335]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 997 KiB/s wr, 351 op/s 2024-06-06T08:45:26.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:26 smithi203 ceph-mon[36579]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 997 KiB/s wr, 351 op/s 2024-06-06T08:45:28.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:28 smithi176 ceph-mon[36786]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 369 KiB/s wr, 212 op/s 2024-06-06T08:45:28.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:28 smithi027 ceph-mon[32335]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 369 KiB/s wr, 212 op/s 2024-06-06T08:45:28.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:28 smithi203 ceph-mon[36579]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 369 KiB/s wr, 212 op/s 2024-06-06T08:45:30.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:30 smithi027 ceph-mon[32335]: pgmap v160: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 370 KiB/s wr, 216 op/s 2024-06-06T08:45:30.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:30 smithi203 ceph-mon[36579]: pgmap v160: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 370 KiB/s wr, 216 op/s 2024-06-06T08:45:31.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:30 smithi176 ceph-mon[36786]: pgmap v160: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 370 KiB/s wr, 216 op/s 2024-06-06T08:45:32.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:32 smithi027 ceph-mon[32335]: pgmap v161: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 140 KiB/s wr, 125 op/s 2024-06-06T08:45:32.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:32 smithi203 ceph-mon[36579]: pgmap v161: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 140 KiB/s wr, 125 op/s 2024-06-06T08:45:33.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:32 smithi176 ceph-mon[36786]: pgmap v161: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 140 KiB/s wr, 125 op/s 2024-06-06T08:45:34.887 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:45:34.917 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:34 smithi027 ceph-mon[32335]: pgmap v162: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 140 KiB/s wr, 127 op/s 2024-06-06T08:45:34.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:34 smithi203 ceph-mon[36579]: pgmap v162: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 140 KiB/s wr, 127 op/s 2024-06-06T08:45:35.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:34 smithi176 ceph-mon[36786]: pgmap v162: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 140 KiB/s wr, 127 op/s 2024-06-06T08:45:35.155 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:45:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.318+0000 7fcf0b01c640 1 -- 172.21.15.27:0/1728305399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf04106890 msgr2=0x7fcf04106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.318+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/1728305399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf04106890 0x7fcf04106c70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009930 tx=0x7fcef802a420 comp rx=0 tx=0).stop 2024-06-06T08:45:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.318+0000 7fcf0b01c640 1 -- 172.21.15.27:0/1728305399 shutdown_connections 2024-06-06T08:45:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.318+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/1728305399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf04101290 0x7fcf0410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.318+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/1728305399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf041008d0 0x7fcf04100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.318+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/1728305399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf04106890 0x7fcf04106c70 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.318+0000 7fcf0b01c640 1 -- 172.21.15.27:0/1728305399 >> 172.21.15.27:0/1728305399 conn(0x7fcf04077a20 msgr2=0x7fcf04077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.318+0000 7fcf0b01c640 1 -- 172.21.15.27:0/1728305399 shutdown_connections 2024-06-06T08:45:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.318+0000 7fcf0b01c640 1 -- 172.21.15.27:0/1728305399 wait complete. 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf0b01c640 1 Processor -- start 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf0b01c640 1 -- start start 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf0b01c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf041008d0 0x7fcf041982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf0b01c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf04101290 0x7fcf041987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf0b01c640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf04106890 0x7fcf0419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf0b01c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcf04110c60 con 0x7fcf041008d0 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf0b01c640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcf04110ae0 con 0x7fcf04106890 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf0b01c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf04110de0 con 0x7fcf04101290 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf08db8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf041008d0 0x7fcf041982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf08db8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf041008d0 0x7fcf041982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33314/0 (socket says 172.21.15.27:33314) 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf08db8640 1 -- 172.21.15.27:0/2854389161 learned_addr learned my addr 172.21.15.27:0/2854389161 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf03fff640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf04101290 0x7fcf041987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf095b9640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf04106890 0x7fcf0419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf08db8640 1 -- 172.21.15.27:0/2854389161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf04101290 msgr2=0x7fcf041987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf08db8640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf04101290 0x7fcf041987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf08db8640 1 -- 172.21.15.27:0/2854389161 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf04106890 msgr2=0x7fcf0419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.319+0000 7fcf08db8640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf04106890 0x7fcf0419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.320+0000 7fcf08db8640 1 -- 172.21.15.27:0/2854389161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0419d2a0 con 0x7fcf041008d0 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.320+0000 7fcf08db8640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf041008d0 0x7fcf041982b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009a50 tx=0x7fcef8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.320+0000 7fcf01ffb640 1 -- 172.21.15.27:0/2854389161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef803f030 con 0x7fcf041008d0 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.320+0000 7fcf01ffb640 1 -- 172.21.15.27:0/2854389161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef8003ce0 con 0x7fcf041008d0 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.320+0000 7fcf0b01c640 1 -- 172.21.15.27:0/2854389161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0419d530 con 0x7fcf041008d0 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.320+0000 7fcf01ffb640 1 -- 172.21.15.27:0/2854389161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8035070 con 0x7fcf041008d0 2024-06-06T08:45:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.320+0000 7fcf0b01c640 1 -- 172.21.15.27:0/2854389161 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf041a4e20 con 0x7fcf041008d0 2024-06-06T08:45:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.323+0000 7fcf0de0a740 1 Processor -- start 2024-06-06T08:45:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.323+0000 7fcf01ffb640 1 -- 172.21.15.27:0/2854389161 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcef8042030 con 0x7fcf041008d0 2024-06-06T08:45:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.323+0000 7fcf0de0a740 1 -- start start 2024-06-06T08:45:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.323+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295a170 0x557fe295a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.323+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295aa90 0x557fe2964e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.323+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2967740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.323+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557fe28be4f0 con 0x557fe2965350 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.323+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557fe285c870 con 0x557fe295aa90 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.323+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557fe28ba4b0 con 0x557fe295a170 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2967740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2967740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33318/0 (socket says 172.21.15.27:33318) 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf03fff640 1 -- 172.21.15.27:0/992733683 learned_addr learned my addr 172.21.15.27:0/992733683 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf095b9640 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295a170 0x557fe295a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf08db8640 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295aa90 0x557fe2964e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf03fff640 1 -- 172.21.15.27:0/992733683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295a170 msgr2=0x557fe295a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf03fff640 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295a170 0x557fe295a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf03fff640 1 -- 172.21.15.27:0/992733683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295aa90 msgr2=0x557fe2964e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf03fff640 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295aa90 0x557fe2964e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf03fff640 1 -- 172.21.15.27:0/992733683 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fe28789b0 con 0x557fe2965350 2024-06-06T08:45:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf01ffb640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcef40a2da0 0x7fcef40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf08db8640 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295aa90 0x557fe2964e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf03fff640 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2967740 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcef000abf0 tx=0x7fcef000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d9f7b8c2c8124e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf095b9640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcef40a2da0 0x7fcef40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fcf01ffb640 1 -- 172.21.15.27:0/2854389161 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fcef80efce0 con 0x7fcf041008d0 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fceeeffd640 1 -- 172.21.15.27:0/992733683 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef001d070 con 0x557fe2965350 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fceeeffd640 1 -- 172.21.15.27:0/992733683 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef0018b20 con 0x557fe2965350 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.324+0000 7fceeeffd640 1 -- 172.21.15.27:0/992733683 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0010ae0 con 0x557fe2965350 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 -- 172.21.15.27:0/992733683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 msgr2=0x557fe2967740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf095b9640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcef40a2da0 0x7fcef40a5260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00ae90 tx=0x7fcefc00a4d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2967740 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcef000abf0 tx=0x7fcef000f790 comp rx=0 tx=0).stop 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 -- 172.21.15.27:0/992733683 shutdown_connections 2024-06-06T08:45:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2967740 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295aa90 0x557fe2964e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/992733683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295a170 0x557fe295a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 -- 172.21.15.27:0/992733683 >> 172.21.15.27:0/992733683 conn(0x557fe2956680 msgr2=0x557fe2956a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 -- 172.21.15.27:0/992733683 shutdown_connections 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 -- 172.21.15.27:0/992733683 wait complete. 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 -- start start 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.325+0000 7fcf0de0a740 1 Processor -- start 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe2962350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe2962890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe2965350 0x557fe2962dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557fe2967f80 con 0x557fe295aa90 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x557fe2847ae0 con 0x557fe295a170 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x557fe2968100 con 0x557fe2965350 2024-06-06T08:45:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf0de0a740 10 client.0 _ll_register_callbacks cb 0x557fe28809b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe2962890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe2962890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33330/0 (socket says 172.21.15.27:33330) 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf03fff640 1 -- 172.21.15.27:0/3905419071 learned_addr learned my addr 172.21.15.27:0/3905419071 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf08db8640 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe2962350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf095b9640 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe2965350 0x557fe2962dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf03fff640 1 -- 172.21.15.27:0/3905419071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe2965350 msgr2=0x557fe2962dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf03fff640 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe2965350 0x557fe2962dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf03fff640 1 -- 172.21.15.27:0/3905419071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 msgr2=0x557fe2962350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf03fff640 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe2962350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf03fff640 1 -- 172.21.15.27:0/3905419071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fe25d2210 con 0x557fe295aa90 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcf03fff640 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe2962890 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcef000a7c0 tx=0x7fcef0018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2601fb34005cf24 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.326+0000 7fcee3fff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcee3fff640 1 -- 172.21.15.27:0/3905419071 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef001d050 con 0x557fe295aa90 2024-06-06T08:45:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcee3fff640 1 -- 172.21.15.27:0/3905419071 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fcef0010860 con 0x557fe295aa90 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3905419071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef0018b20 con 0x557fe295aa90 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcee3fff640 1 -- 172.21.15.27:0/3905419071 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcef0031050 con 0x557fe295aa90 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcf0de0a740 10 client.16551 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3905419071 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557fe269e620 con 0x557fe295aa90 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcee3fff640 1 -- 172.21.15.27:0/3905419071 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef0018b20 con 0x557fe295aa90 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcee3fff640 1 -- 172.21.15.27:0/3905419071 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcef00046c0 con 0x557fe295aa90 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcf0de0a740 10 client.16551 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcf0de0a740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcf0de0a740 20 client.16551 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557fe29753e0 0x557fe29777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcf0de0a740 4 client.16551 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.327+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3905419071 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fe28d1f10 con 0x557fe29753e0 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.328+0000 7fcf08db8640 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557fe29753e0 0x557fe29777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.328+0000 7fcf08db8640 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557fe29753e0 0x557fe29777d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9449b6d215deb25a server_cookie=bd25667750419440 in_seq=0 out_seq=0 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.328+0000 7fcee3fff640 10 client.16551 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcee3fff640 1 -- 172.21.15.27:0/3905419071 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557fe28d1f10 con 0x557fe29753e0 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcee3fff640 10 client.16551 handle_command_reply: tid=0 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 client.16551 shutdown 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3905419071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 msgr2=0x557fe2962890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe2962890 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcef000a7c0 tx=0x7fcef0018670 comp rx=0 tx=0).stop 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3905419071 shutdown_connections 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557fe29753e0 0x557fe29777d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe2965350 0x557fe2962dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe2962890 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcee3fff640 0 client.16551 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3905419071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe2962350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3905419071 >> 172.21.15.27:0/3905419071 conn(0x557fe2956680 msgr2=0x557fe295c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3905419071 shutdown_connections 2024-06-06T08:45:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3905419071 wait complete. 2024-06-06T08:45:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.329+0000 7fcf0de0a740 20 client.16551 trim_cache size 0 max 16384 2024-06-06T08:45:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf0de0a740 1 Processor -- start 2024-06-06T08:45:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf0de0a740 1 -- start start 2024-06-06T08:45:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe297ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe297d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe2965350 0x557fe295f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041b1870 con 0x557fe295aa90 2024-06-06T08:45:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041af080 con 0x557fe295a170 2024-06-06T08:45:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041aced0 con 0x557fe2965350 2024-06-06T08:45:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe297d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe297d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33342/0 (socket says 172.21.15.27:33342) 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf03fff640 1 -- 172.21.15.27:0/3016655691 learned_addr learned my addr 172.21.15.27:0/3016655691 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf03fff640 1 -- 172.21.15.27:0/3016655691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe2965350 msgr2=0x557fe295f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf03fff640 1 --2- 172.21.15.27:0/3016655691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe2965350 0x557fe295f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf08db8640 1 --2- 172.21.15.27:0/3016655691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe297ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf03fff640 1 -- 172.21.15.27:0/3016655691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 msgr2=0x557fe297ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf03fff640 1 --2- 172.21.15.27:0/3016655691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe297ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf03fff640 1 -- 172.21.15.27:0/3016655691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fe29711e0 con 0x557fe295aa90 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf08db8640 1 --2- 172.21.15.27:0/3016655691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe297ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.345+0000 7fcf03fff640 1 --2- 172.21.15.27:0/3016655691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe297d3a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcef000fe40 tx=0x7fcef000aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2584543f8f5d82a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fceee7fc640 1 -- 172.21.15.27:0/3016655691 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef001d070 con 0x557fe295aa90 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fceee7fc640 1 -- 172.21.15.27:0/3016655691 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef001e040 con 0x557fe295aa90 2024-06-06T08:45:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fceee7fc640 1 -- 172.21.15.27:0/3016655691 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0004ed0 con 0x557fe295aa90 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3016655691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 msgr2=0x557fe297d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3016655691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe297d3a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcef000fe40 tx=0x7fcef000aa40 comp rx=0 tx=0).stop 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3016655691 shutdown_connections 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3016655691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe2965350 0x557fe295f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3016655691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe295aa90 0x557fe297d3a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/3016655691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe297ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3016655691 >> 172.21.15.27:0/3016655691 conn(0x557fe2956680 msgr2=0x557fe2960c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3016655691 shutdown_connections 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.346+0000 7fcf0de0a740 1 -- 172.21.15.27:0/3016655691 wait complete. 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf0de0a740 1 -- start start 2024-06-06T08:45:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf0de0a740 1 Processor -- start 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe296d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295aa90 0x557fe296dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf0de0a740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2968b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041ad540 con 0x557fe2965350 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041ae9b0 con 0x557fe295a170 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf0de0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041adc10 con 0x557fe295aa90 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf0de0a740 10 client.0 _ll_register_callbacks cb 0x7fcef001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf095b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2968b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf095b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2968b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33344/0 (socket says 172.21.15.27:33344) 2024-06-06T08:45:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf095b9640 1 -- 172.21.15.27:0/2801806926 learned_addr learned my addr 172.21.15.27:0/2801806926 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.347+0000 7fcf03fff640 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295aa90 0x557fe296dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf08db8640 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe296d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf095b9640 1 -- 172.21.15.27:0/2801806926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295aa90 msgr2=0x557fe296dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf095b9640 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295aa90 0x557fe296dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf095b9640 1 -- 172.21.15.27:0/2801806926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 msgr2=0x557fe296d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf095b9640 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe296d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf095b9640 1 -- 172.21.15.27:0/2801806926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fe2969350 con 0x557fe2965350 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf095b9640 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2968b20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcefc005fd0 tx=0x7fcefc01ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b06e2ae15fd91ca server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcee3fff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcee3fff640 1 -- 172.21.15.27:0/2801806926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc01cab0 con 0x557fe2965350 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcee3fff640 1 -- 172.21.15.27:0/2801806926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fcefc016030 con 0x557fe2965350 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf0de0a740 1 -- 172.21.15.27:0/2801806926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef001e040 con 0x557fe2965350 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcee3fff640 1 -- 172.21.15.27:0/2801806926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcefc034050 con 0x557fe2965350 2024-06-06T08:45:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf0de0a740 10 client.16557 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf0de0a740 1 -- 172.21.15.27:0/2801806926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557fe29dfa10 con 0x557fe2965350 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcee3fff640 1 -- 172.21.15.27:0/2801806926 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef001e040 con 0x557fe2965350 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcee3fff640 1 -- 172.21.15.27:0/2801806926 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcefc02f030 con 0x557fe2965350 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf0de0a740 10 client.16557 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf0de0a740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.348+0000 7fcf0de0a740 20 client.16557 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.349+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557fe29e0180 0x557fe29e25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.349+0000 7fcf0de0a740 4 client.16557 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.349+0000 7fcf0de0a740 1 -- 172.21.15.27:0/2801806926 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557fe28cbc20 con 0x557fe29e0180 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.349+0000 7fcf08db8640 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557fe29e0180 0x557fe29e25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.349+0000 7fcee3fff640 10 client.16557 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.349+0000 7fcf08db8640 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557fe29e0180 0x557fe29e25a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bfdffb931494d7e server_cookie=16c2d06fc4f9e960 in_seq=0 out_seq=0 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcee3fff640 1 -- 172.21.15.27:0/2801806926 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557fe28cbc20 con 0x557fe29e0180 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcee3fff640 10 client.16557 handle_command_reply: tid=0 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 client.16557 shutdown 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 -- 172.21.15.27:0/2801806926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 msgr2=0x557fe2968b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2968b20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcefc005fd0 tx=0x7fcefc01ec20 comp rx=0 tx=0).stop 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 -- 172.21.15.27:0/2801806926 shutdown_connections 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x557fe29e0180 0x557fe29e25a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557fe2965350 0x557fe2968b20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcee3fff640 0 client.16557 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557fe295aa90 0x557fe296dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 --2- 172.21.15.27:0/2801806926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557fe295a170 0x557fe296d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 -- 172.21.15.27:0/2801806926 >> 172.21.15.27:0/2801806926 conn(0x557fe2956680 msgr2=0x557fe295bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 -- 172.21.15.27:0/2801806926 shutdown_connections 2024-06-06T08:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 1 -- 172.21.15.27:0/2801806926 wait complete. 2024-06-06T08:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.370+0000 7fcf0de0a740 20 client.16557 trim_cache size 0 max 16384 2024-06-06T08:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:45:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:45:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 -- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcef40a2da0 msgr2=0x7fcef40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcef40a2da0 0x7fcef40a5260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00ae90 tx=0x7fcefc00a4d0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 -- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf041008d0 msgr2=0x7fcf041982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf041008d0 0x7fcf041982b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009a50 tx=0x7fcef8004270 comp rx=0 tx=0).stop 2024-06-06T08:45:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 -- 172.21.15.27:0/2854389161 shutdown_connections 2024-06-06T08:45:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcef40a2da0 0x7fcef40a5260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf04106890 0x7fcf0419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf04101290 0x7fcf041987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 --2- 172.21.15.27:0/2854389161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf041008d0 0x7fcf041982b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 -- 172.21.15.27:0/2854389161 >> 172.21.15.27:0/2854389161 conn(0x7fcf04077a20 msgr2=0x7fcf04069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 -- 172.21.15.27:0/2854389161 shutdown_connections 2024-06-06T08:45:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:35.371+0000 7fcf0b01c640 1 -- 172.21.15.27:0/2854389161 wait complete. 2024-06-06T08:45:35.528 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:35 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:45:35.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:35 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:45:36.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:35 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:45:36.695 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:36 smithi027 ceph-mon[32335]: pgmap v163: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 99 KiB/s wr, 90 op/s 2024-06-06T08:45:36.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:36 smithi203 ceph-mon[36579]: pgmap v163: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 99 KiB/s wr, 90 op/s 2024-06-06T08:45:37.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:36 smithi176 ceph-mon[36786]: pgmap v163: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 99 KiB/s wr, 90 op/s 2024-06-06T08:45:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 -- 172.21.15.27:0/3192413534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 msgr2=0x7fedac107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 --2- 172.21.15.27:0/3192413534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 0x7fedac107110 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0099c0 tx=0x7fed9c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 -- 172.21.15.27:0/3192413534 shutdown_connections 2024-06-06T08:45:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 --2- 172.21.15.27:0/3192413534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedac107840 0x7fedac10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 --2- 172.21.15.27:0/3192413534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac102740 0x7fedac102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 --2- 172.21.15.27:0/3192413534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 0x7fedac107110 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 -- 172.21.15.27:0/3192413534 >> 172.21.15.27:0/3192413534 conn(0x7fedac0795f0 msgr2=0x7fedac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 -- 172.21.15.27:0/3192413534 shutdown_connections 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 -- 172.21.15.27:0/3192413534 wait complete. 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 Processor -- start 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.241+0000 7fedb347a700 1 -- start start 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb347a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedac102740 0x7fedac074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb347a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 0x7fedac074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb347a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac107840 0x7fedac075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb347a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fedac112770 con 0x7fedac1030f0 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb347a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fedac1125f0 con 0x7fedac107840 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb347a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fedac1128f0 con 0x7fedac102740 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb0a15700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 0x7fedac074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb0a15700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 0x7fedac074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33356/0 (socket says 172.21.15.27:33356) 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb0a15700 1 -- 172.21.15.27:0/2847478592 learned_addr learned my addr 172.21.15.27:0/2847478592 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb1216700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedac102740 0x7fedac074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb1a17700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac107840 0x7fedac075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb0a15700 1 -- 172.21.15.27:0/2847478592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedac102740 msgr2=0x7fedac074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb0a15700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedac102740 0x7fedac074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb0a15700 1 -- 172.21.15.27:0/2847478592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac107840 msgr2=0x7fedac075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb0a15700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac107840 0x7fedac075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb0a15700 1 -- 172.21.15.27:0/2847478592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac1a70d0 con 0x7fedac1030f0 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb1a17700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac107840 0x7fedac075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.242+0000 7fedb0a15700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 0x7fedac074cd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0059d0 tx=0x7fed9c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.243+0000 7feda27fc700 1 -- 172.21.15.27:0/2847478592 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c027070 con 0x7fedac1030f0 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.243+0000 7feda27fc700 1 -- 172.21.15.27:0/2847478592 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9c004050 con 0x7fedac1030f0 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.243+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedac1a7360 con 0x7fedac1030f0 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.243+0000 7feda27fc700 1 -- 172.21.15.27:0/2847478592 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c022db0 con 0x7fedac1030f0 2024-06-06T08:45:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.243+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedac1a7810 con 0x7fedac1030f0 2024-06-06T08:45:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.244+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedac069180 con 0x7fedac1030f0 2024-06-06T08:45:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.245+0000 7feda27fc700 1 -- 172.21.15.27:0/2847478592 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fed9c02a070 con 0x7fedac1030f0 2024-06-06T08:45:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.245+0000 7feda27fc700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed980a2e10 0x7fed980a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.246+0000 7fedb1216700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed980a2e10 0x7fed980a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.246+0000 7feda27fc700 1 -- 172.21.15.27:0/2847478592 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fed9c0d0af0 con 0x7fedac1030f0 2024-06-06T08:45:37.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.246+0000 7fedb1216700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed980a2e10 0x7fed980a52d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feda8005950 tx=0x7feda800b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.249+0000 7feda27fc700 1 -- 172.21.15.27:0/2847478592 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed9c031020 con 0x7fedac1030f0 2024-06-06T08:45:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.508+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedac02dca0 con 0x7fedac1030f0 2024-06-06T08:45:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.509+0000 7feda27fc700 1 -- 172.21.15.27:0/2847478592 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11896 (secure 0 0 0) 0x7fed9c013070 con 0x7fedac1030f0 2024-06-06T08:45:37.508 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:45:37.508 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:43:25.179611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:45:37.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed980a2e10 msgr2=0x7fed980a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:37.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed980a2e10 0x7fed980a52d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feda8005950 tx=0x7feda800b410 comp rx=0 tx=0).stop 2024-06-06T08:45:37.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 msgr2=0x7fedac074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:37.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 0x7fedac074cd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0059d0 tx=0x7fed9c0038b0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 shutdown_connections 2024-06-06T08:45:37.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fed980a2e10 0x7fed980a52d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac107840 0x7fedac075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedac102740 0x7fedac074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 --2- 172.21.15.27:0/2847478592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fedac1030f0 0x7fedac074cd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:37.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 >> 172.21.15.27:0/2847478592 conn(0x7fedac0795f0 msgr2=0x7fedac101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:37.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.511+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 shutdown_connections 2024-06-06T08:45:37.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:37.512+0000 7fedb347a700 1 -- 172.21.15.27:0/2847478592 wait complete. 2024-06-06T08:45:37.511 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-06-06T08:45:37.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:37 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:45:37.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:37 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:45:37.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:37 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:45:37.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:37 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2847478592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:37.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:37 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:45:37.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:37 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:45:37.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:37 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:45:37.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:37 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2847478592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:38.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:37 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:45:38.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:37 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:45:38.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:37 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:45:38.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:37 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2847478592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:38.333 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id db363435 --release --await' 2024-06-06T08:45:38.334 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --set-id db363435 --release --await 2024-06-06T08:45:38.632 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:45:39.068 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:38 smithi027 ceph-mon[32335]: pgmap v164: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-06T08:45:39.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:38 smithi203 ceph-mon[36579]: pgmap v164: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-06T08:45:39.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:38 smithi176 ceph-mon[36786]: pgmap v164: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-06T08:45:40.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 -- 172.21.15.27:0/33694052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 msgr2=0x7f2658101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 --2- 172.21.15.27:0/33694052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 0x7f2658101a00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f26540099c0 tx=0x7f2654009cd0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 -- 172.21.15.27:0/33694052 shutdown_connections 2024-06-06T08:45:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 --2- 172.21.15.27:0/33694052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f26581066b0 0x7f265810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 --2- 172.21.15.27:0/33694052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 0x7f2658101a00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 --2- 172.21.15.27:0/33694052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2658101fd0 0x7f2658105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 -- 172.21.15.27:0/33694052 >> 172.21.15.27:0/33694052 conn(0x7f2658078340 msgr2=0x7f2658078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 -- 172.21.15.27:0/33694052 shutdown_connections 2024-06-06T08:45:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 -- 172.21.15.27:0/33694052 wait complete. 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.421+0000 7f265fb68700 1 Processor -- start 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265fb68700 1 -- start start 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265fb68700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 0x7f2658199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265fb68700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2658101fd0 0x7f265819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265fb68700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26581066b0 0x7f26581a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265fb68700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2658111790 con 0x7f2658101620 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265fb68700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2658111610 con 0x7f2658101fd0 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265fb68700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2658111910 con 0x7f26581066b0 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265d904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 0x7f2658199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265d904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 0x7f2658199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51606/0 (socket says 172.21.15.27:51606) 2024-06-06T08:45:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265d904700 1 -- 172.21.15.27:0/2661783634 learned_addr learned my addr 172.21.15.27:0/2661783634 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265d103700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2658101fd0 0x7f265819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265e105700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26581066b0 0x7f26581a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265d904700 1 -- 172.21.15.27:0/2661783634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26581066b0 msgr2=0x7f26581a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265d904700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26581066b0 0x7f26581a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265d904700 1 -- 172.21.15.27:0/2661783634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2658101fd0 msgr2=0x7f265819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265d904700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2658101fd0 0x7f265819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.422+0000 7f265d904700 1 -- 172.21.15.27:0/2661783634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26581a51f0 con 0x7f2658101620 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.423+0000 7f265d904700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 0x7f2658199c30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2654009fd0 tx=0x7f2654003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.423+0000 7f264effd700 1 -- 172.21.15.27:0/2661783634 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654027070 con 0x7f2658101620 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.423+0000 7f265fb68700 1 -- 172.21.15.27:0/2661783634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26581a53c0 con 0x7f2658101620 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.423+0000 7f264effd700 1 -- 172.21.15.27:0/2661783634 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26540040d0 con 0x7f2658101620 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.423+0000 7f265fb68700 1 -- 172.21.15.27:0/2661783634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581a5870 con 0x7f2658101620 2024-06-06T08:45:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.423+0000 7f264effd700 1 -- 172.21.15.27:0/2661783634 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265400f460 con 0x7f2658101620 2024-06-06T08:45:40.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.425+0000 7f264effd700 1 -- 172.21.15.27:0/2661783634 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f265402a070 con 0x7f2658101620 2024-06-06T08:45:40.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.426+0000 7f264effd700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26440a2e10 0x7f26440a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.426+0000 7f264effd700 1 -- 172.21.15.27:0/2661783634 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f26540d06a0 con 0x7f2658101620 2024-06-06T08:45:40.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.426+0000 7f265d103700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26440a2e10 0x7f26440a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.426+0000 7f265d103700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26440a2e10 0x7f26440a52d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26581010c0 tx=0x7f2648009190 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:40.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.426+0000 7f266a3ebb80 1 Processor -- start 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f266a3ebb80 1 -- start start 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc4423c560 0x55bc44235e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc442363d0 0x55bc44236810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bc44187910 con 0x55bc4423c560 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bc4412f820 con 0x55bc4423bc40 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bc441838d0 con 0x55bc442363d0 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f265d103700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc4423c560 0x55bc44235e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f265d103700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc4423c560 0x55bc44235e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51608/0 (socket says 172.21.15.27:51608) 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f265d103700 1 -- 172.21.15.27:0/1889601789 learned_addr learned my addr 172.21.15.27:0/1889601789 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f265d904700 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f265e105700 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc442363d0 0x55bc44236810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f265d103700 1 -- 172.21.15.27:0/1889601789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc442363d0 msgr2=0x55bc44236810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f265d103700 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc442363d0 0x55bc44236810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f265d103700 1 -- 172.21.15.27:0/1889601789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 msgr2=0x55bc4423c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.427+0000 7f265d103700 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.428+0000 7f265d103700 1 -- 172.21.15.27:0/1889601789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc44126050 con 0x55bc4423c560 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.428+0000 7f265d904700 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.428+0000 7f265d103700 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc4423c560 0x55bc44235e90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2648009fd0 tx=0x7f2648011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c00f3c5f5061dcb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.428+0000 7f2643fff700 1 -- 172.21.15.27:0/1889601789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264801b040 con 0x55bc4423c560 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.428+0000 7f2643fff700 1 -- 172.21.15.27:0/1889601789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2648024070 con 0x55bc4423c560 2024-06-06T08:45:40.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.428+0000 7f2643fff700 1 -- 172.21.15.27:0/1889601789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480136b0 con 0x55bc4423c560 2024-06-06T08:45:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 -- 172.21.15.27:0/1889601789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc4423c560 msgr2=0x55bc44235e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc4423c560 0x55bc44235e90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2648009fd0 tx=0x7f2648011000 comp rx=0 tx=0).stop 2024-06-06T08:45:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 -- 172.21.15.27:0/1889601789 shutdown_connections 2024-06-06T08:45:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc442363d0 0x55bc44236810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/1889601789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc4423c560 0x55bc44235e90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 -- 172.21.15.27:0/1889601789 >> 172.21.15.27:0/1889601789 conn(0x55bc44160dc0 msgr2=0x55bc44231120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 -- 172.21.15.27:0/1889601789 shutdown_connections 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 -- 172.21.15.27:0/1889601789 wait complete. 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 -- start start 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.429+0000 7f266a3ebb80 1 Processor -- start 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44242240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 0x55bc4423fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bc44242fd0 con 0x55bc442363d0 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x55bc44127cf0 con 0x55bc4423bc40 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55bc44243150 con 0x55bc4423c560 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44242240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f266a3ebb80 10 client.0 _ll_register_callbacks cb 0x55bc44121a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44242240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51614/0 (socket says 172.21.15.27:51614) 2024-06-06T08:45:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d904700 1 -- 172.21.15.27:0/2809808286 learned_addr learned my addr 172.21.15.27:0/2809808286 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265e105700 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 0x55bc4423fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d103700 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d904700 1 -- 172.21.15.27:0/2809808286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 msgr2=0x55bc4423fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d904700 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 0x55bc4423fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d904700 1 -- 172.21.15.27:0/2809808286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 msgr2=0x55bc4423f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d904700 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d904700 1 -- 172.21.15.27:0/2809808286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc440d69d0 con 0x55bc442363d0 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d103700 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f265d904700 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44242240 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2654098cf0 tx=0x7f265409d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ae034427fa1809 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f26417fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.430+0000 7f26417fa700 1 -- 172.21.15.27:0/2809808286 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26540b5060 con 0x55bc442363d0 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f26417fa700 1 -- 172.21.15.27:0/2809808286 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f26540d0280 con 0x55bc442363d0 2024-06-06T08:45:40.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f266a3ebb80 1 -- 172.21.15.27:0/2809808286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2648024070 con 0x55bc442363d0 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f26417fa700 1 -- 172.21.15.27:0/2809808286 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f26540a0c40 con 0x55bc442363d0 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f266a3ebb80 10 client.16575 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f266a3ebb80 1 -- 172.21.15.27:0/2809808286 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bc4424d8a0 con 0x55bc442363d0 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f26417fa700 1 -- 172.21.15.27:0/2809808286 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2648024070 con 0x55bc442363d0 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f26417fa700 1 -- 172.21.15.27:0/2809808286 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f26540a4fe0 con 0x55bc442363d0 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f266a3ebb80 10 client.16575 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f266a3ebb80 10 client.16575 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f266a3ebb80 20 client.16575 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bc4424fe40 0x55bc442522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f266a3ebb80 4 client.16575 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.431+0000 7f266a3ebb80 1 -- 172.21.15.27:0/2809808286 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc44194e10 con 0x55bc4424fe40 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.432+0000 7f265d103700 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bc4424fe40 0x55bc442522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.432+0000 7f265d103700 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bc4424fe40 0x55bc442522a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfecb5a6db76e43 server_cookie=b0cc8aedc63ab557 in_seq=0 out_seq=0 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.432+0000 7f26417fa700 10 client.16575 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f26417fa700 1 -- 172.21.15.27:0/2809808286 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bc44194e10 con 0x55bc4424fe40 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f26417fa700 10 client.16575 handle_command_reply: tid=0 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 client.16575 shutdown 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 -- 172.21.15.27:0/2809808286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 msgr2=0x55bc44242240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44242240 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2654098cf0 tx=0x7f265409d010 comp rx=0 tx=0).stop 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 -- 172.21.15.27:0/2809808286 shutdown_connections 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 0x55bc4423fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bc4424fe40 0x55bc442522a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44242240 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/2809808286 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc4423f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f26417fa700 0 client.16575 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 -- 172.21.15.27:0/2809808286 >> 172.21.15.27:0/2809808286 conn(0x55bc44160dc0 msgr2=0x55bc44230d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 -- 172.21.15.27:0/2809808286 shutdown_connections 2024-06-06T08:45:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 1 -- 172.21.15.27:0/2809808286 wait complete. 2024-06-06T08:45:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.433+0000 7f266a3ebb80 20 client.16575 trim_cache size 0 max 16384 2024-06-06T08:45:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.453+0000 7f266a3ebb80 1 Processor -- start 2024-06-06T08:45:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f266a3ebb80 1 -- start start 2024-06-06T08:45:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44235e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423bc40 0x55bc4423fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423c560 0x55bc44240160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f263c011680 con 0x55bc442363d0 2024-06-06T08:45:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f263c00ee50 con 0x55bc4423c560 2024-06-06T08:45:40.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f263c00cca0 con 0x55bc4423bc40 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44235e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44235e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51626/0 (socket says 172.21.15.27:51626) 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d904700 1 -- 172.21.15.27:0/3717940484 learned_addr learned my addr 172.21.15.27:0/3717940484 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d103700 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423bc40 0x55bc4423fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d904700 1 -- 172.21.15.27:0/3717940484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423bc40 msgr2=0x55bc4423fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265e105700 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423c560 0x55bc44240160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d904700 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423bc40 0x55bc4423fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d904700 1 -- 172.21.15.27:0/3717940484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423c560 msgr2=0x55bc44240160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d904700 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423c560 0x55bc44240160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d904700 1 -- 172.21.15.27:0/3717940484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc44252920 con 0x55bc442363d0 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265e105700 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423c560 0x55bc44240160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.454+0000 7f265d904700 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44235e90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f265409ab40 tx=0x7f265409c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d5c7530cce7eb8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:40.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f26437fe700 1 -- 172.21.15.27:0/3717940484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26540a30b0 con 0x55bc442363d0 2024-06-06T08:45:40.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f26437fe700 1 -- 172.21.15.27:0/3717940484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26540cf030 con 0x55bc442363d0 2024-06-06T08:45:40.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f26437fe700 1 -- 172.21.15.27:0/3717940484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265409cd50 con 0x55bc442363d0 2024-06-06T08:45:40.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f266a3ebb80 1 -- 172.21.15.27:0/3717940484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 msgr2=0x55bc44235e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:40.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44235e90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f265409ab40 tx=0x7f265409c500 comp rx=0 tx=0).stop 2024-06-06T08:45:40.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f266a3ebb80 1 -- 172.21.15.27:0/3717940484 shutdown_connections 2024-06-06T08:45:40.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423c560 0x55bc44240160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44235e90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/3717940484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423bc40 0x55bc4423fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f266a3ebb80 1 -- 172.21.15.27:0/3717940484 >> 172.21.15.27:0/3717940484 conn(0x55bc44160dc0 msgr2=0x55bc4423c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f266a3ebb80 1 -- 172.21.15.27:0/3717940484 shutdown_connections 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.455+0000 7f266a3ebb80 1 -- 172.21.15.27:0/3717940484 wait complete. 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f266a3ebb80 1 -- start start 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f266a3ebb80 1 Processor -- start 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44258160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc44256ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f266a3ebb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 0x55bc442570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f263c00d310 con 0x55bc442363d0 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f263c00e780 con 0x55bc4423bc40 2024-06-06T08:45:40.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f266a3ebb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f263c00d9e0 con 0x55bc4423c560 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f265d904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44258160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f265d904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44258160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51640/0 (socket says 172.21.15.27:51640) 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f265d904700 1 -- 172.21.15.27:0/54847870 learned_addr learned my addr 172.21.15.27:0/54847870 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f266a3ebb80 10 client.0 _ll_register_callbacks cb 0x55bc44121a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f265d103700 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc44256ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.456+0000 7f265e105700 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 0x55bc442570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f265d904700 1 -- 172.21.15.27:0/54847870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 msgr2=0x55bc442570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f265d904700 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 0x55bc442570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f265d904700 1 -- 172.21.15.27:0/54847870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 msgr2=0x55bc44256ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f265d904700 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc44256ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f265d904700 1 -- 172.21.15.27:0/54847870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc44257720 con 0x55bc442363d0 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f265d103700 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc44256ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:40.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f265d904700 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44258160 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f26540cdae0 tx=0x7f265409d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7251b4ff5b3ecddd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f26417fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f26417fa700 1 -- 172.21.15.27:0/54847870 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265409cd00 con 0x55bc442363d0 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f26417fa700 1 -- 172.21.15.27:0/54847870 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f265409bce0 con 0x55bc442363d0 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f266a3ebb80 1 -- 172.21.15.27:0/54847870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26540cf030 con 0x55bc442363d0 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f26417fa700 1 -- 172.21.15.27:0/54847870 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f265409bee0 con 0x55bc442363d0 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f266a3ebb80 10 client.16581 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f266a3ebb80 1 -- 172.21.15.27:0/54847870 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bc44257720 con 0x55bc442363d0 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.457+0000 7f26417fa700 1 -- 172.21.15.27:0/54847870 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26540cf030 con 0x55bc442363d0 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f26417fa700 1 -- 172.21.15.27:0/54847870 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f26540985e0 con 0x55bc442363d0 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f266a3ebb80 10 client.16581 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f266a3ebb80 10 client.16581 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f266a3ebb80 20 client.16581 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:40.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bc442bfbd0 0x55bc442c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:40.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f266a3ebb80 4 client.16581 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "db363435", "release": true, "await": true}] 2024-06-06T08:45:40.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f266a3ebb80 1 -- 172.21.15.27:0/54847870 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "quiesce db", "set_id": "db363435", "release": true, "await": true}) -- 0x55bc4418f390 con 0x55bc442bfbd0 2024-06-06T08:45:40.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f265d103700 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bc442bfbd0 0x55bc442c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:40.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f265d103700 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bc442bfbd0 0x55bc442c1ff0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7048bd89ae2ee97 server_cookie=a3517a80e73fdf5b in_seq=0 out_seq=0 2024-06-06T08:45:40.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.458+0000 7f26417fa700 10 client.16581 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:40.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.568+0000 7f26417fa700 1 -- 172.21.15.27:0/54847870 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bc4418f390 con 0x55bc442bfbd0 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.568+0000 7f26417fa700 10 client.16581 handle_command_reply: tid=0 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.568+0000 7f266a3ebb80 1 client.16581 shutdown 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.568+0000 7f266a3ebb80 1 -- 172.21.15.27:0/54847870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 msgr2=0x55bc44258160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.568+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44258160 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f26540cdae0 tx=0x7f265409d010 comp rx=0 tx=0).stop 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f266a3ebb80 1 -- 172.21.15.27:0/54847870 shutdown_connections 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc4423c560 0x55bc442570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x55bc442bfbd0 0x55bc442c1ff0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bc442363d0 0x55bc44258160 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f266a3ebb80 1 --2- 172.21.15.27:0/54847870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bc4423bc40 0x55bc44256ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f26417fa700 0 client.16581 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f266a3ebb80 1 -- 172.21.15.27:0/54847870 >> 172.21.15.27:0/54847870 conn(0x55bc44160dc0 msgr2=0x55bc4423a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f266a3ebb80 1 -- 172.21.15.27:0/54847870 shutdown_connections 2024-06-06T08:45:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f266a3ebb80 1 -- 172.21.15.27:0/54847870 wait complete. 2024-06-06T08:45:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f266a3ebb80 20 client.16581 trim_cache size 0 max 16384 2024-06-06T08:45:40.569 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:45:40.569 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-06-06T08:45:40.569 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24482, 2024-06-06T08:45:40.569 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 16, 2024-06-06T08:45:40.569 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-06-06T08:45:40.569 INFO:teuthology.orchestra.run.smithi027.stdout: "db363435": { 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 16, 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 145.2, 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-06-06T08:45:40.570 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-06-06T08:45:40.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:40.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:40.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:40.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:40.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:45:40.571 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:45:40.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 -- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26440a2e10 msgr2=0x7f26440a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26440a2e10 0x7f26440a52d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26581010c0 tx=0x7f2648009190 comp rx=0 tx=0).stop 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 -- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 msgr2=0x7f2658199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 0x7f2658199c30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2654009fd0 tx=0x7f2654003680 comp rx=0 tx=0).stop 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 -- 172.21.15.27:0/2661783634 shutdown_connections 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26581066b0 0x7f26581a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2658101620 0x7f2658199c30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f26440a2e10 0x7f26440a52d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 --2- 172.21.15.27:0/2661783634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2658101fd0 0x7f265819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.569+0000 7f265fb68700 1 -- 172.21.15.27:0/2661783634 >> 172.21.15.27:0/2661783634 conn(0x7f2658078340 msgr2=0x7f2658105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.570+0000 7f265fb68700 1 -- 172.21.15.27:0/2661783634 shutdown_connections 2024-06-06T08:45:40.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:40.570+0000 7f265fb68700 1 -- 172.21.15.27:0/2661783634 wait complete. 2024-06-06T08:45:41.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:40 smithi027 ceph-mon[32335]: pgmap v165: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-06-06T08:45:41.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:40 smithi203 ceph-mon[36579]: pgmap v165: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-06-06T08:45:41.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:40 smithi176 ceph-mon[36786]: pgmap v165: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-06-06T08:45:41.377 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'db363435', total seconds elapsed: 31.7 2024-06-06T08:45:43.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:42 smithi027 ceph-mon[32335]: pgmap v166: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-06-06T08:45:43.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:42 smithi203 ceph-mon[36579]: pgmap v166: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-06-06T08:45:43.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:42 smithi176 ceph-mon[36786]: pgmap v166: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-06-06T08:45:45.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:44 smithi027 ceph-mon[32335]: pgmap v167: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 27 op/s 2024-06-06T08:45:45.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:44 smithi203 ceph-mon[36579]: pgmap v167: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 27 op/s 2024-06-06T08:45:45.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:44 smithi176 ceph-mon[36786]: pgmap v167: 129 pgs: 129 active+clean; 1.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 27 op/s 2024-06-06T08:45:47.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:46 smithi027 ceph-mon[32335]: pgmap v168: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 387 KiB/s wr, 101 op/s 2024-06-06T08:45:47.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:46 smithi203 ceph-mon[36579]: pgmap v168: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 387 KiB/s wr, 101 op/s 2024-06-06T08:45:47.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:46 smithi176 ceph-mon[36786]: pgmap v168: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 387 KiB/s wr, 101 op/s 2024-06-06T08:45:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.463+0000 7fea79d16640 1 -- 172.21.15.27:0/3532662887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea7410bf70 msgr2=0x7fea7410e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 --2- 172.21.15.27:0/3532662887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea7410bf70 0x7fea7410e410 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009ca0 tx=0x7fea6c02a9e0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 -- 172.21.15.27:0/3532662887 shutdown_connections 2024-06-06T08:45:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 --2- 172.21.15.27:0/3532662887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea7410bf70 0x7fea7410e410 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 --2- 172.21.15.27:0/3532662887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea74103550 0x7fea7410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 --2- 172.21.15.27:0/3532662887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea74102c30 0x7fea74103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 -- 172.21.15.27:0/3532662887 >> 172.21.15.27:0/3532662887 conn(0x7fea74077a20 msgr2=0x7fea74077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 -- 172.21.15.27:0/3532662887 shutdown_connections 2024-06-06T08:45:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 -- 172.21.15.27:0/3532662887 wait complete. 2024-06-06T08:45:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 Processor -- start 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.464+0000 7fea79d16640 1 -- start start 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea79d16640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea74102c30 0x7fea741981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea79d16640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea74103550 0x7fea74198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea79d16640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7410bf70 0x7fea7419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea79d16640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea74111d70 con 0x7fea74103550 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea79d16640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fea74111bf0 con 0x7fea7410bf70 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea79d16640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea74111ef0 con 0x7fea74102c30 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea72ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea74103550 0x7fea74198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea72ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea74103550 0x7fea74198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51660/0 (socket says 172.21.15.27:51660) 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea72ffd640 1 -- 172.21.15.27:0/3906025164 learned_addr learned my addr 172.21.15.27:0/3906025164 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea73fff640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7410bf70 0x7fea7419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea737fe640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea74102c30 0x7fea741981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea72ffd640 1 -- 172.21.15.27:0/3906025164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea74102c30 msgr2=0x7fea741981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea72ffd640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea74102c30 0x7fea741981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea72ffd640 1 -- 172.21.15.27:0/3906025164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7410bf70 msgr2=0x7fea7419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea72ffd640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7410bf70 0x7fea7419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea72ffd640 1 -- 172.21.15.27:0/3906025164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7419d170 con 0x7fea74103550 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea73fff640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7410bf70 0x7fea7419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea72ffd640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea74103550 0x7fea74198710 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e9f0 tx=0x7fea6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea70ff9640 1 -- 172.21.15.27:0/3906025164 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6800cd80 con 0x7fea74103550 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea79d16640 1 -- 172.21.15.27:0/3906025164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7419d460 con 0x7fea74103550 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea70ff9640 1 -- 172.21.15.27:0/3906025164 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea68010910 con 0x7fea74103550 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.465+0000 7fea70ff9640 1 -- 172.21.15.27:0/3906025164 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea680188f0 con 0x7fea74103550 2024-06-06T08:45:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.466+0000 7fea79d16640 1 -- 172.21.15.27:0/3906025164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea741a4cf0 con 0x7fea74103550 2024-06-06T08:45:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.468+0000 7fea70ff9640 1 -- 172.21.15.27:0/3906025164 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fea68010ac0 con 0x7fea74103550 2024-06-06T08:45:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.468+0000 7fea7ca19740 1 Processor -- start 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.468+0000 7fea7ca19740 1 -- start start 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.468+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aace170 0x556d0aace550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.468+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aad8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.468+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aad9350 0x556d0aadb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea7ca19740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556d0aa324f0 con 0x556d0aace170 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea7ca19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556d0a9d0870 con 0x556d0aacea90 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea7ca19740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556d0aa2e4b0 con 0x556d0aad9350 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aace170 0x556d0aace550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aace170 0x556d0aace550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51674/0 (socket says 172.21.15.27:51674) 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 -- 172.21.15.27:0/3172105460 learned_addr learned my addr 172.21.15.27:0/3172105460 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea70ff9640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fea5c0a2da0 0x7fea5c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea737fe640 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aad8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea72ffd640 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aad9350 0x556d0aadb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fea5c0a2da0 0x7fea5c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 -- 172.21.15.27:0/3172105460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aad9350 msgr2=0x556d0aadb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aad9350 0x556d0aadb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea70ff9640 1 -- 172.21.15.27:0/3906025164 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fea680cfc00 con 0x7fea74103550 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 -- 172.21.15.27:0/3172105460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 msgr2=0x556d0aad8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aad8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 -- 172.21.15.27:0/3172105460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d0a9ec9b0 con 0x556d0aace170 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea737fe640 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aad8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aace170 0x556d0aace550 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009ca0 tx=0x7fea6c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708de46924e77d0c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea59ffb640 1 -- 172.21.15.27:0/3172105460 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c04b030 con 0x556d0aace170 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea59ffb640 1 -- 172.21.15.27:0/3172105460 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6c02e8e0 con 0x556d0aace170 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea59ffb640 1 -- 172.21.15.27:0/3172105460 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c033050 con 0x556d0aace170 2024-06-06T08:45:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.469+0000 7fea73fff640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fea5c0a2da0 0x7fea5c0a5260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea6c04e000 tx=0x7fea6c04c000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.470+0000 7fea7ca19740 1 -- 172.21.15.27:0/3172105460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aace170 msgr2=0x556d0aace550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.470+0000 7fea7ca19740 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aace170 0x556d0aace550 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009ca0 tx=0x7fea6c003a70 comp rx=0 tx=0).stop 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.470+0000 7fea7ca19740 1 -- 172.21.15.27:0/3172105460 shutdown_connections 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.470+0000 7fea7ca19740 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aad9350 0x556d0aadb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.470+0000 7fea7ca19740 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aad8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.470+0000 7fea7ca19740 1 --2- 172.21.15.27:0/3172105460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aace170 0x556d0aace550 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.470+0000 7fea7ca19740 1 -- 172.21.15.27:0/3172105460 >> 172.21.15.27:0/3172105460 conn(0x556d0aaca680 msgr2=0x556d0aacaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.470+0000 7fea7ca19740 1 -- 172.21.15.27:0/3172105460 shutdown_connections 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.470+0000 7fea7ca19740 1 -- 172.21.15.27:0/3172105460 wait complete. 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea7ca19740 1 -- start start 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea7ca19740 1 Processor -- start 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aace170 0x556d0aad62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aacea90 0x556d0aad67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aad6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea7ca19740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556d0aadbf80 con 0x556d0aad9350 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea7ca19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556d0a9bbae0 con 0x556d0aace170 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea7ca19740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556d0aadc100 con 0x556d0aacea90 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea7ca19740 10 client.0 _ll_register_callbacks cb 0x556d0a9f49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aad6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aad6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51688/0 (socket says 172.21.15.27:51688) 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea73fff640 1 -- 172.21.15.27:0/1778686710 learned_addr learned my addr 172.21.15.27:0/1778686710 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea737fe640 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aace170 0x556d0aad62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.471+0000 7fea72ffd640 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aacea90 0x556d0aad67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea73fff640 1 -- 172.21.15.27:0/1778686710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aacea90 msgr2=0x556d0aad67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea73fff640 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aacea90 0x556d0aad67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea73fff640 1 -- 172.21.15.27:0/1778686710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aace170 msgr2=0x556d0aad62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea73fff640 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aace170 0x556d0aad62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea73fff640 1 -- 172.21.15.27:0/1778686710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d0a746210 con 0x556d0aad9350 2024-06-06T08:45:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea737fe640 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aace170 0x556d0aad62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:45:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea73fff640 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aad6d20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0093a0 tx=0x7fea6c003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78880ff877828b36 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea52ffd640 1 -- 172.21.15.27:0/1778686710 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c05f060 con 0x556d0aad9350 2024-06-06T08:45:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea52ffd640 1 -- 172.21.15.27:0/1778686710 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fea6c0044c0 con 0x556d0aad9350 2024-06-06T08:45:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea7ca19740 1 -- 172.21.15.27:0/1778686710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea6c02e8e0 con 0x556d0aad9350 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea52ffd640 1 -- 172.21.15.27:0/1778686710 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea6c04b050 con 0x556d0aad9350 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea7ca19740 10 client.16593 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.472+0000 7fea7ca19740 1 -- 172.21.15.27:0/1778686710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556d0a812620 con 0x556d0aad9350 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea52ffd640 1 -- 172.21.15.27:0/1778686710 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6c02e8e0 con 0x556d0aad9350 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea52ffd640 1 -- 172.21.15.27:0/1778686710 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fea6c036930 con 0x556d0aad9350 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea7ca19740 10 client.16593 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea7ca19740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea7ca19740 20 client.16593 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea7ca19740 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556d0aae9310 0x556d0aaeb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea7ca19740 4 client.16593 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea7ca19740 1 -- 172.21.15.27:0/1778686710 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d0aa45f10 con 0x556d0aae9310 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea737fe640 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556d0aae9310 0x556d0aaeb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea737fe640 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556d0aae9310 0x556d0aaeb700 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf74ddd6c5508a70 server_cookie=2a5427344c639fbf in_seq=0 out_seq=0 2024-06-06T08:45:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.473+0000 7fea52ffd640 10 client.16593 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.474+0000 7fea52ffd640 1 -- 172.21.15.27:0/1778686710 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556d0aa45f10 con 0x556d0aae9310 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.474+0000 7fea52ffd640 10 client.16593 handle_command_reply: tid=0 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.474+0000 7fea7ca19740 1 client.16593 shutdown 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.474+0000 7fea7ca19740 1 -- 172.21.15.27:0/1778686710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 msgr2=0x556d0aad6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aad6d20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0093a0 tx=0x7fea6c003de0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 1 -- 172.21.15.27:0/1778686710 shutdown_connections 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556d0aae9310 0x556d0aaeb700 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aad6d20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aacea90 0x556d0aad67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 1 --2- 172.21.15.27:0/1778686710 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aace170 0x556d0aad62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea52ffd640 0 client.16593 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 1 -- 172.21.15.27:0/1778686710 >> 172.21.15.27:0/1778686710 conn(0x556d0aaca680 msgr2=0x556d0aad0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 1 -- 172.21.15.27:0/1778686710 shutdown_connections 2024-06-06T08:45:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 1 -- 172.21.15.27:0/1778686710 wait complete. 2024-06-06T08:45:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.475+0000 7fea7ca19740 20 client.16593 trim_cache size 0 max 16384 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.490+0000 7fea7ca19740 1 Processor -- start 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.490+0000 7fea7ca19740 1 -- start start 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.490+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 0x556d0aad0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.490+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aad81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.490+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aaf0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.490+0000 7fea7ca19740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea741b1750 con 0x556d0aad9350 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.490+0000 7fea7ca19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fea741aef60 con 0x556d0aacea90 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.490+0000 7fea7ca19740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea741acdb0 con 0x556d0aace170 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aaf0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aaf0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51696/0 (socket says 172.21.15.27:51696) 2024-06-06T08:45:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea73fff640 1 -- 172.21.15.27:0/2276207473 learned_addr learned my addr 172.21.15.27:0/2276207473 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea72ffd640 1 --2- 172.21.15.27:0/2276207473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aad81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea737fe640 1 --2- 172.21.15.27:0/2276207473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 0x556d0aad0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea73fff640 1 -- 172.21.15.27:0/2276207473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 msgr2=0x556d0aad0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea73fff640 1 --2- 172.21.15.27:0/2276207473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 0x556d0aad0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea73fff640 1 -- 172.21.15.27:0/2276207473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 msgr2=0x556d0aad81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea73fff640 1 --2- 172.21.15.27:0/2276207473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aad81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea73fff640 1 -- 172.21.15.27:0/2276207473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d0aace8c0 con 0x556d0aad9350 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea73fff640 1 --2- 172.21.15.27:0/2276207473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aaf0d80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009a60 tx=0x7fea6c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee9c597efb24413 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea597fa640 1 -- 172.21.15.27:0/2276207473 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c033070 con 0x556d0aad9350 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea597fa640 1 -- 172.21.15.27:0/2276207473 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6c02e8e0 con 0x556d0aad9350 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.491+0000 7fea597fa640 1 -- 172.21.15.27:0/2276207473 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c05f040 con 0x556d0aad9350 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 -- 172.21.15.27:0/2276207473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 msgr2=0x556d0aaf0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2276207473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aaf0d80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009a60 tx=0x7fea6c003a70 comp rx=0 tx=0).stop 2024-06-06T08:45:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 -- 172.21.15.27:0/2276207473 shutdown_connections 2024-06-06T08:45:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2276207473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0aaf0d80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2276207473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aad81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2276207473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 0x556d0aad0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 -- 172.21.15.27:0/2276207473 >> 172.21.15.27:0/2276207473 conn(0x556d0aaca680 msgr2=0x556d0aad7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 -- 172.21.15.27:0/2276207473 shutdown_connections 2024-06-06T08:45:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 -- 172.21.15.27:0/2276207473 wait complete. 2024-06-06T08:45:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 -- start start 2024-06-06T08:45:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.492+0000 7fea7ca19740 1 Processor -- start 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 0x556d0aadcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aadd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea7ca19740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0ab51d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea7ca19740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea741ad420 con 0x556d0aad9350 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea7ca19740 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fea741ae890 con 0x556d0aacea90 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea7ca19740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea741adaf0 con 0x556d0aace170 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea7ca19740 10 client.0 _ll_register_callbacks cb 0x7fea64009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0ab51d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0ab51d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51702/0 (socket says 172.21.15.27:51702) 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea73fff640 1 -- 172.21.15.27:0/2844157638 learned_addr learned my addr 172.21.15.27:0/2844157638 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea72ffd640 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aadd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea737fe640 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 0x556d0aadcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea73fff640 1 -- 172.21.15.27:0/2844157638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 msgr2=0x556d0aadcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea73fff640 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 0x556d0aadcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea73fff640 1 -- 172.21.15.27:0/2844157638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 msgr2=0x556d0aadd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea73fff640 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aadd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea73fff640 1 -- 172.21.15.27:0/2844157638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d0ab52380 con 0x556d0aad9350 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea737fe640 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 0x556d0aadcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea73fff640 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0ab51d40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009ba0 tx=0x7fea6c02ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2602d1f06e3f6b4f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.493+0000 7fea52ffd640 1 -- 172.21.15.27:0/2844157638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c06e020 con 0x556d0aad9350 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea7ca19740 1 -- 172.21.15.27:0/2844157638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea6c02e8e0 con 0x556d0aad9350 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea52ffd640 1 -- 172.21.15.27:0/2844157638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7fea6c0046d0 con 0x556d0aad9350 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea52ffd640 1 -- 172.21.15.27:0/2844157638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea6c05f070 con 0x556d0aad9350 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea7ca19740 10 client.16599 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea7ca19740 1 -- 172.21.15.27:0/2844157638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556d0ab53690 con 0x556d0aad9350 2024-06-06T08:45:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea52ffd640 1 -- 172.21.15.27:0/2844157638 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6c02e8e0 con 0x556d0aad9350 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea52ffd640 1 -- 172.21.15.27:0/2844157638 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fea6c05a710 con 0x556d0aad9350 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea7ca19740 10 client.16599 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea7ca19740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea7ca19740 20 client.16599 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556d0ab53ba0 0x556d0ab55fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea7ca19740 4 client.16599 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea7ca19740 1 -- 172.21.15.27:0/2844157638 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d0aad1ea0 con 0x556d0ab53ba0 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.494+0000 7fea737fe640 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556d0ab53ba0 0x556d0ab55fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.495+0000 7fea737fe640 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556d0ab53ba0 0x556d0ab55fc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c62989aa899d9c3 server_cookie=328bb2cc98e10bc2 in_seq=0 out_seq=0 2024-06-06T08:45:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.495+0000 7fea52ffd640 10 client.16599 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.933+0000 7fea52ffd640 1 -- 172.21.15.27:0/2844157638 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556d0aad1ea0 con 0x556d0ab53ba0 2024-06-06T08:45:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.933+0000 7fea52ffd640 10 client.16599 handle_command_reply: tid=0 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 client.16599 shutdown 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 -- 172.21.15.27:0/2844157638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 msgr2=0x556d0ab51d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0ab51d40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009ba0 tx=0x7fea6c02ea90 comp rx=0 tx=0).stop 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 -- 172.21.15.27:0/2844157638 shutdown_connections 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556d0ab53ba0 0x556d0ab55fc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea52ffd640 0 client.16599 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556d0aad9350 0x556d0ab51d40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d0aacea90 0x556d0aadd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 --2- 172.21.15.27:0/2844157638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d0aace170 0x556d0aadcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 -- 172.21.15.27:0/2844157638 >> 172.21.15.27:0/2844157638 conn(0x556d0aaca680 msgr2=0x556d0aad8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 -- 172.21.15.27:0/2844157638 shutdown_connections 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 1 -- 172.21.15.27:0/2844157638 wait complete. 2024-06-06T08:45:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.934+0000 7fea7ca19740 20 client.16599 trim_cache size 0 max 16384 2024-06-06T08:45:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:{ 2024-06-06T08:45:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout: "return_code": 0 2024-06-06T08:45:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stdout:} 2024-06-06T08:45:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 -- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fea5c0a2da0 msgr2=0x7fea5c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fea5c0a2da0 0x7fea5c0a5260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fea6c04e000 tx=0x7fea6c04c000 comp rx=0 tx=0).stop 2024-06-06T08:45:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 -- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea74103550 msgr2=0x7fea74198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea74103550 0x7fea74198710 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e9f0 tx=0x7fea6800eeb0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 -- 172.21.15.27:0/3906025164 shutdown_connections 2024-06-06T08:45:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fea5c0a2da0 0x7fea5c0a5260 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7410bf70 0x7fea7419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea74103550 0x7fea74198710 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 --2- 172.21.15.27:0/3906025164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea74102c30 0x7fea741981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 -- 172.21.15.27:0/3906025164 >> 172.21.15.27:0/3906025164 conn(0x7fea74077a20 msgr2=0x7fea740fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 -- 172.21.15.27:0/3906025164 shutdown_connections 2024-06-06T08:45:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi027.front.sepia.ceph.com.smithi027.stderr:2024-06-06T08:45:48.935+0000 7fea79d16640 1 -- 172.21.15.27:0/3906025164 wait complete. 2024-06-06T08:45:49.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:48 smithi027 ceph-mon[32335]: pgmap v169: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 383 KiB/s wr, 99 op/s 2024-06-06T08:45:49.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:48 smithi203 ceph-mon[36579]: pgmap v169: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 383 KiB/s wr, 99 op/s 2024-06-06T08:45:49.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:48 smithi176 ceph-mon[36786]: pgmap v169: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 383 KiB/s wr, 99 op/s 2024-06-06T08:45:50.257 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-06-06T08:45:50.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-06T08:45:50.606 DEBUG:teuthology.parallel:result is None 2024-06-06T08:45:50.607 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-06T08:45:50.638 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-06T08:45:50.639 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-06-06T08:45:50.651 INFO:tasks.quiescer:joining Quiescers 2024-06-06T08:45:50.651 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-06-06T08:45:50.651 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:45:50.929 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:45:50.943 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:50 smithi027 ceph-mon[32335]: pgmap v170: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.0 MiB/s wr, 301 op/s 2024-06-06T08:45:51.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:50 smithi203 ceph-mon[36579]: pgmap v170: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.0 MiB/s wr, 301 op/s 2024-06-06T08:45:51.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:50 smithi176 ceph-mon[36786]: pgmap v170: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.0 MiB/s wr, 301 op/s 2024-06-06T08:45:53.053 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:52 smithi027 ceph-mon[32335]: pgmap v171: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1015 KiB/s wr, 294 op/s 2024-06-06T08:45:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.161+0000 7f4da7772700 1 -- 172.21.15.27:0/3659206544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101390 msgr2=0x7f4da0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.161+0000 7f4da7772700 1 --2- 172.21.15.27:0/3659206544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101390 0x7f4da0101810 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009990 tx=0x7f4d94009ca0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.161+0000 7f4da7772700 1 -- 172.21.15.27:0/3659206544 shutdown_connections 2024-06-06T08:45:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.161+0000 7f4da7772700 1 --2- 172.21.15.27:0/3659206544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4da0101d50 0x7f4da010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.161+0000 7f4da7772700 1 --2- 172.21.15.27:0/3659206544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da0107400 0x7f4da01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.161+0000 7f4da7772700 1 --2- 172.21.15.27:0/3659206544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101390 0x7f4da0101810 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.161+0000 7f4da7772700 1 -- 172.21.15.27:0/3659206544 >> 172.21.15.27:0/3659206544 conn(0x7f4da0078340 msgr2=0x7f4da0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:53.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.161+0000 7f4da7772700 1 -- 172.21.15.27:0/3659206544 shutdown_connections 2024-06-06T08:45:53.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.162+0000 7f4da7772700 1 -- 172.21.15.27:0/3659206544 wait complete. 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.162+0000 7f4da7772700 1 Processor -- start 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.162+0000 7f4da7772700 1 -- start start 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da7772700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4da0101390 0x7f4da019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da7772700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101d50 0x7f4da019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da7772700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da0107400 0x7f4da0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da7772700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0111690 con 0x7f4da0101d50 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da7772700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0111510 con 0x7f4da0101390 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da7772700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0111810 con 0x7f4da0107400 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da4d0d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101d50 0x7f4da019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da4d0d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101d50 0x7f4da019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53002/0 (socket says 172.21.15.27:53002) 2024-06-06T08:45:53.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da4d0d700 1 -- 172.21.15.27:0/1267080132 learned_addr learned my addr 172.21.15.27:0/1267080132 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da550e700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4da0101390 0x7f4da019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da5d0f700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da0107400 0x7f4da0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da4d0d700 1 -- 172.21.15.27:0/1267080132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da0107400 msgr2=0x7f4da0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da4d0d700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da0107400 0x7f4da0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da4d0d700 1 -- 172.21.15.27:0/1267080132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4da0101390 msgr2=0x7f4da019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da4d0d700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4da0101390 0x7f4da019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da4d0d700 1 -- 172.21.15.27:0/1267080132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da0199260 con 0x7f4da0101d50 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da5d0f700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da0107400 0x7f4da0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da4d0d700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101d50 0x7f4da019ee30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4d94000c00 tx=0x7f4d940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4d927fc700 1 -- 172.21.15.27:0/1267080132 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94027070 con 0x7f4da0101d50 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4d927fc700 1 -- 172.21.15.27:0/1267080132 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d94003780 con 0x7f4da0101d50 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da01994f0 con 0x7f4da0101d50 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.163+0000 7f4d927fc700 1 -- 172.21.15.27:0/1267080132 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94022dd0 con 0x7f4da0101d50 2024-06-06T08:45:53.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.164+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da01a57c0 con 0x7f4da0101d50 2024-06-06T08:45:53.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.164+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da0069180 con 0x7f4da0101d50 2024-06-06T08:45:53.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.166+0000 7f4d927fc700 1 -- 172.21.15.27:0/1267080132 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4d9402a070 con 0x7f4da0101d50 2024-06-06T08:45:53.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.167+0000 7f4d927fc700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d8c0a2e10 0x7f4d8c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:53.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.167+0000 7f4da550e700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d8c0a2e10 0x7f4d8c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:53.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.167+0000 7f4d927fc700 1 -- 172.21.15.27:0/1267080132 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f4d940d0850 con 0x7f4da0101d50 2024-06-06T08:45:53.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.168+0000 7f4da550e700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d8c0a2e10 0x7f4d8c0a52d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4da00ffb00 tx=0x7f4d9c00b3c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:53.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.171+0000 7f4d927fc700 1 -- 172.21.15.27:0/1267080132 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4d94013070 con 0x7f4da0101d50 2024-06-06T08:45:53.242 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:52 smithi203 ceph-mon[36579]: pgmap v171: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1015 KiB/s wr, 294 op/s 2024-06-06T08:45:53.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:52 smithi176 ceph-mon[36786]: pgmap v171: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 1015 KiB/s wr, 294 op/s 2024-06-06T08:45:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.428+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4da0199fe0 con 0x7f4da0101d50 2024-06-06T08:45:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.429+0000 7f4d927fc700 1 -- 172.21.15.27:0/1267080132 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11896 (secure 0 0 0) 0x7f4da0199fe0 con 0x7f4da0101d50 2024-06-06T08:45:53.428 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:45:53.428 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:43:25.179611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d8c0a2e10 msgr2=0x7f4d8c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d8c0a2e10 0x7f4d8c0a52d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4da00ffb00 tx=0x7f4d9c00b3c0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101d50 msgr2=0x7f4da019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101d50 0x7f4da019ee30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4d94000c00 tx=0x7f4d940047b0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 shutdown_connections 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da0107400 0x7f4da0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4d8c0a2e10 0x7f4d8c0a52d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4da0101390 0x7f4da019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 --2- 172.21.15.27:0/1267080132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4da0101d50 0x7f4da019ee30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 >> 172.21.15.27:0/1267080132 conn(0x7f4da0078340 msgr2=0x7f4da0105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:53.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 shutdown_connections 2024-06-06T08:45:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:53.431+0000 7f4da7772700 1 -- 172.21.15.27:0/1267080132 wait complete. 2024-06-06T08:45:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-06-06T08:45:54.025 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:53 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1267080132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:54.059 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --cancel --all' 2024-06-06T08:45:54.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24482 quiesce db --cancel --all 2024-06-06T08:45:54.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:53 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1267080132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:54.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:53 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1267080132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:45:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:45:55.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:54 smithi027 ceph-mon[32335]: pgmap v172: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.0 MiB/s wr, 373 op/s 2024-06-06T08:45:55.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:54 smithi203 ceph-mon[36579]: pgmap v172: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.0 MiB/s wr, 373 op/s 2024-06-06T08:45:55.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:54 smithi176 ceph-mon[36786]: pgmap v172: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.0 MiB/s wr, 373 op/s 2024-06-06T08:45:56.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 -- 172.21.15.27:0/2551416830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c102740 msgr2=0x7f4f5c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:56.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 --2- 172.21.15.27:0/2551416830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c102740 0x7f4f5c102b20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0099c0 tx=0x7f4f4c009cd0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 -- 172.21.15.27:0/2551416830 shutdown_connections 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 --2- 172.21.15.27:0/2551416830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f5c107840 0x7f4f5c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 --2- 172.21.15.27:0/2551416830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c102740 0x7f4f5c102b20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 --2- 172.21.15.27:0/2551416830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5c1030f0 0x7f4f5c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 -- 172.21.15.27:0/2551416830 >> 172.21.15.27:0/2551416830 conn(0x7f4f5c0795f0 msgr2=0x7f4f5c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 -- 172.21.15.27:0/2551416830 shutdown_connections 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 -- 172.21.15.27:0/2551416830 wait complete. 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 Processor -- start 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.093+0000 7f4f626b7700 1 -- start start 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f626b7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f5c102740 0x7f4f5c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f626b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c1030f0 0x7f4f5c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f626b7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5c107840 0x7f4f5c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f626b7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c112760 con 0x7f4f5c1030f0 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f626b7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1125e0 con 0x7f4f5c102740 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f626b7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1128e0 con 0x7f4f5c107840 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c1030f0 0x7f4f5c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c1030f0 0x7f4f5c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53012/0 (socket says 172.21.15.27:53012) 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f53fff700 1 -- 172.21.15.27:0/1329645327 learned_addr learned my addr 172.21.15.27:0/1329645327 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f5c102740 0x7f4f5c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f60c54700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5c107840 0x7f4f5c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f53fff700 1 -- 172.21.15.27:0/1329645327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5c107840 msgr2=0x7f4f5c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f53fff700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5c107840 0x7f4f5c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f53fff700 1 -- 172.21.15.27:0/1329645327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f5c102740 msgr2=0x7f4f5c19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f53fff700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f5c102740 0x7f4f5c19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f53fff700 1 -- 172.21.15.27:0/1329645327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c199040 con 0x7f4f5c1030f0 2024-06-06T08:45:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f53fff700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c1030f0 0x7f4f5c19ec60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c9b0 tx=0x7f4f4400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:56.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.094+0000 7f4f59ffb700 1 -- 172.21.15.27:0/1329645327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44007b00 con 0x7f4f5c1030f0 2024-06-06T08:45:56.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.095+0000 7f4f59ffb700 1 -- 172.21.15.27:0/1329645327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f44007ca0 con 0x7f4f5c1030f0 2024-06-06T08:45:56.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.095+0000 7f4f626b7700 1 -- 172.21.15.27:0/1329645327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5c199330 con 0x7f4f5c1030f0 2024-06-06T08:45:56.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.095+0000 7f4f59ffb700 1 -- 172.21.15.27:0/1329645327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44018730 con 0x7f4f5c1030f0 2024-06-06T08:45:56.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.095+0000 7f4f626b7700 1 -- 172.21.15.27:0/1329645327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5c1a55f0 con 0x7f4f5c1030f0 2024-06-06T08:45:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.097+0000 7f4f59ffb700 1 -- 172.21.15.27:0/1329645327 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4f440188d0 con 0x7f4f5c1030f0 2024-06-06T08:45:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.098+0000 7f4f59ffb700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4f480a2d90 0x7f4f480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.098+0000 7f4f59ffb700 1 -- 172.21.15.27:0/1329645327 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f4f440cf880 con 0x7f4f5c1030f0 2024-06-06T08:45:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.098+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4f480a2d90 0x7f4f480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.098+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4f480a2d90 0x7f4f480a5250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c000c00 tx=0x7f4f4c005fb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f6cf3ab80 1 Processor -- start 2024-06-06T08:45:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f6cf3ab80 1 -- start start 2024-06-06T08:45:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 0x556969dd3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3d90 0x556969dcd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dcdbc0 0x556969dcdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556969cbffd0 con 0x556969dd3d90 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556969d1e8c0 con 0x556969dd3470 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556969c864d0 con 0x556969dcdbc0 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3d90 0x556969dcd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3d90 0x556969dcd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53016/0 (socket says 172.21.15.27:53016) 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f53fff700 1 -- 172.21.15.27:0/375230291 learned_addr learned my addr 172.21.15.27:0/375230291 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 0x556969dd3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f60c54700 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dcdbc0 0x556969dcdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f53fff700 1 -- 172.21.15.27:0/375230291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dcdbc0 msgr2=0x556969dcdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f53fff700 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dcdbc0 0x556969dcdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f53fff700 1 -- 172.21.15.27:0/375230291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 msgr2=0x556969dd3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f53fff700 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 0x556969dd3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f53fff700 1 -- 172.21.15.27:0/375230291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556969c6d940 con 0x556969dd3d90 2024-06-06T08:45:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f60c54700 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dcdbc0 0x556969dcdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:45:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.099+0000 7f4f53fff700 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3d90 0x556969dcd680 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4f440a40c0 tx=0x7f4f44097d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cb7252944ffbc6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f527fc700 1 -- 172.21.15.27:0/375230291 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f440b3020 con 0x556969dd3d90 2024-06-06T08:45:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f527fc700 1 -- 172.21.15.27:0/375230291 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4409fd30 con 0x556969dd3d90 2024-06-06T08:45:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f527fc700 1 -- 172.21.15.27:0/375230291 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4409bbe0 con 0x556969dd3d90 2024-06-06T08:45:56.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/375230291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3d90 msgr2=0x556969dcd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:56.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3d90 0x556969dcd680 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4f440a40c0 tx=0x7f4f44097d50 comp rx=0 tx=0).stop 2024-06-06T08:45:56.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/375230291 shutdown_connections 2024-06-06T08:45:56.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dcdbc0 0x556969dcdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 0x556969dd3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/375230291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3d90 0x556969dcd680 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/375230291 >> 172.21.15.27:0/375230291 conn(0x556969cfa0f0 msgr2=0x556969dc8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.100+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/375230291 shutdown_connections 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/375230291 wait complete. 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 1 -- start start 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 1 Processor -- start 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dcdbc0 0x556969dd9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3470 0x556969dd6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 0x556969dd73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556969cbf030 con 0x556969dd3470 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x556969d1a880 con 0x556969dcdbc0 2024-06-06T08:45:56.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556969dda770 con 0x556969dd3d90 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.101+0000 7f4f6cf3ab80 10 client.0 _ll_register_callbacks cb 0x556969cb4620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3470 0x556969dd6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f53fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3470 0x556969dd6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53018/0 (socket says 172.21.15.27:53018) 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f53fff700 1 -- 172.21.15.27:0/751440116 learned_addr learned my addr 172.21.15.27:0/751440116 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f60c54700 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 0x556969dd73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dcdbc0 0x556969dd9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f53fff700 1 -- 172.21.15.27:0/751440116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 msgr2=0x556969dd73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f53fff700 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 0x556969dd73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f53fff700 1 -- 172.21.15.27:0/751440116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dcdbc0 msgr2=0x556969dd9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f53fff700 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dcdbc0 0x556969dd9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f53fff700 1 -- 172.21.15.27:0/751440116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556969ce47c0 con 0x556969dd3470 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f53fff700 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3470 0x556969dd6e80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4f4409b680 tx=0x7f4f4400fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d198b34839e553f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:56.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/751440116 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f440b3050 con 0x556969dd3470 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/751440116 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f4f440ba310 con 0x556969dd3470 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/751440116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4409fd30 con 0x556969dd3470 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/751440116 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f4409bbe0 con 0x556969dd3470 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f6cf3ab80 10 client.16617 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.102+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/751440116 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556969de4f40 con 0x556969dd3470 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.103+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/751440116 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4409fd30 con 0x556969dd3470 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.103+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/751440116 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f44018bc0 con 0x556969dd3470 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.103+0000 7f4f6cf3ab80 10 client.16617 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.103+0000 7f4f6cf3ab80 10 client.16617 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.103+0000 7f4f6cf3ab80 20 client.16617 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.103+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556969de7410 0x556969de9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.103+0000 7f4f6cf3ab80 4 client.16617 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T08:45:56.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.103+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/751440116 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556969d2bdc0 con 0x556969de7410 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.103+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556969de7410 0x556969de9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.104+0000 7f4f3b7fe700 10 client.16617 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.104+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556969de7410 0x556969de9800 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c25d0021c9ad25 server_cookie=7ba4704362fcc515 in_seq=0 out_seq=0 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.104+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/751440116 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556969d2bdc0 con 0x556969de7410 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.104+0000 7f4f3b7fe700 10 client.16617 handle_command_reply: tid=0 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.104+0000 7f4f6cf3ab80 1 client.16617 shutdown 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/751440116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3470 msgr2=0x556969dd6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3470 0x556969dd6e80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4f4409b680 tx=0x7f4f4400fd30 comp rx=0 tx=0).stop 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/751440116 shutdown_connections 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 0x556969dd73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dcdbc0 0x556969dd9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556969de7410 0x556969de9800 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/751440116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dd3470 0x556969dd6e80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/751440116 >> 172.21.15.27:0/751440116 conn(0x556969cfa0f0 msgr2=0x556969dc7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:56.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f3b7fe700 0 client.16617 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:56.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/751440116 shutdown_connections 2024-06-06T08:45:56.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/751440116 wait complete. 2024-06-06T08:45:56.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.105+0000 7f4f6cf3ab80 20 client.16617 trim_cache size 0 max 16384 2024-06-06T08:45:56.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.125+0000 7f4f6cf3ab80 1 Processor -- start 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.125+0000 7f4f6cf3ab80 1 -- start start 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dd5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 0x556969dd96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 0x556969dc82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1b0e80 con 0x556969dcdbc0 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1ae650 con 0x556969dd3470 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1ac4a0 con 0x556969dd3d90 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f5bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dd5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f5bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dd5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53030/0 (socket says 172.21.15.27:53030) 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f5bfff700 1 -- 172.21.15.27:0/545050525 learned_addr learned my addr 172.21.15.27:0/545050525 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f53fff700 1 --2- 172.21.15.27:0/545050525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 0x556969dd96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f60c54700 1 --2- 172.21.15.27:0/545050525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 0x556969dc82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f5bfff700 1 -- 172.21.15.27:0/545050525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 msgr2=0x556969dc82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:56.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/545050525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 0x556969dc82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f5bfff700 1 -- 172.21.15.27:0/545050525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 msgr2=0x556969dd96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:56.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/545050525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 0x556969dd96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f5bfff700 1 -- 172.21.15.27:0/545050525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556969dd4900 con 0x556969dcdbc0 2024-06-06T08:45:56.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.126+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/545050525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dd5de0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b580 tx=0x7f4f4c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d5f61e6ccd779d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f51ffb700 1 -- 172.21.15.27:0/545050525 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00f8e0 con 0x556969dcdbc0 2024-06-06T08:45:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f51ffb700 1 -- 172.21.15.27:0/545050525 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c033430 con 0x556969dcdbc0 2024-06-06T08:45:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f51ffb700 1 -- 172.21.15.27:0/545050525 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c024670 con 0x556969dcdbc0 2024-06-06T08:45:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/545050525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 msgr2=0x556969dd5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/545050525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dd5de0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b580 tx=0x7f4f4c023000 comp rx=0 tx=0).stop 2024-06-06T08:45:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/545050525 shutdown_connections 2024-06-06T08:45:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/545050525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3d90 0x556969dc82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/545050525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dd5de0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/545050525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3470 0x556969dd96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/545050525 >> 172.21.15.27:0/545050525 conn(0x556969cfa0f0 msgr2=0x556969dd2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/545050525 shutdown_connections 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.127+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/545050525 wait complete. 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f6cf3ab80 1 -- start start 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f6cf3ab80 1 Processor -- start 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dea850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3470 0x556969dead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f6cf3ab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3d90 0x556969e50d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1acb10 con 0x556969dcdbc0 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1adf80 con 0x556969dd3d90 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f6cf3ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1ad1e0 con 0x556969dd3470 2024-06-06T08:45:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f6cf3ab80 10 client.0 _ll_register_callbacks cb 0x556969cb4620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f5bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f5bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dea850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53044/0 (socket says 172.21.15.27:53044) 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.128+0000 7f4f5bfff700 1 -- 172.21.15.27:0/2145644954 learned_addr learned my addr 172.21.15.27:0/2145644954 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f53fff700 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3470 0x556969dead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f5bfff700 1 -- 172.21.15.27:0/2145644954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3470 msgr2=0x556969dead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3470 0x556969dead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f5bfff700 1 -- 172.21.15.27:0/2145644954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3d90 msgr2=0x556969e50d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3d90 0x556969e50d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f5bfff700 1 -- 172.21.15.27:0/2145644954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556969e51290 con 0x556969dcdbc0 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f5bfff700 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dea850 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009d10 tx=0x7f4f4c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e05b45398fa90d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/2145644954 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c040b10 con 0x556969dcdbc0 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/2145644954 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9474+0+0 (secure 0 0 0) 0x7f4f4c040cb0 con 0x556969dcdbc0 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/2145644954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4c033430 con 0x556969dcdbc0 2024-06-06T08:45:56.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.129+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/2145644954 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f4c0335e0 con 0x556969dcdbc0 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f6cf3ab80 10 client.16623 fetch_fsmap learned FSMap version 34 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/2145644954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556969e524a0 con 0x556969dcdbc0 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/2145644954 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c033430 con 0x556969dcdbc0 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/2145644954 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f4c03a020 con 0x556969dcdbc0 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f6cf3ab80 10 client.16623 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f6cf3ab80 10 client.16623 resolve_mds: validated gid 24482 aka daemon mds.b 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f6cf3ab80 20 client.16623 populate_metadata read hostname 'smithi027' 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556969e52ae0 0x556969e54f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f6cf3ab80 4 client.16623 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/2145644954 --> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x556969d26340 con 0x556969e52ae0 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.130+0000 7f4f53fff700 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556969e52ae0 0x556969e54f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f53fff700 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556969e52ae0 0x556969e54f20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf6cc4651df3d973 server_cookie=e948f39343be62eb in_seq=0 out_seq=0 2024-06-06T08:45:56.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f3b7fe700 10 client.16623 ms_handle_connect on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:56.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f3b7fe700 1 -- 172.21.15.27:0/2145644954 <== mds.0 v2:172.21.15.176:6838/1702529484 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x556969d26340 con 0x556969e52ae0 2024-06-06T08:45:56.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f3b7fe700 10 client.16623 handle_command_reply: tid=0 2024-06-06T08:45:56.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f6cf3ab80 1 client.16623 shutdown 2024-06-06T08:45:56.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/2145644954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 msgr2=0x556969dea850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:56.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dea850 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009d10 tx=0x7f4f4c00b990 comp rx=0 tx=0).stop 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/2145644954 shutdown_connections 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556969dd3d90 0x556969e50d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.131+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.176:6838/1702529484,v1:172.21.15.176:6839/1702529484] conn(0x556969e52ae0 0x556969e54f20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556969dcdbc0 0x556969dea850 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f6cf3ab80 1 --2- 172.21.15.27:0/2145644954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556969dd3470 0x556969dead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f3b7fe700 0 client.16623 ms_handle_reset on v2:172.21.15.176:6838/1702529484 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/2145644954 >> 172.21.15.27:0/2145644954 conn(0x556969cfa0f0 msgr2=0x556969dcc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/2145644954 shutdown_connections 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f6cf3ab80 1 -- 172.21.15.27:0/2145644954 wait complete. 2024-06-06T08:45:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f6cf3ab80 20 client.16623 trim_cache size 0 max 16384 2024-06-06T08:45:56.135 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:45:56.135 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-06-06T08:45:56.135 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24482, 2024-06-06T08:45:56.135 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 16, 2024-06-06T08:45:56.135 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": {} 2024-06-06T08:45:56.135 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:45:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f626b7700 1 -- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4f480a2d90 msgr2=0x7f4f480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f626b7700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4f480a2d90 0x7f4f480a5250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c000c00 tx=0x7f4f4c005fb0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f626b7700 1 -- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c1030f0 msgr2=0x7f4f5c19ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.132+0000 7f4f626b7700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c1030f0 0x7f4f5c19ec60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c9b0 tx=0x7f4f4400ccc0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.133+0000 7f4f626b7700 1 -- 172.21.15.27:0/1329645327 shutdown_connections 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.133+0000 7f4f626b7700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f5c107840 0x7f4f5c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.133+0000 7f4f626b7700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f4f480a2d90 0x7f4f480a5250 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.133+0000 7f4f626b7700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4f5c102740 0x7f4f5c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.133+0000 7f4f626b7700 1 --2- 172.21.15.27:0/1329645327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f5c1030f0 0x7f4f5c19ec60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.133+0000 7f4f626b7700 1 -- 172.21.15.27:0/1329645327 >> 172.21.15.27:0/1329645327 conn(0x7f4f5c0795f0 msgr2=0x7f4f5c1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.133+0000 7f4f626b7700 1 -- 172.21.15.27:0/1329645327 shutdown_connections 2024-06-06T08:45:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:45:56.133+0000 7f4f626b7700 1 -- 172.21.15.27:0/1329645327 wait complete. 2024-06-06T08:45:56.995 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:56 smithi027 ceph-mon[32335]: pgmap v173: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 544 op/s 2024-06-06T08:45:56.997 INFO:tasks.quiescer:done joining Quiescers 2024-06-06T08:45:56.997 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-06T08:45:57.009 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:45:57.019 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-06-06T08:45:57.045 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-06-06T08:45:57.045 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T08:45:57.045 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:45:57.045 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-06-06T08:45:57.093 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-06-06T08:45:57.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:56 smithi203 ceph-mon[36579]: pgmap v173: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 544 op/s 2024-06-06T08:45:57.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:56 smithi176 ceph-mon[36786]: pgmap v173: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 544 op/s 2024-06-06T08:45:59.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:45:58 smithi203 ceph-mon[36579]: pgmap v174: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 993 KiB/s wr, 470 op/s 2024-06-06T08:45:59.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:45:58 smithi176 ceph-mon[36786]: pgmap v174: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 993 KiB/s wr, 470 op/s 2024-06-06T08:45:59.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:45:58 smithi027 ceph-mon[32335]: pgmap v174: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 993 KiB/s wr, 470 op/s 2024-06-06T08:46:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:00 smithi203 ceph-mon[36579]: pgmap v175: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1002 KiB/s wr, 474 op/s 2024-06-06T08:46:01.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:00 smithi176 ceph-mon[36786]: pgmap v175: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1002 KiB/s wr, 474 op/s 2024-06-06T08:46:01.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:00 smithi027 ceph-mon[32335]: pgmap v175: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1002 KiB/s wr, 474 op/s 2024-06-06T08:46:03.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:02 smithi203 ceph-mon[36579]: pgmap v176: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 356 KiB/s wr, 272 op/s 2024-06-06T08:46:03.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:02 smithi176 ceph-mon[36786]: pgmap v176: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 356 KiB/s wr, 272 op/s 2024-06-06T08:46:03.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:02 smithi027 ceph-mon[32335]: pgmap v176: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 356 KiB/s wr, 272 op/s 2024-06-06T08:46:05.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:04 smithi203 ceph-mon[36579]: pgmap v177: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 356 KiB/s wr, 273 op/s 2024-06-06T08:46:05.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:04 smithi176 ceph-mon[36786]: pgmap v177: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 356 KiB/s wr, 273 op/s 2024-06-06T08:46:05.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:04 smithi027 ceph-mon[32335]: pgmap v177: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 356 KiB/s wr, 273 op/s 2024-06-06T08:46:07.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:06 smithi203 ceph-mon[36579]: pgmap v178: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 321 KiB/s wr, 211 op/s 2024-06-06T08:46:07.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:06 smithi176 ceph-mon[36786]: pgmap v178: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 321 KiB/s wr, 211 op/s 2024-06-06T08:46:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:06 smithi027 ceph-mon[32335]: pgmap v178: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 321 KiB/s wr, 211 op/s 2024-06-06T08:46:09.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:08 smithi203 ceph-mon[36579]: pgmap v179: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-06-06T08:46:09.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:08 smithi176 ceph-mon[36786]: pgmap v179: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-06-06T08:46:09.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:08 smithi027 ceph-mon[32335]: pgmap v179: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-06-06T08:46:11.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:10 smithi203 ceph-mon[36579]: pgmap v180: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-06-06T08:46:11.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:10 smithi176 ceph-mon[36786]: pgmap v180: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-06-06T08:46:11.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:10 smithi027 ceph-mon[32335]: pgmap v180: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-06-06T08:46:13.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:12 smithi176 ceph-mon[36786]: pgmap v181: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-06-06T08:46:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:12 smithi027 ceph-mon[32335]: pgmap v181: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-06-06T08:46:13.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:12 smithi203 ceph-mon[36579]: pgmap v181: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-06-06T08:46:15.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:14 smithi176 ceph-mon[36786]: pgmap v182: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:46:15.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:14 smithi027 ceph-mon[32335]: pgmap v182: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:46:15.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:14 smithi203 ceph-mon[36579]: pgmap v182: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:46:17.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:16 smithi176 ceph-mon[36786]: pgmap v183: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:46:17.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:16 smithi027 ceph-mon[32335]: pgmap v183: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:46:17.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:16 smithi203 ceph-mon[36579]: pgmap v183: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-06-06T08:46:19.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:19 smithi176 ceph-mon[36786]: pgmap v184: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-06-06T08:46:19.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:19 smithi027 ceph-mon[32335]: pgmap v184: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-06-06T08:46:19.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:19 smithi203 ceph-mon[36579]: pgmap v184: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-06-06T08:46:21.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:21 smithi176 ceph-mon[36786]: pgmap v185: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-06-06T08:46:21.258 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:21 smithi027 ceph-mon[32335]: pgmap v185: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-06-06T08:46:21.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:21 smithi203 ceph-mon[36579]: pgmap v185: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-06-06T08:46:22.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:22 smithi027 ceph-mon[32335]: pgmap v186: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T08:46:22.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:22 smithi203 ceph-mon[36579]: pgmap v186: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T08:46:22.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:22 smithi176 ceph-mon[36786]: pgmap v186: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T08:46:24.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:24 smithi027 ceph-mon[32335]: pgmap v187: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:46:24.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:24 smithi203 ceph-mon[36579]: pgmap v187: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:46:24.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:24 smithi176 ceph-mon[36786]: pgmap v187: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:46:26.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:26 smithi027 ceph-mon[32335]: pgmap v188: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:46:26.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:26 smithi203 ceph-mon[36579]: pgmap v188: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:46:26.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:26 smithi176 ceph-mon[36786]: pgmap v188: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:46:28.706 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:28 smithi027 ceph-mon[32335]: pgmap v189: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:28.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:28 smithi203 ceph-mon[36579]: pgmap v189: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:28.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:28 smithi176 ceph-mon[36786]: pgmap v189: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:31.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:30 smithi203 ceph-mon[36579]: pgmap v190: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:46:31.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:30 smithi176 ceph-mon[36786]: pgmap v190: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:46:31.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:30 smithi027 ceph-mon[32335]: pgmap v190: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:46:33.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:32 smithi203 ceph-mon[36579]: pgmap v191: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:33.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:32 smithi176 ceph-mon[36786]: pgmap v191: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:33.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:32 smithi027 ceph-mon[32335]: pgmap v191: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:35.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:34 smithi203 ceph-mon[36579]: pgmap v192: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:46:35.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:34 smithi176 ceph-mon[36786]: pgmap v192: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:46:35.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:34 smithi027 ceph-mon[32335]: pgmap v192: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:46:37.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:36 smithi203 ceph-mon[36579]: pgmap v193: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:46:37.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:36 smithi176 ceph-mon[36786]: pgmap v193: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:46:37.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:36 smithi027 ceph-mon[32335]: pgmap v193: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:46:38.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:37 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:46:38.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:37 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:46:38.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:37 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:46:38.258 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:37 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:46:38.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:37 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:46:38.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:37 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:46:38.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:37 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:46:38.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:37 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:46:38.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:37 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:46:38.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:37 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:46:38.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:37 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:46:38.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:37 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:46:39.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:38 smithi176 ceph-mon[36786]: pgmap v194: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:39.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:38 smithi027 ceph-mon[32335]: pgmap v194: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:39.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:38 smithi203 ceph-mon[36579]: pgmap v194: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:41.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:40 smithi176 ceph-mon[36786]: pgmap v195: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:46:41.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:40 smithi027 ceph-mon[32335]: pgmap v195: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:46:41.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:40 smithi203 ceph-mon[36579]: pgmap v195: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:46:43.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:43 smithi176 ceph-mon[36786]: pgmap v196: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:43.292 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:43 smithi203 ceph-mon[36579]: pgmap v196: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:43 smithi027 ceph-mon[32335]: pgmap v196: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:45.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:45 smithi176 ceph-mon[36786]: pgmap v197: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:46:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:45 smithi027 ceph-mon[32335]: pgmap v197: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:46:45.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:45 smithi203 ceph-mon[36579]: pgmap v197: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:46:47.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:47 smithi176 ceph-mon[36786]: pgmap v198: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:46:47.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:47 smithi027 ceph-mon[32335]: pgmap v198: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:46:47.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:47 smithi203 ceph-mon[36579]: pgmap v198: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:46:49.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:49 smithi027 ceph-mon[32335]: pgmap v199: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:49.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:49 smithi203 ceph-mon[36579]: pgmap v199: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:49.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:49 smithi176 ceph-mon[36786]: pgmap v199: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:51.276 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:51 smithi027 ceph-mon[32335]: pgmap v200: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:46:51.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:51 smithi203 ceph-mon[36579]: pgmap v200: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:46:51.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:51 smithi176 ceph-mon[36786]: pgmap v200: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:46:52.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:52 smithi027 ceph-mon[32335]: pgmap v201: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:52.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:52 smithi203 ceph-mon[36579]: pgmap v201: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:52.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:52 smithi176 ceph-mon[36786]: pgmap v201: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:46:54.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:54 smithi203 ceph-mon[36579]: pgmap v202: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:46:54.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:54 smithi176 ceph-mon[36786]: pgmap v202: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:46:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:54 smithi027 ceph-mon[32335]: pgmap v202: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:46:57.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:56 smithi203 ceph-mon[36579]: pgmap v203: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-06T08:46:57.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:56 smithi176 ceph-mon[36786]: pgmap v203: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-06T08:46:57.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:56 smithi027 ceph-mon[32335]: pgmap v203: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-06T08:46:59.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:46:58 smithi203 ceph-mon[36579]: pgmap v204: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:46:59.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:46:58 smithi176 ceph-mon[36786]: pgmap v204: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:46:59.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:46:58 smithi027 ceph-mon[32335]: pgmap v204: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:47:01.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:00 smithi203 ceph-mon[36579]: pgmap v205: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:47:01.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:00 smithi176 ceph-mon[36786]: pgmap v205: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:47:01.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:00 smithi027 ceph-mon[32335]: pgmap v205: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:47:02.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:01 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:47:02.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:01 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:47:02.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:01 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:47:03.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:03 smithi176 ceph-mon[36786]: pgmap v206: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:47:03.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:03 smithi027 ceph-mon[32335]: pgmap v206: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:47:03.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:03 smithi203 ceph-mon[36579]: pgmap v206: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T08:47:05.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:05 smithi176 ceph-mon[36786]: pgmap v207: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:47:05.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:05 smithi027 ceph-mon[32335]: pgmap v207: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:47:05.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:05 smithi203 ceph-mon[36579]: pgmap v207: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:47:07.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:07 smithi176 ceph-mon[36786]: pgmap v208: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-06-06T08:47:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:07 smithi027 ceph-mon[32335]: pgmap v208: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-06-06T08:47:07.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:07 smithi203 ceph-mon[36579]: pgmap v208: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-06-06T08:47:09.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:09 smithi027 ceph-mon[32335]: pgmap v209: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-06-06T08:47:09.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:09 smithi203 ceph-mon[36579]: pgmap v209: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-06-06T08:47:09.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:09 smithi176 ceph-mon[36786]: pgmap v209: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-06-06T08:47:11.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:11 smithi027 ceph-mon[32335]: pgmap v210: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-06-06T08:47:11.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:11 smithi203 ceph-mon[36579]: pgmap v210: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-06-06T08:47:11.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:11 smithi176 ceph-mon[36786]: pgmap v210: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-06-06T08:47:12.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:12 smithi027 ceph-mon[32335]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T08:47:12.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:12 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:47:12.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:12 smithi027 ceph-mon[32335]: pgmap v212: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:47:12.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:12 smithi203 ceph-mon[36579]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T08:47:12.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:12 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:47:12.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:12 smithi203 ceph-mon[36579]: pgmap v212: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:47:12.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:12 smithi176 ceph-mon[36786]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T08:47:12.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:12 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:47:12.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:12 smithi176 ceph-mon[36786]: pgmap v212: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:47:14.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:14 smithi203 ceph-mon[36579]: pgmap v213: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:47:14.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:14 smithi176 ceph-mon[36786]: pgmap v213: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:47:14.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:14 smithi027 ceph-mon[32335]: pgmap v213: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:47:16.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:16 smithi203 ceph-mon[36579]: pgmap v214: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:16.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:16 smithi176 ceph-mon[36786]: pgmap v214: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:16.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:16 smithi027 ceph-mon[32335]: pgmap v214: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:18.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:18 smithi203 ceph-mon[36579]: pgmap v215: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:18.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:18 smithi176 ceph-mon[36786]: pgmap v215: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:18.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:18 smithi027 ceph-mon[32335]: pgmap v215: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:21.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:20 smithi203 ceph-mon[36579]: pgmap v216: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:21.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:20 smithi203 ceph-mon[36579]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T08:47:21.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:20 smithi176 ceph-mon[36786]: pgmap v216: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:21.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:20 smithi176 ceph-mon[36786]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T08:47:21.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:20 smithi027 ceph-mon[32335]: pgmap v216: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:21.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:20 smithi027 ceph-mon[32335]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T08:47:22.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:22 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:47:22.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:21 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:47:22.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:22 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:47:23.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:23 smithi176 ceph-mon[36786]: pgmap v218: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:23.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:23 smithi027 ceph-mon[32335]: pgmap v218: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:23.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:23 smithi203 ceph-mon[36579]: pgmap v218: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:25.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:25 smithi176 ceph-mon[36786]: pgmap v219: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:25.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:25 smithi027 ceph-mon[32335]: pgmap v219: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:25.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:25 smithi203 ceph-mon[36579]: pgmap v219: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:47:27.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:27 smithi176 ceph-mon[36786]: pgmap v220: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:47:27.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:27 smithi027 ceph-mon[32335]: pgmap v220: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:47:27.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:27 smithi203 ceph-mon[36579]: pgmap v220: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:47:29.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:29 smithi027 ceph-mon[32335]: pgmap v221: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:47:29.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:29 smithi203 ceph-mon[36579]: pgmap v221: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:47:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:29 smithi176 ceph-mon[36786]: pgmap v221: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:47:31.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:31 smithi027 ceph-mon[32335]: pgmap v222: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-06-06T08:47:31.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:31 smithi203 ceph-mon[36579]: pgmap v222: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-06-06T08:47:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:31 smithi176 ceph-mon[36786]: pgmap v222: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-06-06T08:47:33.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:33 smithi027 ceph-mon[32335]: pgmap v223: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-06-06T08:47:33.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:33 smithi203 ceph-mon[36579]: pgmap v223: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-06-06T08:47:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:33 smithi176 ceph-mon[36786]: pgmap v223: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-06-06T08:47:34.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:34 smithi027 ceph-mon[32335]: pgmap v224: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:47:34.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:34 smithi203 ceph-mon[36579]: pgmap v224: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:47:34.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:34 smithi176 ceph-mon[36786]: pgmap v224: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:47:36.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:36 smithi203 ceph-mon[36579]: pgmap v225: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:47:36.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:36 smithi176 ceph-mon[36786]: pgmap v225: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:47:36.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:36 smithi027 ceph-mon[32335]: pgmap v225: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:47:38.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:38 smithi203 ceph-mon[36579]: pgmap v226: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:38.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:38 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:47:38.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:38 smithi176 ceph-mon[36786]: pgmap v226: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:38.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:38 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:47:38.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:38 smithi027 ceph-mon[32335]: pgmap v226: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:38.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:38 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:47:39.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:39 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:47:39.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:39 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:47:39.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:39 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:47:39.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:39 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:47:39.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:39 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:47:39.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:39 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:47:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:39 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:47:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:39 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:47:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:39 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:47:40.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:40 smithi027 ceph-mon[32335]: pgmap v227: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:47:40.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:40 smithi203 ceph-mon[36579]: pgmap v227: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:47:41.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:40 smithi176 ceph-mon[36786]: pgmap v227: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:47:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:42 smithi027 ceph-mon[32335]: pgmap v228: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:42.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:42 smithi203 ceph-mon[36579]: pgmap v228: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:43.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:42 smithi176 ceph-mon[36786]: pgmap v228: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:44.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:44 smithi027 ceph-mon[32335]: pgmap v229: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:47:44.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:44 smithi203 ceph-mon[36579]: pgmap v229: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:47:45.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:44 smithi176 ceph-mon[36786]: pgmap v229: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:47:46.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:46 smithi027 ceph-mon[32335]: pgmap v230: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:47:46.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:46 smithi203 ceph-mon[36579]: pgmap v230: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:47:47.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:46 smithi176 ceph-mon[36786]: pgmap v230: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:47:48.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:48 smithi027 ceph-mon[32335]: pgmap v231: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:48.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:48 smithi203 ceph-mon[36579]: pgmap v231: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:49.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:48 smithi176 ceph-mon[36786]: pgmap v231: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:50.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:50 smithi027 ceph-mon[32335]: pgmap v232: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:47:50.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:50 smithi203 ceph-mon[36579]: pgmap v232: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:47:51.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:50 smithi176 ceph-mon[36786]: pgmap v232: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:47:52.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:52 smithi027 ceph-mon[32335]: pgmap v233: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:52.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:52 smithi203 ceph-mon[36579]: pgmap v233: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:53.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:52 smithi176 ceph-mon[36786]: pgmap v233: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:54 smithi027 ceph-mon[32335]: pgmap v234: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:47:54.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:54 smithi203 ceph-mon[36579]: pgmap v234: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:47:55.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:54 smithi176 ceph-mon[36786]: pgmap v234: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:47:56.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:56 smithi027 ceph-mon[32335]: pgmap v235: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:47:56.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:56 smithi203 ceph-mon[36579]: pgmap v235: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:47:57.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:56 smithi176 ceph-mon[36786]: pgmap v235: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:47:58.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:47:58 smithi027 ceph-mon[32335]: pgmap v236: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:58.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:47:58 smithi203 ceph-mon[36579]: pgmap v236: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:47:59.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:47:58 smithi176 ceph-mon[36786]: pgmap v236: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:00 smithi027 ceph-mon[32335]: pgmap v237: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:48:00.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:00 smithi203 ceph-mon[36579]: pgmap v237: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:48:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:00 smithi176 ceph-mon[36786]: pgmap v237: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:48:02.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:02 smithi027 ceph-mon[32335]: pgmap v238: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:02.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:02 smithi203 ceph-mon[36579]: pgmap v238: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:03.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:02 smithi176 ceph-mon[36786]: pgmap v238: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:04.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:04 smithi027 ceph-mon[32335]: pgmap v239: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:48:04.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:04 smithi203 ceph-mon[36579]: pgmap v239: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:48:05.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:04 smithi176 ceph-mon[36786]: pgmap v239: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:48:07.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:06 smithi176 ceph-mon[36786]: pgmap v240: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-06-06T08:48:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:06 smithi027 ceph-mon[32335]: pgmap v240: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-06-06T08:48:07.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:06 smithi203 ceph-mon[36579]: pgmap v240: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-06-06T08:48:09.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:09 smithi176 ceph-mon[36786]: pgmap v241: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-06-06T08:48:09.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:09 smithi027 ceph-mon[32335]: pgmap v241: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-06-06T08:48:09.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:09 smithi203 ceph-mon[36579]: pgmap v241: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-06-06T08:48:11.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:11 smithi176 ceph-mon[36786]: pgmap v242: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-06-06T08:48:11.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:11 smithi027 ceph-mon[32335]: pgmap v242: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-06-06T08:48:11.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:11 smithi203 ceph-mon[36579]: pgmap v242: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-06-06T08:48:12.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:12 smithi027 ceph-mon[32335]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T08:48:12.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:12 smithi203 ceph-mon[36579]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T08:48:12.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:12 smithi176 ceph-mon[36786]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T08:48:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:13 smithi027 ceph-mon[32335]: pgmap v244: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:48:13.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:13 smithi203 ceph-mon[36579]: pgmap v244: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:48:13.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:13 smithi176 ceph-mon[36786]: pgmap v244: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:48:14.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:14 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:48:14.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:14 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:48:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:14 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:48:15.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:15 smithi027 ceph-mon[32335]: pgmap v245: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:48:15.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:15 smithi203 ceph-mon[36579]: pgmap v245: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:48:15.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:15 smithi176 ceph-mon[36786]: pgmap v245: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:48:16.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:16 smithi027 ceph-mon[32335]: pgmap v246: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:16.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:16 smithi203 ceph-mon[36579]: pgmap v246: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:16.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:16 smithi176 ceph-mon[36786]: pgmap v246: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:18.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:18 smithi176 ceph-mon[36786]: pgmap v247: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:18.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:18 smithi027 ceph-mon[32335]: pgmap v247: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:18.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:18 smithi203 ceph-mon[36579]: pgmap v247: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:21.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:21 smithi176 ceph-mon[36786]: pgmap v248: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:21.257 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:21 smithi176 ceph-mon[36786]: osdmap e105: 12 total, 12 up, 12 in 2024-06-06T08:48:21.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:21 smithi027 ceph-mon[32335]: pgmap v248: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:21.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:21 smithi027 ceph-mon[32335]: osdmap e105: 12 total, 12 up, 12 in 2024-06-06T08:48:21.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:21 smithi203 ceph-mon[36579]: pgmap v248: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:21.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:21 smithi203 ceph-mon[36579]: osdmap e105: 12 total, 12 up, 12 in 2024-06-06T08:48:23.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:23 smithi176 ceph-mon[36786]: pgmap v250: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:23.258 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:23 smithi027 ceph-mon[32335]: pgmap v250: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:23.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:23 smithi203 ceph-mon[36579]: pgmap v250: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:25.256 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:25 smithi176 ceph-mon[36786]: pgmap v251: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:25.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:25 smithi027 ceph-mon[32335]: pgmap v251: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:25.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:25 smithi203 ceph-mon[36579]: pgmap v251: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:48:27.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:27 smithi027 ceph-mon[32335]: pgmap v252: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:48:27.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:27 smithi203 ceph-mon[36579]: pgmap v252: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:48:27.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:27 smithi176 ceph-mon[36786]: pgmap v252: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:48:29.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:29 smithi027 ceph-mon[32335]: pgmap v253: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:48:29.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:29 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:48:29.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:29 smithi203 ceph-mon[36579]: pgmap v253: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:48:29.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:29 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:48:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:29 smithi176 ceph-mon[36786]: pgmap v253: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:48:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:29 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:48:31.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:31 smithi027 ceph-mon[32335]: pgmap v254: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T08:48:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:31 smithi203 ceph-mon[36579]: pgmap v254: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T08:48:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:31 smithi176 ceph-mon[36786]: pgmap v254: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T08:48:32.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:32 smithi027 ceph-mon[32335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:48:32.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:32 smithi203 ceph-mon[36579]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:48:32.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:32 smithi176 ceph-mon[36786]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T08:48:33.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:33 smithi027 ceph-mon[32335]: pgmap v255: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-06-06T08:48:33.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:33 smithi203 ceph-mon[36579]: pgmap v255: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-06-06T08:48:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:33 smithi176 ceph-mon[36786]: pgmap v255: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-06-06T08:48:34.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:34 smithi027 ceph-mon[32335]: pgmap v256: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:48:34.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:34 smithi203 ceph-mon[36579]: pgmap v256: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:48:34.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:34 smithi176 ceph-mon[36786]: pgmap v256: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:48:36.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:36 smithi176 ceph-mon[36786]: pgmap v257: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:48:36.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:36 smithi027 ceph-mon[32335]: pgmap v257: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:48:36.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:36 smithi203 ceph-mon[36579]: pgmap v257: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:48:38.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:38 smithi176 ceph-mon[36786]: pgmap v258: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:38.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:38 smithi027 ceph-mon[32335]: pgmap v258: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:38.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:38 smithi203 ceph-mon[36579]: pgmap v258: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:39.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:39 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:48:39.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:39 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:48:39.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:39 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:48:39.757 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:39 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:48:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:39 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:48:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:39 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:48:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:39 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:48:39.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:39 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:48:39.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:39 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:48:39.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:39 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:48:39.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:39 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:48:39.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:39 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:48:40.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:40 smithi176 ceph-mon[36786]: pgmap v259: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:48:40.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:40 smithi027 ceph-mon[32335]: pgmap v259: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:48:40.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:40 smithi203 ceph-mon[36579]: pgmap v259: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:48:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:42 smithi027 ceph-mon[32335]: pgmap v260: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:42.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:42 smithi203 ceph-mon[36579]: pgmap v260: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:43.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:42 smithi176 ceph-mon[36786]: pgmap v260: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:44.792 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:44 smithi203 ceph-mon[36579]: pgmap v261: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:48:44.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:44 smithi027 ceph-mon[32335]: pgmap v261: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:48:45.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:44 smithi176 ceph-mon[36786]: pgmap v261: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:48:46.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:46 smithi027 ceph-mon[32335]: pgmap v262: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:48:46.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:46 smithi203 ceph-mon[36579]: pgmap v262: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:48:47.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:46 smithi176 ceph-mon[36786]: pgmap v262: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:48:48.809 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:48 smithi176 ceph-mon[36786]: pgmap v263: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:48.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:48 smithi027 ceph-mon[32335]: pgmap v263: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:48.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:48 smithi203 ceph-mon[36579]: pgmap v263: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:50.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:50 smithi027 ceph-mon[32335]: pgmap v264: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:48:50.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:50 smithi203 ceph-mon[36579]: pgmap v264: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:48:51.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:50 smithi176 ceph-mon[36786]: pgmap v264: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:48:52.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:52 smithi027 ceph-mon[32335]: pgmap v265: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:52.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:52 smithi203 ceph-mon[36579]: pgmap v265: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:53.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:52 smithi176 ceph-mon[36786]: pgmap v265: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:54 smithi027 ceph-mon[32335]: pgmap v266: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:48:54.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:54 smithi203 ceph-mon[36579]: pgmap v266: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:48:55.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:54 smithi176 ceph-mon[36786]: pgmap v266: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:48:56.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:56 smithi027 ceph-mon[32335]: pgmap v267: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:48:56.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:56 smithi203 ceph-mon[36579]: pgmap v267: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:48:57.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:56 smithi176 ceph-mon[36786]: pgmap v267: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:48:58.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:48:58 smithi027 ceph-mon[32335]: pgmap v268: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:58.992 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:48:58 smithi203 ceph-mon[36579]: pgmap v268: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:48:59.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:48:58 smithi176 ceph-mon[36786]: pgmap v268: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:00 smithi027 ceph-mon[32335]: pgmap v269: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:49:00.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:00 smithi203 ceph-mon[36579]: pgmap v269: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:49:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:00 smithi176 ceph-mon[36786]: pgmap v269: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:49:02.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:02 smithi027 ceph-mon[32335]: pgmap v270: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:02.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:02 smithi203 ceph-mon[36579]: pgmap v270: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:03.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:02 smithi176 ceph-mon[36786]: pgmap v270: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:04.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:04 smithi027 ceph-mon[32335]: pgmap v271: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:49:04.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:04 smithi203 ceph-mon[36579]: pgmap v271: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:49:05.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:04 smithi176 ceph-mon[36786]: pgmap v271: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:49:06.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:06 smithi027 ceph-mon[32335]: pgmap v272: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-06-06T08:49:06.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:06 smithi203 ceph-mon[36579]: pgmap v272: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-06-06T08:49:07.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:06 smithi176 ceph-mon[36786]: pgmap v272: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-06-06T08:49:08.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:08 smithi027 ceph-mon[32335]: pgmap v273: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-06-06T08:49:08.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:08 smithi203 ceph-mon[36579]: pgmap v273: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-06-06T08:49:09.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:08 smithi176 ceph-mon[36786]: pgmap v273: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-06-06T08:49:10.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:10 smithi027 ceph-mon[32335]: pgmap v274: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-06-06T08:49:10.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:10 smithi203 ceph-mon[36579]: pgmap v274: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-06-06T08:49:11.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:10 smithi176 ceph-mon[36786]: pgmap v274: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-06-06T08:49:11.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:11 smithi027 ceph-mon[32335]: osdmap e106: 12 total, 12 up, 12 in 2024-06-06T08:49:11.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:11 smithi203 ceph-mon[36579]: osdmap e106: 12 total, 12 up, 12 in 2024-06-06T08:49:12.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:11 smithi176 ceph-mon[36786]: osdmap e106: 12 total, 12 up, 12 in 2024-06-06T08:49:12.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:12 smithi027 ceph-mon[32335]: pgmap v276: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:49:12.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:12 smithi203 ceph-mon[36579]: pgmap v276: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:49:13.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:12 smithi176 ceph-mon[36786]: pgmap v276: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-06-06T08:49:14.958 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:14 smithi027 ceph-mon[32335]: pgmap v277: 129 pgs: 6 active+clean+snaptrim_wait, 119 active+clean, 4 active+clean+snaptrim; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:49:14.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:14 smithi203 ceph-mon[36579]: pgmap v277: 129 pgs: 6 active+clean+snaptrim_wait, 119 active+clean, 4 active+clean+snaptrim; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:49:15.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:14 smithi176 ceph-mon[36786]: pgmap v277: 129 pgs: 6 active+clean+snaptrim_wait, 119 active+clean, 4 active+clean+snaptrim; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:49:16.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:16 smithi027 ceph-mon[32335]: pgmap v278: 129 pgs: 66 active+clean+snaptrim_wait, 41 active+clean, 22 active+clean+snaptrim; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:16.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:16 smithi203 ceph-mon[36579]: pgmap v278: 129 pgs: 66 active+clean+snaptrim_wait, 41 active+clean, 22 active+clean+snaptrim; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:17.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:16 smithi176 ceph-mon[36786]: pgmap v278: 129 pgs: 66 active+clean+snaptrim_wait, 41 active+clean, 22 active+clean+snaptrim; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:18.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:18 smithi027 ceph-mon[32335]: pgmap v279: 129 pgs: 66 active+clean+snaptrim_wait, 41 active+clean, 22 active+clean+snaptrim; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:18.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:18 smithi203 ceph-mon[36579]: pgmap v279: 129 pgs: 66 active+clean+snaptrim_wait, 41 active+clean, 22 active+clean+snaptrim; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:19.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:18 smithi176 ceph-mon[36786]: pgmap v279: 129 pgs: 66 active+clean+snaptrim_wait, 41 active+clean, 22 active+clean+snaptrim; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:21.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:21 smithi027 ceph-mon[32335]: pgmap v280: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:21.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:21 smithi027 ceph-mon[32335]: osdmap e107: 12 total, 12 up, 12 in 2024-06-06T08:49:21.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:21 smithi203 ceph-mon[36579]: pgmap v280: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:21.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:21 smithi203 ceph-mon[36579]: osdmap e107: 12 total, 12 up, 12 in 2024-06-06T08:49:21.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:21 smithi176 ceph-mon[36786]: pgmap v280: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:21.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:21 smithi176 ceph-mon[36786]: osdmap e107: 12 total, 12 up, 12 in 2024-06-06T08:49:23.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:23 smithi027 ceph-mon[32335]: pgmap v282: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:23.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:23 smithi203 ceph-mon[36579]: pgmap v282: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:23.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:23 smithi176 ceph-mon[36786]: pgmap v282: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:25.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:25 smithi027 ceph-mon[32335]: pgmap v283: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:25.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:25 smithi203 ceph-mon[36579]: pgmap v283: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:25.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:25 smithi176 ceph-mon[36786]: pgmap v283: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:49:27.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:27 smithi027 ceph-mon[32335]: pgmap v284: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:49:27.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:27 smithi203 ceph-mon[36579]: pgmap v284: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:49:27.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:27 smithi176 ceph-mon[36786]: pgmap v284: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:49:29.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:29 smithi027 ceph-mon[32335]: pgmap v285: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:49:29.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:29 smithi203 ceph-mon[36579]: pgmap v285: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:49:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:29 smithi176 ceph-mon[36786]: pgmap v285: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-06-06T08:49:31.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:31 smithi027 ceph-mon[32335]: pgmap v286: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-06-06T08:49:31.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:31 smithi203 ceph-mon[36579]: pgmap v286: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-06-06T08:49:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:31 smithi176 ceph-mon[36786]: pgmap v286: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-06-06T08:49:33.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:33 smithi027 ceph-mon[32335]: pgmap v287: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-06-06T08:49:33.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:33 smithi203 ceph-mon[36579]: pgmap v287: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-06-06T08:49:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:33 smithi176 ceph-mon[36786]: pgmap v287: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-06-06T08:49:35.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:35 smithi027 ceph-mon[32335]: pgmap v288: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:49:35.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:35 smithi203 ceph-mon[36579]: pgmap v288: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:49:35.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:35 smithi176 ceph-mon[36786]: pgmap v288: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:49:36.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:36 smithi027 ceph-mon[32335]: pgmap v289: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-06-06T08:49:36.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:36 smithi203 ceph-mon[36579]: pgmap v289: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-06-06T08:49:36.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:36 smithi176 ceph-mon[36786]: pgmap v289: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-06-06T08:49:38.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:38 smithi027 ceph-mon[32335]: pgmap v290: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-06-06T08:49:38.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:38 smithi203 ceph-mon[36579]: pgmap v290: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-06-06T08:49:39.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:38 smithi176 ceph-mon[36786]: pgmap v290: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-06-06T08:49:40.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:40 smithi027 ceph-mon[32335]: pgmap v291: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:49:40.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:40 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:49:40.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:40 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:49:40.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:40 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:49:40.957 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:40 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:49:40.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:40 smithi203 ceph-mon[36579]: pgmap v291: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:49:40.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:40 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:49:40.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:40 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:49:40.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:40 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:49:40.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:40 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:49:41.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:40 smithi176 ceph-mon[36786]: pgmap v291: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:49:41.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:40 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:49:41.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:40 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:49:41.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:40 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:49:41.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:40 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:49:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:42 smithi027 ceph-mon[32335]: pgmap v292: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-06-06T08:49:42.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:42 smithi203 ceph-mon[36579]: pgmap v292: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-06-06T08:49:43.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:42 smithi176 ceph-mon[36786]: pgmap v292: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-06-06T08:49:44.791 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:44 smithi203 ceph-mon[36579]: pgmap v293: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:49:44.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:44 smithi027 ceph-mon[32335]: pgmap v293: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:49:45.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:44 smithi176 ceph-mon[36786]: pgmap v293: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:49:46.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:46 smithi027 ceph-mon[32335]: pgmap v294: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-06-06T08:49:46.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:46 smithi203 ceph-mon[36579]: pgmap v294: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-06-06T08:49:47.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:46 smithi176 ceph-mon[36786]: pgmap v294: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-06-06T08:49:48.809 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:48 smithi176 ceph-mon[36786]: pgmap v295: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:48.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:48 smithi027 ceph-mon[32335]: pgmap v295: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:48.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:48 smithi203 ceph-mon[36579]: pgmap v295: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:50.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:50 smithi027 ceph-mon[32335]: pgmap v296: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:49:50.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:50 smithi203 ceph-mon[36579]: pgmap v296: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:49:51.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:50 smithi176 ceph-mon[36786]: pgmap v296: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:49:52.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:52 smithi027 ceph-mon[32335]: pgmap v297: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:52.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:52 smithi203 ceph-mon[36579]: pgmap v297: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:53.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:52 smithi176 ceph-mon[36786]: pgmap v297: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:54 smithi027 ceph-mon[32335]: pgmap v298: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:49:54.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:54 smithi203 ceph-mon[36579]: pgmap v298: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:49:55.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:54 smithi176 ceph-mon[36786]: pgmap v298: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T08:49:56.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:56 smithi027 ceph-mon[32335]: pgmap v299: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:49:56.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:56 smithi203 ceph-mon[36579]: pgmap v299: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:49:57.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:56 smithi176 ceph-mon[36786]: pgmap v299: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:49:58.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:49:58 smithi027 ceph-mon[32335]: pgmap v300: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:58.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:49:58 smithi203 ceph-mon[36579]: pgmap v300: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:49:59.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:49:58 smithi176 ceph-mon[36786]: pgmap v300: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[32331]: 2024-06-06T08:49:59.999+0000 7fc898df5700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-06-06T08:50:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[32331]: 2024-06-06T08:49:59.999+0000 7fc898df5700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-06-06T08:50:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[32331]: 2024-06-06T08:49:59.999+0000 7fc898df5700 -1 log_channel(cluster) log [ERR] : mds.h(mds.2): Metadata damage detected 2024-06-06T08:50:00.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[32331]: 2024-06-06T08:49:59.999+0000 7fc898df5700 -1 log_channel(cluster) log [ERR] : mds.i(mds.1): Metadata damage detected 2024-06-06T08:50:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:00 smithi027 ceph-mon[32335]: pgmap v301: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:50:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:00 smithi027 ceph-mon[32335]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-06-06T08:50:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:00 smithi027 ceph-mon[32335]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-06-06T08:50:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:00 smithi027 ceph-mon[32335]: mds.h(mds.2): Metadata damage detected 2024-06-06T08:50:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:00 smithi027 ceph-mon[32335]: mds.i(mds.1): Metadata damage detected 2024-06-06T08:50:00.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:00 smithi203 ceph-mon[36579]: pgmap v301: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:50:00.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:00 smithi203 ceph-mon[36579]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-06-06T08:50:00.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:00 smithi203 ceph-mon[36579]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-06-06T08:50:00.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:00 smithi203 ceph-mon[36579]: mds.h(mds.2): Metadata damage detected 2024-06-06T08:50:00.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:00 smithi203 ceph-mon[36579]: mds.i(mds.1): Metadata damage detected 2024-06-06T08:50:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:00 smithi176 ceph-mon[36786]: pgmap v301: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:50:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:00 smithi176 ceph-mon[36786]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-06-06T08:50:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:00 smithi176 ceph-mon[36786]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-06-06T08:50:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:00 smithi176 ceph-mon[36786]: mds.h(mds.2): Metadata damage detected 2024-06-06T08:50:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:00 smithi176 ceph-mon[36786]: mds.i(mds.1): Metadata damage detected 2024-06-06T08:50:02.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:02 smithi027 ceph-mon[32335]: pgmap v302: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:02.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:02 smithi203 ceph-mon[36579]: pgmap v302: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:03.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:02 smithi176 ceph-mon[36786]: pgmap v302: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:04.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:04 smithi027 ceph-mon[32335]: pgmap v303: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:50:04.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:04 smithi203 ceph-mon[36579]: pgmap v303: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:50:05.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:04 smithi176 ceph-mon[36786]: pgmap v303: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:50:07.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:07 smithi027 ceph-mon[32335]: pgmap v304: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-06-06T08:50:07.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:07 smithi203 ceph-mon[36579]: pgmap v304: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-06-06T08:50:07.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:07 smithi176 ceph-mon[36786]: pgmap v304: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-06-06T08:50:09.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:09 smithi027 ceph-mon[32335]: pgmap v305: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-06-06T08:50:09.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:09 smithi203 ceph-mon[36579]: pgmap v305: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-06-06T08:50:09.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:09 smithi176 ceph-mon[36786]: pgmap v305: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-06-06T08:50:11.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:11 smithi027 ceph-mon[32335]: pgmap v306: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-06-06T08:50:11.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:11 smithi203 ceph-mon[36579]: pgmap v306: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-06-06T08:50:11.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:11 smithi176 ceph-mon[36786]: pgmap v306: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-06-06T08:50:12.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:12 smithi027 ceph-mon[32335]: osdmap e108: 12 total, 12 up, 12 in 2024-06-06T08:50:12.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:12 smithi203 ceph-mon[36579]: osdmap e108: 12 total, 12 up, 12 in 2024-06-06T08:50:12.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:12 smithi176 ceph-mon[36786]: osdmap e108: 12 total, 12 up, 12 in 2024-06-06T08:50:13.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:13 smithi027 ceph-mon[32335]: pgmap v308: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:50:13.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:13 smithi203 ceph-mon[36579]: pgmap v308: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:50:13.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:13 smithi176 ceph-mon[36786]: pgmap v308: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:50:14.390 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:14 smithi203 ceph-mon[36579]: pgmap v309: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:50:14.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:14 smithi027 ceph-mon[32335]: pgmap v309: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:50:14.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:14 smithi176 ceph-mon[36786]: pgmap v309: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:50:16.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:16 smithi027 ceph-mon[32335]: pgmap v310: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:50:16.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:16 smithi203 ceph-mon[36579]: pgmap v310: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:50:17.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:16 smithi176 ceph-mon[36786]: pgmap v310: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:50:17.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:50:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:50:17.557+0000 7f90a0c4a700 -1 calc_pg_upmaps abort due to max <= 0 2024-06-06T08:50:17.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:50:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:50:17.557+0000 7f90a0c4a700 -1 calc_pg_upmaps abort due to max <= 0 2024-06-06T08:50:18.808 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:18 smithi176 ceph-mon[36786]: pgmap v311: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:50:18.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:18 smithi027 ceph-mon[32335]: pgmap v311: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:50:18.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:18 smithi203 ceph-mon[36579]: pgmap v311: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:50:20.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:20 smithi027 ceph-mon[32335]: pgmap v312: 129 pgs: 16 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 59 active+clean; 460 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:50:20.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:20 smithi203 ceph-mon[36579]: pgmap v312: 129 pgs: 16 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 59 active+clean; 460 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:50:21.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:20 smithi176 ceph-mon[36786]: pgmap v312: 129 pgs: 16 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 59 active+clean; 460 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:50:22.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:22 smithi027 ceph-mon[32335]: pgmap v313: 129 pgs: 16 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 59 active+clean; 460 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 196 B/s wr, 7 op/s 2024-06-06T08:50:22.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:22 smithi203 ceph-mon[36579]: pgmap v313: 129 pgs: 16 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 59 active+clean; 460 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 196 B/s wr, 7 op/s 2024-06-06T08:50:23.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:22 smithi176 ceph-mon[36786]: pgmap v313: 129 pgs: 16 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 59 active+clean; 460 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 196 B/s wr, 7 op/s 2024-06-06T08:50:24.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:24 smithi027 ceph-mon[32335]: pgmap v314: 129 pgs: 14 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 67 active+clean; 426 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-06-06T08:50:24.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:24 smithi203 ceph-mon[36579]: pgmap v314: 129 pgs: 14 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 67 active+clean; 426 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-06-06T08:50:25.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:24 smithi176 ceph-mon[36786]: pgmap v314: 129 pgs: 14 active+clean+snaptrim, 48 active+clean+snaptrim_wait, 67 active+clean; 426 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-06-06T08:50:26.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:26 smithi027 ceph-mon[32335]: pgmap v315: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-06-06T08:50:26.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:26 smithi203 ceph-mon[36579]: pgmap v315: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-06-06T08:50:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:26 smithi176 ceph-mon[36786]: pgmap v315: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-06-06T08:50:28.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:28 smithi027 ceph-mon[32335]: pgmap v316: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T08:50:28.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:28 smithi203 ceph-mon[36579]: pgmap v316: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T08:50:29.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:28 smithi176 ceph-mon[36786]: pgmap v316: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T08:50:31.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:31 smithi027 ceph-mon[32335]: pgmap v317: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T08:50:31.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:31 smithi027 ceph-mon[32335]: osdmap e109: 12 total, 12 up, 12 in 2024-06-06T08:50:31.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:31 smithi203 ceph-mon[36579]: pgmap v317: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T08:50:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:31 smithi203 ceph-mon[36579]: osdmap e109: 12 total, 12 up, 12 in 2024-06-06T08:50:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:31 smithi176 ceph-mon[36786]: pgmap v317: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-06T08:50:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:31 smithi176 ceph-mon[36786]: osdmap e109: 12 total, 12 up, 12 in 2024-06-06T08:50:33.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-06T08:50:33.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.289+0000 7f3cba361640 1 -- 172.21.15.27:0/2216719670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4105840 msgr2=0x7f3cb4105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.289+0000 7f3cba361640 1 --2- 172.21.15.27:0/2216719670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4105840 0x7f3cb4105c20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009900 tx=0x7f3ca402a410 comp rx=0 tx=0).stop 2024-06-06T08:50:33.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.289+0000 7f3cba361640 1 -- 172.21.15.27:0/2216719670 shutdown_connections 2024-06-06T08:50:33.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.289+0000 7f3cba361640 1 --2- 172.21.15.27:0/2216719670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb41015b0 0x7f3cb410d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.289+0000 7f3cba361640 1 --2- 172.21.15.27:0/2216719670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cb4100bf0 0x7f3cb4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.289+0000 7f3cba361640 1 --2- 172.21.15.27:0/2216719670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4105840 0x7f3cb4105c20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.289+0000 7f3cba361640 1 -- 172.21.15.27:0/2216719670 >> 172.21.15.27:0/2216719670 conn(0x7f3cb40766a0 msgr2=0x7f3cb4076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:33.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.289+0000 7f3cba361640 1 -- 172.21.15.27:0/2216719670 shutdown_connections 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.289+0000 7f3cba361640 1 -- 172.21.15.27:0/2216719670 wait complete. 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.290+0000 7f3cba361640 1 Processor -- start 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.290+0000 7f3cba361640 1 -- start start 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.290+0000 7f3cba361640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4100bf0 0x7f3cb419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cba361640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cb41015b0 0x7f3cb419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cba361640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb4105840 0x7f3cb419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cba361640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4110c30 con 0x7f3cb4100bf0 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cba361640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4110ab0 con 0x7f3cb41015b0 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cba361640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4110db0 con 0x7f3cb4105840 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4100bf0 0x7f3cb419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:33.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4100bf0 0x7f3cb419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35162/0 (socket says 172.21.15.27:35162) 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb3fff640 1 -- 172.21.15.27:0/284193482 learned_addr learned my addr 172.21.15.27:0/284193482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb37fe640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cb41015b0 0x7f3cb419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb88fe640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb4105840 0x7f3cb419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb3fff640 1 -- 172.21.15.27:0/284193482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb4105840 msgr2=0x7f3cb419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb3fff640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb4105840 0x7f3cb419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb3fff640 1 -- 172.21.15.27:0/284193482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cb41015b0 msgr2=0x7f3cb419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb3fff640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cb41015b0 0x7f3cb419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb3fff640 1 -- 172.21.15.27:0/284193482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb419f2e0 con 0x7f3cb4100bf0 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb37fe640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cb41015b0 0x7f3cb419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb3fff640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4100bf0 0x7f3cb419a2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4006010 tx=0x7f3ca40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb17fa640 1 -- 172.21.15.27:0/284193482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca403f030 con 0x7f3cb4100bf0 2024-06-06T08:50:33.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb17fa640 1 -- 172.21.15.27:0/284193482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca4003ce0 con 0x7f3cb4100bf0 2024-06-06T08:50:33.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb419f570 con 0x7f3cb4100bf0 2024-06-06T08:50:33.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cb17fa640 1 -- 172.21.15.27:0/284193482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4035070 con 0x7f3cb4100bf0 2024-06-06T08:50:33.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.291+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb41a6e60 con 0x7f3cb4100bf0 2024-06-06T08:50:33.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.292+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb4069a20 con 0x7f3cb4100bf0 2024-06-06T08:50:33.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.294+0000 7f3cb17fa640 1 -- 172.21.15.27:0/284193482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3ca4042030 con 0x7f3cb4100bf0 2024-06-06T08:50:33.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.294+0000 7f3cb17fa640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3c9c0a2da0 0x7f3c9c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:33.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.295+0000 7f3cb37fe640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3c9c0a2da0 0x7f3c9c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:33.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.295+0000 7f3cb17fa640 1 -- 172.21.15.27:0/284193482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f3ca40f0170 con 0x7f3cb4100bf0 2024-06-06T08:50:33.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.295+0000 7f3cb37fe640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3c9c0a2da0 0x7f3c9c0a5260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80058c0 tx=0x7f3ca8005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:33.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.298+0000 7f3cb17fa640 1 -- 172.21.15.27:0/284193482 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ca40bca40 con 0x7f3cb4100bf0 2024-06-06T08:50:33.455 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:33 smithi027 ceph-mon[32335]: pgmap v319: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-06-06T08:50:33.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:33 smithi203 ceph-mon[36579]: pgmap v319: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-06-06T08:50:33.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.508+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3cb406a8e0 con 0x7f3c9c0a2da0 2024-06-06T08:50:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:33 smithi176 ceph-mon[36786]: pgmap v319: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-06-06T08:50:33.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.516+0000 7f3cb17fa640 1 -- 172.21.15.27:0/284193482 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f3cb406a8e0 con 0x7f3c9c0a2da0 2024-06-06T08:50:33.514 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-06T00:00:00", "created": "2024-06-06T08:40:31", "first": "2024-06-06T08:41:00", "last": "2024-06-06T08:50:00", "last_pruned": "2024-06-06T08:50:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-06-06T08:50:33.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.517+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3c9c0a2da0 msgr2=0x7f3c9c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.517+0000 7f3cba361640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3c9c0a2da0 0x7f3c9c0a5260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80058c0 tx=0x7f3ca8005850 comp rx=0 tx=0).stop 2024-06-06T08:50:33.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4100bf0 msgr2=0x7f3cb419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4100bf0 0x7f3cb419a2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4006010 tx=0x7f3ca40042f0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 shutdown_connections 2024-06-06T08:50:33.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3c9c0a2da0 0x7f3c9c0a5260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cb4105840 0x7f3cb419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cb41015b0 0x7f3cb419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 --2- 172.21.15.27:0/284193482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb4100bf0 0x7f3cb419a2f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 >> 172.21.15.27:0/284193482 conn(0x7f3cb40766a0 msgr2=0x7f3cb406a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:33.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 shutdown_connections 2024-06-06T08:50:33.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.518+0000 7f3cba361640 1 -- 172.21.15.27:0/284193482 wait complete. 2024-06-06T08:50:33.527 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-06T08:50:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.635+0000 7f79db942640 1 -- 172.21.15.27:0/2701836063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 msgr2=0x7f79d4106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.635+0000 7f79db942640 1 --2- 172.21.15.27:0/2701836063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 0x7f79d4106cb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009cd0 tx=0x7f79c402a8e0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.635+0000 7f79db942640 1 -- 172.21.15.27:0/2701836063 shutdown_connections 2024-06-06T08:50:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.635+0000 7f79db942640 1 --2- 172.21.15.27:0/2701836063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4101280 0x7f79d410d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.635+0000 7f79db942640 1 --2- 172.21.15.27:0/2701836063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79d41008c0 0x7f79d4100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.635+0000 7f79db942640 1 --2- 172.21.15.27:0/2701836063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 0x7f79d4106cb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.635+0000 7f79db942640 1 -- 172.21.15.27:0/2701836063 >> 172.21.15.27:0/2701836063 conn(0x7f79d4076c20 msgr2=0x7f79d4077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.635+0000 7f79db942640 1 -- 172.21.15.27:0/2701836063 shutdown_connections 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.635+0000 7f79db942640 1 -- 172.21.15.27:0/2701836063 wait complete. 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79db942640 1 Processor -- start 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79db942640 1 -- start start 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79db942640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79d41008c0 0x7f79d4102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79db942640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4101280 0x7f79d41029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79db942640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 0x7f79d4102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79db942640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4110ce0 con 0x7f79d41068d0 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79db942640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4110b60 con 0x7f79d41008c0 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79db942640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4110e60 con 0x7f79d4101280 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d9edf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 0x7f79d4102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d9edf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 0x7f79d4102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35178/0 (socket says 172.21.15.27:35178) 2024-06-06T08:50:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d9edf640 1 -- 172.21.15.27:0/3413675554 learned_addr learned my addr 172.21.15.27:0/3413675554 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d96de640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79d41008c0 0x7f79d4102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d8edd640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4101280 0x7f79d41029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d9edf640 1 -- 172.21.15.27:0/3413675554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4101280 msgr2=0x7f79d41029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d9edf640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4101280 0x7f79d41029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d9edf640 1 -- 172.21.15.27:0/3413675554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79d41008c0 msgr2=0x7f79d4102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d9edf640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79d41008c0 0x7f79d4102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.636+0000 7f79d9edf640 1 -- 172.21.15.27:0/3413675554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d41a6770 con 0x7f79d41068d0 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.637+0000 7f79d9edf640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 0x7f79d4102f10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00c970 tx=0x7f79cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.637+0000 7f79d27fc640 1 -- 172.21.15.27:0/3413675554 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc007bb0 con 0x7f79d41068d0 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.637+0000 7f79d27fc640 1 -- 172.21.15.27:0/3413675554 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79cc007d50 con 0x7f79d41068d0 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.637+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d41a6a60 con 0x7f79d41068d0 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.637+0000 7f79d27fc640 1 -- 172.21.15.27:0/3413675554 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00f450 con 0x7f79d41068d0 2024-06-06T08:50:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.637+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d41a7000 con 0x7f79d41068d0 2024-06-06T08:50:33.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.638+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d404f760 con 0x7f79d41068d0 2024-06-06T08:50:33.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.639+0000 7f79d27fc640 1 -- 172.21.15.27:0/3413675554 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f79cc004f20 con 0x7f79d41068d0 2024-06-06T08:50:33.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.640+0000 7f79d27fc640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79bc0a2e20 0x7f79bc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:33.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.640+0000 7f79d96de640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79bc0a2e20 0x7f79bc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:33.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.640+0000 7f79d27fc640 1 -- 172.21.15.27:0/3413675554 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f79cc0cf3a0 con 0x7f79d41068d0 2024-06-06T08:50:33.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.640+0000 7f79d96de640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79bc0a2e20 0x7f79bc0a52e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006010 tx=0x7f79c4009360 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:33.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.644+0000 7f79d27fc640 1 -- 172.21.15.27:0/3413675554 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f79cc014030 con 0x7f79d41068d0 2024-06-06T08:50:33.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.853+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 --> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f79d40fe460 con 0x7f79bc0a2e20 2024-06-06T08:50:33.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.860+0000 7f79d27fc640 1 -- 172.21.15.27:0/3413675554 <== mgr.15201 v2:172.21.15.27:6800/1196153485 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f79d40fe460 con 0x7f79bc0a2e20 2024-06-06T08:50:33.859 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-06-06T08:50:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.862+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79bc0a2e20 msgr2=0x7f79bc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.862+0000 7f79db942640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79bc0a2e20 0x7f79bc0a52e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006010 tx=0x7f79c4009360 comp rx=0 tx=0).stop 2024-06-06T08:50:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.862+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 msgr2=0x7f79d4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.862+0000 7f79db942640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 0x7f79d4102f10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00c970 tx=0x7f79cc00ce30 comp rx=0 tx=0).stop 2024-06-06T08:50:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.863+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 shutdown_connections 2024-06-06T08:50:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.863+0000 7f79db942640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f79bc0a2e20 0x7f79bc0a52e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.863+0000 7f79db942640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d41068d0 0x7f79d4102f10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.863+0000 7f79db942640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d4101280 0x7f79d41029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.863+0000 7f79db942640 1 --2- 172.21.15.27:0/3413675554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79d41008c0 0x7f79d4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.863+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 >> 172.21.15.27:0/3413675554 conn(0x7f79d4076c20 msgr2=0x7f79d4104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.863+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 shutdown_connections 2024-06-06T08:50:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:33.863+0000 7f79db942640 1 -- 172.21.15.27:0/3413675554 wait complete. 2024-06-06T08:50:33.872 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-06T08:50:33.883 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-06T08:50:33.883 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-06-06T08:50:33.884 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-06T08:50:33.927 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-06T08:50:34.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.036+0000 7fe81d990640 1 -- 172.21.15.27:0/2450601976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818106890 msgr2=0x7fe818106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.036+0000 7fe81d990640 1 --2- 172.21.15.27:0/2450601976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818106890 0x7fe818106c70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe808009930 tx=0x7fe80802a420 comp rx=0 tx=0).stop 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.036+0000 7fe81d990640 1 -- 172.21.15.27:0/2450601976 shutdown_connections 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.036+0000 7fe81d990640 1 --2- 172.21.15.27:0/2450601976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe818101240 0x7fe81810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.036+0000 7fe81d990640 1 --2- 172.21.15.27:0/2450601976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe818100880 0x7fe818100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.036+0000 7fe81d990640 1 --2- 172.21.15.27:0/2450601976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818106890 0x7fe818106c70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.036+0000 7fe81d990640 1 -- 172.21.15.27:0/2450601976 >> 172.21.15.27:0/2450601976 conn(0x7fe8180779a0 msgr2=0x7fe818077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.036+0000 7fe81d990640 1 -- 172.21.15.27:0/2450601976 shutdown_connections 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.036+0000 7fe81d990640 1 -- 172.21.15.27:0/2450601976 wait complete. 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe81d990640 1 Processor -- start 2024-06-06T08:50:34.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe81d990640 1 -- start start 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe81d990640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818100880 0x7fe818198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe81d990640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe818101240 0x7fe818198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe81d990640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe818106890 0x7fe81819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe81d990640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe818110ca0 con 0x7fe818100880 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe81d990640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fe818110b20 con 0x7fe818101240 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe81d990640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe818110e20 con 0x7fe818106890 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe816ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818100880 0x7fe818198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe816ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818100880 0x7fe818198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35190/0 (socket says 172.21.15.27:35190) 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe816ffd640 1 -- 172.21.15.27:0/2820043866 learned_addr learned my addr 172.21.15.27:0/2820043866 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe8167fc640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe818101240 0x7fe818198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe8177fe640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe818106890 0x7fe81819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe816ffd640 1 -- 172.21.15.27:0/2820043866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe818106890 msgr2=0x7fe81819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe816ffd640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe818106890 0x7fe81819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe816ffd640 1 -- 172.21.15.27:0/2820043866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe818101240 msgr2=0x7fe818198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe816ffd640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe818101240 0x7fe818198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.037+0000 7fe816ffd640 1 -- 172.21.15.27:0/2820043866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81819d210 con 0x7fe818100880 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.038+0000 7fe816ffd640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818100880 0x7fe818198220 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe808009230 tx=0x7fe808004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.038+0000 7fe81c98e640 1 -- 172.21.15.27:0/2820043866 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80803f030 con 0x7fe818100880 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.038+0000 7fe81c98e640 1 -- 172.21.15.27:0/2820043866 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe808003ad0 con 0x7fe818100880 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.038+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81819d4a0 con 0x7fe818100880 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.038+0000 7fe81c98e640 1 -- 172.21.15.27:0/2820043866 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808035070 con 0x7fe818100880 2024-06-06T08:50:34.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.038+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8181a4d90 con 0x7fe818100880 2024-06-06T08:50:34.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.039+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8180687c0 con 0x7fe818100880 2024-06-06T08:50:34.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.040+0000 7fe81c98e640 1 -- 172.21.15.27:0/2820043866 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe808042030 con 0x7fe818100880 2024-06-06T08:50:34.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.041+0000 7fe81c98e640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe8000a2d00 0x7fe8000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.041+0000 7fe8167fc640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe8000a2d00 0x7fe8000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.041+0000 7fe81c98e640 1 -- 172.21.15.27:0/2820043866 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7fe8080efbd0 con 0x7fe818100880 2024-06-06T08:50:34.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.041+0000 7fe8167fc640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe8000a2d00 0x7fe8000a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe818102b50 tx=0x7fe80c005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:34.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.045+0000 7fe81c98e640 1 -- 172.21.15.27:0/2820043866 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8080bbd90 con 0x7fe818100880 2024-06-06T08:50:34.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.231+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe818109e60 con 0x7fe818100880 2024-06-06T08:50:34.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.232+0000 7fe81c98e640 1 -- 172.21.15.27:0/2820043866 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+25398 (secure 0 0 0) 0x7fe8080c0090 con 0x7fe818100880 2024-06-06T08:50:34.231 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:50:34.232 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":109,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:50:30.024349+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/876705499":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/550980984":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/4174349888":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3053481437":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:50:34.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.234+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe8000a2d00 msgr2=0x7fe8000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.234+0000 7fe81d990640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe8000a2d00 0x7fe8000a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe818102b50 tx=0x7fe80c005f90 comp rx=0 tx=0).stop 2024-06-06T08:50:34.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.234+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818100880 msgr2=0x7fe818198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.234+0000 7fe81d990640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818100880 0x7fe818198220 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe808009230 tx=0x7fe808004270 comp rx=0 tx=0).stop 2024-06-06T08:50:34.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.235+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 shutdown_connections 2024-06-06T08:50:34.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.235+0000 7fe81d990640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fe8000a2d00 0x7fe8000a51c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.235+0000 7fe81d990640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe818106890 0x7fe81819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.235+0000 7fe81d990640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe818101240 0x7fe818198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.235+0000 7fe81d990640 1 --2- 172.21.15.27:0/2820043866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe818100880 0x7fe818198220 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.235+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 >> 172.21.15.27:0/2820043866 conn(0x7fe8180779a0 msgr2=0x7fe818068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:34.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.235+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 shutdown_connections 2024-06-06T08:50:34.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.235+0000 7fe81d990640 1 -- 172.21.15.27:0/2820043866 wait complete. 2024-06-06T08:50:34.245 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T08:31:22.877337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-06T08:36:16.098725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-06T08:36:19.818146+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-06T08:36:30.341458+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-06T08:50:34.245 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-06T08:50:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.356+0000 7f3bc89dc640 1 -- 172.21.15.27:0/1541485396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 msgr2=0x7f3bc410d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.356+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/1541485396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 0x7f3bc410d710 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009900 tx=0x7f3bbc02a480 comp rx=0 tx=0).stop 2024-06-06T08:50:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.356+0000 7f3bc89dc640 1 -- 172.21.15.27:0/1541485396 shutdown_connections 2024-06-06T08:50:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.356+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/1541485396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 0x7f3bc410d710 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.356+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/1541485396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc4100fd0 0x7f3bc4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.356+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/1541485396 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc41068c0 0x7f3bc4106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.356+0000 7f3bc89dc640 1 -- 172.21.15.27:0/1541485396 >> 172.21.15.27:0/1541485396 conn(0x7f3bc4076e30 msgr2=0x7f3bc40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.356+0000 7f3bc89dc640 1 -- 172.21.15.27:0/1541485396 shutdown_connections 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.356+0000 7f3bc89dc640 1 -- 172.21.15.27:0/1541485396 wait complete. 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc89dc640 1 Processor -- start 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc89dc640 1 -- start start 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc89dc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc4100fd0 0x7f3bc4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc89dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 0x7f3bc41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc89dc640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc41068c0 0x7f3bc419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc89dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4110c90 con 0x7f3bc4101990 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc89dc640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4110b10 con 0x7f3bc41068c0 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc89dc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4110e10 con 0x7f3bc4100fd0 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 0x7f3bc41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 0x7f3bc41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35218/0 (socket says 172.21.15.27:35218) 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc1d9b640 1 -- 172.21.15.27:0/3752652873 learned_addr learned my addr 172.21.15.27:0/3752652873 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc2d9d640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc41068c0 0x7f3bc419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc259c640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc4100fd0 0x7f3bc4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc1d9b640 1 -- 172.21.15.27:0/3752652873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc4100fd0 msgr2=0x7f3bc4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc1d9b640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc4100fd0 0x7f3bc4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc1d9b640 1 -- 172.21.15.27:0/3752652873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc41068c0 msgr2=0x7f3bc419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc1d9b640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc41068c0 0x7f3bc419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc1d9b640 1 -- 172.21.15.27:0/3752652873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc419d110 con 0x7f3bc4101990 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.357+0000 7f3bc259c640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc4100fd0 0x7f3bc4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.358+0000 7f3bc1d9b640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 0x7f3bc41986b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400c920 tx=0x7f3bb400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.358+0000 7f3bbb7fe640 1 -- 172.21.15.27:0/3752652873 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4007c70 con 0x7f3bc4101990 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.358+0000 7f3bbb7fe640 1 -- 172.21.15.27:0/3752652873 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bb4007e10 con 0x7f3bc4101990 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.358+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc419d400 con 0x7f3bc4101990 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.358+0000 7f3bbb7fe640 1 -- 172.21.15.27:0/3752652873 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4018890 con 0x7f3bc4101990 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.358+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41a4c90 con 0x7f3bc4101990 2024-06-06T08:50:34.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.359+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc4068c10 con 0x7f3bc4101990 2024-06-06T08:50:34.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.361+0000 7f3bbb7fe640 1 -- 172.21.15.27:0/3752652873 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3bb4018a50 con 0x7f3bc4101990 2024-06-06T08:50:34.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.361+0000 7f3bbb7fe640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3bb00a2d00 0x7f3bb00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.362+0000 7f3bc259c640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3bb00a2d00 0x7f3bb00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.362+0000 7f3bbb7fe640 1 -- 172.21.15.27:0/3752652873 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f3bb40cfc90 con 0x7f3bc4101990 2024-06-06T08:50:34.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.362+0000 7f3bc259c640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3bb00a2d00 0x7f3bb00a51c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3bac005fd0 tx=0x7f3bac0093c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:34.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.365+0000 7f3bbb7fe640 1 -- 172.21.15.27:0/3752652873 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3bb4014030 con 0x7f3bc4101990 2024-06-06T08:50:34.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.549+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3bc41a4f90 con 0x7f3bc4101990 2024-06-06T08:50:34.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.549+0000 7f3bbb7fe640 1 -- 172.21.15.27:0/3752652873 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f3bb409c530 con 0x7f3bc4101990 2024-06-06T08:50:34.548 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-06-06T08:50:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3bb00a2d00 msgr2=0x7f3bb00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3bb00a2d00 0x7f3bb00a51c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3bac005fd0 tx=0x7f3bac0093c0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 msgr2=0x7f3bc41986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 0x7f3bc41986b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400c920 tx=0x7f3bb400cde0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 shutdown_connections 2024-06-06T08:50:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f3bb00a2d00 0x7f3bb00a51c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc41068c0 0x7f3bc419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc4101990 0x7f3bc41986b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 --2- 172.21.15.27:0/3752652873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc4100fd0 0x7f3bc4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.551+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 >> 172.21.15.27:0/3752652873 conn(0x7f3bc4076e30 msgr2=0x7f3bc40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:34.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.552+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 shutdown_connections 2024-06-06T08:50:34.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.552+0000 7f3bc89dc640 1 -- 172.21.15.27:0/3752652873 wait complete. 2024-06-06T08:50:34.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-06T08:50:34.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.669+0000 7f76fb5a9640 1 -- 172.21.15.27:0/3480373828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 msgr2=0x7f76f410e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.669+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/3480373828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 0x7f76f410e840 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009930 tx=0x7f76ec02a420 comp rx=0 tx=0).stop 2024-06-06T08:50:34.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 -- 172.21.15.27:0/3480373828 shutdown_connections 2024-06-06T08:50:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/3480373828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 0x7f76f410e840 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/3480373828 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76f4101970 0x7f76f4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/3480373828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f4107980 0x7f76f4107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 -- 172.21.15.27:0/3480373828 >> 172.21.15.27:0/3480373828 conn(0x7f76f40fd3e0 msgr2=0x7f76f40ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 -- 172.21.15.27:0/3480373828 shutdown_connections 2024-06-06T08:50:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 -- 172.21.15.27:0/3480373828 wait complete. 2024-06-06T08:50:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 Processor -- start 2024-06-06T08:50:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 -- start start 2024-06-06T08:50:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76f4101970 0x7f76f4073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 0x7f76f40741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.670+0000 7f76fb5a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f4107980 0x7f76f4074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76fb5a9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4111d90 con 0x7f76f4102330 2024-06-06T08:50:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76fb5a9640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4111c10 con 0x7f76f4101970 2024-06-06T08:50:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76fb5a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4111f10 con 0x7f76f4107980 2024-06-06T08:50:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f8b44640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 0x7f76f40741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f8b44640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 0x7f76f40741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35236/0 (socket says 172.21.15.27:35236) 2024-06-06T08:50:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f8b44640 1 -- 172.21.15.27:0/2333512953 learned_addr learned my addr 172.21.15.27:0/2333512953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f9345640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76f4101970 0x7f76f4073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f9b46640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f4107980 0x7f76f4074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f8b44640 1 -- 172.21.15.27:0/2333512953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f4107980 msgr2=0x7f76f4074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f8b44640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f4107980 0x7f76f4074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f8b44640 1 -- 172.21.15.27:0/2333512953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76f4101970 msgr2=0x7f76f4073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f8b44640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76f4101970 0x7f76f4073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f8b44640 1 -- 172.21.15.27:0/2333512953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f41a6750 con 0x7f76f4102330 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f9b46640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f4107980 0x7f76f4074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f8b44640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 0x7f76f40741e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f76e800d950 tx=0x7f76e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f27fc640 1 -- 172.21.15.27:0/2333512953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8017070 con 0x7f76f4102330 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f27fc640 1 -- 172.21.15.27:0/2333512953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e8009c30 con 0x7f76f4102330 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f41a6a40 con 0x7f76f4102330 2024-06-06T08:50:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76f27fc640 1 -- 172.21.15.27:0/2333512953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e800f6b0 con 0x7f76f4102330 2024-06-06T08:50:34.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.671+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41a6f90 con 0x7f76f4102330 2024-06-06T08:50:34.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.672+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f4068c10 con 0x7f76f4102330 2024-06-06T08:50:34.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.674+0000 7f76f27fc640 1 -- 172.21.15.27:0/2333512953 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f76e80109d0 con 0x7f76f4102330 2024-06-06T08:50:34.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.674+0000 7f76f27fc640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f76dc0a2d00 0x7f76dc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.674+0000 7f76f9345640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f76dc0a2d00 0x7f76dc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.674+0000 7f76f27fc640 1 -- 172.21.15.27:0/2333512953 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f76e80cf490 con 0x7f76f4102330 2024-06-06T08:50:34.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.675+0000 7f76f9345640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f76dc0a2d00 0x7f76dc0a51c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005fd0 tx=0x7f76e4009680 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:34.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.678+0000 7f76f27fc640 1 -- 172.21.15.27:0/2333512953 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f76e8009de0 con 0x7f76f4102330 2024-06-06T08:50:34.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.861+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76f4109ee0 con 0x7f76f4102330 2024-06-06T08:50:34.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.861+0000 7f76f27fc640 1 -- 172.21.15.27:0/2333512953 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7f76e809bca0 con 0x7f76f4102330 2024-06-06T08:50:34.860 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-06-06T08:50:34.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f76dc0a2d00 msgr2=0x7f76dc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f76dc0a2d00 0x7f76dc0a51c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005fd0 tx=0x7f76e4009680 comp rx=0 tx=0).stop 2024-06-06T08:50:34.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 msgr2=0x7f76f40741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 0x7f76f40741e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f76e800d950 tx=0x7f76e800de10 comp rx=0 tx=0).stop 2024-06-06T08:50:34.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 shutdown_connections 2024-06-06T08:50:34.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f76dc0a2d00 0x7f76dc0a51c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f4107980 0x7f76f4074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f4102330 0x7f76f40741e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 --2- 172.21.15.27:0/2333512953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76f4101970 0x7f76f4073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 >> 172.21.15.27:0/2333512953 conn(0x7f76f40fd3e0 msgr2=0x7f76f40fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 shutdown_connections 2024-06-06T08:50:34.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.863+0000 7f76fb5a9640 1 -- 172.21.15.27:0/2333512953 wait complete. 2024-06-06T08:50:34.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-06T08:50:34.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.982+0000 7f10941b7640 1 -- 172.21.15.27:0/3755392528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c107980 msgr2=0x7f108c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.982+0000 7f10941b7640 1 --2- 172.21.15.27:0/3755392528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c107980 0x7f108c107d60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1088009930 tx=0x7f108802a420 comp rx=0 tx=0).stop 2024-06-06T08:50:34.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.982+0000 7f10941b7640 1 -- 172.21.15.27:0/3755392528 shutdown_connections 2024-06-06T08:50:34.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.982+0000 7f10941b7640 1 --2- 172.21.15.27:0/3755392528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c06a960 0x7f108c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.982+0000 7f10941b7640 1 --2- 172.21.15.27:0/3755392528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f108c069fa0 0x7f108c06a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.982+0000 7f10941b7640 1 --2- 172.21.15.27:0/3755392528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c107980 0x7f108c107d60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.982+0000 7f10941b7640 1 -- 172.21.15.27:0/3755392528 >> 172.21.15.27:0/3755392528 conn(0x7f108c0766a0 msgr2=0x7f108c076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.982+0000 7f10941b7640 1 -- 172.21.15.27:0/3755392528 shutdown_connections 2024-06-06T08:50:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.982+0000 7f10941b7640 1 -- 172.21.15.27:0/3755392528 wait complete. 2024-06-06T08:50:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f10941b7640 1 Processor -- start 2024-06-06T08:50:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f10941b7640 1 -- start start 2024-06-06T08:50:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f10941b7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c069fa0 0x7f108c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f10941b7640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f108c06a960 0x7f108c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f10941b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c107980 0x7f108c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f10941b7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f108c111de0 con 0x7f108c069fa0 2024-06-06T08:50:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f10941b7640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f108c111c60 con 0x7f108c06a960 2024-06-06T08:50:34.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f10941b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f108c111f60 con 0x7f108c107980 2024-06-06T08:50:34.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1091f53640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c069fa0 0x7f108c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1091f53640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c069fa0 0x7f108c1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35246/0 (socket says 172.21.15.27:35246) 2024-06-06T08:50:34.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1091f53640 1 -- 172.21.15.27:0/1619590312 learned_addr learned my addr 172.21.15.27:0/1619590312 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:50:34.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1091752640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f108c06a960 0x7f108c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1092754640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c107980 0x7f108c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1091f53640 1 -- 172.21.15.27:0/1619590312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c107980 msgr2=0x7f108c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1091f53640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c107980 0x7f108c19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1091f53640 1 -- 172.21.15.27:0/1619590312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f108c06a960 msgr2=0x7f108c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1091f53640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f108c06a960 0x7f108c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.983+0000 7f1091f53640 1 -- 172.21.15.27:0/1619590312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c19d2b0 con 0x7f108c069fa0 2024-06-06T08:50:34.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.984+0000 7f1091f53640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c069fa0 0x7f108c1982c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f108802a940 tx=0x7f1088003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.984+0000 7f1082ffd640 1 -- 172.21.15.27:0/1619590312 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108803f070 con 0x7f108c069fa0 2024-06-06T08:50:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.984+0000 7f1082ffd640 1 -- 172.21.15.27:0/1619590312 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108803b460 con 0x7f108c069fa0 2024-06-06T08:50:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.984+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108c19d540 con 0x7f108c069fa0 2024-06-06T08:50:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.984+0000 7f1082ffd640 1 -- 172.21.15.27:0/1619590312 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088035030 con 0x7f108c069fa0 2024-06-06T08:50:34.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.984+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108c1a4e30 con 0x7f108c069fa0 2024-06-06T08:50:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.985+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108c1030d0 con 0x7f108c069fa0 2024-06-06T08:50:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.986+0000 7f1082ffd640 1 -- 172.21.15.27:0/1619590312 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1088042070 con 0x7f108c069fa0 2024-06-06T08:50:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.987+0000 7f1082ffd640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f10780a2d00 0x7f10780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.987+0000 7f1091752640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f10780a2d00 0x7f10780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.987+0000 7f1082ffd640 1 -- 172.21.15.27:0/1619590312 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f10880efdf0 con 0x7f108c069fa0 2024-06-06T08:50:34.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.987+0000 7f1091752640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f10780a2d00 0x7f10780a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f108c100730 tx=0x7f107c00a4c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:34.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:34.991+0000 7f1082ffd640 1 -- 172.21.15.27:0/1619590312 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f10880bc600 con 0x7f108c069fa0 2024-06-06T08:50:35.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.174+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f108c02d2e0 con 0x7f108c069fa0 2024-06-06T08:50:35.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.175+0000 7f1082ffd640 1 -- 172.21.15.27:0/1619590312 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7f10880bbf00 con 0x7f108c069fa0 2024-06-06T08:50:35.173 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-06-06T08:50:35.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f10780a2d00 msgr2=0x7f10780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f10780a2d00 0x7f10780a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f108c100730 tx=0x7f107c00a4c0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c069fa0 msgr2=0x7f108c1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c069fa0 0x7f108c1982c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f108802a940 tx=0x7f1088003c00 comp rx=0 tx=0).stop 2024-06-06T08:50:35.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 shutdown_connections 2024-06-06T08:50:35.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f10780a2d00 0x7f10780a51c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108c107980 0x7f108c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f108c06a960 0x7f108c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 --2- 172.21.15.27:0/1619590312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f108c069fa0 0x7f108c1982c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 >> 172.21.15.27:0/1619590312 conn(0x7f108c0766a0 msgr2=0x7f108c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:35.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 shutdown_connections 2024-06-06T08:50:35.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.177+0000 7f10941b7640 1 -- 172.21.15.27:0/1619590312 wait complete. 2024-06-06T08:50:35.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:35 smithi027 ceph-mon[32335]: pgmap v320: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T08:50:35.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:35 smithi027 ceph-mon[32335]: from='client.16629 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:50:35.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:35 smithi027 ceph-mon[32335]: from='client.16635 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:50:35.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:35 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2820043866' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:50:35.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:35 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3752652873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:50:35.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:35 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2333512953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T08:50:35.186 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-06T08:50:35.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 -- 172.21.15.27:0/4027812148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818100870 msgr2=0x7f6818100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 --2- 172.21.15.27:0/4027812148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818100870 0x7f6818100cf0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f680c009930 tx=0x7f680c02a420 comp rx=0 tx=0).stop 2024-06-06T08:50:35.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 -- 172.21.15.27:0/4027812148 shutdown_connections 2024-06-06T08:50:35.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 --2- 172.21.15.27:0/4027812148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818101230 0x7f681810d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 --2- 172.21.15.27:0/4027812148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818100870 0x7f6818100cf0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 --2- 172.21.15.27:0/4027812148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6818106880 0x7f6818106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 -- 172.21.15.27:0/4027812148 >> 172.21.15.27:0/4027812148 conn(0x7f68180779a0 msgr2=0x7f6818077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:35.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 -- 172.21.15.27:0/4027812148 shutdown_connections 2024-06-06T08:50:35.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 -- 172.21.15.27:0/4027812148 wait complete. 2024-06-06T08:50:35.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.295+0000 7f681db34640 1 Processor -- start 2024-06-06T08:50:35.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f681db34640 1 -- start start 2024-06-06T08:50:35.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f681db34640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818100870 0x7f6818197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:35.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f681db34640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6818101230 0x7f6818197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:35.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f681db34640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818106880 0x7f681819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:35.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f681db34640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6818110c90 con 0x7f6818106880 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f681db34640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f6818110b10 con 0x7f6818100870 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f681db34640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6818110e10 con 0x7f6818101230 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6817fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818106880 0x7f681819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6817fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818106880 0x7f681819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35264/0 (socket says 172.21.15.27:35264) 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6817fff640 1 -- 172.21.15.27:0/3310070313 learned_addr learned my addr 172.21.15.27:0/3310070313 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6816ffd640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6818101230 0x7f6818197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f68177fe640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818100870 0x7f6818197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6817fff640 1 -- 172.21.15.27:0/3310070313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6818101230 msgr2=0x7f6818197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6817fff640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6818101230 0x7f6818197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6817fff640 1 -- 172.21.15.27:0/3310070313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818100870 msgr2=0x7f6818197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6817fff640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818100870 0x7f6818197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6817fff640 1 -- 172.21.15.27:0/3310070313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681819c220 con 0x7f6818106880 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6816ffd640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6818101230 0x7f6818197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:50:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6817fff640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818106880 0x7f681819baf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f681000cad0 tx=0x7f681000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.296+0000 7f6814ff9640 1 -- 172.21.15.27:0/3310070313 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681000ae20 con 0x7f6818106880 2024-06-06T08:50:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.297+0000 7f6814ff9640 1 -- 172.21.15.27:0/3310070313 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6810005020 con 0x7f6818106880 2024-06-06T08:50:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.297+0000 7f6814ff9640 1 -- 172.21.15.27:0/3310070313 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681000f6f0 con 0x7f6818106880 2024-06-06T08:50:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.297+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681819c510 con 0x7f6818106880 2024-06-06T08:50:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.297+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6818068b30 con 0x7f6818106880 2024-06-06T08:50:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.297+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6818101fb0 con 0x7f6818106880 2024-06-06T08:50:35.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.299+0000 7f6814ff9640 1 -- 172.21.15.27:0/3310070313 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f68100051d0 con 0x7f6818106880 2024-06-06T08:50:35.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.300+0000 7f6814ff9640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f68000a2d00 0x7f68000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:35.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.300+0000 7f68177fe640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f68000a2d00 0x7f68000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:35.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.300+0000 7f6814ff9640 1 -- 172.21.15.27:0/3310070313 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f68100cfd20 con 0x7f6818106880 2024-06-06T08:50:35.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.300+0000 7f68177fe640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f68000a2d00 0x7f68000a51c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6808005fd0 tx=0x7f680800b510 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:35.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.304+0000 7f6814ff9640 1 -- 172.21.15.27:0/3310070313 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6810014030 con 0x7f6818106880 2024-06-06T08:50:35.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.488+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f68180695b0 con 0x7f6818106880 2024-06-06T08:50:35.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.488+0000 7f6814ff9640 1 -- 172.21.15.27:0/3310070313 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v109) ==== 103+0+11 (secure 0 0 0) 0x7f681009c530 con 0x7f6818106880 2024-06-06T08:50:35.487 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-06-06T08:50:35.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f68000a2d00 msgr2=0x7f68000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f68000a2d00 0x7f68000a51c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6808005fd0 tx=0x7f680800b510 comp rx=0 tx=0).stop 2024-06-06T08:50:35.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818106880 msgr2=0x7f681819baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818106880 0x7f681819baf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f681000cad0 tx=0x7f681000cf90 comp rx=0 tx=0).stop 2024-06-06T08:50:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 shutdown_connections 2024-06-06T08:50:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f68000a2d00 0x7f68000a51c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6818106880 0x7f681819baf0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6818101230 0x7f6818197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 --2- 172.21.15.27:0/3310070313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6818100870 0x7f6818197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 >> 172.21.15.27:0/3310070313 conn(0x7f68180779a0 msgr2=0x7f6818103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:35.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 shutdown_connections 2024-06-06T08:50:35.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.490+0000 7f681db34640 1 -- 172.21.15.27:0/3310070313 wait complete. 2024-06-06T08:50:35.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:35 smithi203 ceph-mon[36579]: pgmap v320: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T08:50:35.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:35 smithi203 ceph-mon[36579]: from='client.16629 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:50:35.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:35 smithi203 ceph-mon[36579]: from='client.16635 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:50:35.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:35 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2820043866' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:50:35.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:35 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3752652873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:50:35.492 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:35 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2333512953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T08:50:35.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-06T08:50:35.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:35 smithi176 ceph-mon[36786]: pgmap v320: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T08:50:35.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:35 smithi176 ceph-mon[36786]: from='client.16629 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:50:35.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:35 smithi176 ceph-mon[36786]: from='client.16635 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T08:50:35.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:35 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2820043866' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:50:35.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:35 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3752652873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T08:50:35.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:35 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2333512953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T08:50:35.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 -- 172.21.15.27:0/2064247728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04341035a0 msgr2=0x7f043410ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 --2- 172.21.15.27:0/2064247728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04341035a0 0x7f043410ba80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0424009900 tx=0x7f042402a480 comp rx=0 tx=0).stop 2024-06-06T08:50:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 -- 172.21.15.27:0/2064247728 shutdown_connections 2024-06-06T08:50:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 --2- 172.21.15.27:0/2064247728 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f043410bfc0 0x7f043410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 --2- 172.21.15.27:0/2064247728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04341035a0 0x7f043410ba80 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 --2- 172.21.15.27:0/2064247728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0434102c80 0x7f0434103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 -- 172.21.15.27:0/2064247728 >> 172.21.15.27:0/2064247728 conn(0x7f0434077a00 msgr2=0x7f0434077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 -- 172.21.15.27:0/2064247728 shutdown_connections 2024-06-06T08:50:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 -- 172.21.15.27:0/2064247728 wait complete. 2024-06-06T08:50:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.606+0000 7f0439761640 1 Processor -- start 2024-06-06T08:50:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0439761640 1 -- start start 2024-06-06T08:50:35.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0439761640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0434102c80 0x7f04341008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:35.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0439761640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04341035a0 0x7f0434100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:35.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0439761640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043410bfc0 0x7f0434101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:35.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0439761640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0434111d90 con 0x7f0434102c80 2024-06-06T08:50:35.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0439761640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f0434111c10 con 0x7f04341035a0 2024-06-06T08:50:35.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0439761640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0434111f10 con 0x7f043410bfc0 2024-06-06T08:50:35.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0432ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0434102c80 0x7f04341008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0432ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0434102c80 0x7f04341008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35288/0 (socket says 172.21.15.27:35288) 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0432ffd640 1 -- 172.21.15.27:0/3265178681 learned_addr learned my addr 172.21.15.27:0/3265178681 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f04337fe640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043410bfc0 0x7f0434101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f04327fc640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04341035a0 0x7f0434100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0432ffd640 1 -- 172.21.15.27:0/3265178681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043410bfc0 msgr2=0x7f0434101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0432ffd640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043410bfc0 0x7f0434101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0432ffd640 1 -- 172.21.15.27:0/3265178681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04341035a0 msgr2=0x7f0434100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0432ffd640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04341035a0 0x7f0434100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0432ffd640 1 -- 172.21.15.27:0/3265178681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04341a66f0 con 0x7f0434102c80 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f04337fe640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043410bfc0 0x7f0434101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f0432ffd640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0434102c80 0x7f04341008d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f041c00c920 tx=0x7f041c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.607+0000 7f042bfff640 1 -- 172.21.15.27:0/3265178681 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c007c70 con 0x7f0434102c80 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.608+0000 7f042bfff640 1 -- 172.21.15.27:0/3265178681 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f041c007e10 con 0x7f0434102c80 2024-06-06T08:50:35.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.608+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04341a69e0 con 0x7f0434102c80 2024-06-06T08:50:35.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.608+0000 7f042bfff640 1 -- 172.21.15.27:0/3265178681 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c018890 con 0x7f0434102c80 2024-06-06T08:50:35.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.608+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04341a6f80 con 0x7f0434102c80 2024-06-06T08:50:35.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.609+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0434068c10 con 0x7f0434102c80 2024-06-06T08:50:35.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.610+0000 7f042bfff640 1 -- 172.21.15.27:0/3265178681 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f041c018a70 con 0x7f0434102c80 2024-06-06T08:50:35.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.611+0000 7f042bfff640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f04200a2d80 0x7f04200a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:50:35.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.611+0000 7f042bfff640 1 -- 172.21.15.27:0/3265178681 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f041c0cf9c0 con 0x7f0434102c80 2024-06-06T08:50:35.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.611+0000 7f04327fc640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f04200a2d80 0x7f04200a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:50:35.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.611+0000 7f04327fc640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f04200a2d80 0x7f04200a5240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0434102360 tx=0x7f0424031490 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:50:35.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.615+0000 7f042bfff640 1 -- 172.21.15.27:0/3265178681 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f041c0d1e20 con 0x7f0434102c80 2024-06-06T08:50:35.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.849+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f04341a7210 con 0x7f0434102c80 2024-06-06T08:50:35.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.857+0000 7f042bfff640 1 -- 172.21.15.27:0/3265178681 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740904 (secure 0 0 0) 0x7f041c014030 con 0x7f0434102c80 2024-06-06T08:50:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:50:35.865 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:50:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 26, 2024-06-06T08:50:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "active_gid": 15201, 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "active_name": "x", 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addrs": { 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1196153485 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-06-06T08:50:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1196153485 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addr": "172.21.15.27:6801/1196153485", 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "active_change": "2024-06-06T08:40:17.281117+0000", 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "active_mgr_features": 4540701547738038271, 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "available": true, 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "standbys": [ 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24883, 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "z", 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540701547738038271, 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-06-06T08:50:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.871 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.873 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-06-06T08:50:35.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.880 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:35.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.882 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.884 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.887 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-06-06T08:50:35.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:35.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.890 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.891 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:35.892 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.894 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-06-06T08:50:35.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-06-06T08:50:35.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.899 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:35.900 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.901 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.903 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T08:50:35.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T08:50:35.906 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-06-06T08:50:35.907 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.908 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.911 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.912 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.914 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.915 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-06-06T08:50:35.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.920 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-06-06T08:50:35.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-06-06T08:50:35.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-06-06T08:50:35.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-06-06T08:50:35.924 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-06-06T08:50:35.925 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-06-06T08:50:35.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-06-06T08:50:35.927 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T08:50:35.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.932 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T08:50:35.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T08:50:35.940 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:35.941 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.942 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.943 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.944 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.950 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-06-06T08:50:35.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-06-06T08:50:35.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-06-06T08:50:35.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-06-06T08:50:35.956 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.959 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-06-06T08:50:35.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.962 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:35.963 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.964 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.965 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:35.967 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.968 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.970 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T08:50:35.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T08:50:35.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T08:50:35.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.977 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T08:50:35.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T08:50:35.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T08:50:35.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T08:50:35.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T08:50:35.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T08:50:35.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T08:50:35.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.985 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.991 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.992 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:35.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.005 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-06-06T08:50:36.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.011 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:36.014 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.020 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.025 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.026 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.032 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.034 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.037 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.038 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.039 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.041 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.046 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.050 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.051 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.051 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.051 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.051 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.051 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.051 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.056 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.059 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-06-06T08:50:36.060 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.061 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.062 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.064 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.065 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.066 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.067 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.068 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.069 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.072 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:50:36.073 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.074 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.075 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.077 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.079 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.080 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.081 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-06-06T08:50:36.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-06-06T08:50:36.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.088 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.090 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.093 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.094 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.097 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.098 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.100 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.101 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-06-06T08:50:36.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-06-06T08:50:36.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.105 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.106 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.110 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.113 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.116 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.117 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.118 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.119 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-06-06T08:50:36.120 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.122 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.124 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.125 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-06-06T08:50:36.127 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.128 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.129 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.132 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.134 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.135 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.136 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.137 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.138 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.139 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.140 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.141 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.145 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.146 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.147 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.149 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.150 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.154 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.155 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-06-06T08:50:36.157 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-06-06T08:50:36.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.161 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.162 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-06-06T08:50:36.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.167 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-06-06T08:50:36.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.173 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.177 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.180 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.182 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.183 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.186 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.187 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.191 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.193 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.198 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.201 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.202 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-06-06T08:50:36.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-06-06T08:50:36.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.212 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:36.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.217 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.228 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.229 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.232 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.233 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.234 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.239 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.242 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-06-06T08:50:36.244 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.245 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T08:50:36.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.249 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.258 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.261 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.262 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.267 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.275 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.281 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.285 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24887, 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "y", 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540701547738038271, 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-06-06T08:50:36.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.296 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.298 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.304 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-06-06T08:50:36.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-06-06T08:50:36.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.310 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.311 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.316 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.317 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.318 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T08:50:36.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.323 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-06-06T08:50:36.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-06-06T08:50:36.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.329 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-06-06T08:50:36.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-06-06T08:50:36.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.334 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T08:50:36.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T08:50:36.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T08:50:36.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T08:50:36.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.348 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T08:50:36.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T08:50:36.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T08:50:36.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T08:50:36.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.359 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-06-06T08:50:36.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T08:50:36.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-06-06T08:50:36.364 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T08:50:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T08:50:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-06-06T08:50:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-06-06T08:50:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T08:50:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T08:50:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T08:50:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T08:50:36.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-06-06T08:50:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-06-06T08:50:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.513 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.527 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.528 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.529 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.533 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-06-06T08:50:36.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-06-06T08:50:36.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-06-06T08:50:36.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-06-06T08:50:36.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.544 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.545 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-06-06T08:50:36.549 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-06-06T08:50:36.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-06-06T08:50:36.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.556 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-06-06T08:50:36.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.563 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.565 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.567 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.568 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.569 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.573 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.576 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T08:50:36.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.578 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.579 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.581 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-06-06T08:50:36.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.588 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.590 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-06-06T08:50:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.595 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.598 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.603 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-06-06T08:50:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-06-06T08:50:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-06-06T08:50:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-06-06T08:50:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-06-06T08:50:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-06-06T08:50:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.678 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.679 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.681 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.682 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-06-06T08:50:36.683 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-06-06T08:50:36.684 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.686 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.687 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.689 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.691 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.692 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.694 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.695 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.697 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.698 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.699 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.703 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-06-06T08:50:36.704 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-06-06T08:50:36.705 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.706 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.707 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.711 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-06-06T08:50:36.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.713 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.714 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.715 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.717 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.718 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.719 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.721 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.722 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.724 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.725 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.727 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.729 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.730 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.731 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.732 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.733 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T08:50:36.735 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.736 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.737 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.739 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.740 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.741 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.743 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.744 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.745 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.746 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.747 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.748 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.749 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.750 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.751 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.753 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-06-06T08:50:36.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-06-06T08:50:36.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.756 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.757 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.758 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.759 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.760 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.761 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.764 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.767 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.768 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.771 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.777 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.778 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.780 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.783 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.784 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.786 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.787 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.788 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.789 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.790 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.791 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: "modules": [ 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard", 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: "iostat", 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: "nfs", 2024-06-06T08:50:36.792 INFO:teuthology.orchestra.run.smithi027.stdout: "restful", 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_schedule" 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.793 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.795 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.796 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.797 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.798 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-06-06T08:50:36.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.800 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-06-06T08:50:36.801 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.804 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T08:50:36.806 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-06-06T08:50:36.807 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.808 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-06-06T08:50:36.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-06-06T08:50:36.811 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-06-06T08:50:36.812 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.813 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-06-06T08:50:36.814 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-06-06T08:50:36.815 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.816 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-06-06T08:50:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:36.826 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.827 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.828 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.829 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.830 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-06-06T08:50:36.831 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-06-06T08:50:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-06-06T08:50:36.833 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.834 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-06-06T08:50:36.835 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.836 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.837 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.838 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:36.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-06-06T08:50:36.840 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.841 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.842 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.843 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.844 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.845 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.847 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.848 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.849 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.850 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.851 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.853 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.854 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.855 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.856 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-06-06T08:50:36.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-06-06T08:50:36.859 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.860 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.861 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.863 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.864 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.866 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.867 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.868 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.870 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.871 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.871 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.871 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.871 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.873 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.876 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T08:50:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-06-06T08:50:36.880 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.881 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.882 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T08:50:36.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T08:50:36.884 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T08:50:36.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T08:50:36.886 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T08:50:36.887 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-06-06T08:50:36.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T08:50:36.889 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T08:50:36.890 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T08:50:36.891 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T08:50:36.892 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.893 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.896 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.898 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.900 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.903 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T08:50:36.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.906 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.909 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.910 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.911 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.912 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.916 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.917 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.918 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T08:50:36.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.924 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.926 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.931 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.934 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.937 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.938 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.939 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.941 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.942 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.943 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.944 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.945 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.947 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.949 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.953 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.956 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.957 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.959 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-06-06T08:50:36.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.961 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.962 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-06-06T08:50:36.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.966 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.967 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.968 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.972 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-06-06T08:50:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.983 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.985 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.990 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-06-06T08:50:36.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-06-06T08:50:36.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-06-06T08:50:36.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.994 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.996 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:36.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:36.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.000 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T08:50:37.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.010 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.013 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.015 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.016 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.021 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.025 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.030 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.034 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.037 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.038 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.039 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.041 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.043 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.046 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.048 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.049 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.051 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.052 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.053 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-06-06T08:50:37.054 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.056 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.057 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.058 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.059 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-06-06T08:50:37.061 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.062 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.063 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.064 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.066 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.071 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.072 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.074 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.075 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.075 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.077 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.080 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.081 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-06-06T08:50:37.083 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.084 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.085 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.087 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.089 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.090 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.091 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.094 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.096 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.098 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.101 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.102 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.103 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.105 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.106 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.109 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.110 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.111 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.112 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.114 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.115 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.117 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.119 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.122 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.123 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.124 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.125 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.127 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.130 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.132 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.133 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.135 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.136 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.140 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.141 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.144 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.145 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.148 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.149 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.150 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.154 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.157 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.161 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.163 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.168 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.169 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.172 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.178 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.180 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.181 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.182 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.183 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.184 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.185 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.188 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-06-06T08:50:37.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.190 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.191 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.193 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.198 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.200 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.202 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.204 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.209 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.211 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.212 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.213 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T08:50:37.217 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-06-06T08:50:37.221 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.222 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-06-06T08:50:37.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-06-06T08:50:37.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.225 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-06-06T08:50:37.227 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.228 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-06-06T08:50:37.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.230 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.231 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "services": { 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard": "https://172.21.15.27:8443/" 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "always_on_modules": { 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "octopus": [ 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-06-06T08:50:37.233 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "pacific": [ 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-06-06T08:50:37.234 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "quincy": [ 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-06-06T08:50:37.235 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "reef": [ 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "squid": [ 2024-06-06T08:50:37.236 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "last_failure_osd_epoch": 100, 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "active_clients": [ 2024-06-06T08:50:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3188351915 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "libcephsqlite", 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-06-06T08:50:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3723051644 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2162985586 2024-06-06T08:50:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3043485527 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2434943926 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-06-06T08:50:37.241 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-06-06T08:50:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3235169508 2024-06-06T08:50:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:37.242 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-06-06T08:50:37.242 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:50:37.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:36 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/1619590312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T08:50:37.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:36 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3310070313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T08:50:37.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:36 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3265178681' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T08:50:37.244 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:36 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/1619590312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T08:50:37.244 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:36 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3310070313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T08:50:37.244 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:36 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3265178681' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f04200a2d80 msgr2=0x7f04200a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f04200a2d80 0x7f04200a5240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0434102360 tx=0x7f0424031490 comp rx=0 tx=0).stop 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0434102c80 msgr2=0x7f04341008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0434102c80 0x7f04341008d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f041c00c920 tx=0x7f041c00cde0 comp rx=0 tx=0).stop 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 shutdown_connections 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f04200a2d80 0x7f04200a5240 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043410bfc0 0x7f0434101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04341035a0 0x7f0434100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 --2- 172.21.15.27:0/3265178681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0434102c80 0x7f04341008d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.862+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 >> 172.21.15.27:0/3265178681 conn(0x7f0434077a00 msgr2=0x7f04340fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.863+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 shutdown_connections 2024-06-06T08:50:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:50:35.863+0000 7f0439761640 1 -- 172.21.15.27:0/3265178681 wait complete. 2024-06-06T08:50:37.307 DEBUG:tasks.check_counter:Getting stats from a 2024-06-06T08:50:37.307 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-06T08:50:37.307 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:36 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/1619590312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T08:50:37.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:36 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3310070313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T08:50:37.308 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:36 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3265178681' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T08:50:37.335 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:37 smithi027 ceph-mon[32335]: pgmap v321: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:50:37.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:37 smithi203 ceph-mon[36579]: pgmap v321: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:50:37.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:37 smithi176 ceph-mon[36786]: pgmap v321: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:50:37.569 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:50:39.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:39 smithi027 ceph-mon[32335]: pgmap v322: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:50:39.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:39 smithi203 ceph-mon[36579]: pgmap v322: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:50:39.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:39 smithi176 ceph-mon[36786]: pgmap v322: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:50:39.965 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:50:39.965 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 246, 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 235, 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1587568, 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 144704, 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.048193756, 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.015637826, 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.035233179, 2024-06-06T08:50:39.966 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.004609454, 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 234, 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.011216555, 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000047933 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1587568, 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 144704 2024-06-06T08:50:39.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 6, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 182, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 3392, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 188048, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.015394356, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.010521544, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.002338859, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:39.968 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 180, 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.013320660, 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000074003 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 188048 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.969 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.000422429, 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000356388, 2024-06-06T08:50:39.970 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:50:39.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:50:39.972 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:50:39.973 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-06-06T08:50:39.974 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148731, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 31, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-06-06T08:50:39.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-06-06T08:50:39.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-06-06T08:50:39.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:50:39.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:50:39.976 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-06-06T08:50:39.976 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-06-06T08:50:39.976 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-06-06T08:50:39.976 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 248, 2024-06-06T08:50:39.977 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1552883, 2024-06-06T08:50:39.978 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:39.978 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 248, 2024-06-06T08:50:39.978 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-06-06T08:50:39.978 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-06-06T08:50:39.978 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 248, 2024-06-06T08:50:39.978 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1552883, 2024-06-06T08:50:39.978 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-06-06T08:50:39.978 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:39.978 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:39.979 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:39.979 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:39.979 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:39.979 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:50:40.706 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-06-06T08:50:40.706 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-06-06T08:50:40.707 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-06-06T08:50:40.707 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-06-06T08:50:40.707 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-06-06T08:50:40.707 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-06-06T08:50:40.707 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-06-06T08:50:40.707 DEBUG:tasks.check_counter:Getting stats from d 2024-06-06T08:50:40.707 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-06T08:50:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:50:41.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:41 smithi203 ceph-mon[36579]: pgmap v323: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:41 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:50:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:41 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:41 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:41 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:41 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:41 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:41 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.312 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:41 smithi027 ceph-mon[32335]: pgmap v323: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:41.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:41 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:50:41.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:41 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:41 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:41 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:41 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:41 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:41 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:41 smithi176 ceph-mon[36786]: pgmap v323: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:41.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:41 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T08:50:41.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:41 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:41 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:41 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:41 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:41 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:41.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:41 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:42.783 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:50:42.783 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 4, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 4, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 3072, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 448, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.001583989, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000037681, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000599666, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3, 2024-06-06T08:50:42.784 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000800442, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000266814 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 184, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 188304, 2024-06-06T08:50:42.785 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.015365567, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.010971211, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.001824162, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 181, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.015415656, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000085169 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:42.786 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 188304 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 253, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 242, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1590096, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 148880, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.049025400, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.015837319, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.037239001, 2024-06-06T08:50:42.787 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.004767801, 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 241, 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.011154020, 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000046282 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1590096, 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 148880 2024-06-06T08:50:42.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:50:42.789 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:50:42.790 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:50:42.791 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148714, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 30, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:50:42.792 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-06-06T08:50:42.793 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 255, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1554430, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 255, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 255, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1554430, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:42.794 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:50:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:43 smithi027 ceph-mon[32335]: pgmap v324: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:50:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:43 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:43 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:43 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:43 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:43 smithi203 ceph-mon[36579]: pgmap v324: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:50:43.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:43 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:43 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:43 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:43 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:43 smithi176 ceph-mon[36786]: pgmap v324: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-06T08:50:43.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:43 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:43 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:43 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:43.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:43 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:44.104 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-06T08:50:44.104 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-06T08:50:44.104 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-06-06T08:50:44.104 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-06-06T08:50:44.104 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-06-06T08:50:44.104 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-06-06T08:50:44.104 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-06-06T08:50:44.104 DEBUG:tasks.check_counter:Getting stats from g 2024-06-06T08:50:44.105 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-06T08:50:44.386 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:50:44.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:44 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:44 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:44 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:44 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:44 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:44 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:44 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.391 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:44 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:44 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:44 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:44 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:44 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:44 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:44 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:44 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:44 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:44 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:44 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:44 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:44 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:44 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:44 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:44 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:44.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:44 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:45.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:45 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi176 to 176.2M 2024-06-06T08:50:45.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:45 smithi203 ceph-mon[36579]: Unable to set osd_memory_target on smithi176 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-06-06T08:50:45.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:45 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-06-06T08:50:45.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:45 smithi203 ceph-mon[36579]: Unable to set osd_memory_target on smithi203 to 184758067: error parsing value: Value '184758067' is below minimum 939524096 2024-06-06T08:50:45.403 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:45 smithi203 ceph-mon[36579]: pgmap v325: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:45.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:45 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:45.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:45 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:45 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi176 to 176.2M 2024-06-06T08:50:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:45 smithi027 ceph-mon[32335]: Unable to set osd_memory_target on smithi176 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-06-06T08:50:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:45 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-06-06T08:50:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:45 smithi027 ceph-mon[32335]: Unable to set osd_memory_target on smithi203 to 184758067: error parsing value: Value '184758067' is below minimum 939524096 2024-06-06T08:50:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:45 smithi027 ceph-mon[32335]: pgmap v325: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:45 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:45.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:45 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:45.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:45 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi176 to 176.2M 2024-06-06T08:50:45.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:45 smithi176 ceph-mon[36786]: Unable to set osd_memory_target on smithi176 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-06-06T08:50:45.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:45 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-06-06T08:50:45.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:45 smithi176 ceph-mon[36786]: Unable to set osd_memory_target on smithi203 to 184758067: error parsing value: Value '184758067' is below minimum 939524096 2024-06-06T08:50:45.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:45 smithi176 ceph-mon[36786]: pgmap v325: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:45.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:45 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:45.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:45 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:46.113 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:46 smithi027 ceph-mon[32335]: pgmap v326: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:46.114 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:50:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:50:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 252, 2024-06-06T08:50:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 239, 2024-06-06T08:50:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1591184, 2024-06-06T08:50:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 146432, 2024-06-06T08:50:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-06-06T08:50:46.114 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.044354082, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.014495279, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.032580769, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.004601816, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 239, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.011831872, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000049505 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:46.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1591184, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 146432 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 182, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 187952, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.014547677, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.010166049, 2024-06-06T08:50:46.116 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.001792855, 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 179, 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.017743784, 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000099127 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 187952 2024-06-06T08:50:46.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.000446077, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000374203, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:46.118 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.119 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:50:46.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:50:46.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148714, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 30, 2024-06-06T08:50:46.122 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-06-06T08:50:46.123 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 250, 2024-06-06T08:50:46.124 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1553143, 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 250, 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 250, 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1553143, 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:46.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:46.126 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:50:46.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:46 smithi203 ceph-mon[36579]: pgmap v326: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:46.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:46 smithi176 ceph-mon[36786]: pgmap v326: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:46.774 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-06-06T08:50:46.774 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-06-06T08:50:46.774 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-06-06T08:50:46.775 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-06-06T08:50:46.775 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-06-06T08:50:46.775 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-06-06T08:50:46.775 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-06-06T08:50:46.775 DEBUG:tasks.check_counter:Getting stats from j 2024-06-06T08:50:46.775 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-06T08:50:47.052 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:50:47.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:47 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:47 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:47 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.406 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:47 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.406 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:47 smithi027 ceph-mon[32335]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-06-06T08:50:47.406 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:47 smithi027 ceph-mon[32335]: Unable to set osd_memory_target on smithi027 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T08:50:47.406 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:47 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:50:47.406 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:47 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:50:47.406 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:47 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:47.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:47 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:47 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:47 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:47 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:47 smithi203 ceph-mon[36579]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-06-06T08:50:47.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:47 smithi203 ceph-mon[36579]: Unable to set osd_memory_target on smithi027 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T08:50:47.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:47 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:50:47.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:47 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:50:47.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:47 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:47.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:47 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:47 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:47 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:47 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T08:50:47.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:47 smithi176 ceph-mon[36786]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-06-06T08:50:47.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:47 smithi176 ceph-mon[36786]: Unable to set osd_memory_target on smithi027 to 184710041: error parsing value: Value '184710041' is below minimum 939524096 2024-06-06T08:50:47.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:47 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T08:50:47.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:47 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T08:50:47.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:47 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' 2024-06-06T08:50:48.202 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:48 smithi027 ceph-mon[32335]: pgmap v327: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-06T08:50:48.408 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:48 smithi176 ceph-mon[36786]: pgmap v327: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-06T08:50:48.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:48 smithi203 ceph-mon[36579]: pgmap v327: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-06T08:50:48.771 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-06-06T08:50:48.771 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:50:48.771 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.001340502, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000369343, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.772 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 187, 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-06-06T08:50:48.773 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 188832, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.016377007, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.011560214, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.001958656, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 184, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.016296964, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000088570 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.774 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 188832 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 259, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 246, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1593712, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 150608, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-06-06T08:50:48.775 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.049761629, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.016262769, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.037693365, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.004966482, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 246, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.013768836, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000055970 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:48.776 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1593712, 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 150608 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:50:48.777 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-06-06T08:50:48.778 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:50:48.779 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148731, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 31, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-06-06T08:50:48.780 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-06-06T08:50:48.781 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 257, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1554690, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 257, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-06-06T08:50:48.782 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-06-06T08:50:48.783 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 257, 2024-06-06T08:50:48.783 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1554690, 2024-06-06T08:50:48.783 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-06-06T08:50:48.783 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-06-06T08:50:48.783 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-06-06T08:50:48.783 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-06-06T08:50:48.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:48.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-06-06T08:50:48.783 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-06-06T08:50:49.477 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-06T08:50:49.477 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-06T08:50:49.477 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-06-06T08:50:49.477 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-06-06T08:50:49.477 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-06-06T08:50:49.477 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-06-06T08:50:49.477 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-06-06T08:50:49.477 DEBUG:tasks.check_counter:Getting stats from b 2024-06-06T08:50:49.477 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-06T08:50:49.725 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:50:50.884 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:50 smithi176 ceph-mon[36786]: pgmap v328: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:50.955 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:50 smithi027 ceph-mon[32335]: pgmap v328: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:50.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:50 smithi203 ceph-mon[36579]: pgmap v328: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 47740, 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 62307, 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 14796718, 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 110119560, 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 19, 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 4, 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 3.340165392, 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 1.354699737, 2024-06-06T08:50:51.604 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 1.260705218, 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.725820337, 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 62308, 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 15.174168799, 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000243534 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 61026, 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.011500810, 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000188 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 354304, 2024-06-06T08:50:51.605 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 90112 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 45822, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 61199, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 15088223, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 93811533, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 100, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 8, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 3.706235724, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 1.493803371, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 1.292047608, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.947532527, 2024-06-06T08:50:51.606 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 61199, 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 19.672801349, 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000321456 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 58240, 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.011227283, 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000192 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 488384 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:50:51.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 94264, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 149158, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 34657195, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 130553398, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 23, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 8, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 5.546091925, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 2.379733197, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 15.929849179, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.804249405, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 149152, 2024-06-06T08:50:51.608 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 25.905698244, 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000173686 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 169085, 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.026788142, 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000158 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1817040, 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 195504 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 1, 2024-06-06T08:50:51.609 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-MDSRank": { 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 18431, 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 51.351930688, 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002786171 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-PurgeQueue": { 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-06-06T08:50:51.610 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 18202, 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 3.847470381, 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000211376 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "request": 39542, 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "reply": 39348, 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "reply_latency": { 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 39348, 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 240.110315489, 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.006102224 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.611 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_reply": 0, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "forward": 194, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_complete": 15, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_keys": 1, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_commit": 5143, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_split": 14, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_merge": 0, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes": 25387, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_top": 16067, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_bottom": 8886, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pin_tail": 434, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pinned": 2058, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_expired": 3238, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_with_caps": 3, 2024-06-06T08:50:51.612 INFO:teuthology.orchestra.run.smithi176.stdout: "caps": 3, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "subtrees": 42, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse": 55191, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_hit": 51639, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_forward": 106, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_discover": 325, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_remote_ino": 2, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_lock": 20, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "load_cent": 0, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "q": 0, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "exported": 217, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "exported_inodes": 2487, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "imported": 112, 2024-06-06T08:50:51.613 INFO:teuthology.orchestra.run.smithi176.stdout: "imported_inodes": 1631, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_dir_fetch": 0, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_backtrace_fetch": 9887, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_peer_discover": 2, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rfiles": 4, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rbytes": 234, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rsnaps": 6, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_fetch": 9887, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_set_tag": 0, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_inodes": 1205, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_base_inodes": 5, 2024-06-06T08:50:51.614 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dirfrag_rstats": 1200, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_file_inodes": 8682, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_inode_file_caps": 277, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_revoke": 14404, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_grant": 28000, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_trunc": 188, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps": 30127, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps_dirty": 15557, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_cap_release": 509, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "process_request_cap_release": 17638 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache": { 2024-06-06T08:50:51.615 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update": 2605, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update_receipt": 509, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_try_discover": 338, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_send_discover": 513, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_handle_discover": 5724, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays": 1423, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_delayed": 0, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_created": 11539, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_enqueued": 10074, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_reintegrated": 84, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_migrated": 5, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_processing": 0, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:50:51.616 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_started": 0, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_completed": 0, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_path": 4, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_inode": 16161, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_enqueue_scrub": 5, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_exportdir": 218, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_flush": 0, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragmentdir": 14, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragstats": 0, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_inodestats": 0 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log": { 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "evlrg": 0, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "evadd": 61263, 2024-06-06T08:50:51.617 INFO:teuthology.orchestra.run.smithi176.stdout: "evex": 60344, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "evtrm": 60344, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "ev": 919, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "evexg": 0, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "evexd": 3262, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "segadd": 98, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "segex": 97, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "segtrm": 97, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "seg": 1, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "segmjr": 1, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "segexg": 0, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "segexd": 6, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "expos": 168807228, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "wrpos": 171174501, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "rdpos": 0, 2024-06-06T08:50:51.618 INFO:teuthology.orchestra.run.smithi176.stdout: "jlat": { 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 10277, 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 597.236604894, 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.058113905 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "replayed": 0 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mem": { 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "ino": 11994, 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "ino+": 24778, 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "ino-": 12784, 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "dir": 1459, 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "dir+": 2003, 2024-06-06T08:50:51.619 INFO:teuthology.orchestra.run.smithi176.stdout: "dir-": 544, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "dn": 25387, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "dn+": 39643, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "dn-": 14256, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "cap": 3, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "cap+": 23824, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "cap-": 23821, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "rss": 206640, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "heap": 231804 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server": { 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_client_request": 50498, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_server_request": 2287, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_request": 39542, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_session": 468, 2024-06-06T08:50:51.620 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_peer_request": 4595, 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "req_create_latency": { 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 11396, 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 28.460598073, 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002497419 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getattr_latency": { 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 268, 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 5.743593733, 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.021431319 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getfilelock_latency": { 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "req_link_latency": { 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 180, 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 18.652725325, 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.103626251 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookup_latency": { 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 11074, 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 25.999935120, 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002347835 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookuphash_latency": { 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.622 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupino_latency": { 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000198838, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000198838 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupname_latency": { 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupparent_latency": { 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.623 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.624 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupsnap_latency": { 2024-06-06T08:50:51.624 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.624 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.624 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.624 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lssnap_latency": { 2024-06-06T08:50:51.624 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 10, 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.004104100, 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000410410 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mkdir_latency": { 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1631, 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 20.346373451, 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.012474784 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mknod_latency": { 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 383, 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1.139228240, 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002974486 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.625 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mksnap_latency": { 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 10, 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1.932021226, 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.193202122 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "req_open_latency": { 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 53, 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.040995511, 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000773500 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "req_readdir_latency": { 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1533, 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 2.735852228, 2024-06-06T08:50:51.626 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.001784639 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rename_latency": { 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 423, 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 9.908393806, 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.023424098 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "req_renamesnap_latency": { 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "req_snapdiff_latency": { 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmdir_latency": { 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1616, 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 71.750083689, 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.044399804 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmsnap_latency": { 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 4, 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.019009923, 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.004752480 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmxattr_latency": { 2024-06-06T08:50:51.628 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setattr_latency": { 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 207, 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 10.624579949, 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.051326473 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setdirlayout_latency": { 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.629 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setfilelock_latency": { 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setlayout_latency": { 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setxattr_latency": { 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 9, 2024-06-06T08:50:51.630 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.022689536, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002521059 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "req_symlink_latency": { 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 368, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1.020659334, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002773530 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "req_unlink_latency": { 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 10182, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 41.709273407, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.004096373 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:50:51.631 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getvxattr_latency": { 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessions": { 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "session_count": 2, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "session_add": 7, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "session_remove": 5, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_open": 2, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_stale": 0, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "total_load": 778, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "average_load": 389, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avg_session_uptime": 613, 2024-06-06T08:50:51.632 INFO:teuthology.orchestra.run.smithi176.stdout: "mdthresh_evicted": 0 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 960, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 960, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:50:51.633 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:50:51.634 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:50:51.635 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 11997489, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 14359, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 9712, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 187, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:50:51.636 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 51117211, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 1272669, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter": { 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "op_active": 0, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "op_laggy": 0, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send": 70108, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send_bytes": 210293794, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "op_resend": 31, 2024-06-06T08:50:51.637 INFO:teuthology.orchestra.run.smithi176.stdout: "op_reply": 70108, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "op_latency": { 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 70108, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 13492.484755130, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.192452854 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "op_inflight": 0, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "oplen_avg": { 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 70108, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 117198 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "op": 70108, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "op_r": 10208, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "op_w": 59900, 2024-06-06T08:50:51.638 INFO:teuthology.orchestra.run.smithi176.stdout: "op_rmw": 0, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "op_pg": 0, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_stat": 3591, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_create": 22883, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_read": 269, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_write": 10541, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writefull": 221, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writesame": 0, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_append": 0, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_zero": 32, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_truncate": 14, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_delete": 20641, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_mapext": 0, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_sparse_read": 0, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_clonerange": 0, 2024-06-06T08:50:51.639 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_getxattr": 9907, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_setxattr": 36763, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_rmxattr": 0, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_call": 0, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_watch": 0, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_notify": 0, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls": 0, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:50:51.640 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_other": 221, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_active": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_send": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_resend": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_ping": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_active": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_send": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_resend": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_active": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_send": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_resend": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_active": 0, 2024-06-06T08:50:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_send": 0, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_resend": 0, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "command_active": 0, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "command_send": 0, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "command_resend": 0, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "map_epoch": 109, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "map_full": 0, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "map_inc": 21, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_sessions": 12, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_open": 12, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_close": 0, 2024-06-06T08:50:51.642 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_laggy": 0, 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_wr": 10316, 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_rd": 63, 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_del": 1736 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "oft": { 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_objs": 1, 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_updates": 0, 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_removes": 0 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "osd-slow-ops": { 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_ops_count": 15653 2024-06-06T08:50:51.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "purge_queue": { 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops": 0, 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops_high_water": 23, 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing": 0, 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_high_water": 16, 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed_ops": 20083, 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed": 10074, 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_item_in_journal": 0 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:51.644 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 187822, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 47566183, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 187822, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 187822, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 47566183, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.645 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_bytes": { 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 70108, 2024-06-06T08:50:51.646 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 211540344, 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 70108, 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 211540344, 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_ops": { 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 1024, 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:51.647 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 70108, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 70108, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 70108, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 70108, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:51.648 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 10074, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 1247946, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 10074, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 264, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 1247946, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:51.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle-0x56463b570500": { 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 61263, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 166980197, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 61263, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 10277, 2024-06-06T08:50:51.650 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 166980197, 2024-06-06T08:50:51.651 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:51.651 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:51.651 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:51.651 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:51.651 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:51.651 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:51.651 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-06-06T08:50:52.323 INFO:tasks.check_counter:Daemon mds.b mds.exported=217 2024-06-06T08:50:52.324 INFO:tasks.check_counter:Daemon mds.b mds.imported=112 2024-06-06T08:50:52.324 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=2605 2024-06-06T08:50:52.324 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=509 2024-06-06T08:50:52.324 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-06-06T08:50:52.324 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-06-06T08:50:52.324 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-06-06T08:50:52.324 DEBUG:tasks.check_counter:Getting stats from e 2024-06-06T08:50:52.324 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-06T08:50:52.566 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:50:52.687 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:52 smithi176 ceph-mon[36786]: pgmap v329: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:52.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:52 smithi027 ceph-mon[32335]: pgmap v329: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:52.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:52 smithi203 ceph-mon[36579]: pgmap v329: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:54.344 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-06-06T08:50:54.344 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:50:54.344 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 1935, 2024-06-06T08:50:54.344 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 1935, 2024-06-06T08:50:54.344 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 17341479, 2024-06-06T08:50:54.344 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 614513, 2024-06-06T08:50:54.344 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 6, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 2, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.173456988, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.054826400, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.051787051, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.052335527, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1934, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.070186351, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000036290 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.345 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 357, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 329, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 5899773, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 174790, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 4, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 3, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.061660006, 2024-06-06T08:50:54.346 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.016712343, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.046650675, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.007625820, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 328, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.015526329, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000047336 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1687696, 2024-06-06T08:50:54.347 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 153104 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 370, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 555, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 360865, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 584592, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 6, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 4, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.042166361, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.015976849, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.010268613, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.011025137, 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:54.348 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 552, 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.027861054, 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000050472 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 467568 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 1, 2024-06-06T08:50:54.349 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-MDSRank": { 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 3979, 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.099644630, 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000025042 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-PurgeQueue": { 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-06-06T08:50:54.350 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "request": 0, 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "reply": 0, 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "reply_latency": { 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_reply": 0, 2024-06-06T08:50:54.351 INFO:teuthology.orchestra.run.smithi176.stdout: "forward": 0, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_complete": 0, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_keys": 0, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_commit": 0, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_split": 0, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_merge": 0, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes": 4916, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_top": 2, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_bottom": 4914, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pin_tail": 0, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pinned": 4913, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_expired": 1, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_with_caps": 0, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "caps": 0, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "subtrees": 1, 2024-06-06T08:50:54.352 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_hit": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_forward": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_discover": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_remote_ino": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_lock": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "load_cent": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "q": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "exported": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "exported_inodes": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "imported": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "imported_inodes": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_dir_fetch": 0, 2024-06-06T08:50:54.353 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_backtrace_fetch": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_peer_discover": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rfiles": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rbytes": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rsnaps": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_set_tag": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_inodes": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_file_inodes": 0, 2024-06-06T08:50:54.354 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_inode_file_caps": 0, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_grant": 0, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps": 0, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps_dirty": 0, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_cap_release": 0, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "process_request_cap_release": 0 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache": { 2024-06-06T08:50:54.355 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update_receipt": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_try_discover": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_send_discover": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_handle_discover": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_delayed": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_created": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_enqueued": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_reintegrated": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_migrated": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_processing": 0, 2024-06-06T08:50:54.356 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_started": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_completed": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_path": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_inode": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_exportdir": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_flush": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragmentdir": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragstats": 0, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_inodestats": 0 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log": { 2024-06-06T08:50:54.357 INFO:teuthology.orchestra.run.smithi176.stdout: "evlrg": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "evadd": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "evex": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "evtrm": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "ev": 7745, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "evexg": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "evexd": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "segadd": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "segex": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "segtrm": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "seg": 12, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "segmjr": 2, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "segexg": 0, 2024-06-06T08:50:54.358 INFO:teuthology.orchestra.run.smithi176.stdout: "segexd": 0, 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "expos": 4194304, 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "wrpos": 25430939, 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "rdpos": 25430340, 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "jlat": { 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "replayed": 7745 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mem": { 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "ino": 1464, 2024-06-06T08:50:54.359 INFO:teuthology.orchestra.run.smithi176.stdout: "ino+": 3900, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "ino-": 2436, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "dir": 149, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "dir+": 439, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "dir-": 290, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "dn": 4916, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "dn+": 6257, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "dn-": 1341, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "cap": 0, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "cap+": 0, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "cap-": 0, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "rss": 67024, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "heap": 215420 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.360 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server": { 2024-06-06T08:50:54.361 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_client_request": 0, 2024-06-06T08:50:54.361 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_server_request": 0, 2024-06-06T08:50:54.361 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_request": 0, 2024-06-06T08:50:54.361 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_session": 0, 2024-06-06T08:50:54.361 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_peer_request": 0, 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "req_create_latency": { 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getattr_latency": { 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getfilelock_latency": { 2024-06-06T08:50:54.362 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.363 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.363 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.363 INFO:teuthology.orchestra.run.smithi176.stdout: "req_link_latency": { 2024-06-06T08:50:54.363 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.363 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.363 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.363 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookup_latency": { 2024-06-06T08:50:54.364 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.364 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.364 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.364 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookuphash_latency": { 2024-06-06T08:50:54.364 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.364 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.364 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupino_latency": { 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupname_latency": { 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupparent_latency": { 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.365 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupsnap_latency": { 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lssnap_latency": { 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.367 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mkdir_latency": { 2024-06-06T08:50:54.367 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.367 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.367 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mknod_latency": { 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mksnap_latency": { 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "req_open_latency": { 2024-06-06T08:50:54.368 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "req_readdir_latency": { 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rename_latency": { 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "req_renamesnap_latency": { 2024-06-06T08:50:54.369 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "req_snapdiff_latency": { 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmdir_latency": { 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.370 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmsnap_latency": { 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmxattr_latency": { 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setattr_latency": { 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setdirlayout_latency": { 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setfilelock_latency": { 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setlayout_latency": { 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.372 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setxattr_latency": { 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "req_symlink_latency": { 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "req_unlink_latency": { 2024-06-06T08:50:54.373 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getvxattr_latency": { 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessions": { 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "session_count": 2, 2024-06-06T08:50:54.374 INFO:teuthology.orchestra.run.smithi176.stdout: "session_add": 3, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "session_remove": 1, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_open": 3, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_stale": 0, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "total_load": 0, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "average_load": 0, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "avg_session_uptime": 2834, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "mdthresh_evicted": 0 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:50:54.375 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:50:54.376 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2024-06-06T08:50:54.377 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 251551, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 1692, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2024-06-06T08:50:54.378 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 9712, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 187, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 7409469, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 247383, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.379 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter": { 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "op_active": 0, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "op_laggy": 0, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send": 2367, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send_bytes": 21, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "op_resend": 3, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "op_reply": 2367, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "op_latency": { 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 2367, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1.735986453, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000733412 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "op_inflight": 0, 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "oplen_avg": { 2024-06-06T08:50:54.380 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 2367, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 2368 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "op": 2367, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "op_r": 2367, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "op_w": 0, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "op_rmw": 0, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "op_pg": 0, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_stat": 1444, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_create": 0, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_read": 921, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_write": 0, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writefull": 0, 2024-06-06T08:50:54.381 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writesame": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_append": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_zero": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_truncate": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_delete": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_mapext": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_sparse_read": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_clonerange": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_getxattr": 1, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_setxattr": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_rmxattr": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_call": 0, 2024-06-06T08:50:54.382 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_watch": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_notify": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_other": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_active": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_send": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_resend": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_ping": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_active": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_send": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_resend": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_active": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_send": 0, 2024-06-06T08:50:54.383 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_resend": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_active": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_send": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_resend": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "command_active": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "command_send": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "command_resend": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "map_epoch": 109, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "map_full": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "map_inc": 12, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_sessions": 5, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_open": 5, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_close": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_laggy": 0, 2024-06-06T08:50:54.384 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_wr": 0, 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_rd": 2, 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_del": 0 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "oft": { 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_objs": 0, 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_updates": 0, 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_removes": 0 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "osd-slow-ops": { 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_ops_count": 0 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "purge_queue": { 2024-06-06T08:50:54.385 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops": 0, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing": 0, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_high_water": 0, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed_ops": 0, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed": 0, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_item_in_journal": 0 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 2658, 2024-06-06T08:50:54.386 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 23345108, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 2658, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 2658, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 23345108, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_bytes": { 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:54.387 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 2367, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 21236933, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 2367, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 21236933, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_ops": { 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 1024, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 2367, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 2367, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 2367, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 2367, 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:54.389 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:54.390 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 0, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 0, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle-0x5555b75f06e0": { 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-06-06T08:50:54.391 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 0, 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 0, 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:54.392 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-06-06T08:50:54.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:54 smithi176 ceph-mon[36786]: pgmap v330: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:50:54.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:54 smithi027 ceph-mon[32335]: pgmap v330: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:50:54.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:54 smithi203 ceph-mon[36579]: pgmap v330: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T08:50:55.179 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-06T08:50:55.179 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-06T08:50:55.179 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-06-06T08:50:55.179 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-06-06T08:50:55.179 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-06-06T08:50:55.179 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:50:55.179 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:50:55.179 DEBUG:tasks.check_counter:Getting stats from h 2024-06-06T08:50:55.180 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-06T08:50:55.420 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:50:56.877 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:56 smithi176 ceph-mon[36786]: pgmap v331: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:50:56.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:56 smithi027 ceph-mon[32335]: pgmap v331: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:50:56.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:56 smithi203 ceph-mon[36579]: pgmap v331: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-06-06T08:50:57.130 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 39055, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 23768, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 17923885, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 10730101, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 19, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 7, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 1.762464710, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.772749347, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 1.223854487, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.079711652, 2024-06-06T08:50:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 23767, 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.674399038, 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000028375 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 58456, 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.008422656, 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000144 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:50:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 2140, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 2242, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 2177567, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 5693887, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 13, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 5, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.190723837, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.063460224, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.071992558, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.053882059, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 2241, 2024-06-06T08:50:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.157464616, 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000070265 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 15, 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000002275, 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000151 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1699184, 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 527072 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 12305, 2024-06-06T08:50:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 16334, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 4426851, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 12121520, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 14, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 6, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.677173887, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.292688768, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.412716652, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.041549287, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 16331, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.481961923, 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000029512 2024-06-06T08:50:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 23284, 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.004227034, 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000181 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 6480, 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 160608 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 1, 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-MDSRank": { 2024-06-06T08:50:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 3349, 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 2.710025400, 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000809204 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-PurgeQueue": { 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 2079, 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.235644684, 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000113345 2024-06-06T08:50:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "request": 2858, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "reply": 2586, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "reply_latency": { 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 2586, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 23.839313978, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.009218605 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_reply": 0, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "forward": 272, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_complete": 0, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_keys": 0, 2024-06-06T08:50:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_commit": 11, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_split": 0, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_merge": 0, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes": 1526, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_top": 6, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_bottom": 6, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pin_tail": 1514, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pinned": 1517, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_expired": 4228, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_with_caps": 0, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "caps": 0, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "subtrees": 7, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse": 8768, 2024-06-06T08:50:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_hit": 5638, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_forward": 251, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_discover": 1747, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_remote_ino": 3, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_lock": 13, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "load_cent": 0, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "q": 0, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "exported": 60, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "exported_inodes": 931, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "imported": 157, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "imported_inodes": 1150, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_dir_fetch": 0, 2024-06-06T08:50:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_backtrace_fetch": 895, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_peer_discover": 9, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rfiles": 4076, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rbytes": 26140844, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rsnaps": 4, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_fetch": 895, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_set_tag": 0, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_inodes": 114, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dirfrag_rstats": 114, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_file_inodes": 781, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_inode_file_caps": 74, 2024-06-06T08:50:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_revoke": 1515, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_grant": 1836, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_trunc": 10, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps": 2011, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps_dirty": 506, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_cap_release": 92, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "process_request_cap_release": 2062 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache": { 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update": 228, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update_receipt": 4472, 2024-06-06T08:50:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_try_discover": 2998, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_send_discover": 3232, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_handle_discover": 508, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays": 142, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_delayed": 0, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_created": 1146, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_enqueued": 1000, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_reintegrated": 6, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_migrated": 4, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_processing": 0, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:50:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_started": 0, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_completed": 0, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_path": 4, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_quiesce_inode": 1460, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_exportdir": 60, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_flush": 0, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragmentdir": 0, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragstats": 0, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_inodestats": 0 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log": { 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "evlrg": 0, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "evadd": 4574, 2024-06-06T08:50:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "evex": 0, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "evtrm": 0, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "ev": 4574, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "evexg": 0, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "evexd": 0, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "segadd": 8, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "segex": 0, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "segtrm": 0, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "seg": 8, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "segmjr": 2, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "segexg": 0, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "segexd": 0, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "expos": 4194304, 2024-06-06T08:50:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "wrpos": 17120282, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "rdpos": 0, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "jlat": { 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1563, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 7.536286914, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.004821680 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "replayed": 0 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mem": { 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "ino": 721, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "ino+": 3814, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "ino-": 3093, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "dir": 174, 2024-06-06T08:50:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "dir+": 1938, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "dir-": 1764, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "dn": 1526, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "dn+": 6843, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "dn-": 5317, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "cap": 0, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "cap+": 2199, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "cap-": 2199, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "rss": 86360, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "heap": 223612 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server": { 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_client_request": 5212, 2024-06-06T08:50:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_server_request": 789, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_request": 2858, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_session": 117, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_peer_request": 1948, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "req_create_latency": { 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 50, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.090787252, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.001815745 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getattr_latency": { 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 24, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.049582302, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002065929 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getfilelock_latency": { 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "req_link_latency": { 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 11, 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.180612227, 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.016419293 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookup_latency": { 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 983, 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.389428917, 2024-06-06T08:50:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000396163 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookuphash_latency": { 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupino_latency": { 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupname_latency": { 2024-06-06T08:50:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupparent_latency": { 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupsnap_latency": { 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lssnap_latency": { 2024-06-06T08:50:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mkdir_latency": { 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 34, 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.063793672, 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.001876284 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mknod_latency": { 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 39, 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.069645164, 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.001785773 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mksnap_latency": { 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "req_open_latency": { 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "req_readdir_latency": { 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 205, 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.184976964, 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000902326 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rename_latency": { 2024-06-06T08:50:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 36, 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 18.636873551, 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.517690931 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "req_renamesnap_latency": { 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "req_snapdiff_latency": { 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmdir_latency": { 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 168, 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 2.151625254, 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.012807293 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmsnap_latency": { 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmxattr_latency": { 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setattr_latency": { 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 10, 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.019939887, 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.001993988 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setdirlayout_latency": { 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setfilelock_latency": { 2024-06-06T08:50:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setlayout_latency": { 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setxattr_latency": { 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "req_symlink_latency": { 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 26, 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.029539892, 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.001136149 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "req_unlink_latency": { 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1000, 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1.972508896, 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.001972508 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:50:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getvxattr_latency": { 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessions": { 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "session_count": 1, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "session_add": 4, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "session_remove": 3, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_open": 1, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_stale": 0, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "total_load": 1064, 2024-06-06T08:50:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "average_load": 1064, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "avg_session_uptime": 2099, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "mdthresh_evicted": 0 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:50:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:50:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:50:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 1468362, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 1087, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 9712, 2024-06-06T08:50:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 187, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 3345492, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 102022, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter": { 2024-06-06T08:50:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "op_active": 0, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "op_laggy": 0, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send": 4855, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send_bytes": 13202489, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "op_resend": 28, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "op_reply": 4855, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "op_latency": { 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 4855, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 22.780822190, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.004692239 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "op_inflight": 0, 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "oplen_avg": { 2024-06-06T08:50:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 4855, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 5002 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "op": 4855, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "op_r": 1013, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "op_w": 3842, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "op_rmw": 0, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "op_pg": 0, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_stat": 0, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_create": 10, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_read": 118, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_write": 1681, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writefull": 82, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writesame": 0, 2024-06-06T08:50:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_append": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_zero": 2, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_truncate": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_delete": 1953, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_mapext": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_sparse_read": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_clonerange": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_getxattr": 895, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_setxattr": 20, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_rmxattr": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_call": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_watch": 0, 2024-06-06T08:50:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_notify": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_other": 12, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_active": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_send": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_resend": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_ping": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_active": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_send": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_resend": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_active": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_send": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_resend": 0, 2024-06-06T08:50:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_active": 0, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_send": 0, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_resend": 0, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "command_active": 0, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "command_send": 0, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "command_resend": 0, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "map_epoch": 109, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "map_full": 0, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "map_inc": 12, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_sessions": 12, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_open": 12, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_close": 0, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_laggy": 0, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_wr": 170, 2024-06-06T08:50:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_rd": 0, 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_del": 59 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "oft": { 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_objs": 1, 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_updates": 0, 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_removes": 0 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "osd-slow-ops": { 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_ops_count": 1465 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "purge_queue": { 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops": 0, 2024-06-06T08:50:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops_high_water": 19, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing": 0, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_high_water": 15, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed_ops": 1940, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed": 1000, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_item_in_journal": 0 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 53496, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 19693262, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 53496, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 53496, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 19693262, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_bytes": { 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-06-06T08:50:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 4855, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 13324737, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 4855, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 13324737, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_ops": { 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 1024, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 4855, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 4855, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 4855, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 4855, 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 1000, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 122248, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 1000, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 118, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 122248, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle-0x561ca984a500": { 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 4574, 2024-06-06T08:50:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 12925978, 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 4574, 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 1563, 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 12925978, 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:57.178 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-06-06T08:50:57.903 INFO:tasks.check_counter:Daemon mds.h mds.exported=60 2024-06-06T08:50:57.903 INFO:tasks.check_counter:Daemon mds.h mds.imported=157 2024-06-06T08:50:57.903 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=228 2024-06-06T08:50:57.903 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=4472 2024-06-06T08:50:57.903 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=4 2024-06-06T08:50:57.903 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:50:57.903 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:50:57.903 DEBUG:tasks.check_counter:Getting stats from k 2024-06-06T08:50:57.903 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-06T08:50:58.139 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T08:50:58.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:50:58 smithi027 ceph-mon[32335]: pgmap v332: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:58.976 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:50:58 smithi176 ceph-mon[36786]: pgmap v332: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:58.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:50:58 smithi203 ceph-mon[36579]: pgmap v332: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:50:59.884 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-06-06T08:50:59.884 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:50:59.884 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 259, 2024-06-06T08:50:59.884 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 248, 2024-06-06T08:50:59.884 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 1591728, 2024-06-06T08:50:59.884 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 152720, 2024-06-06T08:50:59.884 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 4, 2024-06-06T08:50:59.884 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 0, 2024-06-06T08:50:59.884 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.050958971, 2024-06-06T08:50:59.885 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.016213587, 2024-06-06T08:50:59.885 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.036613503, 2024-06-06T08:50:59.885 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.006171118, 2024-06-06T08:50:59.885 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:59.885 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 247, 2024-06-06T08:50:59.885 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.012837566, 2024-06-06T08:50:59.885 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000051973 2024-06-06T08:50:59.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.885 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1591728, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 152720 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 6, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 192, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 3392, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 189808, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 5, 2024-06-06T08:50:59.886 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 3, 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.013064341, 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.008861771, 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.001730132, 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 190, 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.019158348, 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000100833 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:59.887 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 189808 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 0, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 2, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 0, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 2, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 1, 2024-06-06T08:50:59.888 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.000508704, 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.000427213, 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:50:59.889 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 0, 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:50:59.890 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:50:59.891 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:50:59.892 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 148731, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 31, 2024-06-06T08:50:59.893 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2024-06-06T08:50:59.894 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 928, 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 2, 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2024-06-06T08:50:59.895 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 261, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 1555210, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 261, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 261, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 1555210, 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2024-06-06T08:50:59.896 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2024-06-06T08:50:59.897 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2024-06-06T08:50:59.897 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2024-06-06T08:50:59.897 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:59.897 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-06-06T08:50:59.897 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-06-06T08:51:00.599 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-06-06T08:51:00.599 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-06-06T08:51:00.599 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-06-06T08:51:00.600 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-06-06T08:51:00.600 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-06-06T08:51:00.600 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-06-06T08:51:00.600 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-06-06T08:51:00.600 DEBUG:tasks.check_counter:Getting stats from c 2024-06-06T08:51:00.600 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-06T08:51:00.855 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:51:00.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:00 smithi027 ceph-mon[32335]: pgmap v333: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:51:00.976 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:00 smithi203 ceph-mon[36579]: pgmap v333: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:51:01.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:00 smithi176 ceph-mon[36786]: pgmap v333: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-06-06T08:51:02.653 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:02 smithi203 ceph-mon[36579]: pgmap v334: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:51:02.654 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-06-06T08:51:02.654 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:51:02.654 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1066, 2024-06-06T08:51:02.654 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1066, 2024-06-06T08:51:02.654 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 63120892, 2024-06-06T08:51:02.654 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 338174, 2024-06-06T08:51:02.654 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 8, 2024-06-06T08:51:02.654 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.141356887, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.034548482, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.060308714, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.036429016, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1065, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.041395789, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000038869 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.655 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 787, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 773, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 70939365, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 285746, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.105703071, 2024-06-06T08:51:02.656 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.026342594, 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.054241532, 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.019447520, 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 772, 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.030768671, 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000039855 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.657 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 259904, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 89792 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 183, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 290, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 33606420, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 333717, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.039609205, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.009252809, 2024-06-06T08:51:02.658 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.023038609, 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.004914146, 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 288, 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.012901545, 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000044797 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T08:51:02.659 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 275840 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 3033, 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.374076047, 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000123335 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.660 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-06-06T08:51:02.661 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 35656, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 23181, 2024-06-06T08:51:02.662 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 12475, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 0, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 2540, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 2, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:51:02.663 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 4, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 234, 2024-06-06T08:51:02.664 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 6, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2024-06-06T08:51:02.665 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2024-06-06T08:51:02.666 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:51:02.667 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2024-06-06T08:51:02.668 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 61202, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 7, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 16, 2024-06-06T08:51:02.669 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 168807228, 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 171179515, 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 171177244, 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 61202 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 11991, 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 24509, 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 12518, 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 1451, 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 1806, 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 355, 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 35656, 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 38883, 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 3227, 2024-06-06T08:51:02.671 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 130344, 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 215420 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2024-06-06T08:51:02.672 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-06-06T08:51:02.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.674 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.675 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.676 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-06-06T08:51:02.677 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.677 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.677 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.677 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.677 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.679 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.680 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.681 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.683 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.684 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 2, 2024-06-06T08:51:02.685 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 2, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 0, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 0, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 2094, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:51:02.686 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:51:02.687 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-06-06T08:51:02.688 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 769277, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 14437, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-06-06T08:51:02.689 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 9712, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 187, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 57627456, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 1229522, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-06-06T08:51:02.690 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 1860, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 26, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 4, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 1860, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1860, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.879930226, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001548349 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-06-06T08:51:02.691 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1860, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1861 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 1860, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 1860, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 1135, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 722, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2024-06-06T08:51:02.692 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 2, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-06-06T08:51:02.693 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-06-06T08:51:02.694 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 109, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 8, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 11, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 11, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-06-06T08:51:02.695 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-06-06T08:51:02.696 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 0, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 2032, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 167472021, 2024-06-06T08:51:02.697 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 2032, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2032, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 167472021, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1860, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 166885162, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1860, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 166885162, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.699 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1860, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 1860, 2024-06-06T08:51:02.700 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1860, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1860, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:02.701 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55aedb6246e0": { 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:02.702 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:02.703 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:02.704 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:02.704 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-06-06T08:51:02.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:02 smithi176 ceph-mon[36786]: pgmap v334: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:51:02.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:02 smithi027 ceph-mon[32335]: pgmap v334: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T08:51:03.679 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-06T08:51:03.679 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-06-06T08:51:03.679 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-06-06T08:51:03.679 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-06-06T08:51:03.679 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-06-06T08:51:03.679 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:51:03.679 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:51:03.679 DEBUG:tasks.check_counter:Getting stats from f 2024-06-06T08:51:03.679 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-06T08:51:03.931 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:51:04.912 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:04 smithi203 ceph-mon[36579]: pgmap v335: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:51:04.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:04 smithi027 ceph-mon[32335]: pgmap v335: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:51:05.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:04 smithi176 ceph-mon[36786]: pgmap v335: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T08:51:05.668 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-06-06T08:51:05.668 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:51:05.668 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 740, 2024-06-06T08:51:05.668 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 742, 2024-06-06T08:51:05.668 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 4382887, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 235384, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 7, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.069269451, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.020319257, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.020003427, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.021933405, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 740, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.032662321, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000044138 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 781, 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 967, 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 4735938, 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 718703, 2024-06-06T08:51:05.670 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 7, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.083505640, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.031100865, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.021603381, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.022359286, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 964, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.045277406, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000046968 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.671 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 470976 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1248, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1218, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 6194602, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 457886, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2024-06-06T08:51:05.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.159470484, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.047674407, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.069972497, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.041613417, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1218, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.065975865, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000054167 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1691312, 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 154832 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4139, 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.116883070, 2024-06-06T08:51:05.674 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000028239 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2024-06-06T08:51:05.675 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 2519, 2024-06-06T08:51:05.676 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 8, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 2511, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 0, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 2507, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 1, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2024-06-06T08:51:05.677 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 0, 2024-06-06T08:51:05.678 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T08:51:05.679 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2024-06-06T08:51:05.680 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:51:05.681 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-06-06T08:51:05.682 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 4574, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 8, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-06-06T08:51:05.683 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 17120282, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 17117721, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 4574 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 703, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 2407, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 1704, 2024-06-06T08:51:05.684 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 144, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 424, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 280, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 2519, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 3816, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 1297, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 59264, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 215420 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-06-06T08:51:05.685 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.686 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-06-06T08:51:05.687 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.688 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.689 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.690 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.690 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-06-06T08:51:05.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.690 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.690 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.690 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-06-06T08:51:05.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.691 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.691 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.691 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-06-06T08:51:05.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.691 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.691 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.692 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.693 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-06-06T08:51:05.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.696 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.697 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 4, 2024-06-06T08:51:05.698 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 3, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 2097, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:51:05.699 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:51:05.700 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-06-06T08:51:05.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 242799, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 1008, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-06-06T08:51:05.702 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 9712, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 187, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 3889722, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 97889, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-06-06T08:51:05.703 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 2472, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 21, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 3, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 2472, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2472, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.443935797, 2024-06-06T08:51:05.704 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000988647 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2472, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2473 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 2472, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 2472, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 1542, 2024-06-06T08:51:05.705 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 928, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-06-06T08:51:05.706 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 1, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-06-06T08:51:05.707 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-06-06T08:51:05.708 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 109, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 12, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 7, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 7, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2024-06-06T08:51:05.709 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-06-06T08:51:05.710 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 0, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:05.711 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 2765, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 15046684, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 2765, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2765, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 15046684, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.712 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 2472, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 12925983, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2472, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 12925983, 2024-06-06T08:51:05.713 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:05.714 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 2472, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 2472, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2472, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 2472, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-06-06T08:51:05.715 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.716 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x5559be1f86e0": { 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:05.717 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-06-06T08:51:05.718 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-06-06T08:51:05.718 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:05.718 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:05.718 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:05.718 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:05.718 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:05.718 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:05.718 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-06-06T08:51:06.468 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-06T08:51:06.468 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-06-06T08:51:06.468 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-06-06T08:51:06.468 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-06-06T08:51:06.468 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-06-06T08:51:06.468 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:51:06.469 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:51:06.469 DEBUG:tasks.check_counter:Getting stats from i 2024-06-06T08:51:06.469 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-06T08:51:06.712 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:51:06.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:06 smithi203 ceph-mon[36579]: pgmap v336: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-06-06T08:51:06.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:06 smithi027 ceph-mon[32335]: pgmap v336: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-06-06T08:51:07.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:06 smithi176 ceph-mon[36786]: pgmap v336: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-06-06T08:51:08.389 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-06-06T08:51:08.389 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 2856, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 2957, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 2587199, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 4365497, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 9, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.251230523, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.090903117, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.083726948, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.069646051, 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:08.390 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2957, 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.218573195, 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000073917 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1705200, 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 528736 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 47780, 2024-06-06T08:51:08.391 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 32812, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 21590668, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 27078903, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 15, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 7, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 2.084067348, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.817640013, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 1.595061350, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.099380465, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 32809, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.836805824, 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000025505 2024-06-06T08:51:08.392 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.393 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:08.393 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 72967, 2024-06-06T08:51:08.393 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.009574109, 2024-06-06T08:51:08.393 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000131 2024-06-06T08:51:08.393 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.393 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 4864, 2024-06-06T08:51:08.393 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 159680 2024-06-06T08:51:08.393 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.393 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 11049, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 18290, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 3729995, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 10310362, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 21, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.648012516, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.274779707, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.404223572, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.043137856, 2024-06-06T08:51:08.394 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 18288, 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.505747796, 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000027654 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 21865, 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.003586381, 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000164 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-06-06T08:51:08.395 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4590, 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 4.617565503, 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001006005 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-06-06T08:51:08.396 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 3489, 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.604591139, 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000173284 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 5383, 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 5108, 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 5108, 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 28.861805763, 2024-06-06T08:51:08.397 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.005650314 2024-06-06T08:51:08.398 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.398 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-06-06T08:51:08.398 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 275, 2024-06-06T08:51:08.398 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 1, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 11, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 3153, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 3, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 2, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 3148, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 3148, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 4281, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 1, 2024-06-06T08:51:08.399 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 1, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 9, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 12858, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 9352, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 262, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 1783, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 1, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 5, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 20, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 72, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 903, 2024-06-06T08:51:08.400 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 80, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 1540, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 1269, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 6, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 4076, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 26140844, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 4, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 1269, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 43, 2024-06-06T08:51:08.401 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 43, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 1226, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 74, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 2433, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 3275, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 29, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 3786, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 1214, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 117, 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 3121 2024-06-06T08:51:08.402 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 302, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 4547, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 3066, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 3194, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 716, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 162, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 1941, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 1774, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 18, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 1, 2024-06-06T08:51:08.403 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 4, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 2192, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 73, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2024-06-06T08:51:08.404 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 1, 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 7745, 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 7745, 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 12, 2024-06-06T08:51:08.405 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 12, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 25430939, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 0, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1978, 2024-06-06T08:51:08.406 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 8.077860034, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004083852 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 0 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 1471, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 5343, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 3872, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 163, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 1954, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 1791, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 3153, 2024-06-06T08:51:08.407 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 9374, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 6221, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 1, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 3491, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 3490, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 114332, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 223612 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 8139, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 1167, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 5383, 2024-06-06T08:51:08.408 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 183, 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 2596, 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 246, 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 17.012623833, 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.069157007 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 41, 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.088805326, 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002165983 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-06-06T08:51:08.409 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 51, 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.595746102, 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.011681296 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1979, 2024-06-06T08:51:08.410 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.731715274, 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000875045 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1, 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000236977, 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000236977 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.411 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-06-06T08:51:08.412 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.413 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.413 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.413 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.413 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-06-06T08:51:08.413 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 121, 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.192501662, 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001590922 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-06-06T08:51:08.414 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 121, 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.177793508, 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001469367 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 8, 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.005987553, 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000748444 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.415 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 242, 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.317968860, 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001313920 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 114, 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.563953404, 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.022490819 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.416 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 213, 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.308398837, 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.010837553 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.417 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 32, 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.048008684, 2024-06-06T08:51:08.418 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001500271 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-06-06T08:51:08.419 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 127, 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.200297591, 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001577146 2024-06-06T08:51:08.420 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1812, 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 3.617768152, 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001996560 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 2, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 3, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 2, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 1796, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 898, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 627, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:51:08.422 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:51:08.423 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:51:08.424 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 1656695, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 1887, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-06-06T08:51:08.425 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 9712, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 187, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 6321110, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 253177, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-06-06T08:51:08.426 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 7238, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 21669266, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 28, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 7238, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 7238, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 31.731237782, 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004383978 2024-06-06T08:51:08.427 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 7238, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 7406 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 7238, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 1407, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 5831, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 0, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 10, 2024-06-06T08:51:08.428 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 137, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 2115, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 90, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 9, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 3464, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 1270, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 20, 2024-06-06T08:51:08.429 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 34, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-06-06T08:51:08.430 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-06-06T08:51:08.431 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 109, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 12, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 12, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 12, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 183, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 72 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-06-06T08:51:08.432 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 1, 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 2197 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 21, 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-06-06T08:51:08.433 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 12, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 3456, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 1774, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 61681, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 22336338, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 61681, 2024-06-06T08:51:08.434 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 61681, 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 22336338, 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.435 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 7238, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 21884064, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 7238, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 21884064, 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:08.436 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-06-06T08:51:08.437 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 7238, 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 7238, 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 7238, 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 7238, 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.438 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 1774, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 214814, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 1774, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:08.439 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 137, 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 214814, 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x56279197a500": { 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:08.440 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 7745, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 21236635, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 7745, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1978, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 21236635, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:08.441 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:08.442 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:08.442 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-06-06T08:51:08.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:08 smithi203 ceph-mon[36579]: pgmap v337: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-06-06T08:51:08.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:08 smithi027 ceph-mon[32335]: pgmap v337: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-06-06T08:51:09.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:08 smithi176 ceph-mon[36786]: pgmap v337: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-06-06T08:51:09.296 INFO:tasks.check_counter:Daemon mds.i mds.exported=72 2024-06-06T08:51:09.296 INFO:tasks.check_counter:Daemon mds.i mds.imported=80 2024-06-06T08:51:09.296 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=302 2024-06-06T08:51:09.296 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=4547 2024-06-06T08:51:09.296 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=4 2024-06-06T08:51:09.296 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-06-06T08:51:09.296 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T08:51:09.297 DEBUG:tasks.check_counter:Getting stats from l 2024-06-06T08:51:09.297 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-06T08:51:09.531 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T08:51:10.657 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:10 smithi203 ceph-mon[36579]: pgmap v338: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-06-06T08:51:10.955 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:10 smithi027 ceph-mon[32335]: pgmap v338: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-06-06T08:51:11.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:10 smithi176 ceph-mon[36786]: pgmap v338: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-06-06T08:51:11.510 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 0, 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 2, 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 0, 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 0, 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 0, 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.001195508, 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.000408382, 2024-06-06T08:51:11.511 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T08:51:11.512 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 6, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 194, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 3392, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 190160, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 5, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.012970161, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.008672820, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.001735390, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 192, 2024-06-06T08:51:11.513 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.020493825, 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000106738 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 190160 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 263, 2024-06-06T08:51:11.514 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 252, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 1592816, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 155280, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 2, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 1, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.054301200, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.016189192, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.040545797, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.006426442, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 251, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.014824339, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000059061 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.515 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1592816, 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 155280 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 0, 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-06-06T08:51:11.516 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-06-06T08:51:11.517 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-06-06T08:51:11.518 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 148761, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 31, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-06-06T08:51:11.519 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 928, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 2, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-06-06T08:51:11.520 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 265, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 1555730, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 265, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 265, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1555730, 2024-06-06T08:51:11.521 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-06-06T08:51:11.522 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-06-06T08:51:11.522 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-06-06T08:51:11.522 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-06-06T08:51:11.522 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:11.522 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-06-06T08:51:11.522 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-06-06T08:51:11.916 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:11 smithi203 ceph-mon[36579]: osdmap e110: 12 total, 12 up, 12 in 2024-06-06T08:51:11.955 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:11 smithi027 ceph-mon[32335]: osdmap e110: 12 total, 12 up, 12 in 2024-06-06T08:51:12.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:11 smithi176 ceph-mon[36786]: osdmap e110: 12 total, 12 up, 12 in 2024-06-06T08:51:12.865 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:12 smithi203 ceph-mon[36579]: pgmap v340: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:51:12.867 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-06-06T08:51:12.867 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-06-06T08:51:12.867 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-06-06T08:51:12.867 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-06-06T08:51:12.867 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-06-06T08:51:12.867 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-06-06T08:51:12.867 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-06-06T08:51:12.868 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-06T08:51:12.879 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-06T08:51:12.880 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:51:12.880 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T08:51:12.913 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:51:12.913 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T08:51:12.955 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:12 smithi027 ceph-mon[32335]: pgmap v340: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:51:12.985 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T08:51:13.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:12 smithi176 ceph-mon[36786]: pgmap v340: 129 pgs: 129 active+clean; 60 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:51:13.477 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:51:14.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:14 smithi203 ceph-mon[36579]: pgmap v341: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 56 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:51:14.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:14 smithi027 ceph-mon[32335]: pgmap v341: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 56 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:51:15.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:14 smithi176 ceph-mon[36786]: pgmap v341: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 56 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-06-06T08:51:16.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.764+0000 7fce17f78700 1 -- 172.21.15.27:0/1234235782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 msgr2=0x7fce10107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:16.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.764+0000 7fce17f78700 1 --2- 172.21.15.27:0/1234235782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 0x7fce10107130 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fce040099c0 tx=0x7fce04009cd0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.764+0000 7fce17f78700 1 -- 172.21.15.27:0/1234235782 shutdown_connections 2024-06-06T08:51:16.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.764+0000 7fce17f78700 1 --2- 172.21.15.27:0/1234235782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce10107860 0x7fce1010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.764+0000 7fce17f78700 1 --2- 172.21.15.27:0/1234235782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce10102770 0x7fce10102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.764+0000 7fce17f78700 1 --2- 172.21.15.27:0/1234235782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 0x7fce10107130 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.764+0000 7fce17f78700 1 -- 172.21.15.27:0/1234235782 >> 172.21.15.27:0/1234235782 conn(0x7fce10079600 msgr2=0x7fce10079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.765+0000 7fce17f78700 1 -- 172.21.15.27:0/1234235782 shutdown_connections 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.765+0000 7fce17f78700 1 -- 172.21.15.27:0/1234235782 wait complete. 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.765+0000 7fce17f78700 1 Processor -- start 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.765+0000 7fce17f78700 1 -- start start 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce17f78700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce10102770 0x7fce10074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce17f78700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 0x7fce10074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce17f78700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce10107860 0x7fce10075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce17f78700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce101127c0 con 0x7fce10103120 2024-06-06T08:51:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce17f78700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fce10112640 con 0x7fce10102770 2024-06-06T08:51:16.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce17f78700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fce10112940 con 0x7fce10107860 2024-06-06T08:51:16.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15513700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 0x7fce10074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:16.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15513700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 0x7fce10074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44552/0 (socket says 172.21.15.27:44552) 2024-06-06T08:51:16.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15513700 1 -- 172.21.15.27:0/454274717 learned_addr learned my addr 172.21.15.27:0/454274717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:51:16.023 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/876705499 2024-06-07T08:40:17.280879+0000 2024-06-06T08:51:16.023 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/550980984 2024-06-07T08:40:17.280879+0000 2024-06-06T08:51:16.023 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1905066984 2024-06-07T08:28:58.346290+0000 2024-06-06T08:51:16.023 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3763593724 2024-06-07T08:40:17.280879+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/2182206949 2024-06-07T08:28:58.346290+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/2182206949 2024-06-07T08:28:58.346290+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3763593724 2024-06-07T08:40:17.280879+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1183901040 2024-06-07T08:28:31.314626+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1624199721 2024-06-07T08:28:31.314626+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/897818490 2024-06-07T08:28:31.314626+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/4174349888 2024-06-07T08:40:17.280879+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/897818490 2024-06-07T08:28:31.314626+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3611712452 2024-06-07T08:28:58.346290+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1620281536 2024-06-07T08:28:31.314626+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3053481437 2024-06-07T08:40:17.280879+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/752935431 2024-06-07T08:28:58.346290+0000 2024-06-06T08:51:16.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15d14700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce10102770 0x7fce10074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce16515700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce10107860 0x7fce10075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15513700 1 -- 172.21.15.27:0/454274717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce10107860 msgr2=0x7fce10075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15513700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce10107860 0x7fce10075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15513700 1 -- 172.21.15.27:0/454274717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce10102770 msgr2=0x7fce10074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15513700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce10102770 0x7fce10074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15513700 1 -- 172.21.15.27:0/454274717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce101a70f0 con 0x7fce10103120 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce15513700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 0x7fce10074cd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fce04009990 tx=0x7fce04009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce0effd700 1 -- 172.21.15.27:0/454274717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04027070 con 0x7fce10103120 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce0effd700 1 -- 172.21.15.27:0/454274717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce04003e40 con 0x7fce10103120 2024-06-06T08:51:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce101a7380 con 0x7fce10103120 2024-06-06T08:51:16.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.766+0000 7fce0effd700 1 -- 172.21.15.27:0/454274717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04022bf0 con 0x7fce10103120 2024-06-06T08:51:16.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.767+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce101a77d0 con 0x7fce10103120 2024-06-06T08:51:16.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.768+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce10069180 con 0x7fce10103120 2024-06-06T08:51:16.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.769+0000 7fce0effd700 1 -- 172.21.15.27:0/454274717 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fce0402a070 con 0x7fce10103120 2024-06-06T08:51:16.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.770+0000 7fce0effd700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcdf80a2d90 0x7fcdf80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:16.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.770+0000 7fce15d14700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcdf80a2d90 0x7fcdf80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:16.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.770+0000 7fce0effd700 1 -- 172.21.15.27:0/454274717 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9650+0+0 (secure 0 0 0) 0x7fce040d0c30 con 0x7fce10103120 2024-06-06T08:51:16.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.770+0000 7fce15d14700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcdf80a2d90 0x7fcdf80a5250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce0000f440 tx=0x7fce000079e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:16.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.774+0000 7fce0effd700 1 -- 172.21.15.27:0/454274717 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fce04031070 con 0x7fce10103120 2024-06-06T08:51:16.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.995+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce1002dca0 con 0x7fce10103120 2024-06-06T08:51:16.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.995+0000 7fce0effd700 1 -- 172.21.15.27:0/454274717 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v110) ==== 81+0+941 (secure 0 0 0) 0x7fce04013070 con 0x7fce10103120 2024-06-06T08:51:16.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcdf80a2d90 msgr2=0x7fcdf80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:16.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcdf80a2d90 0x7fcdf80a5250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce0000f440 tx=0x7fce000079e0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 msgr2=0x7fce10074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 0x7fce10074cd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fce04009990 tx=0x7fce04009f80 comp rx=0 tx=0).stop 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 shutdown_connections 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce10107860 0x7fce10075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcdf80a2d90 0x7fcdf80a5250 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce10102770 0x7fce10074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 --2- 172.21.15.27:0/454274717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce10103120 0x7fce10074cd0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 >> 172.21.15.27:0/454274717 conn(0x7fce10079600 msgr2=0x7fce10100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 shutdown_connections 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:15.998+0000 7fce17f78700 1 -- 172.21.15.27:0/454274717 wait complete. 2024-06-06T08:51:16.028 INFO:teuthology.orchestra.run.smithi027.stderr:listed 16 entries 2024-06-06T08:51:16.652 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:16 smithi027 ceph-mon[32335]: pgmap v342: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:51:16.652 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:16 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/454274717' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:51:16.678 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-06T08:51:16.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:51:16.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-06T08:51:16.723 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:51:16.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:51:16.723 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-06T08:51:16.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:51:16.778 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-06-06T08:51:16.818 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-06T08:51:16.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:51:16.819 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T08:51:16.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:51:16.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-06-06T08:51:16.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:16 smithi203 ceph-mon[36579]: pgmap v342: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:51:16.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:16 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/454274717' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:51:17.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:16 smithi176 ceph-mon[36786]: pgmap v342: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:51:17.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:16 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/454274717' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T08:51:17.010 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-06T08:51:17.011 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-06-06T08:51:17.011 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T08:51:17.029 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056311,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20828 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/3d530c915985816c5218f654a842e67ddc57e69181e490b3470ac9453969cd11/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/3d530c915985816c5218f654a842e67ddc57e69181e490b3470ac9453969cd11/diff,workdir=/var/lib/containers/storage/overlay/3d530c915985816c5218f654a842e67ddc57e69181e490b3470ac9453969cd11/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1dcf77226f081bd5c95b5f84456fc9e675fb6e58ecf6b59c9dbd8330f6c22b2a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/1dcf77226f081bd5c95b5f84456fc9e675fb6e58ecf6b59c9dbd8330f6c22b2a/diff,workdir=/var/lib/containers/storage/overlay/1dcf77226f081bd5c95b5f84456fc9e675fb6e58ecf6b59c9dbd8330f6c22b2a/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/8b0bea1cabe90991826b3467c888c2f2a2599d0259a726751e09a2a06fae32b1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/8b0bea1cabe90991826b3467c888c2f2a2599d0259a726751e09a2a06fae32b1/diff,workdir=/var/lib/containers/storage/overlay/8b0bea1cabe90991826b3467c888c2f2a2599d0259a726751e09a2a06fae32b1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a68666cd512993969ebc496a2217df6aa3de7d658dfcb116d933cf3ecd1fd9fe/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/a68666cd512993969ebc496a2217df6aa3de7d658dfcb116d933cf3ecd1fd9fe/diff,workdir=/var/lib/containers/storage/overlay/a68666cd512993969ebc496a2217df6aa3de7d658dfcb116d933cf3ecd1fd9fe/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/bafb4b412a701f7588c13275c8ac22218a95d4bcfefb9cd26945852d29e0c511/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/bafb4b412a701f7588c13275c8ac22218a95d4bcfefb9cd26945852d29e0c511/diff,workdir=/var/lib/containers/storage/overlay/bafb4b412a701f7588c13275c8ac22218a95d4bcfefb9cd26945852d29e0c511/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/82d8db50a9e4c114c2e88c27794d1e28fa70da366193608a230b26d14f478401/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/82d8db50a9e4c114c2e88c27794d1e28fa70da366193608a230b26d14f478401/diff,workdir=/var/lib/containers/storage/overlay/82d8db50a9e4c114c2e88c27794d1e28fa70da366193608a230b26d14f478401/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/26db7cf14f198310443e41c74970bfefc16299715f03f081a8f02818f023ffe6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/26db7cf14f198310443e41c74970bfefc16299715f03f081a8f02818f023ffe6/diff,workdir=/var/lib/containers/storage/overlay/26db7cf14f198310443e41c74970bfefc16299715f03f081a8f02818f023ffe6/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1d66b1c7d9519712072b919e68eb1504f8dc037496add7c0deaff0adca34f84b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/1d66b1c7d9519712072b919e68eb1504f8dc037496add7c0deaff0adca34f84b/diff,workdir=/var/lib/containers/storage/overlay/1d66b1c7d9519712072b919e68eb1504f8dc037496add7c0deaff0adca34f84b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a4f2c71ab1d7b0eafe3b4b01131de7649f801f6a52a06dbf54271ec6e1d9e9ba/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/a4f2c71ab1d7b0eafe3b4b01131de7649f801f6a52a06dbf54271ec6e1d9e9ba/diff,workdir=/var/lib/containers/storage/overlay/a4f2c71ab1d7b0eafe3b4b01131de7649f801f6a52a06dbf54271ec6e1d9e9ba/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3f5aa4526724d634d8f5887e5e30bac0e81a00a4076efeec9d6f2dafc6f5add3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/47UAQ5FHU6CMZUWEQUEVXPNBNA:/var/lib/containers/storage/overlay/l/AY4Y43S5KXMODHP3KROZ3RQFJ2,upperdir=/var/lib/containers/storage/overlay/3f5aa4526724d634d8f5887e5e30bac0e81a00a4076efeec9d6f2dafc6f5add3/diff,workdir=/var/lib/containers/storage/overlay/3f5aa4526724d634d8f5887e5e30bac0e81a00a4076efeec9d6f2dafc6f5add3/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755,inode64 0 0 2024-06-06T08:51:17.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:51:17.030 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-06-06T08:51:17.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T08:51:17.086 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-06-06T08:51:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2024-06-06T08:51:17.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:51:17.158 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-06T08:51:17.169 INFO:tasks.cephadm:Teardown begin 2024-06-06T08:51:17.169 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-06T08:51:17.170 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:51:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:51:18.860 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:18 smithi027 ceph-mon[32335]: pgmap v343: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:51:18.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:18 smithi203 ceph-mon[36579]: pgmap v343: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:51:19.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:18 smithi176 ceph-mon[36786]: pgmap v343: 129 pgs: 9 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 101 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-06-06T08:51:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.242+0000 7fb7fe54b700 1 -- 172.21.15.27:0/172332910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f8074fe0 msgr2=0x7fb7f810e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.242+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/172332910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f8074fe0 0x7fb7f810e950 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009990 tx=0x7fb7e0009ca0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.243+0000 7fb7fe54b700 1 -- 172.21.15.27:0/172332910 shutdown_connections 2024-06-06T08:51:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.243+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/172332910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7f810ee90 0x7fb7f8111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.243+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/172332910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7f80746c0 0x7fb7f8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.243+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/172332910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f8074fe0 0x7fb7f810e950 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.243+0000 7fb7fe54b700 1 -- 172.21.15.27:0/172332910 >> 172.21.15.27:0/172332910 conn(0x7fb7f80795f0 msgr2=0x7fb7f8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.243+0000 7fb7fe54b700 1 -- 172.21.15.27:0/172332910 shutdown_connections 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.243+0000 7fb7fe54b700 1 -- 172.21.15.27:0/172332910 wait complete. 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.243+0000 7fb7fe54b700 1 Processor -- start 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.243+0000 7fb7fe54b700 1 -- start start 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fe54b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7f80746c0 0x7fb7f819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fe54b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7f8074fe0 0x7fb7f819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fe54b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f810ee90 0x7fb7f819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fe54b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81149a0 con 0x7fb7f810ee90 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fe54b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8114820 con 0x7fb7f80746c0 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fe54b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8114b20 con 0x7fb7f8074fe0 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fcae8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f810ee90 0x7fb7f819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:19.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7f7fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7f80746c0 0x7fb7f819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:19.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fcae8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f810ee90 0x7fb7f819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44566/0 (socket says 172.21.15.27:44566) 2024-06-06T08:51:19.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fcae8700 1 -- 172.21.15.27:0/3021258579 learned_addr learned my addr 172.21.15.27:0/3021258579 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:51:19.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7effff700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7f8074fe0 0x7fb7f819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:19.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fcae8700 1 -- 172.21.15.27:0/3021258579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7f8074fe0 msgr2=0x7fb7f819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:19.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fcae8700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7f8074fe0 0x7fb7f819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fcae8700 1 -- 172.21.15.27:0/3021258579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7f80746c0 msgr2=0x7fb7f819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:19.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fcae8700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7f80746c0 0x7fb7f819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fcae8700 1 -- 172.21.15.27:0/3021258579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f819fdf0 con 0x7fb7f810ee90 2024-06-06T08:51:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fcae8700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f810ee90 0x7fb7f819f6b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00094f0 tx=0x7fb7f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7f5ffb700 1 -- 172.21.15.27:0/3021258579 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0004db0 con 0x7fb7f810ee90 2024-06-06T08:51:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f81a00e0 con 0x7fb7f810ee90 2024-06-06T08:51:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7f5ffb700 1 -- 172.21.15.27:0/3021258579 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f0009e20 con 0x7fb7f810ee90 2024-06-06T08:51:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.244+0000 7fb7f5ffb700 1 -- 172.21.15.27:0/3021258579 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0015400 con 0x7fb7f810ee90 2024-06-06T08:51:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.245+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81a7960 con 0x7fb7f810ee90 2024-06-06T08:51:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.245+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f8069180 con 0x7fb7f810ee90 2024-06-06T08:51:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.247+0000 7fb7f5ffb700 1 -- 172.21.15.27:0/3021258579 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb7f000db20 con 0x7fb7f810ee90 2024-06-06T08:51:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.247+0000 7fb7f5ffb700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb7e40a2d90 0x7fb7e40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.247+0000 7fb7f7fff700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb7e40a2d90 0x7fb7e40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.247+0000 7fb7f5ffb700 1 -- 172.21.15.27:0/3021258579 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9650+0+0 (secure 0 0 0) 0x7fb7f00cf2e0 con 0x7fb7f810ee90 2024-06-06T08:51:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.248+0000 7fb7f7fff700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb7e40a2d90 0x7fb7e40a5250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8005950 tx=0x7fb7e800b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:19.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.252+0000 7fb7f5ffb700 1 -- 172.21.15.27:0/3021258579 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb7f0010070 con 0x7fb7f810ee90 2024-06-06T08:51:19.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.506+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7f81a7b30 con 0x7fb7f810ee90 2024-06-06T08:51:19.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.507+0000 7fb7f5ffb700 1 -- 172.21.15.27:0/3021258579 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11887 (secure 0 0 0) 0x7fb7f009ba90 con 0x7fb7f810ee90 2024-06-06T08:51:19.505 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:51:19.505 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:48:30.185323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:51:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb7e40a2d90 msgr2=0x7fb7e40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb7e40a2d90 0x7fb7e40a5250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8005950 tx=0x7fb7e800b410 comp rx=0 tx=0).stop 2024-06-06T08:51:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f810ee90 msgr2=0x7fb7f819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f810ee90 0x7fb7f819f6b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00094f0 tx=0x7fb7f000d3b0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 shutdown_connections 2024-06-06T08:51:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7f810ee90 0x7fb7f819f6b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fb7e40a2d90 0x7fb7e40a5250 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7f80746c0 0x7fb7f819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 --2- 172.21.15.27:0/3021258579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7f8074fe0 0x7fb7f819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 >> 172.21.15.27:0/3021258579 conn(0x7fb7f80795f0 msgr2=0x7fb7f80ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 shutdown_connections 2024-06-06T08:51:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:19.509+0000 7fb7fe54b700 1 -- 172.21.15.27:0/3021258579 wait complete. 2024-06-06T08:51:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 40 2024-06-06T08:51:19.945 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:19 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3021258579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:51:19.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:19 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3021258579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:51:20.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:19 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3021258579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:51:20.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T08:51:20.903 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:51:21.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:21 smithi027 ceph-mon[32335]: pgmap v344: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-06T08:51:21.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:21 smithi027 ceph-mon[32335]: osdmap e111: 12 total, 12 up, 12 in 2024-06-06T08:51:21.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:21 smithi203 ceph-mon[36579]: pgmap v344: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-06T08:51:21.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:21 smithi203 ceph-mon[36579]: osdmap e111: 12 total, 12 up, 12 in 2024-06-06T08:51:21.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:21 smithi176 ceph-mon[36786]: pgmap v344: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-06T08:51:21.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:21 smithi176 ceph-mon[36786]: osdmap e111: 12 total, 12 up, 12 in 2024-06-06T08:51:23.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.015+0000 7f5dfde50700 1 -- 172.21.15.27:0/2402041926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 msgr2=0x7f5df8101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:23.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.015+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2402041926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 0x7f5df8101a00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5de80099c0 tx=0x7f5de8009cd0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.015+0000 7f5dfde50700 1 -- 172.21.15.27:0/2402041926 shutdown_connections 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.015+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2402041926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df81066b0 0x7f5df810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.015+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2402041926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 0x7f5df8101a00 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.015+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2402041926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df8101fd0 0x7f5df8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.015+0000 7f5dfde50700 1 -- 172.21.15.27:0/2402041926 >> 172.21.15.27:0/2402041926 conn(0x7f5df8078340 msgr2=0x7f5df8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.015+0000 7f5dfde50700 1 -- 172.21.15.27:0/2402041926 shutdown_connections 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.015+0000 7f5dfde50700 1 -- 172.21.15.27:0/2402041926 wait complete. 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.016+0000 7f5dfde50700 1 Processor -- start 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.016+0000 7f5dfde50700 1 -- start start 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5dfde50700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 0x7f5df819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5dfde50700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df8101fd0 0x7f5df819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5dfde50700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df81066b0 0x7f5df8197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:23.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5dfde50700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8111710 con 0x7f5df8101620 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5dfde50700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8111590 con 0x7f5df81066b0 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5dfde50700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5df8111890 con 0x7f5df8101fd0 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 0x7f5df819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 0x7f5df819d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42424/0 (socket says 172.21.15.27:42424) 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df77fe700 1 -- 172.21.15.27:0/2334590313 learned_addr learned my addr 172.21.15.27:0/2334590313 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df7fff700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df81066b0 0x7f5df8197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df77fe700 1 -- 172.21.15.27:0/2334590313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df8101fd0 msgr2=0x7f5df819dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df77fe700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df8101fd0 0x7f5df819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df77fe700 1 -- 172.21.15.27:0/2334590313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df81066b0 msgr2=0x7f5df8197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df77fe700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df81066b0 0x7f5df8197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df77fe700 1 -- 172.21.15.27:0/2334590313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df8198170 con 0x7f5df8101620 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df7fff700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df81066b0 0x7f5df8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.017+0000 7f5df77fe700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 0x7f5df819d850 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5de8006010 tx=0x7f5de8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:23.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.018+0000 7f5df4ff9700 1 -- 172.21.15.27:0/2334590313 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8027070 con 0x7f5df8101620 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.018+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df8198400 con 0x7f5df8101620 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.018+0000 7f5df4ff9700 1 -- 172.21.15.27:0/2334590313 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de8004050 con 0x7f5df8101620 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.018+0000 7f5df4ff9700 1 -- 172.21.15.27:0/2334590313 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8022e10 con 0x7f5df8101620 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.018+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df81a4720 con 0x7f5df8101620 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.019+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df8102ab0 con 0x7f5df8101620 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.020+0000 7f5df4ff9700 1 -- 172.21.15.27:0/2334590313 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5de802a070 con 0x7f5df8101620 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.021+0000 7f5df4ff9700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5de40a2d90 0x7f5de40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.021+0000 7f5df6ffd700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5de40a2d90 0x7f5de40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.021+0000 7f5df4ff9700 1 -- 172.21.15.27:0/2334590313 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9562+0+0 (secure 0 0 0) 0x7f5de80d0ab0 con 0x7f5df8101620 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.021+0000 7f5df6ffd700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5de40a2d90 0x7f5de40a5250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5df8101010 tx=0x7f5de0006d20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:23.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.024+0000 7f5df4ff9700 1 -- 172.21.15.27:0/2334590313 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5de8031070 con 0x7f5df8101620 2024-06-06T08:51:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.276+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5df8198f40 con 0x7f5df8101620 2024-06-06T08:51:23.275 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:23 smithi027 ceph-mon[32335]: pgmap v346: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-06T08:51:23.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.277+0000 7f5df4ff9700 1 -- 172.21.15.27:0/2334590313 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11887 (secure 0 0 0) 0x7f5de8013070 con 0x7f5df8101620 2024-06-06T08:51:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:51:23.276 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1517816957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1517816957},{"type":"v1","addr":"172.21.15.27:6835","nonce":1517816957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1670386995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":1670386995},{"type":"v1","addr":"172.21.15.27:6837","nonce":1670386995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14562,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/2317631267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":2317631267},{"type":"v1","addr":"172.21.15.27:6839","nonce":2317631267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/1085633536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":1085633536},{"type":"v1","addr":"172.21.15.27:6841","nonce":1085633536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/3789595568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3789595568},{"type":"v1","addr":"172.21.15.203:6833","nonce":3789595568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/1587702098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":1587702098},{"type":"v1","addr":"172.21.15.176:6833","nonce":1587702098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T08:36:23.190518+0000","modified":"2024-06-06T08:48:30.185323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24482,"mds_1":24460,"mds_2":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2823423002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2823423002},{"type":"v1","addr":"172.21.15.203:6835","nonce":2823423002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6835/1368203014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1368203014},{"type":"v1","addr":"172.21.15.176:6835","nonce":1368203014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":112,"addr":"172.21.15.176:6837/3697558303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":3697558303},{"type":"v1","addr":"172.21.15.176:6837","nonce":3697558303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":111,"addr":"172.21.15.203:6839/693474346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":693474346},{"type":"v1","addr":"172.21.15.203:6839","nonce":693474346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.176:6839/1702529484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":1702529484},{"type":"v1","addr":"172.21.15.176:6839","nonce":1702529484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25531":{"gid":25531,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/3498487306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3498487306},{"type":"v1","addr":"172.21.15.203:6837","nonce":3498487306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24460,24482,24458]},"id":1}]} 2024-06-06T08:51:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.279+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5de40a2d90 msgr2=0x7f5de40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.279+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5de40a2d90 0x7f5de40a5250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5df8101010 tx=0x7f5de0006d20 comp rx=0 tx=0).stop 2024-06-06T08:51:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.279+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 msgr2=0x7f5df819d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:23.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.279+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 0x7f5df819d850 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5de8006010 tx=0x7f5de8009f80 comp rx=0 tx=0).stop 2024-06-06T08:51:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.279+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 shutdown_connections 2024-06-06T08:51:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.280+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5de40a2d90 0x7f5de40a5250 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.280+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df81066b0 0x7f5df8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.280+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5df8101620 0x7f5df819d850 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.280+0000 7f5dfde50700 1 --2- 172.21.15.27:0/2334590313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5df8101fd0 0x7f5df819dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.280+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 >> 172.21.15.27:0/2334590313 conn(0x7f5df8078340 msgr2=0x7f5df81052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.280+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 shutdown_connections 2024-06-06T08:51:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:23.280+0000 7f5dfde50700 1 -- 172.21.15.27:0/2334590313 wait complete. 2024-06-06T08:51:23.279 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 40 2024-06-06T08:51:23.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:23 smithi203 ceph-mon[36579]: pgmap v346: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-06T08:51:23.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:23 smithi176 ceph-mon[36786]: pgmap v346: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-06T08:51:23.927 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T08:51:24.179 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:51:24.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:24 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2334590313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:51:24.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:24 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2334590313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:51:24.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:24 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2334590313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T08:51:25.355 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:25 smithi027 ceph-mon[32335]: pgmap v347: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-06T08:51:25.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:25 smithi203 ceph-mon[36579]: pgmap v347: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-06T08:51:25.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:25 smithi176 ceph-mon[36786]: pgmap v347: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-06-06T08:51:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.926+0000 7f5c5f614700 1 -- 172.21.15.27:0/2561096118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c58107400 msgr2=0x7f5c581077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.926+0000 7f5c5f614700 1 --2- 172.21.15.27:0/2561096118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c58107400 0x7f5c581077e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009990 tx=0x7f5c54009ca0 comp rx=0 tx=0).stop 2024-06-06T08:51:25.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.926+0000 7f5c5f614700 1 -- 172.21.15.27:0/2561096118 shutdown_connections 2024-06-06T08:51:25.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.926+0000 7f5c5f614700 1 --2- 172.21.15.27:0/2561096118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c58101d60 0x7f5c5810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:25.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.926+0000 7f5c5f614700 1 --2- 172.21.15.27:0/2561096118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c58107400 0x7f5c581077e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:25.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.926+0000 7f5c5f614700 1 --2- 172.21.15.27:0/2561096118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c581013c0 0x7f5c58101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:25.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.926+0000 7f5c5f614700 1 -- 172.21.15.27:0/2561096118 >> 172.21.15.27:0/2561096118 conn(0x7f5c58078330 msgr2=0x7f5c58078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:25.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.926+0000 7f5c5f614700 1 -- 172.21.15.27:0/2561096118 shutdown_connections 2024-06-06T08:51:25.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.926+0000 7f5c5f614700 1 -- 172.21.15.27:0/2561096118 wait complete. 2024-06-06T08:51:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5f614700 1 Processor -- start 2024-06-06T08:51:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5f614700 1 -- start start 2024-06-06T08:51:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5f614700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c581013c0 0x7f5c5819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5f614700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c58101d60 0x7f5c5819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5f614700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c58107400 0x7f5c58198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5f614700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58111640 con 0x7f5c581013c0 2024-06-06T08:51:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5f614700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581114c0 con 0x7f5c58101d60 2024-06-06T08:51:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5f614700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581117c0 con 0x7f5c58107400 2024-06-06T08:51:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5d3b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c581013c0 0x7f5c5819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5d3b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c581013c0 0x7f5c5819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42438/0 (socket says 172.21.15.27:42438) 2024-06-06T08:51:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5d3b0700 1 -- 172.21.15.27:0/3989564244 learned_addr learned my addr 172.21.15.27:0/3989564244 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:51:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5cbaf700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c58101d60 0x7f5c5819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.927+0000 7f5c5dbb1700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c58107400 0x7f5c58198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c5d3b0700 1 -- 172.21.15.27:0/3989564244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c58107400 msgr2=0x7f5c58198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c5d3b0700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c58107400 0x7f5c58198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c5d3b0700 1 -- 172.21.15.27:0/3989564244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c58101d60 msgr2=0x7f5c5819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:25.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c5d3b0700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c58101d60 0x7f5c5819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c5d3b0700 1 -- 172.21.15.27:0/3989564244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c58199230 con 0x7f5c581013c0 2024-06-06T08:51:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c5cbaf700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c58101d60 0x7f5c5819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T08:51:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c5d3b0700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c581013c0 0x7f5c5819e8c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b5c0 tx=0x7f5c540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c4e7fc700 1 -- 172.21.15.27:0/3989564244 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54027070 con 0x7f5c581013c0 2024-06-06T08:51:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c4e7fc700 1 -- 172.21.15.27:0/3989564244 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c54004030 con 0x7f5c581013c0 2024-06-06T08:51:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c581994c0 con 0x7f5c581013c0 2024-06-06T08:51:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c4e7fc700 1 -- 172.21.15.27:0/3989564244 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54022db0 con 0x7f5c581013c0 2024-06-06T08:51:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.928+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c58102fa0 con 0x7f5c581013c0 2024-06-06T08:51:25.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.929+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c58102ab0 con 0x7f5c581013c0 2024-06-06T08:51:25.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.931+0000 7f5c4e7fc700 1 -- 172.21.15.27:0/3989564244 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5c5402a070 con 0x7f5c581013c0 2024-06-06T08:51:25.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.931+0000 7f5c4e7fc700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c440a2d90 0x7f5c440a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:25.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.931+0000 7f5c5cbaf700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c440a2d90 0x7f5c440a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:25.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.931+0000 7f5c4e7fc700 1 -- 172.21.15.27:0/3989564244 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9562+0+0 (secure 0 0 0) 0x7f5c540d0cc0 con 0x7f5c581013c0 2024-06-06T08:51:25.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.932+0000 7f5c5cbaf700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c440a2d90 0x7f5c440a5250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5c580788e0 tx=0x7f5c4800b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:25.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:25.935+0000 7f5c4e7fc700 1 -- 172.21.15.27:0/3989564244 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c54031070 con 0x7f5c581013c0 2024-06-06T08:51:26.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.142+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c58199fb0 con 0x7f5c581013c0 2024-06-06T08:51:26.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.144+0000 7f5c4e7fc700 1 -- 172.21.15.27:0/3989564244 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+25398 (secure 0 0 0) 0x7f5c54013070 con 0x7f5c581013c0 2024-06-06T08:51:26.144 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-06-06T08:51:26.144 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":111,"fsid":"8da40ada-23de-11ef-bc9b-c7b262605968","created":"2024-06-06T08:28:03.871746+0000","modified":"2024-06-06T08:51:20.036304+0000","last_up_change":"2024-06-06T08:35:14.795029+0000","last_in_change":"2024-06-06T08:34:54.069000+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T08:31:22.877337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T08:36:16.098725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T08:36:19.818146+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T08:36:30.341458+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"70143ff8-83c2-42ee-b943-52fd8388c1c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6803","nonce":1837835073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6805","nonce":1837835073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6809","nonce":1837835073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1837835073},{"type":"v1","addr":"172.21.15.27:6807","nonce":1837835073}]},"public_addr":"172.21.15.27:6803/1837835073","cluster_addr":"172.21.15.27:6805/1837835073","heartbeat_back_addr":"172.21.15.27:6809/1837835073","heartbeat_front_addr":"172.21.15.27:6807/1837835073","state":["exists","up"]},{"osd":1,"uuid":"ab40e570-fc0d-4f20-840d-fb0e56483710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6801","nonce":790447916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6803","nonce":790447916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6807","nonce":790447916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":790447916},{"type":"v1","addr":"172.21.15.176:6805","nonce":790447916}]},"public_addr":"172.21.15.176:6801/790447916","cluster_addr":"172.21.15.176:6803/790447916","heartbeat_back_addr":"172.21.15.176:6807/790447916","heartbeat_front_addr":"172.21.15.176:6805/790447916","state":["exists","up"]},{"osd":2,"uuid":"8c91227d-9be5-4fea-81bc-e6696441777f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6801","nonce":252294479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6803","nonce":252294479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6807","nonce":252294479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":252294479},{"type":"v1","addr":"172.21.15.203:6805","nonce":252294479}]},"public_addr":"172.21.15.203:6801/252294479","cluster_addr":"172.21.15.203:6803/252294479","heartbeat_back_addr":"172.21.15.203:6807/252294479","heartbeat_front_addr":"172.21.15.203:6805/252294479","state":["exists","up"]},{"osd":3,"uuid":"aa6a8c4e-0080-4151-aee8-052550a92b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6811","nonce":3079841108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6813","nonce":3079841108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6817","nonce":3079841108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3079841108},{"type":"v1","addr":"172.21.15.27:6815","nonce":3079841108}]},"public_addr":"172.21.15.27:6811/3079841108","cluster_addr":"172.21.15.27:6813/3079841108","heartbeat_back_addr":"172.21.15.27:6817/3079841108","heartbeat_front_addr":"172.21.15.27:6815/3079841108","state":["exists","up"]},{"osd":4,"uuid":"b29f8a35-7dd1-4418-9aa3-dca189a351b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6809","nonce":275393367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6811","nonce":275393367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6815","nonce":275393367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":275393367},{"type":"v1","addr":"172.21.15.176:6813","nonce":275393367}]},"public_addr":"172.21.15.176:6809/275393367","cluster_addr":"172.21.15.176:6811/275393367","heartbeat_back_addr":"172.21.15.176:6815/275393367","heartbeat_front_addr":"172.21.15.176:6813/275393367","state":["exists","up"]},{"osd":5,"uuid":"a98cab95-b7f2-405a-ba80-8814c9a60ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6809","nonce":3782177772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6811","nonce":3782177772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6815","nonce":3782177772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3782177772},{"type":"v1","addr":"172.21.15.203:6813","nonce":3782177772}]},"public_addr":"172.21.15.203:6809/3782177772","cluster_addr":"172.21.15.203:6811/3782177772","heartbeat_back_addr":"172.21.15.203:6815/3782177772","heartbeat_front_addr":"172.21.15.203:6813/3782177772","state":["exists","up"]},{"osd":6,"uuid":"c4261660-83ed-4dbc-9789-405ffbb4389a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6819","nonce":1876109947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6821","nonce":1876109947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6825","nonce":1876109947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1876109947},{"type":"v1","addr":"172.21.15.27:6823","nonce":1876109947}]},"public_addr":"172.21.15.27:6819/1876109947","cluster_addr":"172.21.15.27:6821/1876109947","heartbeat_back_addr":"172.21.15.27:6825/1876109947","heartbeat_front_addr":"172.21.15.27:6823/1876109947","state":["exists","up"]},{"osd":7,"uuid":"e8f52b13-40bd-443c-a8e1-d3ce46bb4397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6817","nonce":3681627280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6819","nonce":3681627280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6823","nonce":3681627280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":3681627280},{"type":"v1","addr":"172.21.15.176:6821","nonce":3681627280}]},"public_addr":"172.21.15.176:6817/3681627280","cluster_addr":"172.21.15.176:6819/3681627280","heartbeat_back_addr":"172.21.15.176:6823/3681627280","heartbeat_front_addr":"172.21.15.176:6821/3681627280","state":["exists","up"]},{"osd":8,"uuid":"fd6572ce-6f94-49c6-9602-07e0196dada4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6817","nonce":2693614915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6819","nonce":2693614915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6823","nonce":2693614915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2693614915},{"type":"v1","addr":"172.21.15.203:6821","nonce":2693614915}]},"public_addr":"172.21.15.203:6817/2693614915","cluster_addr":"172.21.15.203:6819/2693614915","heartbeat_back_addr":"172.21.15.203:6823/2693614915","heartbeat_front_addr":"172.21.15.203:6821/2693614915","state":["exists","up"]},{"osd":9,"uuid":"3bece157-d4cb-4072-a56d-a3e3d3c09f85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6827","nonce":657558341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6829","nonce":657558341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6833","nonce":657558341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":657558341},{"type":"v1","addr":"172.21.15.27:6831","nonce":657558341}]},"public_addr":"172.21.15.27:6827/657558341","cluster_addr":"172.21.15.27:6829/657558341","heartbeat_back_addr":"172.21.15.27:6833/657558341","heartbeat_front_addr":"172.21.15.27:6831/657558341","state":["exists","up"]},{"osd":10,"uuid":"1044fbd0-7574-4faf-a3b1-da2a76a54e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6825","nonce":985198618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6827","nonce":985198618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6831","nonce":985198618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":985198618},{"type":"v1","addr":"172.21.15.176:6829","nonce":985198618}]},"public_addr":"172.21.15.176:6825/985198618","cluster_addr":"172.21.15.176:6827/985198618","heartbeat_back_addr":"172.21.15.176:6831/985198618","heartbeat_front_addr":"172.21.15.176:6829/985198618","state":["exists","up"]},{"osd":11,"uuid":"0c7f5eef-bc5d-4858-abf0-42d889571c21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6825","nonce":1209624343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6827","nonce":1209624343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6831","nonce":1209624343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1209624343},{"type":"v1","addr":"172.21.15.203:6829","nonce":1209624343}]},"public_addr":"172.21.15.203:6825/1209624343","cluster_addr":"172.21.15.203:6827/1209624343","heartbeat_back_addr":"172.21.15.203:6831/1209624343","heartbeat_front_addr":"172.21.15.203:6829/1209624343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:31.742418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:30:55.491549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:19.016148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:31:45.778399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:10.897985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:32:35.463060+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:01.833322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:27.727489+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:33:52.842317+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:20.079457+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:34:46.270375+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T08:35:12.743111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/876705499":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/550980984":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1905066984":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6800/2182206949":"2024-06-07T08:28:58.346290+0000","172.21.15.27:6801/3763593724":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/1183901040":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/1624199721":"2024-06-07T08:28:31.314626+0000","172.21.15.27:6800/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/4174349888":"2024-06-07T08:40:17.280879+0000","172.21.15.27:6801/897818490":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3611712452":"2024-06-07T08:28:58.346290+0000","172.21.15.27:0/1620281536":"2024-06-07T08:28:31.314626+0000","172.21.15.27:0/3053481437":"2024-06-07T08:40:17.280879+0000","172.21.15.27:0/752935431":"2024-06-07T08:28:58.346290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T08:51:26.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c440a2d90 msgr2=0x7f5c440a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c440a2d90 0x7f5c440a5250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5c580788e0 tx=0x7f5c4800b410 comp rx=0 tx=0).stop 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c581013c0 msgr2=0x7f5c5819e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c581013c0 0x7f5c5819e8c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b5c0 tx=0x7f5c540047b0 comp rx=0 tx=0).stop 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 shutdown_connections 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c58107400 0x7f5c58198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c581013c0 0x7f5c5819e8c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f5c440a2d90 0x7f5c440a5250 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 --2- 172.21.15.27:0/3989564244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c58101d60 0x7f5c5819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 >> 172.21.15.27:0/3989564244 conn(0x7f5c58078330 msgr2=0x7f5c5806ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 shutdown_connections 2024-06-06T08:51:26.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:26.146+0000 7f5c5f614700 1 -- 172.21.15.27:0/3989564244 wait complete. 2024-06-06T08:51:26.860 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-06T08:51:27.111 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:51:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:27 smithi027 ceph-mon[32335]: pgmap v348: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-06T08:51:27.206 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:27 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3989564244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:51:27.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:27 smithi203 ceph-mon[36579]: pgmap v348: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-06T08:51:27.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:27 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3989564244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:51:27.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:27 smithi176 ceph-mon[36786]: pgmap v348: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-06T08:51:27.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:27 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3989564244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T08:51:28.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.778+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2327589116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 msgr2=0x7fcfdc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:28.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.778+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2327589116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 0x7fcfdc102b20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00099c0 tx=0x7fcfd0009cd0 comp rx=0 tx=0).stop 2024-06-06T08:51:28.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.778+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2327589116 shutdown_connections 2024-06-06T08:51:28.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.778+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2327589116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfdc107840 0x7fcfdc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:28.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.778+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2327589116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 0x7fcfdc102b20 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:28.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.778+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2327589116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfdc1030f0 0x7fcfdc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:28.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.778+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2327589116 >> 172.21.15.27:0/2327589116 conn(0x7fcfdc0795f0 msgr2=0x7fcfdc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:28.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.778+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2327589116 shutdown_connections 2024-06-06T08:51:28.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.778+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2327589116 wait complete. 2024-06-06T08:51:28.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe2f62700 1 Processor -- start 2024-06-06T08:51:28.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe2f62700 1 -- start start 2024-06-06T08:51:28.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe2f62700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 0x7fcfdc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:28.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe2f62700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfdc1030f0 0x7fcfdc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:28.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe2f62700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfdc107840 0x7fcfdc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:28.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe2f62700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1127e0 con 0x7fcfdc102740 2024-06-06T08:51:28.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe2f62700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc112660 con 0x7fcfdc107840 2024-06-06T08:51:28.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe2f62700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc112960 con 0x7fcfdc1030f0 2024-06-06T08:51:28.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe0cfe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 0x7fcfdc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe0cfe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 0x7fcfdc19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42460/0 (socket says 172.21.15.27:42460) 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfe0cfe700 1 -- 172.21.15.27:0/2908461797 learned_addr learned my addr 172.21.15.27:0/2908461797 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.779+0000 7fcfdbfff700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfdc1030f0 0x7fcfdc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe14ff700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfdc107840 0x7fcfdc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe0cfe700 1 -- 172.21.15.27:0/2908461797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfdc1030f0 msgr2=0x7fcfdc19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe0cfe700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfdc1030f0 0x7fcfdc19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe0cfe700 1 -- 172.21.15.27:0/2908461797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfdc107840 msgr2=0x7fcfdc198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe0cfe700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfdc107840 0x7fcfdc198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe0cfe700 1 -- 172.21.15.27:0/2908461797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc1991e0 con 0x7fcfdc102740 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe14ff700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfdc107840 0x7fcfdc198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe0cfe700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 0x7fcfdc19e870 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009fd0 tx=0x7fcfd0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:28.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfd9ffb700 1 -- 172.21.15.27:0/2908461797 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0027070 con 0x7fcfdc102740 2024-06-06T08:51:28.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfd9ffb700 1 -- 172.21.15.27:0/2908461797 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd0004150 con 0x7fcfdc102740 2024-06-06T08:51:28.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc199470 con 0x7fcfdc102740 2024-06-06T08:51:28.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfd9ffb700 1 -- 172.21.15.27:0/2908461797 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000f460 con 0x7fcfdc102740 2024-06-06T08:51:28.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.780+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a5740 con 0x7fcfdc102740 2024-06-06T08:51:28.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.781+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc103bd0 con 0x7fcfdc102740 2024-06-06T08:51:28.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.783+0000 7fcfd9ffb700 1 -- 172.21.15.27:0/2908461797 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcfd002a070 con 0x7fcfdc102740 2024-06-06T08:51:28.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.784+0000 7fcfd9ffb700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcfcc0a2d90 0x7fcfcc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:28.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.784+0000 7fcfdbfff700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcfcc0a2d90 0x7fcfcc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:28.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.784+0000 7fcfd9ffb700 1 -- 172.21.15.27:0/2908461797 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9562+0+0 (secure 0 0 0) 0x7fcfd00d0ff0 con 0x7fcfdc102740 2024-06-06T08:51:28.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.784+0000 7fcfdbfff700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcfcc0a2d90 0x7fcfcc0a5250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005ea0 tx=0x7fcfc8005e30 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:28.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:28.788+0000 7fcfd9ffb700 1 -- 172.21.15.27:0/2908461797 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcfd0031070 con 0x7fcfdc102740 2024-06-06T08:51:29.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.048+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcfdc02dca0 con 0x7fcfdc102740 2024-06-06T08:51:29.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.050+0000 7fcfd9ffb700 1 -- 172.21.15.27:0/2908461797 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) ==== 161+0+0 (secure 0 0 0) 0x7fcfd0013070 con 0x7fcfdc102740 2024-06-06T08:51:29.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcfcc0a2d90 msgr2=0x7fcfcc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:29.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcfcc0a2d90 0x7fcfcc0a5250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005ea0 tx=0x7fcfc8005e30 comp rx=0 tx=0).stop 2024-06-06T08:51:29.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 msgr2=0x7fcfdc19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:29.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 0x7fcfdc19e870 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009fd0 tx=0x7fcfd0003730 comp rx=0 tx=0).stop 2024-06-06T08:51:29.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 shutdown_connections 2024-06-06T08:51:29.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fcfcc0a2d90 0x7fcfcc0a5250 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005ea0 tx=0x7fcfc8005e30 comp rx=0 tx=0).stop 2024-06-06T08:51:29.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfdc107840 0x7fcfdc198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:29.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfdc102740 0x7fcfdc19e870 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:29.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 --2- 172.21.15.27:0/2908461797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfdc1030f0 0x7fcfdc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:29.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.054+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 >> 172.21.15.27:0/2908461797 conn(0x7fcfdc0795f0 msgr2=0x7fcfdc100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:29.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.055+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 shutdown_connections 2024-06-06T08:51:29.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:29.055+0000 7fcfe2f62700 1 -- 172.21.15.27:0/2908461797 wait complete. 2024-06-06T08:51:29.055 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-06T08:51:29.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:29 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[32331]: 2024-06-06T08:51:29.049+0000 7fc8965f0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T08:51:29.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:29 smithi027 ceph-mon[32335]: pgmap v349: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-06T08:51:29.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:29 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2908461797' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T08:51:29.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:29 smithi027 ceph-mon[32335]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T08:51:29.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:29 smithi027 ceph-mon[32335]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T08:51:29.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:29 smithi027 ceph-mon[32335]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-06-06T08:51:29.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:29 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/2908461797' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T08:51:29.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-mon[36579]: pgmap v349: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-06T08:51:29.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2908461797' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T08:51:29.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-mon[36579]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T08:51:29.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-mon[36579]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T08:51:29.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-mon[36579]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-06-06T08:51:29.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/2908461797' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T08:51:29.490 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[71528]: ignoring --setuser ceph since I am not root 2024-06-06T08:51:29.490 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[71528]: ignoring --setgroup ceph since I am not root 2024-06-06T08:51:29.490 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-mds[71533]: -- 172.21.15.203:0/3904686211 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b574e97a40 con 0x55b574e6b000 2024-06-06T08:51:29.490 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[71528]: starting mds.c at 2024-06-06T08:51:29.491 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f[66895]: ignoring --setuser ceph since I am not root 2024-06-06T08:51:29.491 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f[66895]: ignoring --setgroup ceph since I am not root 2024-06-06T08:51:29.491 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-mds[66899]: -- 172.21.15.203:0/3324901868 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c88347a40 con 0x564c8831b000 2024-06-06T08:51:29.491 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f[66895]: starting mds.f at 2024-06-06T08:51:29.491 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: ignoring --setuser ceph since I am not root 2024-06-06T08:51:29.491 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: ignoring --setgroup ceph since I am not root 2024-06-06T08:51:29.491 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-mds[67506]: -- 172.21.15.203:0/393623763 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558414ba1a40 con 0x558414b75000 2024-06-06T08:51:29.491 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:51:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: starting mds.i at 2024-06-06T08:51:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-mon[36786]: pgmap v349: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-06T08:51:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2908461797' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T08:51:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-mon[36786]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T08:51:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-mon[36786]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T08:51:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-mon[36786]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-06-06T08:51:29.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/2908461797' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T08:51:29.507 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e[66713]: ignoring --setuser ceph since I am not root 2024-06-06T08:51:29.507 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e[66713]: ignoring --setgroup ceph since I am not root 2024-06-06T08:51:29.507 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e[66713]: starting mds.e at 2024-06-06T08:51:29.507 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b[67321]: ignoring --setuser ceph since I am not root 2024-06-06T08:51:29.507 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b[67321]: ignoring --setgroup ceph since I am not root 2024-06-06T08:51:29.507 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-mds[67325]: -- 172.21.15.176:0/3379082148 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5571ab98fa40 con 0x5571ab962c00 2024-06-06T08:51:29.508 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b[67321]: starting mds.b at 2024-06-06T08:51:29.508 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: ignoring --setuser ceph since I am not root 2024-06-06T08:51:29.508 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: ignoring --setgroup ceph since I am not root 2024-06-06T08:51:29.508 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-mds[67022]: -- 172.21.15.176:0/4065260066 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d76913a40 con 0x561d768e7000 2024-06-06T08:51:29.508 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:51:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: starting mds.h at 2024-06-06T08:51:29.788 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-06T08:51:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:51:30.404 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:30 smithi027 ceph-mon[32335]: osdmap e112: 12 total, 12 up, 12 in 2024-06-06T08:51:30.404 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:30 smithi027 ceph-mon[32335]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-06T08:51:30.404 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:30 smithi027 ceph-mon[32335]: pgmap v351: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:51:30.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:30 smithi203 ceph-mon[36579]: osdmap e112: 12 total, 12 up, 12 in 2024-06-06T08:51:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:30 smithi203 ceph-mon[36579]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-06T08:51:30.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:30 smithi203 ceph-mon[36579]: pgmap v351: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:51:30.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:30 smithi176 ceph-mon[36786]: osdmap e112: 12 total, 12 up, 12 in 2024-06-06T08:51:30.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:30 smithi176 ceph-mon[36786]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-06T08:51:30.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:30 smithi176 ceph-mon[36786]: pgmap v351: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T08:51:31.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6834/2719449881,v1:172.21.15.203:6835/2719449881] up:boot 2024-06-06T08:51:31.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6834/3549596184,v1:172.21.15.176:6835/3549596184] up:boot 2024-06-06T08:51:31.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6836/3598440316,v1:172.21.15.176:6838/3598440316] up:boot 2024-06-06T08:51:31.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.176:6837/4196420894,v1:172.21.15.176:6839/4196420894] up:boot 2024-06-06T08:51:31.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6836/2512006051,v1:172.21.15.203:6837/2512006051] up:boot 2024-06-06T08:51:31.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: mds.? [v2:172.21.15.203:6838/4150769833,v1:172.21.15.203:6839/4150769833] up:boot 2024-06-06T08:51:31.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-06T08:51:31.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:51:31.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:51:31.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:51:31.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:51:31.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:51:31.144 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:31 smithi027 ceph-mon[32335]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:51:31.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6834/2719449881,v1:172.21.15.203:6835/2719449881] up:boot 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6834/3549596184,v1:172.21.15.176:6835/3549596184] up:boot 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6836/3598440316,v1:172.21.15.176:6838/3598440316] up:boot 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.176:6837/4196420894,v1:172.21.15.176:6839/4196420894] up:boot 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6836/2512006051,v1:172.21.15.203:6837/2512006051] up:boot 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: mds.? [v2:172.21.15.203:6838/4150769833,v1:172.21.15.203:6839/4150769833] up:boot 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:51:31.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:51:31.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:51:31.491 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:31 smithi203 ceph-mon[36579]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:51:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6834/2719449881,v1:172.21.15.203:6835/2719449881] up:boot 2024-06-06T08:51:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6834/3549596184,v1:172.21.15.176:6835/3549596184] up:boot 2024-06-06T08:51:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6836/3598440316,v1:172.21.15.176:6838/3598440316] up:boot 2024-06-06T08:51:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.176:6837/4196420894,v1:172.21.15.176:6839/4196420894] up:boot 2024-06-06T08:51:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6836/2512006051,v1:172.21.15.203:6837/2512006051] up:boot 2024-06-06T08:51:31.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: mds.? [v2:172.21.15.203:6838/4150769833,v1:172.21.15.203:6839/4150769833] up:boot 2024-06-06T08:51:31.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-06T08:51:31.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T08:51:31.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T08:51:31.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T08:51:31.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T08:51:31.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T08:51:31.507 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:31 smithi176 ceph-mon[36786]: from='mgr.15201 172.21.15.27:0/533730622' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T08:51:31.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.743+0000 7f13bcb95700 1 -- 172.21.15.27:0/2438971760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8101810 msgr2=0x7f13b810f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:31.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.743+0000 7f13bcb95700 1 --2- 172.21.15.27:0/2438971760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8101810 0x7f13b810f280 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f13b000b0a0 tx=0x7f13b000b3b0 comp rx=0 tx=0).stop 2024-06-06T08:51:31.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.743+0000 7f13bcb95700 1 -- 172.21.15.27:0/2438971760 shutdown_connections 2024-06-06T08:51:31.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.743+0000 7f13bcb95700 1 --2- 172.21.15.27:0/2438971760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8101810 0x7f13b810f280 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:31.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.743+0000 7f13bcb95700 1 --2- 172.21.15.27:0/2438971760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b8107570 0x7f13b8107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:31.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.743+0000 7f13bcb95700 1 --2- 172.21.15.27:0/2438971760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b8100e50 0x7f13b81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:31.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.743+0000 7f13bcb95700 1 -- 172.21.15.27:0/2438971760 >> 172.21.15.27:0/2438971760 conn(0x7f13b80766b0 msgr2=0x7f13b8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:31.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.743+0000 7f13bcb95700 1 -- 172.21.15.27:0/2438971760 shutdown_connections 2024-06-06T08:51:31.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.743+0000 7f13bcb95700 1 -- 172.21.15.27:0/2438971760 wait complete. 2024-06-06T08:51:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13bcb95700 1 Processor -- start 2024-06-06T08:51:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13bcb95700 1 -- start start 2024-06-06T08:51:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13bcb95700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8100e50 0x7f13b819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13bcb95700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b8101810 0x7f13b819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13bcb95700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b8107570 0x7f13b8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13bcb95700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f13b81127d0 con 0x7f13b8100e50 2024-06-06T08:51:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13bcb95700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8112650 con 0x7f13b8107570 2024-06-06T08:51:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13bcb95700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8112950 con 0x7f13b8101810 2024-06-06T08:51:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8100e50 0x7f13b819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8100e50 0x7f13b819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34770/0 (socket says 172.21.15.27:34770) 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13b659c700 1 -- 172.21.15.27:0/3231732660 learned_addr learned my addr 172.21.15.27:0/3231732660 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13b6d9d700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b8107570 0x7f13b8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13b5d9b700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b8101810 0x7f13b819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13b659c700 1 -- 172.21.15.27:0/3231732660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b8101810 msgr2=0x7f13b819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.744+0000 7f13b659c700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b8101810 0x7f13b819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.745+0000 7f13b659c700 1 -- 172.21.15.27:0/3231732660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b8107570 msgr2=0x7f13b8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.745+0000 7f13b659c700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b8107570 0x7f13b8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.745+0000 7f13b659c700 1 -- 172.21.15.27:0/3231732660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b81990f0 con 0x7f13b8100e50 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.745+0000 7f13b659c700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8100e50 0x7f13b819e7d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f13a800d980 tx=0x7f13a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.745+0000 7f13a77fe700 1 -- 172.21.15.27:0/3231732660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8017030 con 0x7f13b8100e50 2024-06-06T08:51:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.745+0000 7f13a77fe700 1 -- 172.21.15.27:0/3231732660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a8004500 con 0x7f13b8100e50 2024-06-06T08:51:31.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.745+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b81993e0 con 0x7f13b8100e50 2024-06-06T08:51:31.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.745+0000 7f13a77fe700 1 -- 172.21.15.27:0/3231732660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8003aa0 con 0x7f13b8100e50 2024-06-06T08:51:31.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.745+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b81a56a0 con 0x7f13b8100e50 2024-06-06T08:51:31.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.746+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b8069180 con 0x7f13b8100e50 2024-06-06T08:51:31.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.747+0000 7f13a77fe700 1 -- 172.21.15.27:0/3231732660 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f13a801a070 con 0x7f13b8100e50 2024-06-06T08:51:31.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.748+0000 7f13a77fe700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13a00a2e10 0x7f13a00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:31.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.748+0000 7f13a77fe700 1 -- 172.21.15.27:0/3231732660 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9732+0+0 (secure 0 0 0) 0x7f13a80cf560 con 0x7f13b8100e50 2024-06-06T08:51:31.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.748+0000 7f13b5d9b700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13a00a2e10 0x7f13a00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:31.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.749+0000 7f13b5d9b700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13a00a2e10 0x7f13a00a52d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f13ac005950 tx=0x7f13ac0058e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:31.753+0000 7f13a77fe700 1 -- 172.21.15.27:0/3231732660 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f13a809bd40 con 0x7f13b8100e50 2024-06-06T08:51:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.002+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f13b8199ef0 con 0x7f13b8100e50 2024-06-06T08:51:32.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.077+0000 7f13a77fe700 1 -- 172.21.15.27:0/3231732660 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) ==== 104+0+0 (secure 0 0 0) 0x7f13b8199ef0 con 0x7f13b8100e50 2024-06-06T08:51:32.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13a00a2e10 msgr2=0x7f13a00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:32.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13a00a2e10 0x7f13a00a52d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f13ac005950 tx=0x7f13ac0058e0 comp rx=0 tx=0).stop 2024-06-06T08:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8100e50 msgr2=0x7f13b819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8100e50 0x7f13b819e7d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f13a800d980 tx=0x7f13a800dc90 comp rx=0 tx=0).stop 2024-06-06T08:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 shutdown_connections 2024-06-06T08:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f13a00a2e10 0x7f13a00a52d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b8107570 0x7f13b8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13b8100e50 0x7f13b819e7d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 --2- 172.21.15.27:0/3231732660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b8101810 0x7f13b819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 >> 172.21.15.27:0/3231732660 conn(0x7f13b80766b0 msgr2=0x7f13b80ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:32.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 shutdown_connections 2024-06-06T08:51:32.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:32.079+0000 7f13bcb95700 1 -- 172.21.15.27:0/3231732660 wait complete. 2024-06-06T08:51:32.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:32 smithi027 ceph-mon[32335]: pgmap v352: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T08:51:32.403 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:32 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3231732660' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:51:32.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:32 smithi203 ceph-mon[36579]: pgmap v352: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T08:51:32.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:32 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3231732660' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:51:32.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:32 smithi176 ceph-mon[36786]: pgmap v352: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-06T08:51:32.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:32 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3231732660' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T08:51:32.760 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-06T08:51:33.015 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:51:33.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:33 smithi027 ceph-mon[32335]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T08:51:33.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:33 smithi027 ceph-mon[32335]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T08:51:33.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:33 smithi027 ceph-mon[32335]: Cluster is now healthy 2024-06-06T08:51:33.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:33 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/3231732660' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T08:51:33.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:33 smithi027 ceph-mon[32335]: fsmap 12 up:standby 2024-06-06T08:51:33.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:33 smithi203 ceph-mon[36579]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T08:51:33.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:33 smithi203 ceph-mon[36579]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T08:51:33.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:33 smithi203 ceph-mon[36579]: Cluster is now healthy 2024-06-06T08:51:33.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:33 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/3231732660' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T08:51:33.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:33 smithi203 ceph-mon[36579]: fsmap 12 up:standby 2024-06-06T08:51:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:33 smithi176 ceph-mon[36786]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T08:51:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:33 smithi176 ceph-mon[36786]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T08:51:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:33 smithi176 ceph-mon[36786]: Cluster is now healthy 2024-06-06T08:51:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:33 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/3231732660' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T08:51:33.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:33 smithi176 ceph-mon[36786]: fsmap 12 up:standby 2024-06-06T08:51:34.171 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:34 smithi027 ceph-mon[32335]: pgmap v353: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-06-06T08:51:34.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:34 smithi203 ceph-mon[36579]: pgmap v353: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-06-06T08:51:34.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:34 smithi176 ceph-mon[36786]: pgmap v353: 129 pgs: 129 active+clean; 39 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.728+0000 7f2f9c867700 1 -- 172.21.15.27:0/2862748877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98101d40 msgr2=0x7f2f9810e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.728+0000 7f2f9c867700 1 --2- 172.21.15.27:0/2862748877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98101d40 0x7f2f9810e230 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000b0a0 tx=0x7f2f9000b3b0 comp rx=0 tx=0).stop 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.728+0000 7f2f9c867700 1 -- 172.21.15.27:0/2862748877 shutdown_connections 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.728+0000 7f2f9c867700 1 --2- 172.21.15.27:0/2862748877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f98101d40 0x7f2f9810e230 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.728+0000 7f2f9c867700 1 --2- 172.21.15.27:0/2862748877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f981073f0 0x7f2f981077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.728+0000 7f2f9c867700 1 --2- 172.21.15.27:0/2862748877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f98101380 0x7f2f98101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.728+0000 7f2f9c867700 1 -- 172.21.15.27:0/2862748877 >> 172.21.15.27:0/2862748877 conn(0x7f2f98078300 msgr2=0x7f2f98078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.728+0000 7f2f9c867700 1 -- 172.21.15.27:0/2862748877 shutdown_connections 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.728+0000 7f2f9c867700 1 -- 172.21.15.27:0/2862748877 wait complete. 2024-06-06T08:51:34.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f9c867700 1 Processor -- start 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f9c867700 1 -- start start 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f9c867700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f98101380 0x7f2f98199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f9c867700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f98101d40 0x7f2f9819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f9c867700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f981073f0 0x7f2f981a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f9c867700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98111660 con 0x7f2f981073f0 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f9c867700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981114e0 con 0x7f2f98101380 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f9c867700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981117e0 con 0x7f2f98101d40 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f96d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f981073f0 0x7f2f981a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f96d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f981073f0 0x7f2f981a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34790/0 (socket says 172.21.15.27:34790) 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f96d9d700 1 -- 172.21.15.27:0/552753715 learned_addr learned my addr 172.21.15.27:0/552753715 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f9659c700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f98101380 0x7f2f98199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:34.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.729+0000 7f2f95d9b700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f98101d40 0x7f2f9819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f96d9d700 1 -- 172.21.15.27:0/552753715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f98101d40 msgr2=0x7f2f9819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f96d9d700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f98101d40 0x7f2f9819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f96d9d700 1 -- 172.21.15.27:0/552753715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f98101380 msgr2=0x7f2f98199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f96d9d700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f98101380 0x7f2f98199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f96d9d700 1 -- 172.21.15.27:0/552753715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f981a50a0 con 0x7f2f981073f0 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f96d9d700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f981073f0 0x7f2f981a4b60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f900062a0 tx=0x7f2f9000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f877fe700 1 -- 172.21.15.27:0/552753715 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90021070 con 0x7f2f981073f0 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f877fe700 1 -- 172.21.15.27:0/552753715 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f90007c80 con 0x7f2f981073f0 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f981a5270 con 0x7f2f981073f0 2024-06-06T08:51:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f877fe700 1 -- 172.21.15.27:0/552753715 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90005420 con 0x7f2f981073f0 2024-06-06T08:51:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.730+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f981a5720 con 0x7f2f981073f0 2024-06-06T08:51:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.731+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f98069180 con 0x7f2f981073f0 2024-06-06T08:51:34.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.732+0000 7f2f877fe700 1 -- 172.21.15.27:0/552753715 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2f90024070 con 0x7f2f981073f0 2024-06-06T08:51:34.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.733+0000 7f2f877fe700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f800a2d90 0x7f2f800a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:34.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.733+0000 7f2f9659c700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f800a2d90 0x7f2f800a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:34.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.733+0000 7f2f877fe700 1 -- 172.21.15.27:0/552753715 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9732+0+0 (secure 0 0 0) 0x7f2f9000f020 con 0x7f2f981073f0 2024-06-06T08:51:34.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.734+0000 7f2f9659c700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f800a2d90 0x7f2f800a5250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005950 tx=0x7f2f8800a300 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:34.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.737+0000 7f2f877fe700 1 -- 172.21.15.27:0/552753715 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f9002b030 con 0x7f2f981073f0 2024-06-06T08:51:34.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:34.943+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2f9802dca0 con 0x7f2f981073f0 2024-06-06T08:51:35.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.083+0000 7f2f877fe700 1 -- 172.21.15.27:0/552753715 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) ==== 181+0+0 (secure 0 0 0) 0x7f2f9009d3e0 con 0x7f2f981073f0 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f800a2d90 msgr2=0x7f2f800a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f800a2d90 0x7f2f800a5250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005950 tx=0x7f2f8800a300 comp rx=0 tx=0).stop 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f981073f0 msgr2=0x7f2f981a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f981073f0 0x7f2f981a4b60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f900062a0 tx=0x7f2f9000be60 comp rx=0 tx=0).stop 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 shutdown_connections 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f981073f0 0x7f2f981a4b60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7f2f800a2d90 0x7f2f800a5250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2f98101380 0x7f2f98199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 --2- 172.21.15.27:0/552753715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f98101d40 0x7f2f9819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 >> 172.21.15.27:0/552753715 conn(0x7f2f98078300 msgr2=0x7f2f980ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 shutdown_connections 2024-06-06T08:51:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:35.087+0000 7f2f9c867700 1 -- 172.21.15.27:0/552753715 wait complete. 2024-06-06T08:51:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-06-06T08:51:35.205 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:35 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/552753715' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:51:35.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:35 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/552753715' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:51:35.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:35 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/552753715' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:51:35.795 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-06T08:51:36.048 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:51:36.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:36 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/552753715' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:51:36.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:36 smithi027 ceph-mon[32335]: osdmap e113: 12 total, 12 up, 12 in 2024-06-06T08:51:36.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:36 smithi027 ceph-mon[32335]: pgmap v355: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:36.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:36 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/552753715' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:51:36.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:36 smithi203 ceph-mon[36579]: osdmap e113: 12 total, 12 up, 12 in 2024-06-06T08:51:36.490 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:36 smithi203 ceph-mon[36579]: pgmap v355: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:36.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:36 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/552753715' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:51:36.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:36 smithi176 ceph-mon[36786]: osdmap e113: 12 total, 12 up, 12 in 2024-06-06T08:51:36.506 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:36 smithi176 ceph-mon[36786]: pgmap v355: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.675+0000 7fbaaabb4700 1 -- 172.21.15.27:0/3820393876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 msgr2=0x7fbaa41077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.675+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/3820393876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 0x7fbaa41077d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b0a0 tx=0x7fba9400b3b0 comp rx=0 tx=0).stop 2024-06-06T08:51:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.675+0000 7fbaaabb4700 1 -- 172.21.15.27:0/3820393876 shutdown_connections 2024-06-06T08:51:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.675+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/3820393876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaa4101d40 0x7fbaa410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.675+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/3820393876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 0x7fbaa41077d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.675+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/3820393876 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaa4101380 0x7fbaa4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.675+0000 7fbaaabb4700 1 -- 172.21.15.27:0/3820393876 >> 172.21.15.27:0/3820393876 conn(0x7fbaa4078300 msgr2=0x7fbaa4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.675+0000 7fbaaabb4700 1 -- 172.21.15.27:0/3820393876 shutdown_connections 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.675+0000 7fbaaabb4700 1 -- 172.21.15.27:0/3820393876 wait complete. 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaaabb4700 1 Processor -- start 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaaabb4700 1 -- start start 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaaabb4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaa4101380 0x7fbaa4103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaaabb4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaa4101d40 0x7fbaa41035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaaabb4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 0x7fbaa4103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaaabb4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa4111660 con 0x7fbaa41073f0 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaaabb4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41114e0 con 0x7fbaa4101380 2024-06-06T08:51:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaaabb4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41117e0 con 0x7fbaa4101d40 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaa9151700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 0x7fbaa4103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaa9151700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 0x7fbaa4103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34812/0 (socket says 172.21.15.27:34812) 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.676+0000 7fbaa9151700 1 -- 172.21.15.27:0/66268346 learned_addr learned my addr 172.21.15.27:0/66268346 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa3fff700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaa4101d40 0x7fbaa41035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa8950700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaa4101380 0x7fbaa4103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa9151700 1 -- 172.21.15.27:0/66268346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaa4101d40 msgr2=0x7fbaa41035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa9151700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaa4101d40 0x7fbaa41035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa9151700 1 -- 172.21.15.27:0/66268346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaa4101380 msgr2=0x7fbaa4103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa9151700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaa4101380 0x7fbaa4103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa9151700 1 -- 172.21.15.27:0/66268346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa41a7120 con 0x7fbaa41073f0 2024-06-06T08:51:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa8950700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaa4101380 0x7fbaa4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:51:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa9151700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 0x7fbaa4103ae0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00c900 tx=0x7fba9c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa1ffb700 1 -- 172.21.15.27:0/66268346 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c007b00 con 0x7fbaa41073f0 2024-06-06T08:51:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa1ffb700 1 -- 172.21.15.27:0/66268346 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba9c00ce50 con 0x7fbaa41073f0 2024-06-06T08:51:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa41a7410 con 0x7fbaa41073f0 2024-06-06T08:51:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaa1ffb700 1 -- 172.21.15.27:0/66268346 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c018660 con 0x7fbaa41073f0 2024-06-06T08:51:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.677+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa41a7950 con 0x7fbaa41073f0 2024-06-06T08:51:37.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.678+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa4069180 con 0x7fbaa41073f0 2024-06-06T08:51:37.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.679+0000 7fbaa1ffb700 1 -- 172.21.15.27:0/66268346 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fba9c00f450 con 0x7fbaa41073f0 2024-06-06T08:51:37.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.680+0000 7fbaa1ffb700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fba900a2e10 0x7fba900a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:37.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.680+0000 7fbaa8950700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fba900a2e10 0x7fba900a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:37.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.680+0000 7fbaa1ffb700 1 -- 172.21.15.27:0/66268346 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9170+0+0 (secure 0 0 0) 0x7fba9c0cf580 con 0x7fbaa41073f0 2024-06-06T08:51:37.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.681+0000 7fbaa8950700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fba900a2e10 0x7fba900a52d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b830 tx=0x7fba94009390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:37.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.685+0000 7fbaa1ffb700 1 -- 172.21.15.27:0/66268346 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba9c014030 con 0x7fbaa41073f0 2024-06-06T08:51:37.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:37.893+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbaa41a7ce0 con 0x7fbaa41073f0 2024-06-06T08:51:38.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.555+0000 7fbaa1ffb700 1 -- 172.21.15.27:0/66268346 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) ==== 169+0+0 (secure 0 0 0) 0x7fba9c09bf10 con 0x7fbaa41073f0 2024-06-06T08:51:38.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.559+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fba900a2e10 msgr2=0x7fba900a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:38.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.559+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fba900a2e10 0x7fba900a52d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b830 tx=0x7fba94009390 comp rx=0 tx=0).stop 2024-06-06T08:51:38.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.559+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 msgr2=0x7fbaa4103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:38.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.559+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 0x7fbaa4103ae0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00c900 tx=0x7fba9c00cc10 comp rx=0 tx=0).stop 2024-06-06T08:51:38.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.560+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 shutdown_connections 2024-06-06T08:51:38.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.560+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa41073f0 0x7fbaa4103ae0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:38.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.560+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fba900a2e10 0x7fba900a52d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:38.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.560+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbaa4101380 0x7fbaa4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:38.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.560+0000 7fbaaabb4700 1 --2- 172.21.15.27:0/66268346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaa4101d40 0x7fbaa41035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:38.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.560+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 >> 172.21.15.27:0/66268346 conn(0x7fbaa4078300 msgr2=0x7fbaa4100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:38.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.561+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 shutdown_connections 2024-06-06T08:51:38.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:38.561+0000 7fbaaabb4700 1 -- 172.21.15.27:0/66268346 wait complete. 2024-06-06T08:51:38.561 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-06-06T08:51:38.948 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:38 smithi027 ceph-mon[32335]: pgmap v356: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:38.948 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:38 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/66268346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:51:38.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:38 smithi203 ceph-mon[36579]: pgmap v356: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:38.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:38 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/66268346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:51:39.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:38 smithi176 ceph-mon[36786]: pgmap v356: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:39.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:38 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/66268346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:51:39.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-06T08:51:39.502 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T08:51:39.838 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:39 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/66268346' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:51:39.838 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:39 smithi027 ceph-mon[32335]: osdmap e114: 12 total, 12 up, 12 in 2024-06-06T08:51:39.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:39 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/66268346' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:51:39.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:39 smithi203 ceph-mon[36579]: osdmap e114: 12 total, 12 up, 12 in 2024-06-06T08:51:40.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:39 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/66268346' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:51:40.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:39 smithi176 ceph-mon[36786]: osdmap e114: 12 total, 12 up, 12 in 2024-06-06T08:51:40.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:40 smithi027 ceph-mon[32335]: pgmap v358: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:40.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:40 smithi203 ceph-mon[36579]: pgmap v358: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:41.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:40 smithi176 ceph-mon[36786]: pgmap v358: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:41.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 -- 172.21.15.27:0/3876651550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc81010b0 msgr2=0x7fabc8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:41.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 --2- 172.21.15.27:0/3876651550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc81010b0 0x7fabc8101490 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009990 tx=0x7fabb8009ca0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 -- 172.21.15.27:0/3876651550 shutdown_connections 2024-06-06T08:51:41.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 --2- 172.21.15.27:0/3876651550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabc8105f50 0x7fabc810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 --2- 172.21.15.27:0/3876651550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc81010b0 0x7fabc8101490 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 --2- 172.21.15.27:0/3876651550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc8101a60 0x7fabc8105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 -- 172.21.15.27:0/3876651550 >> 172.21.15.27:0/3876651550 conn(0x7fabc80766b0 msgr2=0x7fabc8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 -- 172.21.15.27:0/3876651550 shutdown_connections 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 -- 172.21.15.27:0/3876651550 wait complete. 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.081+0000 7fabcd40f700 1 Processor -- start 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabcd40f700 1 -- start start 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabcd40f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabc81010b0 0x7fabc810f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabcd40f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc8101a60 0x7fabc810f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabcd40f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc8105f50 0x7fabc810fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabcd40f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fabc8112870 con 0x7fabc8105f50 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabcd40f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81126f0 con 0x7fabc81010b0 2024-06-06T08:51:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabcd40f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81129f0 con 0x7fabc8101a60 2024-06-06T08:51:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabc77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc8105f50 0x7fabc810fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabc77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc8105f50 0x7fabc810fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55014/0 (socket says 172.21.15.27:55014) 2024-06-06T08:51:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabc77fe700 1 -- 172.21.15.27:0/4238559649 learned_addr learned my addr 172.21.15.27:0/4238559649 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-06-06T08:51:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabc6ffd700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabc81010b0 0x7fabc810f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabc67fc700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc8101a60 0x7fabc810f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabc77fe700 1 -- 172.21.15.27:0/4238559649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc8101a60 msgr2=0x7fabc810f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabc77fe700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc8101a60 0x7fabc810f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.082+0000 7fabc77fe700 1 -- 172.21.15.27:0/4238559649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabc81010b0 msgr2=0x7fabc810f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.083+0000 7fabc77fe700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabc81010b0 0x7fabc810f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.083+0000 7fabc77fe700 1 -- 172.21.15.27:0/4238559649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc81a70c0 con 0x7fabc8105f50 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.083+0000 7fabc67fc700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc8101a60 0x7fabc810f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.083+0000 7fabc77fe700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc8105f50 0x7fabc810fc30 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fabc0007d70 tx=0x7fabc000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.083+0000 7fabaffff700 1 -- 172.21.15.27:0/4238559649 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0016020 con 0x7fabc8105f50 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.083+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc81a73b0 con 0x7fabc8105f50 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.083+0000 7fabaffff700 1 -- 172.21.15.27:0/4238559649 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc000ad40 con 0x7fabc8105f50 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.083+0000 7fabaffff700 1 -- 172.21.15.27:0/4238559649 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc00115c0 con 0x7fabc8105f50 2024-06-06T08:51:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.083+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc81a7940 con 0x7fabc8105f50 2024-06-06T08:51:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.084+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc81090e0 con 0x7fabc8105f50 2024-06-06T08:51:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.085+0000 7fabaffff700 1 -- 172.21.15.27:0/4238559649 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fabc0019020 con 0x7fabc8105f50 2024-06-06T08:51:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.086+0000 7fabaffff700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fabb00a2d90 0x7fabb00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T08:51:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.086+0000 7fabc6ffd700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fabb00a2d90 0x7fabb00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T08:51:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.086+0000 7fabaffff700 1 -- 172.21.15.27:0/4238559649 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8582+0+0 (secure 0 0 0) 0x7fabc00cf2b0 con 0x7fabc8105f50 2024-06-06T08:51:41.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.087+0000 7fabc6ffd700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fabb00a2d90 0x7fabb00a5250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabb800b5c0 tx=0x7fabb8005960 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T08:51:41.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.091+0000 7fabaffff700 1 -- 172.21.15.27:0/4238559649 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fabc0020020 con 0x7fabc8105f50 2024-06-06T08:51:41.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.297+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fabc8108ac0 con 0x7fabc8105f50 2024-06-06T08:51:41.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.569+0000 7fabaffff700 1 -- 172.21.15.27:0/4238559649 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v115) ==== 178+0+0 (secure 0 0 0) 0x7fabc009bfa0 con 0x7fabc8105f50 2024-06-06T08:51:41.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.572+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fabb00a2d90 msgr2=0x7fabb00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:41.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.572+0000 7fabcd40f700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fabb00a2d90 0x7fabb00a5250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabb800b5c0 tx=0x7fabb8005960 comp rx=0 tx=0).stop 2024-06-06T08:51:41.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.572+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc8105f50 msgr2=0x7fabc810fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T08:51:41.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.572+0000 7fabcd40f700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc8105f50 0x7fabc810fc30 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fabc0007d70 tx=0x7fabc000a340 comp rx=0 tx=0).stop 2024-06-06T08:51:41.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.573+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 shutdown_connections 2024-06-06T08:51:41.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.573+0000 7fabcd40f700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabc8105f50 0x7fabc810fc30 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.573+0000 7fabcd40f700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.27:6800/1196153485,v1:172.21.15.27:6801/1196153485] conn(0x7fabb00a2d90 0x7fabb00a5250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.573+0000 7fabcd40f700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabc81010b0 0x7fabc810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.573+0000 7fabcd40f700 1 --2- 172.21.15.27:0/4238559649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fabc8101a60 0x7fabc810f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T08:51:41.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.573+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 >> 172.21.15.27:0/4238559649 conn(0x7fabc80766b0 msgr2=0x7fabc80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T08:51:41.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.573+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 shutdown_connections 2024-06-06T08:51:41.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-06-06T08:51:41.573+0000 7fabcd40f700 1 -- 172.21.15.27:0/4238559649 wait complete. 2024-06-06T08:51:41.573 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' removed 2024-06-06T08:51:41.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:41 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4238559649' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:51:41.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:41 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4238559649' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:51:42.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:41 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4238559649' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T08:51:42.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:51:42.232 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:51:42.265 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:51:42.299 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-06T08:51:42.299 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T08:51:42.317 DEBUG:teuthology.orchestra.run.smithi176:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T08:51:42.333 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T08:51:42.356 INFO:tasks.cephadm:Stopping all daemons... 2024-06-06T08:51:42.357 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-06T08:51:42.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.a 2024-06-06T08:51:42.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:42 smithi027 systemd[1]: Stopping Ceph mon.a for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:51:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:42 smithi027 ceph-mon[32335]: pgmap v359: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:42 smithi027 ceph-mon[32335]: from='client.? 172.21.15.27:0/4238559649' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:51:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:42 smithi027 ceph-mon[32335]: osdmap e115: 12 total, 12 up, 12 in 2024-06-06T08:51:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:42 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[32331]: 2024-06-06T08:51:42.689+0000 7fc89bdfb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:51:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:42 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a[32331]: 2024-06-06T08:51:42.689+0000 7fc89bdfb700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-06T08:51:42.956 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:42 smithi027 podman[110709]: 2024-06-06 08:51:42.721054347 +0000 UTC m=+0.156103026 container died ddea9b723a2d2154bf44eb00b3345d1b0260a202d628926c929137d265f4843a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-06T08:51:42.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:42 smithi203 ceph-mon[36579]: pgmap v359: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:42.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:42 smithi203 ceph-mon[36579]: from='client.? 172.21.15.27:0/4238559649' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:51:42.990 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:42 smithi203 ceph-mon[36579]: osdmap e115: 12 total, 12 up, 12 in 2024-06-06T08:51:43.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:42 smithi176 ceph-mon[36786]: pgmap v359: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T08:51:43.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:42 smithi176 ceph-mon[36786]: from='client.? 172.21.15.27:0/4238559649' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T08:51:43.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:42 smithi176 ceph-mon[36786]: osdmap e115: 12 total, 12 up, 12 in 2024-06-06T08:51:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:43 smithi027 podman[110709]: 2024-06-06 08:51:43.131544627 +0000 UTC m=+0.566593303 container cleanup ddea9b723a2d2154bf44eb00b3345d1b0260a202d628926c929137d265f4843a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-06T08:51:43.456 INFO:journalctl@ceph.mon.a.smithi027.stdout:Jun 06 08:51:43 smithi027 bash[110709]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-a 2024-06-06T08:51:43.672 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-06T08:51:43.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:51:43.724 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-06T08:51:43.724 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-06T08:51:43.724 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.b 2024-06-06T08:51:44.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:43 smithi176 systemd[1]: Stopping Ceph mon.b for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:51:44.456 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:44 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-b[36782]: 2024-06-06T08:51:44.047+0000 7f507f3f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:51:44.456 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:44 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-b[36782]: 2024-06-06T08:51:44.048+0000 7f507f3f1700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-06T08:51:44.456 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:44 smithi176 podman[73506]: 2024-06-06 08:51:44.171236698 +0000 UTC m=+0.247752293 container died e636c7c3892f72f6a915ae818401de738affce058b04e6c0d29c8296db1bf5f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-06T08:51:44.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:44 smithi176 podman[73506]: 2024-06-06 08:51:44.557852607 +0000 UTC m=+0.634368201 container cleanup e636c7c3892f72f6a915ae818401de738affce058b04e6c0d29c8296db1bf5f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-b, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:51:44.756 INFO:journalctl@ceph.mon.b.smithi176.stdout:Jun 06 08:51:44 smithi176 bash[73506]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-b 2024-06-06T08:51:44.925 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-06T08:51:44.992 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:51:44.992 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-06T08:51:44.993 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-06T08:51:44.993 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.c 2024-06-06T08:51:45.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:45 smithi203 systemd[1]: Stopping Ceph mon.c for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:51:45.739 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:45 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-c[36575]: 2024-06-06T08:51:45.290+0000 7f50502eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:51:45.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:45 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-c[36575]: 2024-06-06T08:51:45.290+0000 7f50502eb700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-06T08:51:45.740 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:45 smithi203 podman[74145]: 2024-06-06 08:51:45.444990369 +0000 UTC m=+0.251759366 container died 276655564d2fc611c839927ffd967345da9739347f4d67cfb84a114381d4d734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-c, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:51:46.108 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:45 smithi203 podman[74145]: 2024-06-06 08:51:45.850275918 +0000 UTC m=+0.657044876 container cleanup 276655564d2fc611c839927ffd967345da9739347f4d67cfb84a114381d4d734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-06T08:51:46.108 INFO:journalctl@ceph.mon.c.smithi203.stdout:Jun 06 08:51:45 smithi203 bash[74145]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mon-c 2024-06-06T08:51:46.285 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-06T08:51:46.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:51:46.329 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-06T08:51:46.329 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-06T08:51:46.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T08:51:46.652 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:51:46 smithi027 systemd[1]: Stopping Ceph mgr.x for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:51:46.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:51:46 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x[32567]: 2024-06-06T08:51:46.653+0000 7f9105e2b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:51:46.956 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:51:46 smithi027 podman[110828]: 2024-06-06 08:51:46.685396501 +0000 UTC m=+0.161306048 container died e42a772f7b3d3c767939c76a22f6c558d37807f78ec1d1c8bc1ffbf6c2bc8b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-06T08:51:47.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:51:47 smithi027 podman[110828]: 2024-06-06 08:51:47.156975007 +0000 UTC m=+0.632884522 container cleanup e42a772f7b3d3c767939c76a22f6c558d37807f78ec1d1c8bc1ffbf6c2bc8b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-06T08:51:47.456 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Jun 06 08:51:47 smithi027 bash[110828]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-x 2024-06-06T08:51:47.646 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-06T08:51:47.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:51:47.683 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-06T08:51:47.683 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-06T08:51:47.684 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.y 2024-06-06T08:51:48.006 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:51:47 smithi176 systemd[1]: Stopping Ceph mgr.y for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:51:48.311 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:51:48 smithi176 podman[73628]: 2024-06-06 08:51:48.008276466 +0000 UTC m=+0.129098605 container died 26d069ee88a772a8e634b75c19d4b502a6d166684abb588671c50fa64aa16b37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc) 2024-06-06T08:51:48.688 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:51:48 smithi176 podman[73628]: 2024-06-06 08:51:48.41333294 +0000 UTC m=+0.534155071 container cleanup 26d069ee88a772a8e634b75c19d4b502a6d166684abb588671c50fa64aa16b37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc) 2024-06-06T08:51:48.688 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Jun 06 08:51:48 smithi176 bash[73628]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-y 2024-06-06T08:51:48.845 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-06T08:51:48.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:51:48.879 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-06T08:51:48.880 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-06T08:51:48.880 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.z 2024-06-06T08:51:49.224 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:51:48 smithi203 systemd[1]: Stopping Ceph mgr.z for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:51:49.489 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:51:49 smithi203 podman[74264]: 2024-06-06 08:51:49.227045044 +0000 UTC m=+0.149220385 container died a7190f80265abf2a453ac436f682ad3c34b8b8f7fa61167e9f7e1197952df9bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-06T08:51:49.906 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:51:49 smithi203 podman[74264]: 2024-06-06 08:51:49.607061118 +0000 UTC m=+0.529236466 container cleanup a7190f80265abf2a453ac436f682ad3c34b8b8f7fa61167e9f7e1197952df9bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:51:49.906 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Jun 06 08:51:49 smithi203 bash[74264]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mgr-z 2024-06-06T08:51:50.033 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-06T08:51:50.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:51:50.068 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-06T08:51:50.068 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-06T08:51:50.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.0 2024-06-06T08:51:50.426 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:50 smithi027 systemd[1]: Stopping Ceph osd.0 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:51:50.706 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:50 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0[46879]: 2024-06-06T08:51:50.428+0000 7f55e912f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:51:50.706 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:50 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0[46879]: 2024-06-06T08:51:50.428+0000 7f55e912f700 -1 osd.0 115 *** Got signal Terminated *** 2024-06-06T08:51:50.706 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:50 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0[46879]: 2024-06-06T08:51:50.428+0000 7f55e912f700 -1 osd.0 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:51:56.086 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:55 smithi027 podman[110949]: 2024-06-06 08:51:55.7851775 +0000 UTC m=+5.509673879 container died 041bb80bb98c3fc9f9276fee51ba6eed61db863047eec2dc2456458005f909b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-06T08:51:56.456 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:56 smithi027 podman[110949]: 2024-06-06 08:51:56.196518063 +0000 UTC m=+5.921014441 container cleanup 041bb80bb98c3fc9f9276fee51ba6eed61db863047eec2dc2456458005f909b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:51:56.456 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:56 smithi027 bash[110949]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0 2024-06-06T08:51:56.955 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:56 smithi027 podman[110967]: 2024-06-06 08:51:56.548171318 +0000 UTC m=+0.758472026 container remove 041bb80bb98c3fc9f9276fee51ba6eed61db863047eec2dc2456458005f909b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:51:57.275 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:56 smithi027 podman[111057]: 2024-06-06 08:51:56.895165474 +0000 UTC m=+0.017594538 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:51:57.275 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:57 smithi027 podman[111057]: 2024-06-06 08:51:57.024363394 +0000 UTC m=+0.146792433 container create 835ad146a1e6f7b285ceea127717f4532670b3a4df12f304ddcda71d6438a6b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-06-06T08:51:57.597 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:57 smithi027 podman[111057]: 2024-06-06 08:51:57.277446837 +0000 UTC m=+0.399875873 container init 835ad146a1e6f7b285ceea127717f4532670b3a4df12f304ddcda71d6438a6b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T08:51:57.597 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:57 smithi027 podman[111057]: 2024-06-06 08:51:57.281211007 +0000 UTC m=+0.403640058 container start 835ad146a1e6f7b285ceea127717f4532670b3a4df12f304ddcda71d6438a6b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T08:51:57.598 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:57 smithi027 podman[111057]: 2024-06-06 08:51:57.360834607 +0000 UTC m=+0.483263676 container attach 835ad146a1e6f7b285ceea127717f4532670b3a4df12f304ddcda71d6438a6b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-06T08:51:57.895 INFO:journalctl@ceph.osd.0.smithi027.stdout:Jun 06 08:51:57 smithi027 podman[111057]: 2024-06-06 08:51:57.59928098 +0000 UTC m=+0.721710025 container died 835ad146a1e6f7b285ceea127717f4532670b3a4df12f304ddcda71d6438a6b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:51:58.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-06T08:51:58.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:51:58.259 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-06T08:51:58.259 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-06T08:51:58.260 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.1 2024-06-06T08:51:58.574 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:51:58 smithi176 systemd[1]: Stopping Ceph osd.1 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:51:59.006 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:51:58 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1[41353]: 2024-06-06T08:51:58.575+0000 7fec1f5c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:51:59.006 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:51:58 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1[41353]: 2024-06-06T08:51:58.575+0000 7fec1f5c4700 -1 osd.1 115 *** Got signal Terminated *** 2024-06-06T08:51:59.006 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:51:58 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1[41353]: 2024-06-06T08:51:58.575+0000 7fec1f5c4700 -1 osd.1 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:52:04.350 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:04 smithi176 podman[73764]: 2024-06-06 08:52:04.058956488 +0000 UTC m=+5.596023238 container died 993160e51dd310c14503979f927a1d5368e90b504c4e2a463da787d7db41794a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-06T08:52:04.692 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:04 smithi176 podman[73764]: 2024-06-06 08:52:04.451469722 +0000 UTC m=+5.988536456 container cleanup 993160e51dd310c14503979f927a1d5368e90b504c4e2a463da787d7db41794a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:52:04.692 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:04 smithi176 bash[73764]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1 2024-06-06T08:52:05.006 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:04 smithi176 podman[73780]: 2024-06-06 08:52:04.694701286 +0000 UTC m=+0.635722436 container remove 993160e51dd310c14503979f927a1d5368e90b504c4e2a463da787d7db41794a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-06T08:52:05.388 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:05 smithi176 podman[73871]: 2024-06-06 08:52:05.030590833 +0000 UTC m=+0.017093296 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:52:05.388 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:05 smithi176 podman[73871]: 2024-06-06 08:52:05.146076364 +0000 UTC m=+0.132578837 container create 417fc7b1a1525d7ac9ed913e2b71be01ac5b06b1cf285c1b01396968c2e70414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T08:52:05.682 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:05 smithi176 podman[73871]: 2024-06-06 08:52:05.390207008 +0000 UTC m=+0.376709463 container init 417fc7b1a1525d7ac9ed913e2b71be01ac5b06b1cf285c1b01396968c2e70414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-06T08:52:05.682 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:05 smithi176 podman[73871]: 2024-06-06 08:52:05.393632576 +0000 UTC m=+0.380135033 container start 417fc7b1a1525d7ac9ed913e2b71be01ac5b06b1cf285c1b01396968c2e70414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-06T08:52:05.682 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:05 smithi176 podman[73871]: 2024-06-06 08:52:05.490410259 +0000 UTC m=+0.476912714 container attach 417fc7b1a1525d7ac9ed913e2b71be01ac5b06b1cf285c1b01396968c2e70414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-06T08:52:06.006 INFO:journalctl@ceph.osd.1.smithi176.stdout:Jun 06 08:52:05 smithi176 podman[73871]: 2024-06-06 08:52:05.684610746 +0000 UTC m=+0.671113209 container died 417fc7b1a1525d7ac9ed913e2b71be01ac5b06b1cf285c1b01396968c2e70414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True) 2024-06-06T08:52:06.380 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-06T08:52:06.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:52:06.421 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-06T08:52:06.421 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-06T08:52:06.421 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.2 2024-06-06T08:52:06.734 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:06 smithi203 systemd[1]: Stopping Ceph osd.2 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:52:06.989 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2[41696]: 2024-06-06T08:52:06.736+0000 7f836992b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:52:06.989 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2[41696]: 2024-06-06T08:52:06.736+0000 7f836992b700 -1 osd.2 115 *** Got signal Terminated *** 2024-06-06T08:52:06.990 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2[41696]: 2024-06-06T08:52:06.736+0000 7f836992b700 -1 osd.2 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:52:12.610 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:12 smithi203 podman[74389]: 2024-06-06 08:52:12.309988227 +0000 UTC m=+5.696665636 container died b9ee16b5d62ac783e020b45d1b4481036be5b0f466b720977a46509f48e7ad82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-06T08:52:12.945 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:12 smithi203 podman[74389]: 2024-06-06 08:52:12.703766505 +0000 UTC m=+6.090443915 container cleanup b9ee16b5d62ac783e020b45d1b4481036be5b0f466b720977a46509f48e7ad82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:52:12.945 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:12 smithi203 bash[74389]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2 2024-06-06T08:52:13.239 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:12 smithi203 podman[74407]: 2024-06-06 08:52:12.947095226 +0000 UTC m=+0.634643861 container remove b9ee16b5d62ac783e020b45d1b4481036be5b0f466b720977a46509f48e7ad82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc) 2024-06-06T08:52:13.599 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:13 smithi203 podman[74495]: 2024-06-06 08:52:13.187388443 +0000 UTC m=+0.017668050 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:52:13.599 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:13 smithi203 podman[74495]: 2024-06-06 08:52:13.2985212 +0000 UTC m=+0.128800797 container create ea60e8e2652c2d19c8a23ba3629d627d4f1016e8cc2550d89a9b1fb374dbbd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-06T08:52:13.599 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:13 smithi203 podman[74495]: 2024-06-06 08:52:13.509775972 +0000 UTC m=+0.340055579 container init ea60e8e2652c2d19c8a23ba3629d627d4f1016e8cc2550d89a9b1fb374dbbd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:52:13.600 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:13 smithi203 podman[74495]: 2024-06-06 08:52:13.512957992 +0000 UTC m=+0.343237598 container start ea60e8e2652c2d19c8a23ba3629d627d4f1016e8cc2550d89a9b1fb374dbbd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-06T08:52:13.600 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:13 smithi203 podman[74495]: 2024-06-06 08:52:13.601499124 +0000 UTC m=+0.431778743 container attach ea60e8e2652c2d19c8a23ba3629d627d4f1016e8cc2550d89a9b1fb374dbbd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:52:13.990 INFO:journalctl@ceph.osd.2.smithi203.stdout:Jun 06 08:52:13 smithi203 podman[74495]: 2024-06-06 08:52:13.815585217 +0000 UTC m=+0.645864821 container died ea60e8e2652c2d19c8a23ba3629d627d4f1016e8cc2550d89a9b1fb374dbbd6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-06T08:52:14.481 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-06T08:52:14.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:52:14.519 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-06T08:52:14.519 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-06T08:52:14.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.3 2024-06-06T08:52:14.850 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:14 smithi027 systemd[1]: Stopping Ceph osd.3 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:52:15.206 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3[53334]: 2024-06-06T08:52:14.851+0000 7fc6e9901700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:52:15.206 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3[53334]: 2024-06-06T08:52:14.851+0000 7fc6e9901700 -1 osd.3 115 *** Got signal Terminated *** 2024-06-06T08:52:15.206 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:14 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3[53334]: 2024-06-06T08:52:14.851+0000 7fc6e9901700 -1 osd.3 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:52:16.956 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:16 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3[53334]: 2024-06-06T08:52:16.652+0000 7fc6e5ef5700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:51.840562+0000 front 2024-06-06T08:51:51.840591+0000 (oldest deadline 2024-06-06T08:52:15.940889+0000) 2024-06-06T08:52:17.956 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:17 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3[53334]: 2024-06-06T08:52:17.700+0000 7fc6e5ef5700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:51.840562+0000 front 2024-06-06T08:51:51.840591+0000 (oldest deadline 2024-06-06T08:52:15.940889+0000) 2024-06-06T08:52:18.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:17 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:17.767+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:18.955 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:18 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3[53334]: 2024-06-06T08:52:18.674+0000 7fc6e5ef5700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:51.840562+0000 front 2024-06-06T08:51:51.840591+0000 (oldest deadline 2024-06-06T08:52:15.940889+0000) 2024-06-06T08:52:19.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:18 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:18.738+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:19.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:19.284+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:19.956 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:19 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3[53334]: 2024-06-06T08:52:19.648+0000 7fc6e5ef5700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:51.840562+0000 front 2024-06-06T08:51:51.840591+0000 (oldest deadline 2024-06-06T08:52:15.940889+0000) 2024-06-06T08:52:19.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:19 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:19.592+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:20.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:19 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:19.711+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:20.610 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:20 smithi027 podman[111174]: 2024-06-06 08:52:20.309223548 +0000 UTC m=+5.579203725 container died e93a91fcb96745ea8b51f68ddc6c806065aa73e5fd6c9243bf9c653ba6987fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-06T08:52:20.739 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:20.254+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:20.956 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:20 smithi027 podman[111174]: 2024-06-06 08:52:20.701769535 +0000 UTC m=+5.971749700 container cleanup e93a91fcb96745ea8b51f68ddc6c806065aa73e5fd6c9243bf9c653ba6987fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:52:20.956 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:20 smithi027 bash[111174]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3 2024-06-06T08:52:20.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:20 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:20.618+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:21.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:20 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:20.724+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:21.330 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:20 smithi027 podman[111189]: 2024-06-06 08:52:20.969889667 +0000 UTC m=+0.657811239 container remove e93a91fcb96745ea8b51f68ddc6c806065aa73e5fd6c9243bf9c653ba6987fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-06T08:52:21.605 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:21 smithi027 podman[111278]: 2024-06-06 08:52:21.236099346 +0000 UTC m=+0.017681836 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:52:21.605 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:21 smithi027 podman[111278]: 2024-06-06 08:52:21.354344257 +0000 UTC m=+0.135926769 container create 1b1cdc3b798d72784af572ac47e3261a27a7b50e8f2f67220db0bae2d6446f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:52:21.605 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:21 smithi027 podman[111278]: 2024-06-06 08:52:21.607575511 +0000 UTC m=+0.389157997 container init 1b1cdc3b798d72784af572ac47e3261a27a7b50e8f2f67220db0bae2d6446f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-06T08:52:21.739 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:21 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:21.252+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:21.912 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:21 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:21.626+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:21.912 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:21 smithi027 podman[111278]: 2024-06-06 08:52:21.611616047 +0000 UTC m=+0.393198518 container start 1b1cdc3b798d72784af572ac47e3261a27a7b50e8f2f67220db0bae2d6446f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:52:21.912 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:21 smithi027 podman[111278]: 2024-06-06 08:52:21.723960778 +0000 UTC m=+0.505543251 container attach 1b1cdc3b798d72784af572ac47e3261a27a7b50e8f2f67220db0bae2d6446f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-06T08:52:22.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:21 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:21.742+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:22.206 INFO:journalctl@ceph.osd.3.smithi027.stdout:Jun 06 08:52:21 smithi027 podman[111278]: 2024-06-06 08:52:21.914832648 +0000 UTC m=+0.696415139 container died 1b1cdc3b798d72784af572ac47e3261a27a7b50e8f2f67220db0bae2d6446f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-06T08:52:22.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:22.213+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:22.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:22.014+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:22.607 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-06T08:52:22.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:52:22.650 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-06T08:52:22.650 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-06T08:52:22.650 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.4 2024-06-06T08:52:22.953 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:22 smithi176 systemd[1]: Stopping Ceph osd.4 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:52:22.953 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:22 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:22.775+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:22.955 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:22 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:22.661+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:23.256 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:22 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4[48286]: 2024-06-06T08:52:22.954+0000 7fbb65f3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:52:23.256 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:22 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4[48286]: 2024-06-06T08:52:22.954+0000 7fbb65f3f700 -1 osd.4 115 *** Got signal Terminated *** 2024-06-06T08:52:23.257 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:22 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4[48286]: 2024-06-06T08:52:22.954+0000 7fbb65f3f700 -1 osd.4 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:52:23.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:23.046+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:23.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:23.176+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:23.955 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:23 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:23.702+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:24.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:23 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:23.797+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:24.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:24.054+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:24.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:24.186+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:25.205 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:24 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:24.740+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:25.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:24 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:24.767+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:25.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:24 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:24.767+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:25.456 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:25 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:25.069+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:25.456 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:25 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:25.144+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:25.739 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:25 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:25.458+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:25.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:25 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:25.699+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:25.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:25 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:25.699+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:26.239 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:26 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:26.165+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:26.240 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:26 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:26.086+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:26.240 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:26 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:26.086+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:26.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:25 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:25.782+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:26.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:25 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:25.782+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:26.739 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:26 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:26.474+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:26.955 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:26 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:26.693+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:26.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:26 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:26.693+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:27.179 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:26 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:26.788+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:27.179 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:26 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:26.788+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:27.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:27 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:27.174+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:27.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:27 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:27.043+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:27.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:27 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:27.043+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:27.506 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:27 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:27.181+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:27.653 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:27 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:27.247+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:27.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:27 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:27.652+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:27.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:27 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:27.652+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:27.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:27 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:27.516+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:28.193 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:27 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:27.795+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:28.194 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:27 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:27.795+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:28.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:28 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:28.091+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:28.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:28 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:28.091+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:28.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:28 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:28.162+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:28.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:28 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:28.162+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:28.506 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:28 smithi176 podman[74001]: 2024-06-06 08:52:28.387906814 +0000 UTC m=+5.528611896 container died d2a49b94d24fa42870087b09b0ce78b094e6e152ac997807ee335c9476ea0631 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-06T08:52:28.506 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:28 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:28.195+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:28.688 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:28 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:28.272+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:28.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:28 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:28.690+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:28.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:28 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:28.690+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:28.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:28 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:28.557+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:29.006 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:28 smithi176 podman[74001]: 2024-06-06 08:52:28.814017039 +0000 UTC m=+5.954722120 container cleanup d2a49b94d24fa42870087b09b0ce78b094e6e152ac997807ee335c9476ea0631 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:52:29.006 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:28 smithi176 bash[74001]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4 2024-06-06T08:52:29.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:28 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:28.820+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:29.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:28 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:28.820+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:29.480 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:29.212+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:29.481 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:29 smithi176 podman[74019]: 2024-06-06 08:52:29.040562259 +0000 UTC m=+0.643216527 container remove d2a49b94d24fa42870087b09b0ce78b094e6e152ac997807ee335c9476ea0631 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-06T08:52:29.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:29.082+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:29.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:29.082+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:29.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:29.204+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:29.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:29.204+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:29.706 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:29 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:29.299+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:29.756 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:29 smithi176 podman[74107]: 2024-06-06 08:52:29.483332523 +0000 UTC m=+0.105551570 container create bd44b1e4fad1158294a3c606fd2dfcd0f1954e7dc550f343a152ef86c691b998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-06T08:52:29.756 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:29 smithi176 podman[74107]: 2024-06-06 08:52:29.39524117 +0000 UTC m=+0.017460234 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:52:29.756 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:29 smithi176 podman[74107]: 2024-06-06 08:52:29.727629087 +0000 UTC m=+0.349848143 container init bd44b1e4fad1158294a3c606fd2dfcd0f1954e7dc550f343a152ef86c691b998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-06T08:52:29.756 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:29 smithi176 podman[74107]: 2024-06-06 08:52:29.730835324 +0000 UTC m=+0.353054373 container start bd44b1e4fad1158294a3c606fd2dfcd0f1954e7dc550f343a152ef86c691b998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-06T08:52:29.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:29 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:29.602+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:30.205 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:29 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:29.708+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:30.206 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:29 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:29.708+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:30.212 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:29.808+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:30.212 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:29 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:29.808+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:30.213 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:29 smithi176 podman[74107]: 2024-06-06 08:52:29.827795993 +0000 UTC m=+0.450015048 container attach bd44b1e4fad1158294a3c606fd2dfcd0f1954e7dc550f343a152ef86c691b998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-06T08:52:30.213 INFO:journalctl@ceph.osd.4.smithi176.stdout:Jun 06 08:52:30 smithi176 podman[74107]: 2024-06-06 08:52:30.020683499 +0000 UTC m=+0.642902554 container died bd44b1e4fad1158294a3c606fd2dfcd0f1954e7dc550f343a152ef86c691b998 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-06T08:52:30.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:30 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:30.109+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:30.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:30 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:30.109+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:30.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:30 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:30.173+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:30.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:30 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:30.173+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:30.506 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:30 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:30.213+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:30.644 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-06T08:52:30.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:52:30.682 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-06T08:52:30.682 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-06T08:52:30.682 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.5 2024-06-06T08:52:30.705 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:30.267+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:30.982 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:30 smithi203 systemd[1]: Stopping Ceph osd.5 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:52:30.982 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:30 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:30.632+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:31.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:30 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:30.758+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:31.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:30 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:30.758+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:31.205 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:30.718+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:31.206 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:30 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:30.718+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:31.239 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:31 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:31.203+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:31.240 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:31 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:31.203+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:31.240 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:30 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:30.982+0000 7f1a6411b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:52:31.240 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:30 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:30.982+0000 7f1a6411b700 -1 osd.5 115 *** Got signal Terminated *** 2024-06-06T08:52:31.240 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:30 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:30.982+0000 7f1a6411b700 -1 osd.5 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:52:31.240 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:31 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:31.084+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:31.240 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:31 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:31.084+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:31.506 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:31 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:31.234+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:31.675 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:31 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:31.257+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:31.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:31 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:31.677+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:31.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:31 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:31.677+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:31.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:31 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:31.595+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:32.241 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:31 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:31.782+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:32.241 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:31 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:31.782+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:32.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:32 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:32.042+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:32.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:32 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:32.042+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:32.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:32 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:32.206+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:32.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:32 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:32.206+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:32.506 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:32 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:32.242+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:32.673 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:32 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:32.285+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:32.955 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:32 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:32.675+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:32.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:32 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:32.675+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:32.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:32 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:32.601+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:33.228 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:32 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:32.821+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:33.228 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:32 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:32.821+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:33.228 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:32 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:32.821+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:33.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:32 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:32.994+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:33.489 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:32 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:32.994+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:33.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:33 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:33.191+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:33.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:33 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:33.191+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:33.506 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:33 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:33.230+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:33.506 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:33 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:33.230+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:33.688 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:33 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:33.288+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:33.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:33 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:33.690+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:33.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:33 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:33.690+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:33.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:33 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:33.627+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:33.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:33 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:33.627+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:34.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:33 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:33.830+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:34.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:33 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:33.830+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:34.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:33 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:33.830+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:34.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:34 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:34.208+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:34.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:34 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:34.208+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:34.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:34 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:34.038+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:34.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:34 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:34.038+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:34.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:34 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:34.038+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:09.773560+0000 front 2024-06-06T08:52:09.773511+0000 (oldest deadline 2024-06-06T08:52:33.273741+0000) 2024-06-06T08:52:34.642 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:34 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:34.279+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:34.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:34 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:34.279+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:34.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:34 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:34.279+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:34.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:34 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:34.644+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:34.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:34 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:34.644+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:34.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:34 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:34.673+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:34.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:34 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:34.673+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:35.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:34 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:34.796+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:35.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:34 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:34.796+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:35.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:34 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:34.796+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:35.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:35 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:35.204+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:35.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:35 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:35.204+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:35.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:35 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:35.055+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.470586+0000 front 2024-06-06T08:51:53.470614+0000 (oldest deadline 2024-06-06T08:52:21.071487+0000) 2024-06-06T08:52:35.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:35 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:35.055+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.072141+0000 front 2024-06-06T08:52:01.072209+0000 (oldest deadline 2024-06-06T08:52:25.172022+0000) 2024-06-06T08:52:35.490 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:35 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5[48485]: 2024-06-06T08:52:35.055+0000 7f1a6070f700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:09.773560+0000 front 2024-06-06T08:52:09.773511+0000 (oldest deadline 2024-06-06T08:52:33.273741+0000) 2024-06-06T08:52:35.637 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:35 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:35.254+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:35.637 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:35 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:35.254+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.277743+0000 front 2024-06-06T08:52:10.277686+0000 (oldest deadline 2024-06-06T08:52:34.377717+0000) 2024-06-06T08:52:35.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:35 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:35.273+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:35.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:35 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:35.273+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:35.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:35 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:35.639+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:35.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:35 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:35.639+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:35.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:35 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:35.639+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:35.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:35 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:35.715+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:35.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:35 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:35.715+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:36.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:35 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:35.810+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:36.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:35 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:35.810+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:36.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:35 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:35.810+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:36.451 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:36 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:36.193+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:36.452 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:36 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:36.193+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:36.638 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:36.230+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:36.638 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:36.230+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.277743+0000 front 2024-06-06T08:52:10.277686+0000 (oldest deadline 2024-06-06T08:52:34.377717+0000) 2024-06-06T08:52:36.739 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:36 smithi203 podman[74613]: 2024-06-06 08:52:36.454194094 +0000 UTC m=+5.567373327 container died 8d1eb7db8a8835ac996f2978a7c624c4cadd01a391c36ffb52614ba2f49cfe4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T08:52:36.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:36 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:36.298+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:36.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:36 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:36.298+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:36.955 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:36.639+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:36.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:36.639+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:36.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:36.639+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:37.126 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:36 smithi203 podman[74613]: 2024-06-06 08:52:36.835006143 +0000 UTC m=+5.948185376 container cleanup 8d1eb7db8a8835ac996f2978a7c624c4cadd01a391c36ffb52614ba2f49cfe4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-06T08:52:37.127 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:36 smithi203 bash[74613]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5 2024-06-06T08:52:37.127 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:36 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:36.741+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:37.127 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:36 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:36.741+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:37.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:36 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:36.835+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:37.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:36 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:36.835+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:37.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:36 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:36.835+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:37.456 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:37 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:37.193+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:37.456 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:37 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:37.193+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.277743+0000 front 2024-06-06T08:52:10.277686+0000 (oldest deadline 2024-06-06T08:52:34.377717+0000) 2024-06-06T08:52:37.459 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:37 smithi203 podman[74628]: 2024-06-06 08:52:37.12879806 +0000 UTC m=+0.671515606 container remove 8d1eb7db8a8835ac996f2978a7c624c4cadd01a391c36ffb52614ba2f49cfe4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-06T08:52:37.459 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:37 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:37.218+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:37.459 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:37 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:37.218+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:37.740 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:37 smithi203 podman[74715]: 2024-06-06 08:52:37.364161741 +0000 UTC m=+0.017226004 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:52:37.740 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:37 smithi203 podman[74715]: 2024-06-06 08:52:37.471361512 +0000 UTC m=+0.124425807 container create 4a9293d581f438c793b0b24c091857f40d75d61f503fe8b69b88193cf37a9a8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:52:37.740 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:37 smithi203 podman[74715]: 2024-06-06 08:52:37.691304196 +0000 UTC m=+0.344368455 container init 4a9293d581f438c793b0b24c091857f40d75d61f503fe8b69b88193cf37a9a8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-06T08:52:37.740 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:37 smithi203 podman[74715]: 2024-06-06 08:52:37.694510373 +0000 UTC m=+0.347574623 container start 4a9293d581f438c793b0b24c091857f40d75d61f503fe8b69b88193cf37a9a8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-06T08:52:37.740 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:37 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:37.701+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:37.740 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:37 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:37.701+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:37.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:37 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:37.260+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:37.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:37 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:37.260+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:37.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:37 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:37.674+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:37.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:37 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:37.674+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:37.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:37 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:37.674+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:38.224 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:37 smithi203 podman[74715]: 2024-06-06 08:52:37.774386496 +0000 UTC m=+0.427450745 container attach 4a9293d581f438c793b0b24c091857f40d75d61f503fe8b69b88193cf37a9a8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-06T08:52:38.224 INFO:journalctl@ceph.osd.5.smithi203.stdout:Jun 06 08:52:37 smithi203 podman[74715]: 2024-06-06 08:52:37.991318431 +0000 UTC m=+0.644382682 container died 4a9293d581f438c793b0b24c091857f40d75d61f503fe8b69b88193cf37a9a8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528) 2024-06-06T08:52:38.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:37 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:37.848+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:38.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:37 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:37.848+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:38.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:37 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:37.848+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:38.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:38 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:38.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:38.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:38 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:38.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:38.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:38 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:38.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:38.626 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-06T08:52:38.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:52:38.663 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-06T08:52:38.663 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-06T08:52:38.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.6 2024-06-06T08:52:38.691 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:38 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:38.219+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:38.691 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:38 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:38.219+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.277743+0000 front 2024-06-06T08:52:10.277686+0000 (oldest deadline 2024-06-06T08:52:34.377717+0000) 2024-06-06T08:52:38.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:38 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:38.295+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:38.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:38 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:38.295+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:38.956 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:38 smithi027 systemd[1]: Stopping Ceph osd.6 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:52:38.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:38 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:38.708+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:38.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:38 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:38.708+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:38.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:38 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:38.708+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:38.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:38 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:38.730+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:38.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:38 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:38.730+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:39.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:38 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:38.842+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:39.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:38 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:38.842+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:39.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:38 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:38.842+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:39.456 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:39 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:39.005+0000 7f5f159df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:52:39.456 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:39 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:39.005+0000 7f5f159df700 -1 osd.6 115 *** Got signal Terminated *** 2024-06-06T08:52:39.456 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:39 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:39.005+0000 7f5f159df700 -1 osd.6 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:52:39.456 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:39 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:39.242+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:39.456 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:39 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:39.242+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.277743+0000 front 2024-06-06T08:52:10.277686+0000 (oldest deadline 2024-06-06T08:52:34.377717+0000) 2024-06-06T08:52:39.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:39 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:39.195+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:39.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:39 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:39.195+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:39.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:39 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:39.196+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:39.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:39 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:39.319+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:39.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:39 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:39.319+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:39.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:39 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:39.680+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:39.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:39 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:39.680+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:39.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:39 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:39.680+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:39.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:39 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:39.714+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:39.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:39 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:39.714+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:40.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:39 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:39.815+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:40.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:39 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:39.815+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:40.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:39 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:39.815+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:40.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:40 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:40.197+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:40.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:40 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:40.197+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:40.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:40 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:40.197+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:40.662 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:40 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:40.232+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:40.662 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:40 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:40.232+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.277743+0000 front 2024-06-06T08:52:10.277686+0000 (oldest deadline 2024-06-06T08:52:34.377717+0000) 2024-06-06T08:52:40.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:40 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:40.333+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:40.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:40 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:40.333+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:40.955 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:40 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:40.664+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:40.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:40 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:40.664+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:40.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:40 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:40.664+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:41.152 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:40 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:40.748+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:41.153 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:40 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:40.748+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:41.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:40 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:40.834+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:41.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:40 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:40.834+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:41.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:40 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:40.834+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:41.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:41 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:41.154+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:41.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:41 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:41.154+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:41.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:41 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:41.154+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:41.661 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:41 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:41.272+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:41.661 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:41 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:41.272+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.277743+0000 front 2024-06-06T08:52:10.277686+0000 (oldest deadline 2024-06-06T08:52:34.377717+0000) 2024-06-06T08:52:41.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:41 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:41.368+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:41.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:41 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:41.368+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:41.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:41 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:41.662+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:41.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:41 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:41.662+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:41.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:41 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:41.662+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:41.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:41 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:41.662+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:42.200 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:41 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:41.759+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:42.201 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:41 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:41.760+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:42.201 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:41 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:41.760+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:42.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:41 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:41.833+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:42.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:41 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:41.833+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:42.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:41 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:41.833+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:42.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:41 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:41.833+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:42.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:42 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:42.202+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:42.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:42 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:42.202+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:42.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:42 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:42.202+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:42.636 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:42 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:42.251+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:42.636 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:42 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:42.251+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.277743+0000 front 2024-06-06T08:52:10.277686+0000 (oldest deadline 2024-06-06T08:52:34.377717+0000) 2024-06-06T08:52:42.636 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:42 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:42.251+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:17.179027+0000 front 2024-06-06T08:52:17.179320+0000 (oldest deadline 2024-06-06T08:52:41.278962+0000) 2024-06-06T08:52:42.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:42 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:42.376+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:42.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:42 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:42.376+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:42.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:42 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:42.638+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:42.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:42 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:42.638+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:42.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:42 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:42.638+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:42.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:42 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:42.638+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:43.166 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:42 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:42.809+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:43.167 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:42 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:42.809+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:43.167 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:42 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:42.809+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:43.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:42 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:42.861+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:43.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:42 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:42.861+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:43.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:42 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:42.861+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:43.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:42 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:42.861+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:43.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:43 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:43.168+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:43.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:43 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:43.168+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:43.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:43 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:43.168+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:43.620 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:43 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:43.277+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.074869+0000 front 2024-06-06T08:51:58.074710+0000 (oldest deadline 2024-06-06T08:52:26.875627+0000) 2024-06-06T08:52:43.620 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:43 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:43.277+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.277743+0000 front 2024-06-06T08:52:10.277686+0000 (oldest deadline 2024-06-06T08:52:34.377717+0000) 2024-06-06T08:52:43.620 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:43 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6[60282]: 2024-06-06T08:52:43.277+0000 7f5f11fd3700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:17.179027+0000 front 2024-06-06T08:52:17.179320+0000 (oldest deadline 2024-06-06T08:52:41.278962+0000) 2024-06-06T08:52:43.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:43 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:43.383+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:43.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:43 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:43.383+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:43.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:43 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:43.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:43.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:43 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:43.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:43.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:43 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:43.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:43.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:43 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:43.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:44.180 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:43 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:43.811+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:44.180 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:43 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:43.811+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:44.180 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:43 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:43.811+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:44.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:43 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:43.882+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:44.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:43 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:43.882+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:44.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:43 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:43.882+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:44.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:43 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:43.882+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:44.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:44 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:44.182+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:44.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:44 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:44.182+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:44.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:44 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:44.182+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:44.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:44 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:44.407+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:44.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:44 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:44.407+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:44.788 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:44 smithi027 podman[111383]: 2024-06-06 08:52:44.501616626 +0000 UTC m=+5.623845931 container died 537977a48b3ccd5309f79acff478a6ce0a8047e42427791727cc28cc9bc9e335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-06T08:52:44.788 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:44 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:44.589+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:44.788 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:44 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:44.589+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:44.788 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:44 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:44.589+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:44.788 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:44 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:44.589+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:45.189 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:44 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:44.832+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:45.189 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:44 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:44.832+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:45.190 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:44 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:44.832+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:45.206 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:44 smithi027 podman[111383]: 2024-06-06 08:52:44.89775786 +0000 UTC m=+6.019987152 container cleanup 537977a48b3ccd5309f79acff478a6ce0a8047e42427791727cc28cc9bc9e335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD) 2024-06-06T08:52:45.206 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:44 smithi027 bash[111383]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6 2024-06-06T08:52:45.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:44 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:44.905+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:45.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:44 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:44.905+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:45.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:44 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:44.905+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:45.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:44 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:44.905+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:45.461 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:45 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:45.191+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:45.461 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:45 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:45.191+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:45.461 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:45 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:45.191+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:45.604 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:45 smithi027 podman[111398]: 2024-06-06 08:52:45.299193186 +0000 UTC m=+0.792445055 container remove 537977a48b3ccd5309f79acff478a6ce0a8047e42427791727cc28cc9bc9e335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:52:45.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:45 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:45.454+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:45.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:45 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:45.454+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:45.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:45 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:45.454+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:19.222094+0000 front 2024-06-06T08:52:19.222152+0000 (oldest deadline 2024-06-06T08:52:44.522084+0000) 2024-06-06T08:52:45.863 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:45 smithi027 podman[111487]: 2024-06-06 08:52:45.604130086 +0000 UTC m=+0.017621495 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:52:45.863 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:45 smithi027 podman[111487]: 2024-06-06 08:52:45.725127203 +0000 UTC m=+0.138618590 container create 70942e9adc675e298794da517505d458cb9998b4d51ef1702bd4853b428b87a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-06T08:52:45.863 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:45 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:45.605+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:45.863 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:45 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:45.605+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:45.863 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:45 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:45.605+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:45.863 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:45 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:45.605+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:46.181 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:45 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:45.817+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:46.181 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:45 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:45.817+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:46.181 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:45 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:45.817+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:46.206 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:45 smithi027 podman[111487]: 2024-06-06 08:52:45.986039784 +0000 UTC m=+0.399531178 container init 70942e9adc675e298794da517505d458cb9998b4d51ef1702bd4853b428b87a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0) 2024-06-06T08:52:46.206 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:45 smithi027 podman[111487]: 2024-06-06 08:52:45.989557202 +0000 UTC m=+0.403048596 container start 70942e9adc675e298794da517505d458cb9998b4d51ef1702bd4853b428b87a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-06T08:52:46.206 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:46 smithi027 podman[111487]: 2024-06-06 08:52:46.086038225 +0000 UTC m=+0.499529618 container attach 70942e9adc675e298794da517505d458cb9998b4d51ef1702bd4853b428b87a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-06T08:52:46.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:45 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:45.860+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:46.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:45 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:45.860+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:46.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:45 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:45.860+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:46.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:45 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:45.860+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:46.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:46 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:46.183+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:46.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:46 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:46.183+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:46.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:46 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:46.183+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:46.578 INFO:journalctl@ceph.osd.6.smithi027.stdout:Jun 06 08:52:46 smithi027 podman[111487]: 2024-06-06 08:52:46.295586611 +0000 UTC m=+0.709078013 container died 70942e9adc675e298794da517505d458cb9998b4d51ef1702bd4853b428b87a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-06T08:52:46.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:46.476+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:46.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:46.476+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:46.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:46.476+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:19.222094+0000 front 2024-06-06T08:52:19.222152+0000 (oldest deadline 2024-06-06T08:52:44.522084+0000) 2024-06-06T08:52:46.904 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:46 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:46.581+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:46.904 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:46 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:46.581+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:46.905 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:46 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:46.581+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:46.905 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:46 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:46.581+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:46.920 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-06T08:52:47.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:52:47.002 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-06T08:52:47.003 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-06T08:52:47.003 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.7 2024-06-06T08:52:47.206 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:46 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:46.818+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:47.207 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:46 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:46.818+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:47.207 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:46 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:46.818+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:47.256 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:47 smithi176 systemd[1]: Stopping Ceph osd.7 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:52:47.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:46.861+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:47.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:46.861+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:47.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:46.861+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:47.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:46.861+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:47.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:47 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:47.208+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:47.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:47 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:47.208+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:47.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:47 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:47.208+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:47.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:47.326+0000 7f9894666700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:52:47.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:47.326+0000 7f9894666700 -1 osd.7 115 *** Got signal Terminated *** 2024-06-06T08:52:47.757 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:47.326+0000 7f9894666700 -1 osd.7 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:52:47.757 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:47.432+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:47.757 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:47.432+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:47.757 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:47.432+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:19.222094+0000 front 2024-06-06T08:52:19.222152+0000 (oldest deadline 2024-06-06T08:52:44.522084+0000) 2024-06-06T08:52:47.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:47 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:47.616+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:47.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:47 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:47.616+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:47.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:47 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:47.616+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:47.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:47 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:47.616+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:48.222 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:47 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:47.860+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:48.223 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:47 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:47.860+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:48.223 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:47 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:47.860+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:48.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:47.823+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:48.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:47.823+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:48.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:47.823+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:48.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:47 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:47.823+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:48.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:48 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:48.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:48.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:48 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:48.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:48.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:48 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:48.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:48.532 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:48 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:48.393+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:48.533 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:48 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:48.393+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:48.533 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:48 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:48.393+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:19.222094+0000 front 2024-06-06T08:52:19.222152+0000 (oldest deadline 2024-06-06T08:52:44.522084+0000) 2024-06-06T08:52:48.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:48 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:48.612+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:48.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:48 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:48.612+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:48.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:48 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:48.612+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:48.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:48 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:48.612+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:49.196 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:48 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:48.834+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:49.196 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:48 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:48.834+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:49.196 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:48 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:48.834+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:49.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:48 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:48.873+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:49.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:48 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:48.873+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:49.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:48 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:48.873+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:49.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:48 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:48.873+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:49.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:49 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:49.198+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:49.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:49 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:49.198+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:49.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:49 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:49.198+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:49.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:49 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:49.419+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:49.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:49 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:49.419+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:49.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:49 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:49.419+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:19.222094+0000 front 2024-06-06T08:52:19.222152+0000 (oldest deadline 2024-06-06T08:52:44.522084+0000) 2024-06-06T08:52:49.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:49 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:49.578+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:49.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:49 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:49.578+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:49.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:49 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:49.578+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:49.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:49 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:49.578+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:50.147 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:49 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:49.805+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:50.147 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:49 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:49.805+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:50.147 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:49 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:49.805+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:50.148 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:49 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:49.805+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:50.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:49 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:49.915+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:50.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:49 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:49.915+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:50.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:49 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:49.915+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:50.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:49 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:49.915+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:50.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:50 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:50.149+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:50.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:50 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:50.149+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:50.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:50 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:50.149+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:50.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:50.406+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:50.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:50.406+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:50.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:50.406+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:19.222094+0000 front 2024-06-06T08:52:19.222152+0000 (oldest deadline 2024-06-06T08:52:44.522084+0000) 2024-06-06T08:52:50.757 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:50.406+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:24.522655+0000 front 2024-06-06T08:52:24.522612+0000 (oldest deadline 2024-06-06T08:52:49.822733+0000) 2024-06-06T08:52:50.955 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:50 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:50.605+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:50.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:50 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:50.605+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:50.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:50 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:50.605+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:50.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:50 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:50.605+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:51.125 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:50 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:50.760+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:51.125 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:50 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:50.760+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:51.125 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:50 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:50.760+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:51.125 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:50 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:50.760+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:51.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:50.947+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:51.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:50.948+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:51.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:50.948+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:51.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:50.948+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:51.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:50.948+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:52:51.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:51 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:51.127+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:51.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:51 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:51.127+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:51.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:51 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:51.127+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:51.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:51 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:51.396+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:01.719477+0000 front 2024-06-06T08:52:01.719561+0000 (oldest deadline 2024-06-06T08:52:26.419664+0000) 2024-06-06T08:52:51.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:51 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:51.396+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.720775+0000 front 2024-06-06T08:52:11.720709+0000 (oldest deadline 2024-06-06T08:52:32.820649+0000) 2024-06-06T08:52:51.756 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:51 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:51.396+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:19.222094+0000 front 2024-06-06T08:52:19.222152+0000 (oldest deadline 2024-06-06T08:52:44.522084+0000) 2024-06-06T08:52:51.757 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:51 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7[55255]: 2024-06-06T08:52:51.396+0000 7f9890c5a700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:24.522655+0000 front 2024-06-06T08:52:24.522612+0000 (oldest deadline 2024-06-06T08:52:49.822733+0000) 2024-06-06T08:52:51.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:51.592+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:51.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:51.592+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:51.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:51.592+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:51.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:51.592+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:51.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:51 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:51.592+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:52:52.113 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:51 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:51.747+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:52.113 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:51 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:51.747+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:52.113 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:51 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:51.747+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:52.113 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:51 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:51.747+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:52.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:51 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:51.968+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:52.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:51 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:51.968+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:52.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:51 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:51.968+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:52.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:51 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:51.968+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:52.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:51 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:51.968+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:52:52.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:52 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:52.115+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:52.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:52 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:52.115+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:52.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:52 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:52.115+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:52.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:52 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:52.115+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:52:52.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:52 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:52.568+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:52.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:52 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:52.568+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:52.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:52 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:52.568+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:52.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:52 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:52.568+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:52.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:52 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:52.568+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:52:52.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:52 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:52.728+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:52.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:52 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:52.728+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:52.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:52 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:52.728+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:52.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:52 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:52.728+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:53.074 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:52 smithi176 podman[74211]: 2024-06-06 08:52:52.767458613 +0000 UTC m=+5.567371878 container died c2f20824948788b684abd8364942389ff9243c76f59dcb998d5e2b6753e5ceaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:52:53.075 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:53.075 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:53.075 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:53.075 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:53.075 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:52:53.391 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:53 smithi176 podman[74211]: 2024-06-06 08:52:53.175812037 +0000 UTC m=+5.975725303 container cleanup c2f20824948788b684abd8364942389ff9243c76f59dcb998d5e2b6753e5ceaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-06T08:52:53.391 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:53 smithi176 bash[74211]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7 2024-06-06T08:52:53.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:53.076+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:53.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:53.076+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:53.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:53.076+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:53.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:53.076+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:52:53.716 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:53 smithi176 podman[74240]: 2024-06-06 08:52:53.393769688 +0000 UTC m=+0.626023595 container remove c2f20824948788b684abd8364942389ff9243c76f59dcb998d5e2b6753e5ceaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-06T08:52:53.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:53 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:53.590+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:53.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:53 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:53.590+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:53.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:53 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:53.590+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:53.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:53 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:53.590+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:53.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:53 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:53.590+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:52:53.984 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:53 smithi176 podman[74328]: 2024-06-06 08:52:53.622058841 +0000 UTC m=+0.017035221 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:52:53.984 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:53 smithi176 podman[74328]: 2024-06-06 08:52:53.738227324 +0000 UTC m=+0.133203690 container create 24cf09d1f2440374e30dae6098c5a9a2b8d6ac4e701b4fb2f01647b4c8693fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-06T08:52:53.984 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:53 smithi176 podman[74328]: 2024-06-06 08:52:53.955950614 +0000 UTC m=+0.350926986 container init 24cf09d1f2440374e30dae6098c5a9a2b8d6ac4e701b4fb2f01647b4c8693fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:52:53.985 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:53 smithi176 podman[74328]: 2024-06-06 08:52:53.958967753 +0000 UTC m=+0.353944136 container start 24cf09d1f2440374e30dae6098c5a9a2b8d6ac4e701b4fb2f01647b4c8693fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:52:53.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:53.724+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:53.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:53.724+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:53.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:53.724+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:53.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:53 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:53.724+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:54.243 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:54 smithi176 podman[74328]: 2024-06-06 08:52:54.039326476 +0000 UTC m=+0.434302884 container attach 24cf09d1f2440374e30dae6098c5a9a2b8d6ac4e701b4fb2f01647b4c8693fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-06T08:52:54.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:54.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:54.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:54.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:54.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:53 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:53.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:52:54.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:54.095+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:54.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:54.095+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:54.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:54.095+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:54.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:54.095+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:52:54.506 INFO:journalctl@ceph.osd.7.smithi176.stdout:Jun 06 08:52:54 smithi176 podman[74328]: 2024-06-06 08:52:54.246463074 +0000 UTC m=+0.641439460 container died 24cf09d1f2440374e30dae6098c5a9a2b8d6ac4e701b4fb2f01647b4c8693fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:52:54.888 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-06T08:52:54.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:52:54.928 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-06T08:52:54.928 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-06T08:52:54.928 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.8 2024-06-06T08:52:54.955 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:54 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:54.623+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:54.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:54 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:54.623+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:54.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:54 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:54.623+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:54.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:54 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:54.623+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:54.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:54 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:54.623+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:52:54.978 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:54.715+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:54.978 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:54.715+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:54.978 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:54.715+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:54.979 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:54.715+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:55.239 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:54 smithi203 systemd[1]: Stopping Ceph osd.8 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:52:55.240 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:55.110+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:55.240 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:55.110+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:55.240 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:55.110+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:55.240 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:55.110+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:52:55.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:54 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:54.978+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:55.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:54 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:54.978+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:55.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:54 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:54.978+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:55.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:54 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:54.978+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:55.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:54 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:54.978+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:52:55.690 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:55.248+0000 7f37b895f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:52:55.690 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:55.248+0000 7f37b895f700 -1 osd.8 115 *** Got signal Terminated *** 2024-06-06T08:52:55.691 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:55.248+0000 7f37b895f700 -1 osd.8 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:52:55.691 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:55.692+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:55.691 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:55.692+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:55.691 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:55.692+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:55.691 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:55 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:55.692+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:55.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:55 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:55.599+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:55.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:55 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:55.599+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:55.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:55 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:55.599+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:55.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:55 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:55.599+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:55.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:55 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:55.599+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:52:56.453 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:56 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:56.086+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:56.453 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:56 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:56.086+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:56.453 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:56 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:56.086+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:56.453 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:56 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:56.086+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:52:56.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.021+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:56.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.021+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:56.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.021+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:56.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.021+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:56.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.021+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:52:56.857 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:56 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:56.603+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:56.857 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:56 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:56.603+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:56.857 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:56 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:56.603+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:56.857 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:56 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:56.603+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:56.857 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:56 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:56.603+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:52:56.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:56 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:56.730+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:56.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:56 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:56.730+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:56.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:56 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:56.730+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:56.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:56 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:56.730+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:57.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.984+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:57.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.984+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:57.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.984+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:57.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.984+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:57.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:56 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:56.984+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:52:57.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:57 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:57.058+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:57.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:57 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:57.058+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:57.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:57 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:57.058+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:57.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:57 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:57.058+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:52:57.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:57 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:57.633+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:57.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:57 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:57.633+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:57.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:57 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:57.633+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:57.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:57 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:57.633+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:57.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:57 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:57.633+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:52:57.957 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:57 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:57.633+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:52:57.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:57 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:57.701+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:57.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:57 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:57.701+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:57.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:57 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:57.701+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:57.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:57 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:57.701+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:58.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:57 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:57.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:58.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:57 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:57.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:58.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:57 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:57.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:58.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:57 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:57.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:58.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:57 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:57.987+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:52:58.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:58.108+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:58.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:58.108+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:58.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:58.108+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:58.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:58.108+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:52:58.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:58 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:58.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:58.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:58 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:58.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:58.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:58 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:58.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:58.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:58 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:58.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:58.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:58 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:58.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:52:58.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:58 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:58.620+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:52:58.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:58.709+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:58.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:58.709+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:58.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:58.709+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:58.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:58.709+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:58.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:58.709+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.183066+0000 front 2024-06-06T08:52:33.183119+0000 (oldest deadline 2024-06-06T08:52:57.883193+0000) 2024-06-06T08:52:59.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:59.142+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:52:59.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:59.142+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:52:59.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:59.142+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:52:59.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:59.142+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:52:59.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:52:59.142+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:52:59.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:52:59.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:52:59.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:52:59.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:52:59.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.008+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:52:59.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:59 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:59.619+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:52:59.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:59 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:59.619+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:52:59.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:59 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:59.619+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:52:59.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:59 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:59.619+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:52:59.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:59 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:59.619+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:52:59.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:52:59 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:52:59.619+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:52:59.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:59.692+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:59.378371+0000 front 2024-06-06T08:51:59.378398+0000 (oldest deadline 2024-06-06T08:52:24.678280+0000) 2024-06-06T08:52:59.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:59.692+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.579082+0000 front 2024-06-06T08:52:07.579122+0000 (oldest deadline 2024-06-06T08:52:33.379701+0000) 2024-06-06T08:52:59.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:59.692+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.280801+0000 front 2024-06-06T08:52:16.280937+0000 (oldest deadline 2024-06-06T08:52:40.980799+0000) 2024-06-06T08:52:59.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:59.692+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.281852+0000 front 2024-06-06T08:52:26.281862+0000 (oldest deadline 2024-06-06T08:52:49.781742+0000) 2024-06-06T08:52:59.990 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:52:59 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8[55305]: 2024-06-06T08:52:59.692+0000 7f37b4f53700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.183066+0000 front 2024-06-06T08:52:33.183119+0000 (oldest deadline 2024-06-06T08:52:57.883193+0000) 2024-06-06T08:53:00.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:00.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:00.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:00.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:00.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:52:59 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:52:59.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:00.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:00 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:00.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:00.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:00 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:00.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:00.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:00 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:00.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:00.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:00 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:00.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:00.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:00 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:00.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:00.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:00.632+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:53:00.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:00.632+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:53:00.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:00.632+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:53:00.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:00.632+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:53:00.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:00.632+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:53:00.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:00 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:00.632+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:53:00.989 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:00 smithi203 podman[74835]: 2024-06-06 08:53:00.740343638 +0000 UTC m=+5.612919163 container died 3875a46c8e4cebf516b4676b91bc420bfdda59566dfcd443d0b2bac154c6457d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-06T08:53:01.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:00 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:00.947+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:01.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:00 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:00.947+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:01.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:00 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:00.947+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:01.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:00 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:00.947+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:01.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:00 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:00.947+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:01.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:01 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:01.130+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:01.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:01 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:01.130+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:01.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:01 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:01.130+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:01.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:01 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:01.130+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:01.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:01 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:01.130+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:01.490 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:01 smithi203 podman[74835]: 2024-06-06 08:53:01.080447057 +0000 UTC m=+5.953022574 container cleanup 3875a46c8e4cebf516b4676b91bc420bfdda59566dfcd443d0b2bac154c6457d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-06T08:53:01.490 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:01 smithi203 bash[74835]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8 2024-06-06T08:53:01.490 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:01 smithi203 podman[74850]: 2024-06-06 08:53:01.315222763 +0000 UTC m=+0.570406599 container remove 3875a46c8e4cebf516b4676b91bc420bfdda59566dfcd443d0b2bac154c6457d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-06T08:53:01.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:01 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:01.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:53:01.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:01 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:01.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:53:01.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:01 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:01.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:53:01.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:01 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:01.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:53:01.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:01 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:01.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:53:01.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:01 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:01.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:53:02.017 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:01 smithi203 podman[74939]: 2024-06-06 08:53:01.65602807 +0000 UTC m=+0.017274972 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:53:02.017 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:01 smithi203 podman[74939]: 2024-06-06 08:53:01.774681228 +0000 UTC m=+0.135928171 container create f365d32b231955d9dd6595483a57bf236046823552bb45fd181beff273493ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-06T08:53:02.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:01 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:01.974+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:02.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:01 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:01.974+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:02.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:01 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:01.974+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:02.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:01 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:01.974+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:02.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:01 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:01.974+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:02.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:01 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:01.974+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:02.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:02 smithi203 podman[74939]: 2024-06-06 08:53:02.019383917 +0000 UTC m=+0.380630808 container init f365d32b231955d9dd6595483a57bf236046823552bb45fd181beff273493ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-06T08:53:02.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:02 smithi203 podman[74939]: 2024-06-06 08:53:02.022613526 +0000 UTC m=+0.383860430 container start f365d32b231955d9dd6595483a57bf236046823552bb45fd181beff273493ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:53:02.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:02 smithi203 podman[74939]: 2024-06-06 08:53:02.127799109 +0000 UTC m=+0.489046036 container attach f365d32b231955d9dd6595483a57bf236046823552bb45fd181beff273493ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:53:02.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:02 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:02.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:02.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:02 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:02.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:02.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:02 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:02.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:02.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:02 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:02.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:02.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:02 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:02.105+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:02.612 INFO:journalctl@ceph.osd.8.smithi203.stdout:Jun 06 08:53:02 smithi203 podman[74939]: 2024-06-06 08:53:02.324640806 +0000 UTC m=+0.685887704 container died f365d32b231955d9dd6595483a57bf236046823552bb45fd181beff273493ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:53:02.922 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-06T08:53:02.955 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:02 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:02.610+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:53:02.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:02 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:02.610+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:53:02.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:02 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:02.610+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:53:02.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:02 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:02.610+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:53:02.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:02 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:02.610+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:53:02.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:02 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:02.610+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:53:02.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:02.959 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-06T08:53:02.959 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-06T08:53:02.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.9 2024-06-06T08:53:03.225 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:03 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:03.144+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:03.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:03 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:03.144+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:03.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:03 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:03.144+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:03.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:03 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:03.144+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:03.226 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:03 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:03.144+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:03.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:02 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:02.938+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:03.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:02 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:02.938+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:03.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:02 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:02.938+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:03.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:02 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:02.938+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:03.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:02 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:02.938+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:03.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:02 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:02.938+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:03.283 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 systemd[1]: Stopping Ceph osd.9 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:03.646 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:03.284+0000 7fe5d4ebc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:03.646 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:03.284+0000 7fe5d4ebc700 -1 osd.9 115 *** Got signal Terminated *** 2024-06-06T08:53:03.646 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:03.284+0000 7fe5d4ebc700 -1 osd.9 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:53:03.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:03.648+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:53:03.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:03.648+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:53:03.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:03.648+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:53:03.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:03.648+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:53:03.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:03.648+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:53:03.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:03 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:03.648+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:53:04.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:03 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:03.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:04.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:03 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:03.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:04.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:03 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:03.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:04.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:03 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:03.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:04.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:03 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:03.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:04.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:03 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:03.963+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:04.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:04 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:04.101+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:04.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:04 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:04.101+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:04.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:04 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:04.101+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:04.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:04 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:04.101+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:04.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:04 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:04.101+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:04.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:04 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:04.613+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:53:04.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:04 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:04.613+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:53:04.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:04 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:04.613+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:53:04.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:04 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:04.613+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:53:04.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:04 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:04.613+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:53:04.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:04 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:04.613+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:53:05.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:04 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:04.989+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:05.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:04 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:04.989+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:05.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:04 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:04.989+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:05.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:04 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:04.989+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:05.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:04 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:04.989+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:05.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:04 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:04.989+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:05.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:05 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:05.125+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:05.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:05 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:05.125+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:05.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:05 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:05.125+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:05.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:05 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:05.125+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:05.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:05 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:05.125+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:05.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:05 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:05.653+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:53:05.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:05 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:05.653+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:53:05.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:05 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:05.653+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:53:05.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:05 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:05.653+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:53:05.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:05 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:05.653+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:53:05.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:05 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:05.653+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:53:06.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:05 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:05.954+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:06.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:05 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:05.954+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:06.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:05 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:05.954+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:06.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:05 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:05.954+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:06.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:05 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:05.954+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:06.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:05 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:05.954+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:06.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:06.152+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:06.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:06.152+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:06.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:06.152+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:06.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:06.152+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:06.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:06.152+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:06.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:06.152+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:06.922 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:06 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:06.611+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:53:06.922 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:06 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:06.611+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:53:06.923 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:06 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:06.611+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:53:06.923 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:06 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:06.611+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:53:06.923 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:06 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:06.611+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:53:06.923 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:06 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:06.611+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:53:07.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:06 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:06.922+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:07.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:06 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:06.922+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:07.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:06 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:06.922+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:07.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:06 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:06.922+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:07.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:06 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:06.922+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:07.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:06 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:06.922+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:07.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:07 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:07.183+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:07.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:07 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:07.183+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:07.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:07 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:07.183+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:07.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:07 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:07.183+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:07.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:07 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:07.183+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:07.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:07 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:07.183+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:07.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:07 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:07.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:52.970531+0000 front 2024-06-06T08:51:52.970338+0000 (oldest deadline 2024-06-06T08:52:18.870206+0000) 2024-06-06T08:53:07.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:07 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:07.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:51:58.870957+0000 front 2024-06-06T08:51:58.870867+0000 (oldest deadline 2024-06-06T08:52:24.770898+0000) 2024-06-06T08:53:07.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:07 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:07.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:11.673393+0000 front 2024-06-06T08:52:11.673564+0000 (oldest deadline 2024-06-06T08:52:35.173253+0000) 2024-06-06T08:53:07.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:07 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:07.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:16.874131+0000 front 2024-06-06T08:52:16.874165+0000 (oldest deadline 2024-06-06T08:52:40.974298+0000) 2024-06-06T08:53:07.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:07 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:07.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.375778+0000 front 2024-06-06T08:52:27.375935+0000 (oldest deadline 2024-06-06T08:52:51.475764+0000) 2024-06-06T08:53:07.956 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:07 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9[67099]: 2024-06-06T08:53:07.628+0000 7fe5d14b0700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:31.476415+0000 front 2024-06-06T08:52:31.476375+0000 (oldest deadline 2024-06-06T08:52:56.776236+0000) 2024-06-06T08:53:08.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:07 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:07.918+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:08.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:07 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:07.918+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:08.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:07 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:07.918+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:08.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:07 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:07.918+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:08.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:07 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:07.918+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:08.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:07 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:07.918+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:08.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:07 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:07.918+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:41.387190+0000 front 2024-06-06T08:52:41.387098+0000 (oldest deadline 2024-06-06T08:53:07.187486+0000) 2024-06-06T08:53:08.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:08.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:08.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:08.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:08.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:08.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:08.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:08.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:08.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:08.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:08.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:08 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:08.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:09.184 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:08 smithi027 podman[111605]: 2024-06-06 08:53:08.795387568 +0000 UTC m=+5.630165345 container died 6ffe3e694f3c640e265008778ec72b32d25885fe9051b42211870b1e1b508f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-06T08:53:09.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:08 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:08.884+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:09.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:08 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:08.884+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:09.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:08 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:08.884+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:09.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:08 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:08.884+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:09.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:08 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:08.884+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:09.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:08 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:08.884+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:09.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:08 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:08.884+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:41.387190+0000 front 2024-06-06T08:52:41.387098+0000 (oldest deadline 2024-06-06T08:53:07.187486+0000) 2024-06-06T08:53:09.456 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:09 smithi027 podman[111605]: 2024-06-06 08:53:09.267666391 +0000 UTC m=+6.102444157 container cleanup 6ffe3e694f3c640e265008778ec72b32d25885fe9051b42211870b1e1b508f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=) 2024-06-06T08:53:09.456 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:09 smithi027 bash[111605]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9 2024-06-06T08:53:09.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:09 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:09.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:09.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:09 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:09.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:09.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:09 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:09.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:09.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:09 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:09.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:09.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:09 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:09.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:09.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:09 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:09.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:09.902 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:09 smithi027 podman[111622]: 2024-06-06 08:53:09.560830862 +0000 UTC m=+0.757156437 container remove 6ffe3e694f3c640e265008778ec72b32d25885fe9051b42211870b1e1b508f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-06T08:53:10.206 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:09 smithi027 podman[111712]: 2024-06-06 08:53:09.807311156 +0000 UTC m=+0.017918775 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:53:10.206 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:09 smithi027 podman[111712]: 2024-06-06 08:53:09.91160363 +0000 UTC m=+0.122211263 container create 7df2ce14acde5f0cf8fff52bc1c34c0c7462a23b990bcbf700a1aea3cef855a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-06T08:53:10.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:09 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:09.895+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:10.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:09 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:09.895+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:10.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:09 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:09.895+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:10.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:09 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:09.895+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:10.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:09 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:09.895+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:10.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:09 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:09.895+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:10.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:09 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:09.895+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:41.387190+0000 front 2024-06-06T08:52:41.387098+0000 (oldest deadline 2024-06-06T08:53:07.187486+0000) 2024-06-06T08:53:10.562 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:10 smithi027 podman[111712]: 2024-06-06 08:53:10.247691136 +0000 UTC m=+0.458298760 container init 7df2ce14acde5f0cf8fff52bc1c34c0c7462a23b990bcbf700a1aea3cef855a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:53:10.562 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:10 smithi027 podman[111712]: 2024-06-06 08:53:10.2515852 +0000 UTC m=+0.462192814 container start 7df2ce14acde5f0cf8fff52bc1c34c0c7462a23b990bcbf700a1aea3cef855a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-06T08:53:10.562 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:10 smithi027 podman[111712]: 2024-06-06 08:53:10.322076125 +0000 UTC m=+0.532683744 container attach 7df2ce14acde5f0cf8fff52bc1c34c0c7462a23b990bcbf700a1aea3cef855a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:53:10.739 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:10.278+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:10.739 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:10.278+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:10.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:10.278+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:10.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:10.278+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:10.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:10.278+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:10.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:10 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:10.278+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:10.847 INFO:journalctl@ceph.osd.9.smithi027.stdout:Jun 06 08:53:10 smithi027 podman[111712]: 2024-06-06 08:53:10.565197567 +0000 UTC m=+0.775805183 container died 7df2ce14acde5f0cf8fff52bc1c34c0c7462a23b990bcbf700a1aea3cef855a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:53:11.175 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-06T08:53:11.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:11.217 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-06T08:53:11.217 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-06T08:53:11.217 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.10 2024-06-06T08:53:11.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:10.916+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:11.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:10.916+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:11.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:10.916+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:11.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:10.916+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:11.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:10.916+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:11.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:10.916+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:11.244 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:10 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:10.916+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:41.387190+0000 front 2024-06-06T08:52:41.387098+0000 (oldest deadline 2024-06-06T08:53:07.187486+0000) 2024-06-06T08:53:11.506 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 systemd[1]: Stopping Ceph osd.10 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:11.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:11.275+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:11.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:11.275+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:11.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:11.275+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:11.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:11.275+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:11.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:11.275+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:11.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:11 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:11.275+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:11.885 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.528+0000 7f7efb537700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:11.885 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.528+0000 7f7efb537700 -1 osd.10 115 *** Got signal Terminated *** 2024-06-06T08:53:11.885 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.528+0000 7f7efb537700 -1 osd.10 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:53:11.885 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.886+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:11.885 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.886+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:11.886 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.886+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:12.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.886+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:12.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.886+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:12.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.886+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:12.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:11 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:11.886+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:41.387190+0000 front 2024-06-06T08:52:41.387098+0000 (oldest deadline 2024-06-06T08:53:07.187486+0000) 2024-06-06T08:53:12.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:12.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:12.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:12.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:12.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:12.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:12.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:12.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:12.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:12.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:12.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:12 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:12.236+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:13.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:12.887+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:13.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:12.887+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:13.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:12.887+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:13.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:12.887+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:13.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:12.887+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:13.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:12.887+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:13.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:12 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:12.887+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:41.387190+0000 front 2024-06-06T08:52:41.387098+0000 (oldest deadline 2024-06-06T08:53:07.187486+0000) 2024-06-06T08:53:13.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:13.235+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:13.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:13.235+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:13.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:13.235+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:13.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:13.235+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:13.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:13.235+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:13.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:13 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:13.235+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:14.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:13.850+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:14.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:13.850+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:14.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:13.850+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:14.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:13.850+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:14.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:13.850+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:14.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:13.850+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:14.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:13 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:13.850+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:41.387190+0000 front 2024-06-06T08:52:41.387098+0000 (oldest deadline 2024-06-06T08:53:07.187486+0000) 2024-06-06T08:53:14.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:14.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:14.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:14.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:14.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:14.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:14.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:14.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:14.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:14.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:14.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:14.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:14.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:14 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:14.224+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:15.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:14.801+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:15.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:14.801+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:15.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:14.801+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:15.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:14.801+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:15.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:14.801+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:15.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:14.801+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:15.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:14 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:14.801+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:41.387190+0000 front 2024-06-06T08:52:41.387098+0000 (oldest deadline 2024-06-06T08:53:07.187486+0000) 2024-06-06T08:53:15.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:15.211+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:15.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:15.211+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:15.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:15.211+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:15.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:15.211+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:15.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:15.211+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:15.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:15.211+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:15.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:15 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:15.211+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:16.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:15.785+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:55.280297+0000 front 2024-06-06T08:51:55.280364+0000 (oldest deadline 2024-06-06T08:52:17.580403+0000) 2024-06-06T08:53:16.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:15.785+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:03.381918+0000 front 2024-06-06T08:52:03.381892+0000 (oldest deadline 2024-06-06T08:52:24.482041+0000) 2024-06-06T08:53:16.256 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:15.785+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:10.383495+0000 front 2024-06-06T08:52:10.383294+0000 (oldest deadline 2024-06-06T08:52:32.683318+0000) 2024-06-06T08:53:16.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:15.785+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-06-06T08:52:15.584455+0000 front 2024-06-06T08:52:15.584403+0000 (oldest deadline 2024-06-06T08:52:41.484360+0000) 2024-06-06T08:53:16.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:15.785+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:27.285533+0000 front 2024-06-06T08:52:27.285547+0000 (oldest deadline 2024-06-06T08:52:50.185636+0000) 2024-06-06T08:53:16.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:15.785+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:30.186121+0000 front 2024-06-06T08:52:30.186076+0000 (oldest deadline 2024-06-06T08:53:01.386550+0000) 2024-06-06T08:53:16.257 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:15 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10[62117]: 2024-06-06T08:53:15.785+0000 7f7ef3b23700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:41.387190+0000 front 2024-06-06T08:52:41.387098+0000 (oldest deadline 2024-06-06T08:53:07.187486+0000) 2024-06-06T08:53:16.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:16 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:16.201+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:16.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:16 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:16.201+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:16.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:16 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:16.201+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:16.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:16 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:16.201+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:16.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:16 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:16.201+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:16.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:16 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:16.201+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:16.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:16 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:16.201+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:17.294 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:17 smithi176 podman[74434]: 2024-06-06 08:53:17.038119138 +0000 UTC m=+5.628844452 container died 870e9db97e28a7d621a85251ebbedbd7bd5605357c15eb0f27fae8dc2b4f08bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-06T08:53:17.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:17 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:17.209+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:17.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:17 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:17.209+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:17.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:17 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:17.209+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:17.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:17 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:17.209+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:17.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:17 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:17.209+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:17.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:17 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:17.209+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:17.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:17 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:17.209+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:17.594 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:17 smithi176 podman[74434]: 2024-06-06 08:53:17.370189887 +0000 UTC m=+5.960915199 container cleanup 870e9db97e28a7d621a85251ebbedbd7bd5605357c15eb0f27fae8dc2b4f08bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-06T08:53:17.594 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:17 smithi176 bash[74434]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10 2024-06-06T08:53:18.006 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:17 smithi176 podman[74451]: 2024-06-06 08:53:17.596275408 +0000 UTC m=+0.553392852 container remove 870e9db97e28a7d621a85251ebbedbd7bd5605357c15eb0f27fae8dc2b4f08bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T08:53:18.258 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:18 smithi176 podman[74541]: 2024-06-06 08:53:17.94363774 +0000 UTC m=+0.017079198 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:53:18.258 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:18 smithi176 podman[74541]: 2024-06-06 08:53:18.046919119 +0000 UTC m=+0.120360558 container create 1d35ab688cdc839968d7ca137ac1700c65d1e0a95aea19cee671c6f3fbd6665d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-06T08:53:18.258 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:18 smithi176 podman[74541]: 2024-06-06 08:53:18.257555402 +0000 UTC m=+0.330996842 container init 1d35ab688cdc839968d7ca137ac1700c65d1e0a95aea19cee671c6f3fbd6665d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-06T08:53:18.258 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:18 smithi176 podman[74541]: 2024-06-06 08:53:18.260612016 +0000 UTC m=+0.334053459 container start 1d35ab688cdc839968d7ca137ac1700c65d1e0a95aea19cee671c6f3fbd6665d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-06T08:53:18.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:18 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:18.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:18.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:18 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:18.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:18.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:18 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:18.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:18.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:18 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:18.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:18.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:18 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:18.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:18.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:18 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:18.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:18.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:18 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:18.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:18.552 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:18 smithi176 podman[74541]: 2024-06-06 08:53:18.374461333 +0000 UTC m=+0.447902788 container attach 1d35ab688cdc839968d7ca137ac1700c65d1e0a95aea19cee671c6f3fbd6665d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-06T08:53:18.842 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:18 smithi176 podman[74541]: 2024-06-06 08:53:18.555143002 +0000 UTC m=+0.628584453 container died 1d35ab688cdc839968d7ca137ac1700c65d1e0a95aea19cee671c6f3fbd6665d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-06T08:53:19.097 INFO:journalctl@ceph.osd.10.smithi176.stdout:Jun 06 08:53:19 smithi176 podman[74566]: 2024-06-06 08:53:19.086235268 +0000 UTC m=+0.526086803 container remove 1d35ab688cdc839968d7ca137ac1700c65d1e0a95aea19cee671c6f3fbd6665d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True) 2024-06-06T08:53:19.103 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-06T08:53:19.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:19.181 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-06T08:53:19.181 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-06T08:53:19.181 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.11 2024-06-06T08:53:19.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 systemd[1]: Stopping Ceph osd.11 for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:19.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.281+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:19.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.281+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:19.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.281+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:19.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.281+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:19.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.281+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:19.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.281+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:19.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.281+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:19.989 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.508+0000 7fc7790df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:19.990 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.508+0000 7fc7790df700 -1 osd.11 115 *** Got signal Terminated *** 2024-06-06T08:53:19.990 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:19 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:19.508+0000 7fc7790df700 -1 osd.11 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T08:53:20.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:20.332+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:20.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:20.332+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:20.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:20.332+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:20.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:20.332+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:20.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:20.332+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:20.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:20.332+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:20.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:20 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:20.332+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:21.739 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:21 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:21.303+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:21.739 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:21 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:21.303+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:21.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:21 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:21.303+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:21.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:21 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:21.303+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:21.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:21 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:21.303+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:21.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:21 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:21.303+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:21.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:21 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:21.303+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:22.739 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:22.300+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:22.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:22.300+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:22.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:22.300+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:22.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:22.300+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:22.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:22.300+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:22.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:22.300+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:22.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:22.300+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:22.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:22 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:22.300+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6820 osd.8 since back 2024-06-06T08:52:59.288749+0000 front 2024-06-06T08:52:59.288704+0000 (oldest deadline 2024-06-06T08:53:21.588704+0000) 2024-06-06T08:53:23.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:23.271+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:23.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:23.271+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:23.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:23.271+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:23.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:23.271+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:23.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:23.271+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:23.740 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:23.271+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:23.741 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:23.271+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:23.741 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:23 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:23.271+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6820 osd.8 since back 2024-06-06T08:52:59.288749+0000 front 2024-06-06T08:52:59.288704+0000 (oldest deadline 2024-06-06T08:53:21.588704+0000) 2024-06-06T08:53:24.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:24.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-06-06T08:51:53.180575+0000 front 2024-06-06T08:51:53.180724+0000 (oldest deadline 2024-06-06T08:52:19.080677+0000) 2024-06-06T08:53:24.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:24.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2024-06-06T08:52:02.581838+0000 front 2024-06-06T08:52:02.582002+0000 (oldest deadline 2024-06-06T08:52:27.881682+0000) 2024-06-06T08:53:24.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:24.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-06-06T08:52:07.882084+0000 front 2024-06-06T08:52:07.882061+0000 (oldest deadline 2024-06-06T08:52:37.282826+0000) 2024-06-06T08:53:24.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:24.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2024-06-06T08:52:26.584916+0000 front 2024-06-06T08:52:26.584862+0000 (oldest deadline 2024-06-06T08:52:51.284954+0000) 2024-06-06T08:53:24.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:24.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-06-06T08:52:33.585698+0000 front 2024-06-06T08:52:33.585776+0000 (oldest deadline 2024-06-06T08:52:58.885748+0000) 2024-06-06T08:53:24.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:24.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-06-06T08:52:40.586628+0000 front 2024-06-06T08:52:40.586795+0000 (oldest deadline 2024-06-06T08:53:05.886523+0000) 2024-06-06T08:53:24.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:24.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2024-06-06T08:52:51.688217+0000 front 2024-06-06T08:52:51.688169+0000 (oldest deadline 2024-06-06T08:53:13.387989+0000) 2024-06-06T08:53:24.490 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:24 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11[62305]: 2024-06-06T08:53:24.233+0000 7fc7716cb700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.203:6820 osd.8 since back 2024-06-06T08:52:59.288749+0000 front 2024-06-06T08:52:59.288704+0000 (oldest deadline 2024-06-06T08:53:21.588704+0000) 2024-06-06T08:53:25.232 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:24 smithi203 podman[75067]: 2024-06-06 08:53:24.953752061 +0000 UTC m=+5.572839960 container died fc156673dc66fe6dd1aabed08ceffdd25f0eced78d3cff5ee9c2fd6800cafeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T08:53:25.232 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:25 smithi203 podman[75067]: 2024-06-06 08:53:25.234423246 +0000 UTC m=+5.853511138 container cleanup fc156673dc66fe6dd1aabed08ceffdd25f0eced78d3cff5ee9c2fd6800cafeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-06T08:53:25.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:25 smithi203 bash[75067]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11 2024-06-06T08:53:25.489 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:25 smithi203 podman[75083]: 2024-06-06 08:53:25.394047992 +0000 UTC m=+0.436147453 container remove fc156673dc66fe6dd1aabed08ceffdd25f0eced78d3cff5ee9c2fd6800cafeae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-06T08:53:26.054 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:25 smithi203 podman[75173]: 2024-06-06 08:53:25.709379686 +0000 UTC m=+0.017356538 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T08:53:26.054 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:25 smithi203 podman[75173]: 2024-06-06 08:53:25.81996653 +0000 UTC m=+0.127943374 container create ea9407f56b0ed2baa127d059f6f46aea3e28f0b344aca98fbd00acd9e5d100e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11-deactivate, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-06T08:53:26.054 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:26 smithi203 podman[75173]: 2024-06-06 08:53:26.056594055 +0000 UTC m=+0.364570907 container init ea9407f56b0ed2baa127d059f6f46aea3e28f0b344aca98fbd00acd9e5d100e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-06-06T08:53:26.350 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:26 smithi203 podman[75173]: 2024-06-06 08:53:26.059974451 +0000 UTC m=+0.367951310 container start ea9407f56b0ed2baa127d059f6f46aea3e28f0b344aca98fbd00acd9e5d100e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-06-06T08:53:26.350 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:26 smithi203 podman[75173]: 2024-06-06 08:53:26.139913704 +0000 UTC m=+0.447890563 container attach ea9407f56b0ed2baa127d059f6f46aea3e28f0b344aca98fbd00acd9e5d100e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-06T08:53:26.617 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:26 smithi203 podman[75200]: 2024-06-06 08:53:26.380529099 +0000 UTC m=+0.018264517 container died ea9407f56b0ed2baa127d059f6f46aea3e28f0b344aca98fbd00acd9e5d100e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-06T08:53:26.861 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-06T08:53:26.883 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:26 smithi203 podman[75200]: 2024-06-06 08:53:26.843593733 +0000 UTC m=+0.481329138 container remove ea9407f56b0ed2baa127d059f6f46aea3e28f0b344aca98fbd00acd9e5d100e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-osd-11-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T08:53:26.883 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:26 smithi203 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.11.service: Deactivated successfully. 2024-06-06T08:53:26.883 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:26 smithi203 systemd[1]: Stopped Ceph osd.11 for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:53:26.883 INFO:journalctl@ceph.osd.11.smithi203.stdout:Jun 06 08:53:26 smithi203 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@osd.11.service: Consumed 1min 16.376s CPU time. 2024-06-06T08:53:26.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:26.897 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-06T08:53:26.898 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-06T08:53:26.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.a 2024-06-06T08:53:27.206 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:53:26 smithi027 systemd[1]: Stopping Ceph mds.a for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:27.481 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:53:27 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-a[74358]: 2024-06-06T08:53:27.210+0000 7f78d8ba6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:27.481 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:53:27 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-a[74358]: 2024-06-06T08:53:27.210+0000 7f78d8ba6700 -1 mds.a *** got signal Terminated *** 2024-06-06T08:53:30.706 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:53:30 smithi027 podman[111818]: 2024-06-06 08:53:30.451948843 +0000 UTC m=+3.346916596 container died bf14b0f8c55e9df0581459763680854d853dcd4b26c0c8cfa1ec8b8d3e99afd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-a, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-06T08:53:31.070 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:53:30 smithi027 podman[111818]: 2024-06-06 08:53:30.802181366 +0000 UTC m=+3.697149119 container cleanup bf14b0f8c55e9df0581459763680854d853dcd4b26c0c8cfa1ec8b8d3e99afd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-06T08:53:31.070 INFO:journalctl@ceph.mds.a.smithi027.stdout:Jun 06 08:53:30 smithi027 bash[111818]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-a 2024-06-06T08:53:31.238 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-06T08:53:31.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:31.275 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-06T08:53:31.276 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-06T08:53:31.276 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.d 2024-06-06T08:53:31.596 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:53:31 smithi027 systemd[1]: Stopping Ceph mds.d for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:31.956 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:53:31 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d[73104]: 2024-06-06T08:53:31.597+0000 7f4ae9f53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:31.956 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:53:31 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d[73104]: 2024-06-06T08:53:31.597+0000 7f4ae9f53700 -1 mds.d *** got signal Terminated *** 2024-06-06T08:53:33.354 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:53:33 smithi027 podman[111952]: 2024-06-06 08:53:33.106300701 +0000 UTC m=+1.589715797 container died c86825e1d59d746a47ee936f69970038324c3ebdb1b2c1a40a1e7e137e4b85bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-06T08:53:33.698 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:53:33 smithi027 podman[111952]: 2024-06-06 08:53:33.464739124 +0000 UTC m=+1.948154212 container cleanup c86825e1d59d746a47ee936f69970038324c3ebdb1b2c1a40a1e7e137e4b85bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:53:33.698 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:53:33 smithi027 bash[111952]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d 2024-06-06T08:53:33.956 INFO:journalctl@ceph.mds.d.smithi027.stdout:Jun 06 08:53:33 smithi027 podman[111964]: 2024-06-06 08:53:33.699743799 +0000 UTC m=+0.587215850 container remove c86825e1d59d746a47ee936f69970038324c3ebdb1b2c1a40a1e7e137e4b85bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-06-06T08:53:33.961 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-06T08:53:33.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:33.995 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-06T08:53:33.995 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-06T08:53:33.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.g 2024-06-06T08:53:34.334 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:34 smithi027 systemd[1]: Stopping Ceph mds.g for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:34.706 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:34 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-g[74020]: 2024-06-06T08:53:34.335+0000 7ff358f22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:34.706 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:34 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-g[74020]: 2024-06-06T08:53:34.335+0000 7ff358f22700 -1 mds.g *** got signal Terminated *** 2024-06-06T08:53:35.706 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:35 smithi027 podman[112074]: 2024-06-06 08:53:35.444240972 +0000 UTC m=+1.214950349 container died 95bb7bab2ce79df9407cab755f0c6ffb0b85128b99a85ba9be71b30ed1b9bea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-g, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:53:35.968 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:35 smithi027 podman[112074]: 2024-06-06 08:53:35.78592603 +0000 UTC m=+1.556635405 container cleanup 95bb7bab2ce79df9407cab755f0c6ffb0b85128b99a85ba9be71b30ed1b9bea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-g, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-06T08:53:35.968 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:35 smithi027 bash[112074]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-g 2024-06-06T08:53:36.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-06T08:53:36.223 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:35 smithi027 podman[112086]: 2024-06-06 08:53:35.9704114 +0000 UTC m=+0.522218067 container remove 95bb7bab2ce79df9407cab755f0c6ffb0b85128b99a85ba9be71b30ed1b9bea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-06T08:53:36.223 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:36 smithi027 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.g.service: Deactivated successfully. 2024-06-06T08:53:36.223 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:36 smithi027 systemd[1]: Stopped Ceph mds.g for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:53:36.224 INFO:journalctl@ceph.mds.g.smithi027.stdout:Jun 06 08:53:36 smithi027 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.g.service: Consumed 1.129s CPU time. 2024-06-06T08:53:36.234 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:36.235 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-06T08:53:36.235 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-06T08:53:36.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.j 2024-06-06T08:53:36.546 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:36 smithi027 systemd[1]: Stopping Ceph mds.j for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:36.956 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-j[72745]: 2024-06-06T08:53:36.547+0000 7f6345042700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:36.956 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:36 smithi027 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-j[72745]: 2024-06-06T08:53:36.547+0000 7f6345042700 -1 mds.j *** got signal Terminated *** 2024-06-06T08:53:38.179 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:37 smithi027 podman[112197]: 2024-06-06 08:53:37.922766993 +0000 UTC m=+1.463662120 container died 7753269c0d96ff9b67c9f87a9493286dab831c6b67ae75f649760ef7edf2873d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T08:53:38.456 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:38 smithi027 podman[112197]: 2024-06-06 08:53:38.28159101 +0000 UTC m=+1.822486136 container cleanup 7753269c0d96ff9b67c9f87a9493286dab831c6b67ae75f649760ef7edf2873d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-06T08:53:38.456 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:38 smithi027 bash[112197]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-j 2024-06-06T08:53:38.775 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-06T08:53:38.799 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:38 smithi027 podman[112209]: 2024-06-06 08:53:38.599576369 +0000 UTC m=+0.676956043 container remove 7753269c0d96ff9b67c9f87a9493286dab831c6b67ae75f649760ef7edf2873d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528) 2024-06-06T08:53:38.799 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:38 smithi027 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.j.service: Deactivated successfully. 2024-06-06T08:53:38.799 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:38 smithi027 systemd[1]: Stopped Ceph mds.j for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:53:38.799 INFO:journalctl@ceph.mds.j.smithi027.stdout:Jun 06 08:53:38 smithi027 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.117s CPU time. 2024-06-06T08:53:38.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:38.808 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-06T08:53:38.809 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-06T08:53:38.809 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.b 2024-06-06T08:53:39.110 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:53:38 smithi176 systemd[1]: Stopping Ceph mds.b for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:39.506 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:53:39 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b[67321]: 2024-06-06T08:53:39.111+0000 7f95e066c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:39.506 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:53:39 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b[67321]: 2024-06-06T08:53:39.111+0000 7f95e066c700 -1 mds.b *** got signal Terminated *** 2024-06-06T08:53:41.506 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:53:41 smithi176 podman[74657]: 2024-06-06 08:53:41.128358815 +0000 UTC m=+2.120963569 container died 8210361d2b03276a5f83d713b7df95b16f1ae8ecca65b3dd4c41ded734a7287a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-06T08:53:41.952 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:53:41 smithi176 podman[74657]: 2024-06-06 08:53:41.703676374 +0000 UTC m=+2.696281116 container cleanup 8210361d2b03276a5f83d713b7df95b16f1ae8ecca65b3dd4c41ded734a7287a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-06T08:53:41.953 INFO:journalctl@ceph.mds.b.smithi176.stdout:Jun 06 08:53:41 smithi176 bash[74657]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-b 2024-06-06T08:53:42.079 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-06T08:53:42.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:42.112 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-06T08:53:42.112 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-06T08:53:42.112 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.e 2024-06-06T08:53:42.256 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:42 smithi176 systemd[1]: Stopping Ceph mds.e for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:42.756 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:42 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e[66713]: 2024-06-06T08:53:42.440+0000 7fc386fde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:42.756 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:42 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e[66713]: 2024-06-06T08:53:42.440+0000 7fc386fde700 -1 mds.e *** got signal Terminated *** 2024-06-06T08:53:45.456 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:45 smithi176 podman[74777]: 2024-06-06 08:53:45.126052235 +0000 UTC m=+2.792071422 container died 58df65fbc510362d6510c5ca146bdbbab100751a87cae778438e7f2949111561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-06T08:53:45.731 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:45 smithi176 podman[74777]: 2024-06-06 08:53:45.557630669 +0000 UTC m=+3.223649848 container cleanup 58df65fbc510362d6510c5ca146bdbbab100751a87cae778438e7f2949111561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc) 2024-06-06T08:53:45.731 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:45 smithi176 bash[74777]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e 2024-06-06T08:53:45.971 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-06T08:53:45.992 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:45 smithi176 podman[74791]: 2024-06-06 08:53:45.73397699 +0000 UTC m=+0.605929978 container remove 58df65fbc510362d6510c5ca146bdbbab100751a87cae778438e7f2949111561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-06T08:53:45.992 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:45 smithi176 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.e.service: Deactivated successfully. 2024-06-06T08:53:45.992 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:45 smithi176 systemd[1]: Stopped Ceph mds.e for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:53:45.992 INFO:journalctl@ceph.mds.e.smithi176.stdout:Jun 06 08:53:45 smithi176 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.e.service: Consumed 4.121s CPU time. 2024-06-06T08:53:46.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:46.003 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-06T08:53:46.003 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-06T08:53:46.003 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.h 2024-06-06T08:53:46.256 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:46 smithi176 systemd[1]: Stopping Ceph mds.h for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:46.756 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:53:46.344+0000 7ff4c5edd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:46.756 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:46 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h[67018]: 2024-06-06T08:53:46.344+0000 7ff4c5edd700 -1 mds.h *** got signal Terminated *** 2024-06-06T08:53:49.460 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:49 smithi176 podman[74899]: 2024-06-06 08:53:49.130033052 +0000 UTC m=+2.906360350 container died b19b4f193920b50869d5e50b51cc84af4d835bad6072dc6535a128b42d596ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-06T08:53:49.756 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:49 smithi176 podman[74899]: 2024-06-06 08:53:49.569368235 +0000 UTC m=+3.345695530 container cleanup b19b4f193920b50869d5e50b51cc84af4d835bad6072dc6535a128b42d596ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-06T08:53:49.756 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:49 smithi176 bash[74899]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h 2024-06-06T08:53:49.995 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-06T08:53:50.015 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:49 smithi176 podman[74914]: 2024-06-06 08:53:49.762228064 +0000 UTC m=+0.633711341 container remove b19b4f193920b50869d5e50b51cc84af4d835bad6072dc6535a128b42d596ea7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-06T08:53:50.015 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:49 smithi176 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.h.service: Deactivated successfully. 2024-06-06T08:53:50.015 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:49 smithi176 systemd[1]: Stopped Ceph mds.h for 8da40ada-23de-11ef-bc9b-c7b262605968. 2024-06-06T08:53:50.015 INFO:journalctl@ceph.mds.h.smithi176.stdout:Jun 06 08:53:49 smithi176 systemd[1]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.h.service: Consumed 22.277s CPU time. 2024-06-06T08:53:50.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:50.026 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-06T08:53:50.026 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-06T08:53:50.026 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.k 2024-06-06T08:53:50.346 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:53:50 smithi176 systemd[1]: Stopping Ceph mds.k for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:50.756 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:53:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k[66395]: 2024-06-06T08:53:50.347+0000 7efd42d7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:50.756 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:53:50 smithi176 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k[66395]: 2024-06-06T08:53:50.347+0000 7efd42d7e700 -1 mds.k *** got signal Terminated *** 2024-06-06T08:53:54.006 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:53:53 smithi176 podman[75021]: 2024-06-06 08:53:53.75646152 +0000 UTC m=+3.512183155 container died 2543aa97c256160cba8bcb3c053e8e076c1ae34fb63c552373a8e0eada355dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:53:54.500 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:53:54 smithi176 podman[75021]: 2024-06-06 08:53:54.090035229 +0000 UTC m=+3.845756858 container cleanup 2543aa97c256160cba8bcb3c053e8e076c1ae34fb63c552373a8e0eada355dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T08:53:54.500 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:53:54 smithi176 bash[75021]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k 2024-06-06T08:53:54.500 INFO:journalctl@ceph.mds.k.smithi176.stdout:Jun 06 08:53:54 smithi176 podman[75046]: 2024-06-06 08:53:54.283196079 +0000 UTC m=+0.527814739 container remove 2543aa97c256160cba8bcb3c053e8e076c1ae34fb63c552373a8e0eada355dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-06T08:53:54.519 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-06T08:53:54.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:54.552 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-06T08:53:54.552 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-06T08:53:54.552 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.c 2024-06-06T08:53:54.858 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:53:54 smithi203 systemd[1]: Stopping Ceph mds.c for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:55.120 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:53:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[71528]: 2024-06-06T08:53:54.859+0000 7f7224ac5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:55.120 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:53:54 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c[71528]: 2024-06-06T08:53:54.859+0000 7f7224ac5700 -1 mds.c *** got signal Terminated *** 2024-06-06T08:53:57.409 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:53:57 smithi203 podman[75280]: 2024-06-06 08:53:57.126298717 +0000 UTC m=+2.372706817 container died 1faa63afe90c8d67e78f665f766374a336597d59d3cb6deb71d433a0150f0214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T08:53:57.734 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:53:57 smithi203 podman[75280]: 2024-06-06 08:53:57.485778736 +0000 UTC m=+2.732186838 container cleanup 1faa63afe90c8d67e78f665f766374a336597d59d3cb6deb71d433a0150f0214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-06T08:53:57.734 INFO:journalctl@ceph.mds.c.smithi203.stdout:Jun 06 08:53:57 smithi203 bash[75280]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-c 2024-06-06T08:53:57.875 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-06T08:53:57.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:53:57.910 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-06T08:53:57.911 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-06T08:53:57.911 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.f 2024-06-06T08:53:58.239 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:53:57 smithi203 systemd[1]: Stopping Ceph mds.f for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:53:58.239 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:53:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f[66895]: 2024-06-06T08:53:58.222+0000 7fd4e0cc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:53:58.239 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:53:58 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f[66895]: 2024-06-06T08:53:58.222+0000 7fd4e0cc9700 -1 mds.f *** got signal Terminated *** 2024-06-06T08:54:01.396 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:54:01 smithi203 podman[75413]: 2024-06-06 08:54:01.133398215 +0000 UTC m=+3.005653792 container died aaca192b1d3e53627969c00ca43a23f19b30e1b46f2f60127c7e3aa0d8976087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-06T08:54:01.739 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:54:01 smithi203 podman[75413]: 2024-06-06 08:54:01.498247646 +0000 UTC m=+3.370503214 container cleanup aaca192b1d3e53627969c00ca43a23f19b30e1b46f2f60127c7e3aa0d8976087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-06T08:54:01.739 INFO:journalctl@ceph.mds.f.smithi203.stdout:Jun 06 08:54:01 smithi203 bash[75413]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-f 2024-06-06T08:54:01.928 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-06T08:54:01.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:54:01.961 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-06T08:54:01.961 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-06T08:54:01.961 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.i 2024-06-06T08:54:02.283 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:54:02 smithi203 systemd[1]: Stopping Ceph mds.i for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:54:02.739 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:54:02 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:54:02.283+0000 7fc3435b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:54:02.739 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:54:02 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i[67502]: 2024-06-06T08:54:02.283+0000 7fc3435b2700 -1 mds.i *** got signal Terminated *** 2024-06-06T08:54:05.393 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:54:05 smithi203 podman[75532]: 2024-06-06 08:54:05.131430255 +0000 UTC m=+2.953716608 container died 6ec074f172ee9fff2ab35fa00ff2544c98cdcf01e525428c10cc690e56fc2f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-06T08:54:05.726 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:54:05 smithi203 podman[75532]: 2024-06-06 08:54:05.494088791 +0000 UTC m=+3.316375133 container cleanup 6ec074f172ee9fff2ab35fa00ff2544c98cdcf01e525428c10cc690e56fc2f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-06T08:54:05.726 INFO:journalctl@ceph.mds.i.smithi203.stdout:Jun 06 08:54:05 smithi203 bash[75532]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-i 2024-06-06T08:54:05.884 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-06T08:54:05.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:54:05.915 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-06T08:54:05.915 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-06T08:54:05.915 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.l 2024-06-06T08:54:06.239 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:54:05 smithi203 systemd[1]: Stopping Ceph mds.l for 8da40ada-23de-11ef-bc9b-c7b262605968... 2024-06-06T08:54:06.739 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:54:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l[66592]: 2024-06-06T08:54:06.263+0000 7f3a846f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T08:54:06.739 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:54:06 smithi203 ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l[66592]: 2024-06-06T08:54:06.263+0000 7f3a846f1700 -1 mds.l *** got signal Terminated *** 2024-06-06T08:54:10.613 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:54:10 smithi203 podman[75652]: 2024-06-06 08:54:10.271900964 +0000 UTC m=+4.136122414 container died ca8f63a505d194fde10e36358c7263241cab0dabb7469ccc50742cadb452e580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-06T08:54:10.989 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:54:10 smithi203 podman[75652]: 2024-06-06 08:54:10.689753773 +0000 UTC m=+4.553975217 container cleanup ca8f63a505d194fde10e36358c7263241cab0dabb7469ccc50742cadb452e580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True) 2024-06-06T08:54:10.989 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:54:10 smithi203 bash[75652]: ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l 2024-06-06T08:54:10.990 INFO:journalctl@ceph.mds.l.smithi203.stdout:Jun 06 08:54:10 smithi203 podman[75665]: 2024-06-06 08:54:10.849361601 +0000 UTC m=+0.570881525 container remove ca8f63a505d194fde10e36358c7263241cab0dabb7469ccc50742cadb452e580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-8da40ada-23de-11ef-bc9b-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T08:54:11.073 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8da40ada-23de-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-06T08:54:11.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T08:54:11.103 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-06T08:54:11.103 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T08:54:11.305 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:54:13.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T08:54:14.069 INFO:teuthology.orchestra.run.smithi176.stdout:Deleting cluster with fsid: 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:54:16.406 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T08:54:16.597 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:54:18.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:54:19.032 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:54:19.062 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T08:54:19.087 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-06T08:54:19.089 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/remote/smithi027/crash 2024-06-06T08:54:19.090 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T08:54:19.120 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T08:54:19.120 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-06-06T08:54:19.122 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/remote/smithi176/crash 2024-06-06T08:54:19.123 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T08:54:19.286 INFO:teuthology.orchestra.run.smithi176.stderr:tar: /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T08:54:19.286 INFO:teuthology.orchestra.run.smithi176.stderr:tar: Error is not recoverable: exiting now 2024-06-06T08:54:19.289 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/remote/smithi203/crash 2024-06-06T08:54:19.289 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T08:54:19.320 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T08:54:19.320 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2024-06-06T08:54:19.321 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-06T08:54:19.322 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-06T08:54:19.363 INFO:teuthology.orchestra.run.smithi027.stdout:2024-06-06T08:43:10.011723+0000 mds.h (mds.2) 24 : cluster [WRN] Scrub error on inode 0x10000002389 (/volumes/qa/sv_1/97e6dc45-7eb6-4039-a0c7-4313ff988f86/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/connectors/pec/event_generator.c) see mds.h log and `damage ls` output for details 2024-06-06T08:54:19.364 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-06T08:54:19.364 INFO:tasks.cephadm:Compressing logs... 2024-06-06T08:54:19.365 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T08:54:19.406 DEBUG:teuthology.orchestra.run.smithi176:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T08:54:19.409 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T08:54:19.431 INFO:teuthology.orchestra.run.smithi176.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T08:54:19.431 INFO:teuthology.orchestra.run.smithi176.stderr:: No such file or directory 2024-06-06T08:54:19.431 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T08:54:19.431 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T08:54:19.432 INFO:teuthology.orchestra.run.smithi176.stderr:gzip/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-06T08:54:19.432 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-06T08:54:19.432 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-06T08:54:19.432 INFO:teuthology.orchestra.run.smithi176.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:: No such file or directory 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.audit.log: /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-06T08:54:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-06T08:54:19.439 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mgr.x.log: 88.7% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T08:54:19.439 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-06T08:54:19.439 INFO:teuthology.orchestra.run.smithi027.stderr: 90.2% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T08:54:19.441 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-06T08:54:19.441 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.4.log: --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-06T08:54:19.441 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-06T08:54:19.441 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mgr.y.log: 90.3% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T08:54:19.441 INFO:teuthology.orchestra.run.smithi176.stderr: 88.8% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-06T08:54:19.441 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-06-06T08:54:19.441 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T08:54:19.447 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.h.log: /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.10.log: 92.0% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-06T08:54:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T08:54:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:: No such file or directory 2024-06-06T08:54:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T08:54:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-06T08:54:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-06T08:54:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T08:54:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-06T08:54:19.449 INFO:teuthology.orchestra.run.smithi203.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T08:54:19.449 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mgr.z.log: 88.7% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.cephadm.log 90.4% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.11.log: 92.1% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-06T08:54:19.450 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T08:54:19.451 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mon.a.log: 94.6% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-06T08:54:19.451 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T08:54:19.451 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-06T08:54:19.451 INFO:teuthology.orchestra.run.smithi027.stderr: 94.6% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-06T08:54:19.451 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T08:54:19.451 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T08:54:19.452 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-06T08:54:19.452 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-volume.log: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T08:54:19.452 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-06T08:54:19.452 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-06T08:54:19.452 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-06T08:54:19.452 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-06T08:54:19.452 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-06T08:54:19.456 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-06T08:54:19.456 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T08:54:19.456 INFO:teuthology.orchestra.run.smithi027.stderr: 91.1% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T08:54:19.460 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-06T08:54:19.469 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T08:54:19.482 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-06T08:54:19.485 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-volume.log: /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-06T08:54:19.487 INFO:teuthology.orchestra.run.smithi203.stderr: 91.0% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T08:54:19.501 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-06T08:54:19.517 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mon.b.log: /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T08:54:19.555 INFO:teuthology.orchestra.run.smithi176.stderr: 94.6% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-06T08:54:19.842 INFO:teuthology.orchestra.run.smithi027.stderr: 90.3% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-06T08:54:19.873 INFO:teuthology.orchestra.run.smithi203.stderr: 91.8% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-06T08:54:20.050 INFO:teuthology.orchestra.run.smithi176.stderr: 91.7% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-06T08:54:20.053 INFO:teuthology.orchestra.run.smithi203.stderr: 94.0% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-06T08:54:20.443 INFO:teuthology.orchestra.run.smithi176.stderr: 94.5% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-06T08:54:20.498 INFO:teuthology.orchestra.run.smithi027.stderr: 90.4% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-06T08:54:25.418 INFO:teuthology.orchestra.run.smithi203.stderr: 94.9% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-06T08:54:27.386 INFO:teuthology.orchestra.run.smithi176.stderr: 93.9% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-06T08:54:30.258 INFO:teuthology.orchestra.run.smithi203.stderr: 93.8% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-06T08:54:30.285 INFO:teuthology.orchestra.run.smithi203.stderr: 91.7% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-06T08:54:32.420 INFO:teuthology.orchestra.run.smithi027.stderr: 91.5% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-06T08:54:32.514 INFO:teuthology.orchestra.run.smithi176.stderr: 91.8% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-06T08:54:34.759 INFO:teuthology.orchestra.run.smithi176.stderr: 91.5% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-06T08:54:35.146 INFO:teuthology.orchestra.run.smithi027.stderr: 91.5% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-06T08:54:35.283 INFO:teuthology.orchestra.run.smithi027.stderr: 91.4% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-06T08:54:35.580 INFO:teuthology.orchestra.run.smithi203.stderr: 91.7% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-06T08:54:36.061 INFO:teuthology.orchestra.run.smithi027.stderr: 91.5% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-06T08:54:36.063 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-06-06T08:54:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m16.642s 2024-06-06T08:54:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:user 1m0.776s 2024-06-06T08:54:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m1.822s 2024-06-06T08:54:36.176 INFO:teuthology.orchestra.run.smithi176.stderr: 91.9% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-06T08:54:37.578 INFO:teuthology.orchestra.run.smithi203.stderr: 91.7% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-06T08:54:39.099 INFO:teuthology.orchestra.run.smithi203.stderr: 91.4% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-06T08:54:39.101 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-06-06T08:54:39.101 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m19.678s 2024-06-06T08:54:39.101 INFO:teuthology.orchestra.run.smithi203.stderr:user 1m19.963s 2024-06-06T08:54:39.102 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m2.364s 2024-06-06T08:54:40.274 INFO:teuthology.orchestra.run.smithi176.stderr: 91.6% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-06T08:55:18.582 INFO:teuthology.orchestra.run.smithi176.stderr: 93.6% -- replaced with /var/log/ceph/8da40ada-23de-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-06T08:55:18.584 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-06-06T08:55:18.584 INFO:teuthology.orchestra.run.smithi176.stderr:real 0m59.162s 2024-06-06T08:55:18.584 INFO:teuthology.orchestra.run.smithi176.stderr:user 2m9.901s 2024-06-06T08:55:18.584 INFO:teuthology.orchestra.run.smithi176.stderr:sys 0m3.929s 2024-06-06T08:55:18.584 INFO:tasks.cephadm:Archiving logs... 2024-06-06T08:55:18.585 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/remote/smithi027/log 2024-06-06T08:55:18.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T08:55:28.869 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/remote/smithi176/log 2024-06-06T08:55:28.869 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T08:55:47.263 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/remote/smithi203/log 2024-06-06T08:55:47.264 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T08:55:58.593 INFO:tasks.cephadm:Removing cluster... 2024-06-06T08:55:58.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 --force 2024-06-06T08:55:58.799 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:55:59.168 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 --force 2024-06-06T08:55:59.358 INFO:teuthology.orchestra.run.smithi176.stdout:Deleting cluster with fsid: 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:55:59.773 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8da40ada-23de-11ef-bc9b-c7b262605968 --force 2024-06-06T08:55:59.965 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 8da40ada-23de-11ef-bc9b-c7b262605968 2024-06-06T08:56:00.325 INFO:tasks.cephadm:Removing cephadm ... 2024-06-06T08:56:00.326 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T08:56:00.346 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T08:56:00.362 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T08:56:00.377 INFO:tasks.cephadm:Teardown complete 2024-06-06T08:56:00.378 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-06T08:56:00.396 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-06T08:56:00.396 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T08:56:00.399 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T08:56:00.404 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T08:56:00.512 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-06T08:56:00.524 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-06T08:56:00.524 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:56:00.527 DEBUG:teuthology.orchestra.run.smithi176:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:56:00.529 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:56:00.542 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:56:00.542 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:56:00.543 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:56:00.546 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:56:00.547 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-06-06T08:56:00.547 INFO:teuthology.orchestra.run.smithi176.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +91us[ +89us] +/- 57ms 2024-06-06T08:56:00.547 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -2012us[-2015us] +/- 80ms 2024-06-06T08:56:00.547 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +1721us[+1721us] +/- 75ms 2024-06-06T08:56:00.547 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:56:00.549 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:56:00.550 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-06-06T08:56:00.550 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +113us[ +118us] +/- 58ms 2024-06-06T08:56:00.550 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -2008us[-2003us] +/- 80ms 2024-06-06T08:56:00.550 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +1734us[+1739us] +/- 75ms 2024-06-06T08:56:00.550 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:56:00.551 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:56:00.551 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-06-06T08:56:00.551 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +129us[ +111us] +/- 58ms 2024-06-06T08:56:00.551 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -1995us[-2012us] +/- 80ms 2024-06-06T08:56:00.551 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +1749us[+1731us] +/- 75ms 2024-06-06T08:56:00.551 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:56:00.552 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-06T08:56:00.562 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-06T08:56:00.563 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-06T08:56:00.599 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T08:56:00.695 DEBUG:teuthology.orchestra.run.smithi176:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T08:56:00.762 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T08:56:00.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:56:00.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:56:00.859 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:56:00.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:56:00.887 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:56:00.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:56:00.915 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-06T08:56:00.927 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-06T08:56:00.965 INFO:teuthology.task.internal:Duration was 2118.166737 seconds 2024-06-06T08:56:00.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-06T08:56:01.016 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-06T08:56:01.017 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T08:56:01.019 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T08:56:01.022 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T08:56:01.056 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:56:01.058 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:56:01.061 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:56:01.548 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-06T08:56:01.548 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-06-06T08:56:01.548 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T08:56:01.573 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi176.front.sepia.ceph.com 2024-06-06T08:56:01.573 DEBUG:teuthology.orchestra.run.smithi176:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T08:56:01.614 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-06-06T08:56:01.614 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T08:56:01.638 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-06T08:56:01.638 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T08:56:01.641 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T08:56:01.656 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T08:56:01.704 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-06T08:56:01.704 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T08:56:01.711 DEBUG:teuthology.orchestra.run.smithi176:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T08:56:01.723 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T08:56:02.574 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-06T08:56:02.586 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-06T08:56:02.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T08:56:02.612 DEBUG:teuthology.orchestra.run.smithi176:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T08:56:02.638 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T08:56:02.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-06T08:56:02.676 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:56:02.678 DEBUG:teuthology.orchestra.run.smithi176:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:56:02.681 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:56:02.700 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-06-06T08:56:02.702 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = core 2024-06-06T08:56:02.729 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-06-06T08:56:02.769 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:56:02.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:56:02.784 DEBUG:teuthology.orchestra.run.smithi176:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:56:02.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:56:02.799 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:56:02.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:56:02.824 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-06T08:56:02.837 INFO:teuthology.task.internal:Transferring archived files... 2024-06-06T08:56:02.839 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/remote/smithi027 2024-06-06T08:56:02.839 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T08:56:02.896 DEBUG:teuthology.misc:Transferring archived files from smithi176:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/remote/smithi176 2024-06-06T08:56:02.896 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T08:56:02.931 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743249/remote/smithi203 2024-06-06T08:56:02.932 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T08:56:02.969 INFO:teuthology.task.internal:Removing archive directory... 2024-06-06T08:56:02.969 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T08:56:02.972 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T08:56:02.974 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T08:56:03.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-06T08:56:03.033 INFO:teuthology.task.internal:Not uploading archives. 2024-06-06T08:56:03.033 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-06T08:56:03.059 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-06T08:56:03.059 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T08:56:03.062 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T08:56:03.064 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T08:56:03.074 INFO:teuthology.orchestra.run.smithi027.stdout: 266037 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 08:56 /home/ubuntu/cephtest 2024-06-06T08:56:03.077 INFO:teuthology.orchestra.run.smithi176.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 08:56 /home/ubuntu/cephtest 2024-06-06T08:56:03.079 INFO:teuthology.orchestra.run.smithi203.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 08:56 /home/ubuntu/cephtest 2024-06-06T08:56:03.080 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-06T08:56:03.108 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-06T08:56:03.212 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} duration: 2118.1667370796204 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-06-06T08:56:03.212 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T08:56:03.308 INFO:teuthology.run:FAIL